Standards for the artificial intelligence community

Erik Blasch; James Sung

doi:10.1117/12.2589025

13 April 2021 Standards for the artificial intelligence community

Erik Blasch, James Sung

Proceedings Volume 11729, Automatic Target Recognition XXXI; 117290X (2021) https://doi.org/10.1117/12.2589025
Event: SPIE Defense + Commercial Sensing, 2021, Online Only

Abstract

Artificial intelligence (AI) / Machine Learning (ML) applications are widely available for different domains such as commercial, industrial, and intelligence applications. In particular, the use of AI applications for the security environment requires standards to manage expectations for users to understand how the results were derived. A reliance on “black boxes” to generate predictions and inform decisions could lead to errors of analysis. This paper explores the development of potential standards designed for each stage of the development of an AI/ML system to help enable trust, transparency, and explainability. Specifically, the paper utilizes the standards outlined in Intelligence Community Directive 203 (Analytic Standards) to hold machine outputs to the same rigorous accountability standards as performed by humans. Building on the ICD203, the Multi-Source AI Scorecard Table (MAST) was developed to support the community towards test and evaluation of AI/ML techniques. The paper provides discussion towards using MAST to rate a semantic processing tool for processing noisy, unstructured, and complex microtext in the form of streaming chat for video call outs. The scoring is notional, but provides a discussion on how MAST could be used as a standard to compare AI/ML methods that complements datasheets and model cards.

Conference Presentation

Citation Download Citation

Erik Blasch and James Sung "Standards for the artificial intelligence community", Proc. SPIE 11729, Automatic Target Recognition XXXI, 117290X (13 April 2021); https://doi.org/10.1117/12.2589025

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available