START for evaluation of target detection and tracking

John Irvine; Scott K. Ralph; Mark R. Stevens; Jeremy Marvel; Magnus Snorrason; David Gwilt

doi:10.1117/12.603432

19 May 2005 START for evaluation of target detection and tracking

John Irvine, Scott K. Ralph, Mark R. Stevens, Jeremy Marvel, Magnus Snorrason, David Gwilt

Proceedings Volume 5807, Automatic Target Recognition XV; (2005) https://doi.org/10.1117/12.603432
Event: Defense and Security, 2005, Orlando, Florida, United States

Abstract

A major challenge for ATR evaluation is developing an accurate image truth that can be compared to an ATR algo-rithm's decisions to assess performance. While many standard truthing methods and scoring metrics exist for stationary targets in still imagery, techniques for dealing with motion imagery and moving targets are not as prevalent. This is par-tially due to the fact that the moving imagery / moving targets scenario introduces the data association problem of as-signing targets to tracks. Video datasets typically contain far more imagery than static collections, increasing the size of the truthing task. Specifying the types and locations of the targets present for a large number of images is tedious, time consuming, and error prone. In this paper, we present an updated version of a complete truthing system we call the Scoring, Truthing, And Registration Toolkit (START). The application consists of two components: a truthing compo-nents that assists in the automated construction of image truth, and a scoring component that assesses the performance of a given algorithm relative to the specified truth. In motion imagery, both stationary and moving targets can be de-tected and tracked over portions of a motion imagery clip. We summarize the capabilities of START with emphasis on the target tracking and truthing diagnostics. The user manually truths certain key frames, truth for intermediate frames is then inferred and sets of diagnostics verify the quality of the truth. If ambiguous situations are encountered in the inter-mediate frames, diagnostics flag the problem so that the user can intervene manually. This approach can dramatically reduce the effort required for truthing video data, while maintaining high fidelity in the truth data. We present the results of two user evaluations of START, one addressing the accuracy and the other focusing on the human factors aspects of the design.

Citation Download Citation

John Irvine, Scott K. Ralph, Mark R. Stevens, Jeremy Marvel, Magnus Snorrason, and David Gwilt "START for evaluation of target detection and tracking", Proc. SPIE 5807, Automatic Target Recognition XV, (19 May 2005); https://doi.org/10.1117/12.603432

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available