New scientific accuracy measure for performance evaluation of human-computer diagnostic systems

Samuel C. Lee; Elisa T. Lee; Yiming Wang

doi:10.1117/12.441586

25 September 2001 New scientific accuracy measure for performance evaluation of human-computer diagnostic systems

Samuel C. Lee, Elisa T. Lee, Yiming Wang

Proceedings Volume 4553, Visualization and Optimization Techniques; (2001) https://doi.org/10.1117/12.441586
Event: Multispectral Image Processing and Pattern Recognition, 2001, Wuhan, China

Abstract

This paper first presents a new scientific accuracy measure (denoted by G) for assessing/evaluating the performance of computer medical diagnostic (CMD) systems by incorporating the true positives (TP), true negatives (TN), false positives (FP), and false negatives (FN) of human and computer's diagnoses with respect to each other. Based on G, a formula for computing a multi-parameter sensitivity vector S(G), with the assumption that the system parameter percentage variations are small, is then proposed. For a given set of parameter percentage errors, from the expression of S(G), we can compute the error bound of G and assess the reliability of the system with human and/or computer errors being taken into consideration. It has been demonstrated that the new measure G is capable of providing consistent performance evaluation of a CMD system in general. Based on the value of G, a CMD system can be classified as having 'good', 'fair', or 'poor' performance. Even though the proposed basic accuracy measure and its sensitivity study are derived based on the diagnosis using two diagnostic categories (positive and negative) compared by two observers (a human expert and a computer system), however, its methodology can be extended to CMD systems with multiple diagnostic categories and observers. The formulas for measuring the performance of such systems are discussed and present.

Citation Download Citation

Samuel C. Lee, Elisa T. Lee, and Yiming Wang "New scientific accuracy measure for performance evaluation of human-computer diagnostic systems", Proc. SPIE 4553, Visualization and Optimization Techniques, (25 September 2001); https://doi.org/10.1117/12.441586

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available