Comparison of two classifiers when the data sets are imbalanced: the power of the area under the precision-recall curve as the figure of merit versus the area under the ROC curve

Berkman Sahiner; Weijie Chen; Aria Pezeshk; Nicholas Petrick

doi:10.1117/12.2254742

10 March 2017 Comparison of two classifiers when the data sets are imbalanced: the power of the area under the precision-recall curve as the figure of merit versus the area under the ROC curve

Berkman Sahiner, Weijie Chen, Aria Pezeshk, Nicholas Petrick

Author Affiliations +

Proceedings Volume 10136, Medical Imaging 2017: Image Perception, Observer Performance, and Technology Assessment; 101360G (2017) https://doi.org/10.1117/12.2254742
Event: SPIE Medical Imaging, 2017, Orlando, Florida, United States

Abstract

In many two-class problems in automated classification and information retrieval, the classes are imbalanced, and the separation between the positive and negative classes is large. The precision-recall (PR) curve has been suggested as an alternative to the receiver operating characteristic (ROC) curve to characterize the performance of automated systems when the classes are imbalanced, and the area under the precision-recall curve (AUCPR) has been suggested as an alternative performance measure to the area under the ROC curve (AUCROC). AUCPR and AUCROC are distinct measures of performance, even though the relationship between the precision-recall and ROC curves is well-known. In this study, we compared the statistical power of the AUCPR to that of the AUCROC. Our results indicate that the AUCPR can offer a small statistical advantage when the prevalence is low and the separation between the positive and negative classes is large. When the data set is more balanced or the separation between the classes is low or moderate, AUCROC has slightly higher power.

Conference Presentation

Citation Download Citation

Berkman Sahiner, Weijie Chen, Aria Pezeshk, and Nicholas Petrick "Comparison of two classifiers when the data sets are imbalanced: the power of the area under the precision-recall curve as the figure of merit versus the area under the ROC curve", Proc. SPIE 10136, Medical Imaging 2017: Image Perception, Observer Performance, and Technology Assessment, 101360G (10 March 2017); https://doi.org/10.1117/12.2254742

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available