Supplementing training with data from a shifted distribution for machine learning classifiers: adding more cases may not always help

Kenny H. Cha; Alexej Gossmann; Nicholas Petrick; Berkman Sahiner

doi:10.1117/12.2550538

16 March 2020 Supplementing training with data from a shifted distribution for machine learning classifiers: adding more cases may not always help

Kenny H. Cha, Alexej Gossmann, Nicholas Petrick, Berkman Sahiner

Author Affiliations +

Proceedings Volume 11316, Medical Imaging 2020: Image Perception, Observer Performance, and Technology Assessment; 113160S (2020) https://doi.org/10.1117/12.2550538
Event: SPIE Medical Imaging, 2020, Houston, Texas, United States

Abstract

In this study, we show that when a training data set is supplemented by drawing samples from a distribution that is different from that of the target population, the differences in the distributions of the original and supplemental training populations should be considered to maximize the performance of the classifier in the target population. Depending on these distributions, drawing a large number of cases from the supplemental distribution may result in lower performance compared to limiting the number of added cases. This is relevant for medical images when synthetic data is used for training a machine learning algorithm, which may result in a mixed distribution for the training set. We simulated a twoclass classification problem and determined the performance of a linear classifier and a neural network classifier on test cases when trained with cases from only the target distribution, and when cases from a shifted, supplemental distribution are added to a limited number of cases from the target distribution. We show that adding data from a supplemental distribution for machine learning classifier training may improve the performance on the target test distribution. However, given the same number of training cases from a mixed distribution, the performance may not reach the performance of only training on data from the target distribution. In addition, the increase in performance will peak or plateau, depending on the shift in the distribution and the number of cases from the supplemental distribution.

Conference Presentation

Citation Download Citation

Kenny H. Cha, Alexej Gossmann, Nicholas Petrick, and Berkman Sahiner "Supplementing training with data from a shifted distribution for machine learning classifiers: adding more cases may not always help", Proc. SPIE 11316, Medical Imaging 2020: Image Perception, Observer Performance, and Technology Assessment, 113160S (16 March 2020); https://doi.org/10.1117/12.2550538

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available