Contrastive self-supervised learning from 100 million medical images with optional supervision

Florin C. Ghesu; Bogdan Georgescu; Awais Mansoor; Youngjin Yoo; Dominik Neumann; Pragnesh Kumar Patel; Reddappagari Suryanarayana Vishwanath; James M. Balter; Yue Cao; Sasa Grbic; Dorin Comaniciu

doi:10.1117/1.JMI.9.6.064503

30 November 2022 Contrastive self-supervised learning from 100 million medical images with optional supervision

Florin C. Ghesu, Bogdan Georgescu, Awais Mansoor, Youngjin Yoo, Dominik Neumann, Pragnesh Kumar Patel, Reddappagari Suryanarayana Vishwanath, James M. Balter, Yue Cao, Sasa Grbic, Dorin Comaniciu

Author Affiliations +

Journal of Medical Imaging, Vol. 9, Issue 6, 064503 (November 2022). https://doi.org/10.1117/1.JMI.9.6.064503

Abstract

Purpose

Building accurate and robust artificial intelligence systems for medical image assessment requires the creation of large sets of annotated training examples. However, constructing such datasets is very costly due to the complex nature of annotation tasks, which often require expert knowledge (e.g., a radiologist). To counter this limitation, we propose a method to learn from medical images at scale in a self-supervised way.

Approach

Our approach, based on contrastive learning and online feature clustering, leverages training datasets of over 100,000,000 medical images of various modalities, including radiography, computed tomography (CT), magnetic resonance (MR) imaging, and ultrasonography (US). We propose to use the learned features to guide model training in supervised and hybrid self-supervised/supervised regime on various downstream tasks.

Results

We highlight a number of advantages of this strategy on challenging image assessment problems in radiography, CT, and MR: (1) significant increase in accuracy compared to the state-of-the-art (e.g., area under the curve boost of 3% to 7% for detection of abnormalities from chest radiography scans and hemorrhage detection on brain CT); (2) acceleration of model convergence during training by up to 85% compared with using no pretraining (e.g., 83% when training a model for detection of brain metastases in MR scans); and (3) increase in robustness to various image augmentations, such as intensity variations, rotations or scaling reflective of data variation seen in the field.

Conclusions

The proposed approach enables large gains in accuracy and robustness on challenging image assessment problems. The improvement is significant compared with other state-of-the-art approaches trained on medical or vision images (e.g., ImageNet).

Citation Download Citation

Florin C. Ghesu, Bogdan Georgescu, Awais Mansoor, Youngjin Yoo, Dominik Neumann, Pragnesh Kumar Patel, Reddappagari Suryanarayana Vishwanath, James M. Balter, Yue Cao, Sasa Grbic, and Dorin Comaniciu "Contrastive self-supervised learning from 100 million medical images with optional supervision," Journal of Medical Imaging 9(6), 064503 (30 November 2022). https://doi.org/10.1117/1.JMI.9.6.064503

Received: 25 May 2022; Accepted: 14 November 2022; Published: 30 November 2022

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available