Estimating deep learning model uncertainty of breast lesion classification to guide reading strategy in breast cancer screening

Sarah D. Verboom; Jaap Kroes; Mireille J. M. Broeders; Ioannis Sechopoulos

doi:10.1117/12.3006713

3 April 2024 Estimating deep learning model uncertainty of breast lesion classification to guide reading strategy in breast cancer screening

Sarah D. Verboom, Jaap Kroes, Mireille J. M. Broeders, Ioannis Sechopoulos

Author Affiliations +

Proceedings Volume 12927, Medical Imaging 2024: Computer-Aided Diagnosis; 129271Y (2024) https://doi.org/10.1117/12.3006713
Event: SPIE Medical Imaging, 2024, San Diego, California, United States

Conference Poster

Abstract

Estimating model uncertainty of artificial intelligence (AI)-based breast cancer detection algorithms could help guide the reading strategy in breast cancer screening. For example, the recall decision can be made solely by AI when it exhibits high certainty, while cases where the certainty is low should be read by radiologists. This study aims to evaluate two metrics to predict model uncertainty of a lesion characterization network: 1) the variance of a set of outputs generated with stochastic layer depth, and 2) the entropy of the average output. To test these approaches, 367 mammography exams with cancer (333 screen-detected, and 34 interval) and 367 cancer-negative exams from the Dutch Breast Cancer Screening Program were included. Using a commercial lesion detection algorithm operating at high sensitivity, 6,477 suspicious regions were included (14.1% labeled malignant). By varying the uncertainty threshold, the predictions were classified as certain or uncertain by a specified proportion. Radiologists double reading had a sensitivity of 90.9% (95% CI 89.0% – 92.7%) and a specificity of 93.8% (95% CI 93.2% – 96.2%) for all regions. At equal specificity, the network had a sensitivity of 92.1% (95% CI 89.9% – 94.0%) for all regions. The sensitivity of the network was higher for regions with low uncertainty for both approaches; for the top 50% most certain regions the sensitivity was 96.9% (95% CI 94.7% – 98.4%) and 97.1% (95% CI 94.9% – 98.8%) at equal specificity to radiologists. In conclusion, AI-based lesion classification uncertainty of breast regions can be estimated by applying stochastic layer depth during prediction.

Citation Download Citation

Sarah D. Verboom, Jaap Kroes, Mireille J. M. Broeders, and Ioannis Sechopoulos "Estimating deep learning model uncertainty of breast lesion classification to guide reading strategy in breast cancer screening", Proc. SPIE 12927, Medical Imaging 2024: Computer-Aided Diagnosis, 129271Y (3 April 2024); https://doi.org/10.1117/12.3006713

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available