Longitudinal assessment of demographic representativeness in the Medical Imaging and Data Resource Center open data commons

Heather M. Whitney; Natalie Baughan; Kyle J. Myers; Karen Drukker; Judy Gichoya; Brad Bower; Weijie Chen; Nicholas Gruszauskas; Jayashree Kalpathy-Cramer; Sanmi Koyejo; Rui C. Sá; Berkman Sahiner; Zi Zhang; Maryellen L. Giger

doi:10.1117/1.JMI.10.6.061105

18 July 2023 Longitudinal assessment of demographic representativeness in the Medical Imaging and Data Resource Center open data commons

Heather M. Whitney, Natalie Baughan, Kyle J. Myers, Karen Drukker, Judy Gichoya, Brad Bower, Weijie Chen, Nicholas Gruszauskas, Jayashree Kalpathy-Cramer, Sanmi Koyejo, Rui C. Sá, Berkman Sahiner, Zi Zhang, Maryellen L. Giger

Author Affiliations +

Journal of Medical Imaging, Vol. 10, Issue 6, 061105 (July 2023). https://doi.org/10.1117/1.JMI.10.6.061105

Abstract

Purpose

The Medical Imaging and Data Resource Center (MIDRC) open data commons was launched to accelerate the development of artificial intelligence (AI) algorithms to help address the COVID-19 pandemic. The purpose of this study was to quantify longitudinal representativeness of the demographic characteristics of the primary MIDRC dataset compared to the United States general population (US Census) and COVID-19 positive case counts from the Centers for Disease Control and Prevention (CDC).

Approach

The Jensen-Shannon distance (JSD), a measure of similarity of two distributions, was used to longitudinally measure the representativeness of the distribution of (1) all unique patients in the MIDRC data to the 2020 US Census and (2) all unique COVID-19 positive patients in the MIDRC data to the case counts reported by the CDC. The distributions were evaluated in the demographic categories of age at index, sex, race, ethnicity, and the combination of race and ethnicity.

Results

Representativeness of the MIDRC data by ethnicity and the combination of race and ethnicity was impacted by the percentage of CDC case counts for which this was not reported. The distributions by sex and race have retained their level of representativeness over time.

Conclusion

The representativeness of the open medical imaging datasets in the curated public data commons at MIDRC has evolved over time as the number of contributing institutions and overall number of subjects have grown. The use of metrics, such as the JSD support measurement of representativeness, is one step needed for fair and generalizable AI algorithm development.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 International License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation Download Citation

Heather M. Whitney, Natalie Baughan, Kyle J. Myers, Karen Drukker, Judy Gichoya, Brad Bower, Weijie Chen, Nicholas Gruszauskas, Jayashree Kalpathy-Cramer, Sanmi Koyejo, Rui C. Sá, Berkman Sahiner, Zi Zhang, and Maryellen L. Giger "Longitudinal assessment of demographic representativeness in the Medical Imaging and Data Resource Center open data commons," Journal of Medical Imaging 10(6), 061105 (18 July 2023). https://doi.org/10.1117/1.JMI.10.6.061105

Received: 31 January 2023; Accepted: 23 June 2023; Published: 18 July 2023

ACCESS THE FULL ARTICLE

JOURNAL ARTICLE
15 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY