KEYWORDS: Image processing, Quality control, Information science, Compliance, Aliasing, Data processing, Process control, Medical imaging, Imaging systems
Imaging research requires complex data processing and compliance with hospital, state, federal, and international privacy regulations. Such requirements can be particularly challenging for large hospital networks spanning multiple states or countries. In the present study, we combined a governing process for DICOM deidentification and built a software pipeline to support the approved process in a tertiary-care hospital network. This allows researchers to submit large dataset requests and receive fully-compliant files. This study lends evidence that standardized data management at scale is feasible through control policy and fit-to-purpose informatics tools.
Due to variations in unique patient populations, imaging hardware, and drift, artificial intelligence (AI) models perform differently in different clinical practices. The validation of externally developed models is a critical step in implementation, but several challenges related to data de-identification, security, and exchange exist. We created a workflow allowing our clinical radiology practice to safely evaluate external AI models. The workflow encompassed four steps: study selection, extraction, inference, and assessment. A commercially available AI model for intracranial hemorrhage (ICH) was used as a proof of concept. Noncontrast head CT cases were collected using both an internal search engine and a neuroradiologist teaching file that contained 16 exams in which ICH had been missed on the original radiology interpretation. These challenge cases were included to enrich the cohort. Our DICOM de-identification and processing pipeline (D2P2) processed the header and stripped identifiable information. The cleaned data was made available to the external party for AI model processing. The processed results are matched against the ground truth, and performance metrics were calculated, including subgroup analyses. The overall precision was 0.98, the recall was 0.75, and the specificity was 0.97, with an F1 score of 0.85. This result was similar to the original radiologist performance for this enriched challenge cohort. The estimated combined performance of the original radiologist with AI improved the recall to 0.91. Subgroup analyses suggested that combined performance was improved over radiologists or AI alone for subtle hemorrhages or those without mass effect. An integrated workflow to handle the assessment and validation of external deep learning models before implementation is feasible and can aid radiology practices seeking to deploy AI products, particularly in assessing the performance in the local setting and clinical value-added scenarios.
In this work, we present the use of Bayesian networks for radiologist decision support during clinical interpretation. This computational approach has the advantage of avoiding incorrect diagnoses that result from known human cognitive biases such as anchoring bias, framing effect, availability bias, and premature closure. To integrate Bayesian networks into clinical practice, we developed an open-source web application that provides diagnostic support for a variety of radiology disease entities (e.g., basal ganglia diseases, bone lesions). The Clinical tool presents the user with a set of buttons representing clinical and imaging features of interest. These buttons are used to set the value for each observed feature. As features are identified, the conditional probabilities for each possible diagnosis are updated in real time. Additionally, using sensitivity analysis, the interface may be set to inform the user which remaining imaging features provide maximum discriminatory information to choose the most likely diagnosis. The Case Submission tools allow the user to submit a validated case and the associated imaging features to a database, which can then be used for future tuning/testing of the Bayesian networks. These submitted cases are then reviewed by an assigned expert using the provided QC tool. The Research tool presents users with cases with previously labeled features and a chosen diagnosis, for the purpose of performance evaluation. Similarly, the Education page presents cases with known features, but provides real time feedback on feature selection.
In radiology, diagnostic errors occur either through the failure of detection or incorrect interpretation. Errors are
estimated to occur in 30-35% of all exams and contribute to 40-54% of medical malpractice litigations. In this
work, we focus on reducing incorrect interpretation of known imaging features.
Existing literature categorizes cognitive bias leading a radiologist to an incorrect diagnosis despite having correctly
recognized the abnormal imaging features: anchoring bias, framing effect, availability bias, and premature closure.
Computational methods make a unique contribution, as they do not exhibit the same cognitive biases as a human.
Bayesian networks formalize the diagnostic process. They modify pre-test diagnostic probabilities using clinical
and imaging features, arriving at a post-test probability for each possible diagnosis.
To translate Bayesian networks to clinical practice, we implemented an entirely web-based open-source software
tool. In this tool, the radiologist first selects a network of choice (e.g. basal ganglia). Then, large, clearly labeled
buttons displaying salient imaging features are displayed on the screen serving both as a checklist and for input. As
the radiologist inputs the value of an extracted imaging feature, the conditional probabilities of each possible
diagnosis are updated. The software presents its level of diagnostic discrimination using a Pareto distribution chart,
updated with each additional imaging feature.
Active collaboration with the clinical radiologist is a feasible approach to software design and leads to design
decisions closely coupling the complex mathematics of conditional probability in Bayesian networks with practice.
Proceedings Volume Editor (3)
This will count as one of your downloads.
You will have access to both the presentation and article (if available).
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.