Volume 9 Issue 6 | Journal of Medical Imaging

Journal of Medical Imaging

VOL. 9 · NO. 6 | November 2022

CONTENTS

IN THIS ISSUE

Special Series on Global Health, Equity, Bias, and Diversity in AI in Medical Imaging (1)

Image Processing (4)

Computer-Aided Diagnosis (3)

Image-Guided Procedures, Robotic Interventions, and Modeling (2)

Biomedical Applications in Molecular, Structural, and Functional Imaging (3)

Ultrasonic Imaging and Tomography (1)

Digital Pathology (1)

< Previous Issue | Next Issue >

Receive Email Alerts

VIEW ALL ABSTRACTS +

Special Series on Global Health, Equity, Bias, and Diversity in AI in Medical Imaging

Fairness-related performance and explainability effects in deep learning models for brain image analysis

Emma A. Stanley, Matthias Wilms, Pauline Mouches, Nils D. Forkert

Journal of Medical Imaging, Vol. 9, Issue 06, 061102, (August 2022) https://doi.org/10.1117/1.JMI.9.6.061102

TOPICS: Brain, Performance modeling, Brain mapping, Cerebellum, Magnetic resonance imaging, Data modeling, Amygdala, Neuroimaging, Solid modeling, Artificial intelligence

Read Abstract +

Purpose: Explainability and fairness are two key factors for the effective and ethical clinical implementation of deep learning-based machine learning models in healthcare settings. However, there has been limited work on investigating how unfair performance manifests in explainable artificial intelligence (XAI) methods, and how XAI can be used to investigate potential reasons for unfairness. Thus, the aim of this work was to analyze the effects of previously established sociodemographic-related confounders on classifier performance and explainability methods.

Approach: A convolutional neural network (CNN) was trained to predict biological sex from T1-weighted brain MRI datasets of 4547 9- to 10-year-old adolescents from the Adolescent Brain Cognitive Development study. Performance disparities of the trained CNN between White and Black subjects were analyzed and saliency maps were generated for each subgroup at the intersection of sex and race.

Results: The classification model demonstrated a significant difference in the percentage of correctly classified White male (90.3 % ± 1.7 % ) and Black male (81.1 % ± 4.5 % ) children. Conversely, slightly higher performance was found for Black female (89.3 % ± 4.8 % ) compared with White female (86.5 % ± 2.0 % ) children. Saliency maps showed subgroup-specific differences, corresponding to brain regions previously associated with pubertal development. In line with this finding, average pubertal development scores of subjects used in this study were significantly different between Black and White females (p < 0.001) and males (p < 0.001).

Conclusions: We demonstrate that a CNN with significantly different sex classification performance between Black and White adolescents can identify different important brain regions when comparing subgroup saliency maps. Importance scores vary substantially between subgroups within brain structures associated with pubertal development, a race-associated confounder for predicting sex. We illustrate that unfair models can produce different XAI results between subgroups and that these results may explain potential reasons for biased performance.

Image Processing

Efficient labeling of retinal fundus photographs using deep active learning

Samantha Paul, Ian Pan, Warren Sobol

Journal of Medical Imaging, Vol. 9, Issue 06, 064001, (November 2022) https://doi.org/10.1117/1.JMI.9.6.064001

TOPICS: Data modeling, Photography, Binary data, Receivers, Performance modeling, Ophthalmology, Eye models, Visual process modeling, Optical coherence tomography, Statistical modeling

Read Abstract +

Bundle geodesic convolutional neural network for diffusion-weighted imaging segmentation

Renfei Liu, Francois Lauze, Kenny Erleben, Rune W. Berg, Sune Darkner

Journal of Medical Imaging, Vol. 9, Issue 06, 064002, (November 2022) https://doi.org/10.1117/1.JMI.9.6.064002

TOPICS: Data modeling, Diffusion weighted imaging, Convolutional neural networks, Image segmentation, Convolution, Spherical lenses, Optical spheres, Brain, Performance modeling, Spinal cord

Read Abstract +

Contour interpolation by deep learning approach

Chenxi Zhao, Ye Duan, Deshan Yang

Journal of Medical Imaging, Vol. 9, Issue 06, 064003, (December 2022) https://doi.org/10.1117/1.JMI.9.6.064003

TOPICS: Interpolation, Deep learning, Education and training, Contour modeling, Lung, Stomach, Liver, Image segmentation, Data modeling, Kidney

Read Abstract +

Purpose

Contour interpolation is an important tool for expediting manual segmentation of anatomical structures. The process allows users to manually contour on discontinuous slices and then automatically fill in the gaps, therefore saving time and efforts. The most used conventional shape-based interpolation (SBI) algorithm, which operates on shape information, often performs suboptimally near the superior and inferior borders of organs and for the gastrointestinal structures. In this study, we present a generic deep learning solution to improve the robustness and accuracy for contour interpolation, especially for these historically difficult cases.

Approach

A generic deep contour interpolation model was developed and trained using 16,796 publicly available cases from 5 different data libraries, covering 15 organs. The network inputs were a 128 × 128 × 5 image patch and the two-dimensional contour masks for the top and bottom slices of the patch. The outputs were the organ masks for the three middle slices. The performance was evaluated on both dice scores and distance-to-agreement (DTA) values.

Results

The deep contour interpolation model achieved a dice score of 0.95 ± 0.05 and a mean DTA value of 1.09 ± 2.30 mm, averaged on 3167 testing cases of all 15 organs. In a comparison, the results by the conventional SBI method were 0.94 ± 0.08 and 1.50 ± 3.63 mm, respectively. For the difficult cases, the dice score and DTA value were 0.91 ± 0.09 and 1.68 ± 2.28 mm by the deep interpolator, compared with 0.86 ± 0.13 and 3.43 ± 5.89 mm by SBI. The t-test results confirmed that the performance improvements were statistically significant (p < 0.05) for all cases in dice scores and for small organs and difficult cases in DTA values. Ablation studies were also performed.

Conclusions

A deep learning method was developed to enhance the process of contour interpolation. It could be useful for expediting the tasks of manual segmentation of organs and structures in the medical images.

Fractal, recurrent, and dense U-Net architectures with EfficientNet encoder for medical image segmentation

Nahian Siddique, Sidike Paheding, Abel Reyes Angulo, Md. Zahangir Alom, Vijay Devabhaktuni

Journal of Medical Imaging, Vol. 9, Issue 06, 064004, (December 2022) https://doi.org/10.1117/1.JMI.9.6.064004

TOPICS: Fractal analysis, Image segmentation, Data modeling, Performance modeling, Deep learning, Education and training, Statistical modeling, Medical imaging, Neural networks, Network architectures

Read Abstract +

Purpose

U-Net is a deep learning technique that has made significant contributions to medical image segmentation. Although the accomplishments of deep learning algorithms in terms of image processing are evident, many challenges still need to be overcome to achieve human-like performance. One of the main challenges in building deeper U-Nets is black-box problems, such as vanishing gradients. Overcoming this problem allows us to develop neural networks with advanced network designs.

Approach

We propose three U-Net variants, namely efficient R2U-Net, efficient dense U-Net, and efficient fractal U-Net, that can create highly accurate segmentation maps. The first part of our contribution makes use of EfficientNet to distribute resources in the network efficiently. The second part of our work applies the following layer connections to design the U-Net decoders: residual connections, dense connections, and fractal expansion. We apply EfficientNet as the encoder to our three decoders to design three conceivable models.

Results

The aforementioned three proposed deep learning models were tested on four benchmark datasets, including the CHASE DB1 and digital retinal images for vessel extraction (DRIVE) retinal image databases and the ISIC 2018 and HAM10000 dermoscopy image databases. We obtained the highest Dice coefficient of 0.8013, 0.8808, 0.8019, and 0.9295 for CHASE DB1, ISIC 2018, DRIVE, and HAM10000, respectively, and a Jaccard (JAC) score of 0.6686, 0.7870, 0.6694, and 0.8683 for CHASE DB1, ISIC 2018, DRIVE, and HAM10000, respectively. Statistical analysis revealed that the proposed deep learning models achieved better segmentation results compared with the state-of-the-art models.

Conclusions

U-Net is quite an adaptable deep learning framework and can be integrated with other deep learning techniques. The use of recurrent feedback connections, dense convolution, residual skip connections, and fractal convolutional expansions allow for the design of improved deeper U-Net models. With the addition of EfficientNet, we can now leverage the performance of an optimally scaled classifier for U-Net encoders.

Computer-Aided Diagnosis

Comparison of performance between an F18-FDG PET normal brain template and a commercial template using the MIMneuro software

Chanisa Chotipanich, Natdanai Hirata, Attapon Jantarato, Peerapon Kiatkittikul, Dheeratama Siripongsatian, Anchisa Kunawudhi, Chetsadaporn Promteangtrong, Nattakoon Tieojaroenkit, Saiphet Vanprom, Nithi Mahanonda

Journal of Medical Imaging, Vol. 9, Issue 06, 064501, (November 2022) https://doi.org/10.1117/1.JMI.9.6.064501

TOPICS: Brain, Positron emission tomography, Neuroimaging, Databases, Visualization, Visual analytics, Diagnostics, Statistical analysis, Magnetic resonance imaging, Computed tomography

Read Abstract +

Unsupervised and quantitative intestinal ischemia detection using conditional adversarial network in multimodal optical imaging

Yaning Wang, Laura Tiusaba, Shimon Jacobs, Michele Saruwatari, Bo Ning, Marc Levitt, Anthony Sandler, So-Hyun Nam, Jin Kang, Jaepyeong Cha

Journal of Medical Imaging, Vol. 9, Issue 06, 064502, (November 2022) https://doi.org/10.1117/1.JMI.9.6.064502

TOPICS: RGB color model, Laser speckle contrast imaging, Ischemia, Image registration, Surgery, Tissues, Data modeling, Education and training, Speckle, Gallium nitride

Read Abstract +

Purpose

Intraoperative evaluation of bowel perfusion is currently dependent upon subjective assessment. Thus, quantitative and objective methods of bowel viability in intestinal anastomosis are scarce. To address this clinical need, a conditional adversarial network is used to analyze the data from laser speckle contrast imaging (LSCI) paired with a visible-light camera to identify abnormal tissue perfusion regions.

Approach

Our vision platform was based on a dual-modality bench-top imaging system with red-green-blue (RGB) and dye-free LSCI channels. Swine model studies were conducted to collect data on bowel mesenteric vascular structures with normal/abnormal microvascular perfusion to construct the control or experimental group. Subsequently, a deep-learning model based on a conditional generative adversarial network (cGAN) was utilized to perform dual-modality image alignment and learn the distribution of normal datasets for training. Thereafter, abnormal datasets were fed into the predictive model for testing. Ischemic bowel regions could be detected by monitoring the erroneous reconstruction from the latent space. The main advantage is that it is unsupervised and does not require subjective manual annotations. Compared with the conventional qualitative LSCI technique, it provides well-defined segmentation results for different levels of ischemia.

Results

We demonstrated that our model could accurately segment the ischemic intestine images, with a Dice coefficient and accuracy of 90.77% and 93.06%, respectively, in 2560 RGB/LSCI image pairs. The ground truth was labeled by multiple and independent estimations, combining the surgeons’ annotations with fastest gradient descent in suspicious areas of vascular images. The total processing time was 0.05 s for an image size of 256 × 256.

Conclusions

The proposed cGAN can provide pixel-wise and dye-free quantitative analysis of intestinal perfusion, which is an ideal supplement to the traditional LSCI technique. It has potential to help surgeons increase the accuracy of intraoperative diagnosis and improve clinical outcomes of mesenteric ischemia and other gastrointestinal surgeries.

Contrastive self-supervised learning from 100 million medical images with optional supervision

Florin Ghesu, Bogdan Georgescu, Awais Mansoor, Youngjin Yoo, Dominik Neumann, Pragnesh Kumar Patel, Reddappagari Suryanarayana Vishwanath, James Balter, Yue Cao, Sasa Grbic, Dorin Comaniciu

Journal of Medical Imaging, Vol. 9, Issue 06, 064503, (November 2022) https://doi.org/10.1117/1.JMI.9.6.064503

TOPICS: Education and training, Medical imaging, Radiography, Data modeling, Computed tomography, Brain, Magnetic resonance imaging, Neuroimaging, Prototyping, 3D modeling

Read Abstract +

Image-Guided Procedures, Robotic Interventions, and Modeling

Supine magnetic resonance image registration for breast surgery: insights on material mechanics

Morgan Ringel, Winona Richey, Jon Heiselman, Ma Luo, Ingrid M. Meszoely, Michael Miga

Journal of Medical Imaging, Vol. 9, Issue 06, 065001, (November 2022) https://doi.org/10.1117/1.JMI.9.6.065001

TOPICS: Image registration, Breast, Tissues, Data modeling, Magnetic resonance imaging, Surgery, Model-based design, Mechanics, Magnetism, Image segmentation

Read Abstract +

Purpose

Breast conserving surgery (BCS) is a common procedure for early-stage breast cancer patients. Supine preoperative magnetic resonance (MR) breast imaging for visualizing tumor location and extent, while not standard for procedural guidance, is being explored since it more closely represents the surgical presentation compared to conventional diagnostic imaging positions. Despite this preoperative imaging position, deformation is still present between the supine imaging and surgical state. As a result, a fast and accurate image-to-physical registration approach is needed to realize image-guided breast surgery.

Approach

In this study, three registration methods were investigated on healthy volunteers’ breasts (n = 11) with the supine arm-down position simulating preoperative imaging and supine arm-up position simulating intraoperative presentation. The registration methods included (1) point-based rigid registration using synthetic fiducials, (2) nonrigid biomechanical model-based registration using sparse data, and (3) a data-dense three-dimensional diffeomorphic image-based registration from the Advanced Normalization Tools (ANTs) repository. Additionally, deformation metrics (volume change and anisotropy) were calculated from the ANTs deformation field to better understand breast material mechanics.

Results

The average target registration errors (TRE) were 10.4 ± 2.3, 6.4 ± 1.5, and 2.8 ± 1.3 mm (mean ± standard deviation) and the average fiducial registration errors (FRE) were 7.8 ± 1.7, 2.5 ± 1.1, and 3.1 ± 1.1 mm for the point-based rigid, nonrigid biomechanical, and ANTs registrations, respectively. The mechanics-based deformation metrics revealed an overall anisotropic tissue behavior and a statistically significant difference in volume change between glandular and adipose tissue, suggesting that nonrigid modeling methods may be improved by incorporating material heterogeneity and anisotropy.

Conclusions

Overall, registration accuracy significantly improved with increasingly flexible and data-dense registration methods. Analysis of these outcomes may inform the future development of image guidance systems for lumpectomy procedures.

Concentric-ring arrays for forward-viewing ultrasound imaging

Ryosuke Tsumura, Shang Gao, Yichuan Tang, Haichong Zhang

Journal of Medical Imaging, Vol. 9, Issue 06, 065002, (November 2022) https://doi.org/10.1117/1.JMI.9.6.065002

TOPICS: Transducers, Spatial filtering, Ultrasonography, Signal to noise ratio, Image quality, Apodization, Visualization, Design and modelling, Imaging systems, Data acquisition

Read Abstract +

Biomedical Applications in Molecular, Structural, and Functional Imaging

Predicting recurrence risks in lung cancer patients using multimodal radiomics and random survival forests

Jaryd Christie, Omar Daher, Mohamed Abdelrazek, Perrin Romine, Richard Malthaner, Mehdi Qiabi, Rahul Nayak, Sandy Napel, Viswam S. Nair, Sarah Mattonen

Journal of Medical Imaging, Vol. 9, Issue 06, 066001, (November 2022) https://doi.org/10.1117/1.JMI.9.6.066001

TOPICS: Tumors, Image segmentation, Computed tomography, Lung cancer, Positron emission tomography, Bone, Data modeling, Tumor growth modeling, Cancer, Lung

Read Abstract +

Knowledge and practice of radiation protection in the operating theater among orthopedic surgeons

Badera Al Mohammad, Monther Gharaibeh, Maram Al Alakhras

Journal of Medical Imaging, Vol. 9, Issue 06, 066002, (November 2022) https://doi.org/10.1117/1.JMI.9.6.066002

TOPICS: Surgery, X-rays, Personal protective equipment, Lead, Safety, Aluminum, Lead glass, Statistical analysis, Cancer, Medicine

Read Abstract +

Toward the determination of sensitive and reliable whole-lung computed tomography features for robust standard radiomics and delta-radiomics analysis in a nonhuman primate model of coronavirus disease 2019

Marcelo Castro, Syed M. Reza, Winston Chu, Dara Bradley, Ji Hyun Lee, Ian Crozier, Philip Sayre, Byeong Lee, Venkatesh Mani, Thomas Friedrich, David O’Connor, Courtney Finch, Gabriella Worwa, Irwin M. Feuerstein, Jens Kuhn, Jeffrey Solomon

Journal of Medical Imaging, Vol. 9, Issue 06, 066003, (December 2022) https://doi.org/10.1117/1.JMI.9.6.066003

TOPICS: Animals, Radiomics, Cooccurrence matrices, Computed tomography, Reliability, Feature extraction, COVID 19, Image segmentation, Lung, Animal model studies

Read Abstract +

Ultrasonic Imaging and Tomography

Separation of mainlobe and sidelobe contributions to B-mode ultrasound images based on the aperture spectrum

Rehman Ali, Trevor Mitcham, Leandra Brickson, Wentao Hu, Marvin Doyley, Deborah Rubens, Zeljko Ignjatovic, Nebojsa Duric, Jeremy Dahl

Journal of Medical Imaging, Vol. 9, Issue 06, 067001, (November 2022) https://doi.org/10.1117/1.JMI.9.6.067001

Read Abstract +

Digital Pathology

Cell2Grid: an efficient, spatial, and convolutional neural network-ready representation of cell segmentation data

Laurin Herbsthofer, Martina Tomberger, Maria Smolle, Barbara Prietl, Thomas Pieber, Pablo López-García

Journal of Medical Imaging, Vol. 9, Issue 06, 067501, (November 2022) https://doi.org/10.1117/1.JMI.9.6.067501

TOPICS: Image segmentation, Education and training, Neural networks, Tissues, Silicon nitride, Image resolution, Colorectal cancer, Image compression, Data modeling, Tumors

Read Abstract +

Purpose

Approach

Results

Conclusions

Purpose

Approach

Results

Conclusions

Purpose

Approach

Results

Conclusions

Purpose

Approach

Results

Conclusions

Purpose

Approach

Results

Conclusions

Purpose

Approach

Results

Conclusions

Purpose

Approach

Results

Conclusions

Purpose

Approach

Results

Conclusions

Purpose

Approach

Results

Conclusions

Purpose

Approach

Results

Conclusions

Purpose

Approach

Results

Conclusion

Purpose

Approach

Results

Conclusions

Purpose

Approach

Results

Conclusions

Purpose

Approach

Results

Conclusions

Keywords/Phrases

Search In:

Publication Years