We present an in-depth review and analysis of salient methods for computer-aided detection of lung nodules. We evaluate the current methods for detecting lung nodules using literature searches with selection criteria based on validation dataset types, nodule sizes, numbers of cases, types of nodules, extracted features in traditional feature-based classifiers, sensitivity, and false positives (FP)/scans. Our review shows that current detection systems are often optimized for particular datasets and can detect only one or two types of nodules. We conclude that, in addition to achieving high sensitivity and reduced FP/scans, strategies for detecting lung nodules must detect a variety of nodules with high precision to improve the performances of the radiologists. To the best of our knowledge, ours is the first review of the effectiveness of feature extraction using traditional feature-based classifiers. Moreover, we discuss deep-learning methods in detail and conclude that features must be appropriately selected to improve the overall accuracy of the system. We present an analysis of current schemes and highlight constraints and future research areas.

The editorial introduces the JMI Vol. 6, No. 2 Special Section on 3D Printing in Medical Imaging.

The purpose of our study is to develop and evaluate a method for radiopaque 3-D printing (R3P) of soft tissue computed tomography (CT) phantoms with office laser printers. Five laser printers from different vendors are tested for toner CT attenuation. A liver phantom is created by printing CT images of a patient liver on office paper. One thousand eight hundred sixty paper sheets are printed with three repeated prints per page, resulting in a stack of 18.6 cm. The phantom is examined with 12 tube current settings. Images are reconstructed using filtered back projection (FBP) and iterative reconstruction [adaptive iterative dose reduction 3D (AIDR 3D)]. Seven radiologists rated image quality of all acquisitions. Toner attenuation of all investigated printers increased linearly with the print template grayscale. The liver phantom reproduced anatomic detail and attenuation values of the patient (mean  ±  SD HU difference 12.68  ±  7.74). Image quality scores increased with dose but did not vary significantly above a threshold dose for AIDR 3D. Overall, AIDR 3D reconstructed images are rated superior to FBP reconstructions (p  <  0.001). In conclusion, R3P with standard office laser printers can generate soft tissue CT phantoms without hardware manipulations but with limited flexibility regarding attenuation properties of the printed toner material.

We developed three-dimensionally (3D) printed patient-specific coronary phantoms that are capable of sustaining physiological flow and pressure conditions. We assessed the accuracy of these phantoms from coronary CT acquisition, benchtop experimentation, and CT-FFR software. Five patients with coronary artery disease underwent 320-detector row coronary CT angiography (CCTA) (Aquilion ONE, Canon Medical Systems) and a catheter lab procedure to measure fractional flow reserve (FFR). The aortic root and three main coronary arteries were segmented (Vitrea, Vital Images) and 3D printed (Eden 260V, Stratasys). Phantoms were connected into a pulsatile flow loop, which replicated physiological flow and pressure gradients. Contrast was introduced and the phantoms were scanned using the same CT scanner model and CCTA protocol as used for the patients. Image data from the phantoms were input to a CT-FFR research software (Canon Medical Systems) and compared to those derived from the clinical data, along with comparisons between image measurements and benchtop FFR results. Phantom diameter measurements were within 1 mm on average compared to patient measurements. Patient and phantom CT-FFR results had an absolute mean difference of 4.34% and Pearson correlation of 0.95. We have demonstrated the capabilities of 3D printed patient-specific phantoms in a diagnostic software.

Anthropomorphic breast phantoms mimic patient anatomy in order to evaluate clinical mammography and digital breast tomosynthesis system performance. Our goal is to create a modular phantom with an anthropomorphic region to allow for improved lesion and calcification detection as well as a uniform region to evaluate standard quality control (QC) metrics. Previous versions of this phantom used commercial photopolymer inkjet three-dimensional printers to recreate breast anatomy using four surfaces that were fabricated with commercial materials spanning only a limited breast density range of 36% to 64%. We use modified printers to create voxelized, dithered breast phantoms with continuous gradations between glandular and adipose tissues. Moreover, the new phantom replicates the low-end density (representing adipose tissue) using third party material, Jf Flexible, and increases the high-end density to the density of glandular tissue and beyond by either doping Jf Flexible with salts and nanoparticles or using a new commercial resin, VeroPureWhite. An insert design is utilized to add masses, calcifications, and iodinated objects into the phantom for increased utility. The uniform chest wall region provides a space for traditional QC objects such as line pair patterns for measuring resolution and scale bars for measuring printer linearity. Incorporating these distinct design modules enables us to create an improved, complete breast phantom to better evaluate clinical mammography systems for lesion and calcification detection and standard QC performance evaluation.

Three-dimensional (3D) printing has significantly impacted the quality, efficiency, and reproducibility of preclinical magnetic resonance imaging. It has vastly expanded the ability to produce MR-compatible parts that readily permit customization of animal handling, achieve consistent positioning of anatomy and RF coils promptly, and accelerate throughput. It permits the rapid and cost-effective creation of parts customized to a specific imaging study, animal species, animal weight, or even one unique animal, not routinely used in preclinical research. We illustrate the power of this technology by describing five preclinical studies and specific solutions enabled by different 3D printing processes and materials. We describe fixtures, assemblies, and devices that were created to ensure the safety of anesthetized lemurs during an MR examination of their brain or to facilitate localized, contrast-enhanced measurements of white blood cell concentration in a mouse model of pancreatitis. We illustrate expansive use of 3D printing to build a customized birdcage coil and components of a ventilator to enable imaging of pulmonary gas exchange in rats using hyperpolarized Xe129. Finally, we present applications of 3D printing to create high-quality, dual RF coils to accelerate brain connectivity mapping in mouse brain specimens and to increase the throughput of brain tumor examinations in a mouse model of pituitary adenoma.

We aimed to design and fabricate synthetic lung nodules with patient-informed internal heterogeneity to assess the variability and accuracy of measured texture features in CT. To that end, 190 lung nodules from a publicly available database of chest CT images (Lung Image Database Consortium) were selected based on size (>3  mm) and malignancy. The texture features of the nodules were used to train a statistical texture synthesis model based on clustered lumpy background. The model parameters were ascertained based on a genetic optimization of a Mahalanobis distance objective function. The resulting texture model defined internal heterogeneity within 24 anthropomorphic lesion models which were subsequently fabricated into physical phantoms using a multimaterial three-dimensional (3-D) printer. The 3-D-printed lesions were imbedded in an anthropomorphic chest phantom and imaged with a clinical scanner using different acquisition parameters including slice thickness, dose level, and reconstruction kernel. The imaged lesions were analyzed in terms of texture features to ascertain the impact of CT imaging on lesion texture quantification. The texture modeling method produced lesion models with low and stable Mahalanobis distance between real and synthetic textures. The virtual lesions were successfully printed into 3-D phantoms. The accuracy and variability of the measured features extracted from the CT images of the phantoms showed notable influence from the imaging acquisition parameters with contrast, energy, and texture entropy exhibiting most sensitivity in terms of accuracy, and contrast, dissimilarity, and texture entropy most variability. Thinner slice thicknesses yielded more accurate and edge reconstruction kernels more stable results. We conclude that printed textured models of lesions can be developed using a method that can target and minimize the mathematical distance between real and synthetic lesions. The synthetic lesions can be used as the basis to investigate how CT imaging conditions might affect radiomics features derived from CT images.

Terahertz imaging and spectroscopy characterization of freshly excised breast cancer tumors are presented in the range 0.15 to 3.5 THz. Cancerous breast tissues were obtained from partial or full removal of malignant tumors while healthy breast tissues were obtained from breast reduction surgeries. The reflection spectroscopy to obtain the refractive index and absorption coefficient is performed on experimental data at each pixel of the tissue, forming tomographic images. The transmission spectroscopy of the refractive index and absorption coefficient are retrieved from experimental data at few tissue points. The average refractive index and absorption coefficients for cancer, fat, and collagen tissue regions are compared between transmission and reflection modes. The reflection mode offers the advantage of retrieving the electrical properties across a significantly greater number of points without the need for sectioning or altering the freshly excised tissue as in the transmission mode. The terahertz spectral power images and the tomographic images demonstrated good qualitative comparison with pathology.

A dynamic prepatient attenuator can modulate flux in a computed tomography (CT) system along both fan and view angles for reduced dose, scatter, and required detector dynamic range. Reducing the dynamic range requirement is crucial for photon counting detectors. One approach, the piecewise-linear attenuator (Hsieh and Pelc, Med Phys 2013), has shown promising results both in simulations and an initial prototype. Multiple wedges, each covering a different fan angle range, are moved in the axial direction to change thickness seen in an axial slice. We report on an implementation of a filter with precision components and a control algorithm targeted for a tabletop system. Algorithms for optimizing wedge position and mA modulation and for correcting bowtie-specific beam-hardening are proposed. In experiments, the error between expected and observed bowtie transmission was ∼2  %   on average and ∼7  %   at maximum for a chest phantom. Within object boundaries, the observed flux dynamic ranges of 42 for a chest and 25 for an abdomen were achieved, corresponding to a reduction factor of 5 and 11 from the object scans without the bowtie. With beam hardening correction, the CT number in soft tissue regions was improved by 79 HU and deviated by 7 HU on average from clinical scanner CT images. The implemented piecewise-linear attenuator is able to dynamically adjust its thickness with high precision to achieve flexible flux control.

Segmentation of skin lesions is an important step in computer-aided diagnosis of melanoma; it is also a very challenging task due to fuzzy lesion boundaries and heterogeneous lesion textures. We present a fully automatic method for skin lesion segmentation based on deep fully convolutional networks (FCNs). We investigate a shallow encoding network to model clinically valuable prior knowledge, in which spatial filters simulating simple cell receptive fields function in the primary visual cortex (V1) is considered. An effective fusing strategy using skip connections and convolution operators is then leveraged to couple prior knowledge encoded via shallow network with hierarchical data-driven features learned from the FCNs for detailed segmentation of the skin lesions. To our best knowledge, this is the first time the domain-specific hand craft features have been built into a deep network trained in an end-to-end manner for skin lesion segmentation. The method has been evaluated on both ISBI 2016 and ISBI 2017 skin lesion challenge datasets. We provide comparative evidence to demonstrate that our newly designed network can gain accuracy for lesion segmentation by coupling the prior knowledge encoded by the shallow network with the deep FCNs. Our method is robust without the need for data augmentation or comprehensive parameter tuning, and the experimental results show great promise of the method with effective model generalization compared to other state-of-the-art-methods.

As traditional visual-examination-based methods provide neither reliable nor consistent wound assessment, several computer-based approaches for quantitative wound image analysis have been proposed in recent years. However, these methods require either some level of human interaction for proper image processing or that images be captured under controlled conditions. However, to become a practical tool of diabetic patients for wound management, the wound image algorithm needs to be able to correctly locate and detect the wound boundary of images acquired under less-constrained conditions, where the illumination and camera angle can vary within reasonable bounds. We present a wound boundary determination method that is robust to lighting and camera orientation perturbations by applying the associative hierarchical random field (AHRF) framework, which is an improved conditional random field (CRF) model originally applied to natural image multiscale analysis. To validate the robustness of the AHRF framework for wound boundary recognition tasks, we have tested the method on two image datasets: (1) foot and leg ulcer images (for the patients we have tracked for 2 years) that were captured under one of the two conditions, such that 70% of the entire dataset are captured with image capture box to ensure consistent lighting and range and the remaining 30% of the images are captured by a handheld camera under varied conditions of lighting, incident angle, and range and (2) moulage wound images that were captured under similarly varied conditions. Compared to other CRF-based machine learning strategies, our new method provides a determination accuracy with the best global performance rates (specificity: >95  %   and sensitivity: >77  %  .

The segmentation of the dermal–epidermal junction (DEJ) in in vivo confocal images represents a challenging task due to uncertainty in visual labeling and complex dependencies between skin layers. We propose a method to segment the DEJ surface, which combines random forest classification with spatial regularization based on a three-dimensional conditional random field (CRF) to improve the classification robustness. The CRF regularization introduces spatial constraints consistent with skin anatomy and its biological behavior. We propose to specify the interaction potentials between pixels according to their depth and their relative position to each other to model skin biological properties. The proposed approach adds regularity to the classification by prohibiting inconsistent transitions between skin layers. As a result, it improves the sensitivity and specificity of the classification results.

Positron emission tomography (PET) imaging of the lungs is confounded by respiratory motion-induced blurring artifacts that degrade quantitative accuracy. Gating and motion-compensated image reconstruction are frequently used to correct these motion artifacts in PET. In the absence of voxel-by-voxel deformation measures, surrogate signals from external markers are used to track internal motion and generate gated PET images. The objective of our work is to develop a group-level parcellation framework for the lungs to guide the placement of markers depending on the location of the internal target region. We present a data-driven framework based on higher-order singular value decomposition (HOSVD) of deformation tensors that enables identification of synchronous areas inside the torso and on the skin surface. Four-dimensional (4-D) magnetic resonance (MR) imaging based on a specialized radial pulse sequence with a one-dimensional slice-projection navigator was used for motion capture under free-breathing conditions. The deformation tensors were computed by nonrigidly registering the gated MR images. Group-level motion signatures obtained via HOSVD were used to cluster the voxels both inside the volume and on the surface. To characterize the parcellation result, we computed correlation measures across the different regions of interest (ROIs). To assess the robustness of the parcellation technique, leave-one-out cross-validation was performed over the subject cohort, and the dependence of the result on varying numbers of gates and singular value thresholds was examined. Overall, the parcellation results were largely consistent across these test cases with Jaccard indices reflecting high degrees of overlap. Finally, a PET simulation study was performed which showed that, depending on the location of the lesion, the selection of a synchronous ROI may lead to noticeable gains in the recovery coefficient. Accurate quantitative interpretation of PET images is important for lung cancer management. Therefore, a guided motion monitoring approach is of utmost importance in the context of pulmonary PET imaging.

Accurate segmentation of gliomas on routine magnetic resonance image (MRI) scans plays an important role in disease diagnosis, prognosis, and patient treatment planning. We present a fully automated approach, radiomics-based convolutional neural network (RadCNN), for segmenting both high- and low-grade gliomas using multimodal MRI volumes (T1c, T2w, and FLAIR). RadCNN incorporates radiomic texture features (i.e., Haralick, Gabor, and Laws) within DeepMedic [a deep 3-D convolutional neural network (CNN) segmentation framework that uses image intensities; a top performing method in the BraTS 2016 challenge] to further augment the performance of brain tumor subcompartment segmentation. We first identify textural radiomic representations that best separate the different subcompartments [enhancing tumor (ET), whole tumor (WT), and tumor core (TC)] on the training set, and then feed these representations as inputs to the CNN classifier for prediction of different subcompartments. We hypothesize that textural radiomic representations of lesion subcompartments will enhance the separation of subcompartment boundaries, and hence providing these features as inputs to the deep CNN, over and above raw intensity values alone, will improve the subcompartment segmentation. Using a training set of N  =  241 patients, validation set of N  =  44, and test set of N  =  46 patients, RadCNN method achieved Dice similarity coefficient (DSC) scores of 0.71, 0.89, and 0.73 for ET, WT, and TC, respectively. Compared to the DeepMedic model, RadCNN showed improvement in DSC scores for both ET and WT and demonstrated comparable results in segmenting the TC. Similarly, smaller Hausdorff distance measures were obtained with RadCNN as compared to the DeepMedic model across all the subcompartments. Following the segmentation of the different subcompartments, we extracted a set of subcompartment specific radiomic descriptors that capture lesion disorder and assessed their ability in separating patients into different survival cohorts (short-, mid- and long-term survival) based on their overall survival from the date of baseline diagnosis. Using a multilinear regression approach, we achieved accuracies of 0.57, 0.63, and 0.45 for the training, validation, and test cases, respectively.

White blood cells (WBCs) are the most diverse types of cells observed in the healing process of injured skeletal muscles. In the recovery process, WBCs exhibit a dynamic cellular response and undergo multiple changes of the protein expression. The progress of healing can be analyzed by the number of WBCs or by the number of specific proteins observed in light microscopy images obtained at different time points after injury. We propose a deep learning quantification and analysis system called DeepQuantify to analyze WBCs in light microscopy images of uninjured and injured muscles of female mice. The DeepQuantify system features in segmentation using the localized iterative Otsu’s thresholding method, masking postprocessing, and classification of WBCs with a convolutional neural network (CNN) classifier to achieve a high accuracy and a low manpower cost. The proposed two-layer CNN classifier designed based on the optimization hypothesis is evaluated and compared with other CNN classifiers. The DeepQuantify system adopting these CNN classifiers is evaluated for quantifying CD68-positive macrophages and 7/4-positive neutrophils and compared with the state-of-the-art deep learning segmentation architectures. DeepQuantify achieves an accuracy of 90.64% and 89.31% for CD68-positive macrophages and 7/4-positive neutrophils, respectively. The DeepQuantify system employing the proposed two-layer CNN architecture achieves better performance than those deep segmentation architectures. The quantitative analysis of two protein dynamics during muscle recovery is also presented.

Accurate and automated prostate whole gland and central gland segmentations on MR images are essential for aiding any prostate cancer diagnosis system. Our work presents a 2-D orthogonal deep learning method to automatically segment the whole prostate and central gland from T2-weighted axial-only MR images. The proposed method can generate high-density 3-D surfaces from low-resolution (z axis) MR images. In the past, most methods have focused on axial images alone, e.g., 2-D based segmentation of the prostate from each 2-D slice. Those methods suffer the problems of over-segmenting or under-segmenting the prostate at apex and base, which adds a major contribution for errors. The proposed method leverages the orthogonal context to effectively reduce the apex and base segmentation ambiguities. It also overcomes jittering or stair-step surface artifacts when constructing a 3-D surface from 2-D segmentation or direct 3-D segmentation approaches, such as 3-D U-Net. The experimental results demonstrate that the proposed method achieves 92.4  %    ±  3  %   Dice similarity coefficient (DSC) for prostate and DSC of 90.1  %    ±  4.6  %   for central gland without trimming any ending contours at apex and base. The experiments illustrate the feasibility and robustness of the 2-D-based holistically nested networks with short connections method for MR prostate and central gland segmentation. The proposed method achieves segmentation results on par with the current literature.

Radiomics has shown promising results in several medical studies, yet it suffers from a limited discrimination and informative capability as well as a high variation and correlation with the tomographic scanner types, pixel spacing, acquisition protocol, and reconstruction parameters. We propose and compare two methods to transform quantitative image features in order to improve their stability across varying image acquisition parameters while preserving the texture discrimination abilities. In this way, variations in extracted features are representative of true physiopathological tissue changes in the scanned patients. A first approach is based on a two-layer neural network that can learn a nonlinear standardization transformation of various types of features including handcrafted and deep features. Second, domain adversarial training is explored to increase the invariance of the transformed features to the scanner of origin. The generalization of the proposed approach to unseen textures and unseen scanners is demonstrated by a set of experiments using a publicly available computed tomography texture phantom dataset scanned with various imaging devices and parameters.

Changes in the choroid have been suggested to be of critical relevance to some eye diseases, e.g., geographic atrophy (GA) of age-related macular degeneration (AMD). Several groups have worked to develop automated choroidal segmentation algorithms in optical coherence tomography (OCT) images. However, the reported algorithms mainly focus on OCT images with early AMD (or eye diseases with mild/moderate retinal damage). GA, an advanced stage of AMD, frequently presents severe retinal damage and those algorithms may be confounded by the zones of increased choroidal reflectivity due to GA damage. The fundus autofluorescence (FAF) signal from GA regions is more distinguishable and detectable than on OCT images. We propose to use a three-dimensional (3-D) graph-based approach with the complementary information from the companion FAF images to handle the 3-D choroidal segmentation difficulty due to GA. We compare the automated approaches with and without using the FAF-derived GA information against the ground truth. With the FAF-derived information, the segmentation performance regarding both choroidal borders is significantly improved (all p  <  0.01). The robustness of our segmentation approach may be of great value for future large-scale quantitative and longitudinal studies of the choroid, particularly in the setting of atrophic AMD.

A glioma grading method using conventional structural magnetic resonance image (MRI) and molecular data from patients is proposed. The noninvasive grading of glioma tumors is obtained using multiple radiomic texture features including dynamic texture analysis, multifractal detrended fluctuation analysis, and multiresolution fractal Brownian motion in structural MRI. The proposed method is evaluated using two multicenter MRI datasets: (1) the brain tumor segmentation (BRATS-2017) challenge for high-grade versus low-grade (LG) and (2) the cancer imaging archive (TCIA) repository for glioblastoma (GBM) versus LG glioma grading. The grading performance using MRI is compared with that of digital pathology (DP) images in the cancer genome atlas (TCGA) data repository. The results show that the mean area under the receiver operating characteristic curve (AUC) is 0.88 for the BRATS dataset. The classification of tumor grades using MRI and DP images in TCIA/TCGA yields mean AUC of 0.90 and 0.93, respectively. This work further proposes and compares tumor grading performance using molecular alterations (IDH1/2 mutations) along with MRI and DP data, following the most recent World Health Organization grading criteria, respectively. The overall grading performance demonstrates the efficacy of the proposed noninvasive glioma grading approach using structural MRI.

Recent advances in the field of radiomics have enabled the development of a number of prognostic and predictive imaging-based tools for a variety of diseases. However, wider clinical adoption of these tools is contingent on their generalizability across multiple sites and scanners. This may be particularly relevant in the context of radiomic features derived from T1- or T2-weighted magnetic resonance images (MRIs), where signal intensity values are known to lack tissue-specific meaning and vary based on differing acquisition protocols between institutions. We present the first empirical study of benchmarking five different radiomic feature families in terms of both reproducibility and discriminability in a multisite setting, specifically, for identifying prostate tumors in the peripheral zone on MRI. Our cohort comprised 147 patient T2-weighted MRI datasets from four different sites, all of which are first preprocessed to correct for acquisition-related artifacts such as bias field, differing voxel resolutions, and intensity drift (nonstandardness). About 406 three-dimensional voxel-wise radiomic features from five different families (gray, Haralick, gradient, Laws, and Gabor) were evaluated in a cross-site setting to determine (a) how reproducible they are within a relatively homogeneous nontumor tissue region and (b) how well they could discriminate tumor regions from nontumor regions. Our results demonstrate that a majority of the popular Haralick features are reproducible in over 99% of all cross-site comparisons, as well as achieve excellent cross-site discriminability (classification accuracy of ≈0.8). By contrast, a majority of Laws features are highly variable across sites (reproducible in <75  %   of all cross-site comparisons) as well as resulting in low cross-site classifier accuracies (<0.6), likely due to a large number of noisy filter responses that can be extracted. These trends suggest that only a subset of radiomic features and associated parameters may be both reproducible and discriminable enough for use within machine learning classifier schemes.

Brachytherapy, a type of radiotherapy, may be used to place radioactive sources into or in close proximity to tumors, providing a method for conformally escalating dose in the tumor and the local area surrounding the malignancy. High-dose-rate interstitial brachytherapy of vaginal tumors requires precise placement of multiple needles through holes in a plastic perineal template to deliver treatment while optimizing dose and avoiding overexposure of nearby organs at risk (OARs). Despite the importance of needle placement, image guidance for adaptive, intraoperative needle visualization, allowing misdirected needles to be identified and corrected during insertion, is not standard practice. We have developed a 360-deg three-dimensional (3-D) transvaginal ultrasound (TVUS) system using a conventional probe with a template-compatible custom sonolucent vaginal cylinder and propose its use for intraoperative needle guidance during interstitial gynecologic brachytherapy. We describe the 3-D TVUS mechanism and geometric validation, present mock phantom procedure results, and report on needle localization accuracy in patients. For the six patients imaged, landmark anatomical features and all needles were clearly visible. The implementation of 360-deg 3-D TVUS through a sonolucent vaginal cylinder provides a technique for visualizing needles and OARs intraoperatively during interstitial gynecologic brachytherapy, enabling implants to be assessed and providing the potential for image guidance.

We develop a mathematical framework for the design of orbital trajectories that are optimal to a particular imaging task (or tasks) in advanced cone-beam computed tomography systems that have the capability of general source–detector positioning. The framework allows various parameterizations of the orbit as well as constraints based on imaging system capabilities. To accommodate nonstandard system geometries, a model-based iterative reconstruction method is applied. Such algorithms generally complicate the assessment and prediction of reconstructed image properties; however, we leverage efficient implementations of analytical predictors of local noise and spatial resolution that incorporate dependencies of the reconstruction algorithm on patient anatomy, x-ray technique, and geometry. These image property predictors serve as inputs to a task-based performance metric defined by detectability index, which is optimized with respect to the orbital parameters of data acquisition. We investigate the framework of the task-driven trajectory design in several examples to examine the dependence of optimal source–detector trajectories on the imaging task (or tasks), including location and spatial-frequency dependence. A variety of multitask objectives are also investigated, and the advantages to imaging performance are quantified in simulation studies.

Segmentation of the prostate in computed tomography (CT) is used for planning and guidance of prostate treatment procedures. However, due to the low soft-tissue contrast of the images, manual delineation of the prostate on CT is a time-consuming task with high interobserver variability. We developed an automatic, three-dimensional (3-D) prostate segmentation algorithm based on a customized U-Net architecture. Our dataset contained 92 3-D abdominal CT scans from 92 patients, of which 69 images were used for training and validation and the remaining for testing the convolutional neural network model. Compared to manual segmentation by an expert radiologist, our method achieved 83  %    ±  6  %   for Dice similarity coefficient (DSC), 2.3  ±  0.6  mm for mean absolute distance (MAD), and 1.9  ±  4.0  cm³ for signed volume difference (ΔV). The average recorded interexpert difference measured on the same test dataset was 92% (DSC), 1.1 mm (MAD), and 2.1  cm³ (ΔV). The proposed algorithm is fast, accurate, and robust for 3-D segmentation of the prostate on CT images.

We apply the methodology detailed in “Task-driven source–detector trajectories in cone-beam computed tomography: I. Theory and methods” by Stayman et al. for task-driven optimization of source–detector orbits in cone-beam computed tomography (CBCT) to scenarios emulating imaging tasks in interventional neuroradiology. The task-driven imaging framework is used to optimize the CBCT source–detector trajectory by maximizing the detectability index, d^′. The approach was applied to simulated cases of endovascular embolization of an aneurysm and arteriovenous malformation and was translated to real data first using a CBCT test bench followed by implementation on an interventional robotic C-arm. Task-driven trajectories were found to generally favor higher fidelity (i.e., less noisy) views, with an average increase in d^′ ranging from 7% to 28%. Visually, this resulted in improved conspicuity of particular stimuli by reducing the noise and altering the noise correlation to a form distinct from the spatial frequencies associated with the imaging task. The improvements in detectability and the demonstration of the task-driven workflow using a real interventional imaging system show the potential of the task-driven imaging framework to improve imaging performance on motorized, multiaxis C-arms in neuroradiology.

Although a greater extent of tumor resection is important for patients’ survival, complete tumor removal, especially tumor margins, remains challenging due to the lack of sensitivity and specificity of current surgical guidance techniques at the margins. Intraoperative fluorescence imaging with targeted fluorophores is promising for tumor margin delineation. To verify the tumor margins detected by the fluorescence images, it is necessary to register fluorescence with histological images, which provide the ground truth for tumor regions. However, current registration methods compare fluorescence images to a single-layer histological slide, which is selected subjectively and represents a single plane of the three-dimensional tumor. A multistep pipeline is established to correlate fluorescence images to stacked histological images, including fluorescence calibration and multistep registration. Multiple histological slices are integrated as a two-dimensional (2-D) tumor map using optical attenuation model and average intensity projection. A BLZ-100-labeled medulloblastoma mouse model is used to test the whole framework. On average, the synthesized 2-D tumor map outperforms the selected best slide as ground truth [Dice similarity coefficient (DSC): 0.582 versus 0.398, with significant differences; mean area under the curve (AUC) of the receiver operating characteristic curve: 88% versus 85.5%] and the randomly selected slide as ground truth (DSC: 0.582 versus 0.396 with significant differences; mean AUC: 88% versus 84.1% with significant differences), which indicates our pipeline is reliable and can be applied to investigate targeted fluorescence probes in tumor margin detection. Following this proposed pipeline, BLZ-100 shows enhancement in both tumor cores and tumor margins (mean target-to-background ratio: 8.64  ±  5.76 and 4.82  ±  2.79, respectively).

Shoulder arthrography is a diagnostic procedure which involves injecting a contrast agent into the joint space for enhanced visualization of anatomical structures. Typically, a contrast agent is injected under fluoroscopy or computed tomography (CT) guidance, resulting in exposure to ionizing radiation, which should be avoided especially in pediatric patients. The patient then waits for the next available magnetic resonance imaging (MRI) slot for obtaining high-resolution anatomical images for diagnosis, which can result in long procedure times. Performing the contrast agent injection under MRI guidance could overcome both these issues. However, it comes with the challenges of the MRI environment including high magnetic field strength, limited ergonomic patient access, and lack of real-time needle guidance. We present the development of an integrated robotic system to perform shoulder arthrography procedures under intraoperative MRI guidance, eliminating fluoroscopy/CT guidance and patient transportation from the fluoroscopy/CT room to the MRI suite. The average accuracy of the robotic manipulator in benchtop experiments is 0.90 mm and 1.04 deg, whereas the average accuracy of the integrated system in MRI phantom experiments is 1.92 mm and 1.28 deg at the needle tip. Based on the American Society for Testing and Materials (ASTM) tests performed, the system is classified as MR conditional.

We compare a surface-driven, model-based deformation correction method to a clinically relevant rigid registration approach within the application of image-guided microwave ablation for the purpose of demonstrating improved localization and antenna placement in a deformable hepatic phantom. Furthermore, we present preliminary computational modeling of microwave ablation integrated within the navigational environment to lay the groundwork for a more comprehensive procedural planning and guidance framework. To achieve this, we employ a simple, retrospective model of microwave ablation after registration, which allows a preliminary evaluation of the combined therapeutic and navigational framework. When driving registrations with full organ surface data (i.e., as could be available in a percutaneous procedure suite), the deformation correction method improved average ablation antenna registration error by 58.9% compared to rigid registration (i.e., 2.5  ±  1.1  mm, 5.6  ±  2.3  mm of average target error for corrected and rigid registration, respectively) and on average improved volumetric overlap between the modeled and ground-truth ablation zones from 67.0  ±  11.8  %   to 85.6  ±  5.0  %   for rigid and corrected, respectively. Furthermore, when using sparse-surface data (i.e., as is available in an open surgical procedure), the deformation correction improved registration error by 38.3% and volumetric overlap from 64.8  ±  12.4  %   to 77.1  ±  8.0  %   for rigid and corrected, respectively. We demonstrate, in an initial phantom experiment, enhanced navigation in image-guided hepatic ablation procedures and identify a clear multiphysics pathway toward a more comprehensive thermal dose planning and deformation-corrected guidance framework.

Microaneurysms (MAs) play an important role in the diagnosis of clinical diabetic retinopathy at the early stage. Annotation of MAs manually by experts is laborious and so it is essential to develop automatic segmentation methods. Automatic MA segmentation remains a challenging task mainly due to the low local contrast of the image and the small size of MAs. A deep learning-based method called U-Net has become one of the most popular methods for the medical image segmentation task. We propose an architecture for U-Net, named deep recurrent U-Net (DRU-Net), obtained by combining the deep residual model and recurrent convolutional operations into U-Net. In the MA segmentation task, DRU-Net can accumulate effective features much better than the typical U-Net. The proposed method is evaluated on two publicly available datasets: E-Ophtha and IDRiD. Our results show that the proposed DRU-Net achieves the best performance with 0.9999 accuracy value and 0.9943 area under curve (AUC) value on the E-Ophtha dataset. And on the IDRiD dataset, it has achieved 0.987 AUC value (to our knowledge, this is the first result of segmenting MAs on this dataset). Compared with other methods, such as U-Net, FCNN, and ResU-Net, our architecture (DRU-Net) achieves state-of-the-art performance.

Task-based image quality procedures in CT that substitute a human observer with a model observer usually use single-slice images with uniform backgrounds from homogeneous phantoms. However, anatomical structures and inhomogeneities in organs generate noise that can affect the detection performance of human observers. The purpose of this work was to assess the impact of background type, uniform or liver, and the viewing modality, single- or multislice, on the detection performance of human and model observers. We collected abdominal CT scans from patients and homogeneous phantom scans in which we digitally inserted low-contrast signals that mimicked a liver lesion. We ran a rating experiment with the two background conditions with three signal sizes and three human observers presenting images in two reading modalities: single- and multislice. In addition, channelized Hotelling observers (CHO) for single- and multislice detection were implemented and evaluated according to their degree of correlation with the human observer performance. For human observers, there was a small but significant improvement in performance with multislice compared to the single-slice viewing mode. Our data did not reveal a significant difference between uniform and anatomical backgrounds. Model observers demonstrated a good correlation with human observers for both viewing modalities. Human observers have very similar performances in both multi- and single-slice viewing mode. It is therefore preferable to use single-slice CHO as this model is computationally more tractable than multislice CHO. However, using images from a homogeneous phantom can result in overestimating image quality as CHO performance tends to be higher in uniform than anatomical backgrounds, while human observers have similar detection performances.

Images derived from a “virtual phantom” can be useful in characterizing the performance of imaging systems. This has driven the development of virtual breast phantoms implemented in simulation environments. In breast imaging, several such phantoms have been proposed. We analyze the non-Gaussian statistical properties from three classes of virtual breast phantoms and compare them to similar statistics from a database of breast images. These include clustered-blob lumpy backgrounds (CBLBs), truncated binary textures, and the UPenn virtual breast phantoms. We use Laplacian fractional entropy (LFE) as a measure of the non-Gaussian statistical properties of each simulation procedure. Our results show that, despite similar power spectra, the simulation approaches differ considerably in LFE with very low scores for the CBLB to high values for the UPenn phantom at certain frequencies. These results suggest that LFE may have value in developing and tuning virtual phantom simulation procedures.

Detection and characterization of abnormalities in clinical imaging are of utmost importance for patient diagnosis and treatment. We present a comparison of convolutional neural network (CNN) and human observer performance on a simulated lesion detection and characterization task. We apply both conventional performance metrics, including accuracy and nonconventional metrics such as lift charts to perform qualitative and quantitative comparisons of each type of observer. It is determined that the CNN generally outperforms the human observers, particularly at high noise levels. However, high noise correlation reduces the relative performance of the CNN, and human observer performance is comparable to CNN under these conditions. These findings extend into the field of diagnostic radiology, where the adoption of deep learning is starting to become widespread. Consideration of the applications for which deep learning is most effective is of critical importance to this development.

Predicting infarct volume from magnetic resonance perfusion-weighted imaging can provide helpful information to clinicians in deciding how aggressively to treat acute stroke patients. Models have been developed to predict tissue fate, yet these models are mostly built using hand-crafted features (e.g., time-to-maximum) derived from perfusion images, which are sensitive to deconvolution methods. We demonstrate the application of deep convolution neural networks (CNNs) on predicting final stroke infarct volume using only the source perfusion images. We propose a deep CNN architecture that improves feature learning and achieves an area under the curve of 0.871  ±  0.024, outperforming existing tissue fate models. We further validate the proposed deep CNN with existing 2-D and 3-D deep CNNs for images/video classification, showing the importance of the proposed architecture. Our work leverages deep learning techniques in stroke tissue outcome prediction, advancing magnetic resonance imaging perfusion analysis one step closer to an operational decision support tool for stroke treatment guidance.

We present an algorithm for automatic anatomical measurements in tomographic datasets of the knee. The algorithm uses a set of atlases, each consisting of a knee image, surface segmentations of the bones, and locations of landmarks required by the anatomical metrics. A multistage volume-to-volume and surface-to-volume registration is performed to transfer the landmarks from the atlases to the target volume. Manual segmentation of the target volume is not required in this approach. Metrics were computed from the transferred landmarks of a best-matching atlas member (different for each bone), identified based on a mutual information criterion. Leave-one-out validation of the algorithm was performed on 24 scans of the knee obtained using extremity cone-beam computed tomography. Intraclass correlation (ICC) between the algorithm and the expert who generated atlas landmarks was above 0.95 for all metrics. This compares favorably to inter-reader ICC, which varied from 0.19 to 0.95, depending on the metric. Absolute agreement with the expert was also good, with median errors below 0.25 deg for measurements of tibial slope and static alignment, and below 0.2 mm for tibial tuberosity-trochlear groove distance and medial tibial depth. The automatic approach is anticipated to improve measurement workflow and mitigate the effects of operator experience and training on reliability of the metrics.

Distance correlation is a measure that can detect both linear and nonlinear associations. However, applying distance correlation to imaging genetic studies often needs multiple testing correction due to the large number of multiple inferences. As a result, the sensitivity of its detection may be low. We propose a new model, distance canonical correlation analysis (DCCA), which overcomes this problem by searching a combination of features with the highest distance correlation. This is achieved by constructing a distance kernel function followed by solving a subsequent optimization problem. The ability to detect both linear and nonlinear associations makes DCCA suitable for analyzing complex multimodal and imaging-genetic associations. When applied to a brain imaging-genetic study from the Philadelphia Neurodevelopmental Cohort (PNC), DCCA detected several mental disorder-related gene pathways and brain networks. Experiments on brain connectivity found that the default mode network had strong nonlinear connections with other brain networks. When applied to the study of age effects, DCCA revealed that the connections of brain networks were relatively weak in younger groups but became stronger at older age stages. It indicates that adolescence is a vital stage for brain development. DCCA thus reveals a number of interesting findings and demonstrates a powerful new approach for analyzing multimodal brain imaging data.

The availability of massive amounts of data in histopathological whole-slide images (WSIs) has enabled the application of deep learning models and especially convolutional neural networks (CNNs), which have shown a high potential for improvement in cancer diagnosis. However, storage and transmission of large amounts of data such as gigapixel histopathological WSIs are challenging. Exploiting lossy compression algorithms for medical images is controversial but, as long as the clinical diagnosis is not affected, is acceptable. We study the impact of JPEG 2000 compression on our proposed CNN-based algorithm, which has produced performance comparable to that of pathologists and which was ranked second place in the CAMELYON17 challenge. Detecting tumor metastases in hematoxylin and eosin-stained tissue sections of breast lymph nodes is evaluated and compared with the pathologists’ diagnoses in three different experimental setups. Our experiments show that the CNN model is robust against compression ratios up to 24:1 when it is trained on uncompressed high-quality images. We demonstrate that a model trained on lower quality images—i.e., lossy compressed images—depicts a classification performance that is significantly improved for the corresponding compression ratio. Moreover, it is also observed that the model performs equally well on all higher-quality images. These properties will help to design cloud-based computer-aided diagnosis (CAD) systems, e.g., telemedicine that employ deep CNN models that are more robust to image quality variations due to compression required to address data storage and transmission constraints. However, the results presented are specific to the CAD system and application described, and further work is needed to examine whether they generalize to other systems and applications.