Multiple instance learning (MIL) is a supervised learning methodology that aims to allow models to learn instance class labels from bag class labels, where a bag is defined to contain multiple instances. MIL is gaining traction for learning from weak labels but has not been widely applied to 3D medical imaging. MIL is well-suited to clinical CT acquisitions since (1) the highly anisotropic voxels hinder application of traditional 3D networks and (2) patch-based networks have limited ability to learn whole volume labels. In this work, we apply MIL with a deep convolutional neural network to identify whether clinical CT head image volumes possess one or more large hemorrhages (>; 20cm3 ), resulting in a learned 2D model without the need for 2D slice annotations. Individual image volumes are considered separate bags, and the slices in each volume are instances. Such a framework sets the stage for incorporating information obtained in clinical reports to help train a 2D segmentation approach. Within this context, we evaluate the data requirements to enable generalization of MIL by varying the amount of training data. Our results show that a training size of at least 400 patient image volumes was needed to achieve accurate per-slice hemorrhage detection. Over a five-fold cross-validation, the leading model, which made use of the maximum number of training volumes, had an average true positive rate of 98.10%, an average true negative rate of 99.36%, and an average precision of 0.9698. The models have been made available along with source code1 to enabled continued exploration and adaption of MIL in CT neuroimaging.
Machine learning models are becoming commonplace in the domain of medical imaging, and with these methods comes an ever-increasing need for more data. However, to preserve patient anonymity it is frequently impractical or prohibited to transfer protected health information (PHI) between institutions. Additionally, due to the nature of some studies, there may not be a large public dataset available on which to train models. To address this conundrum, we analyze the efficacy of transferring the model itself in lieu of data between different sites. By doing so we accomplish two goals: 1) the model gains access to training on a larger dataset that it could not normally obtain and 2) the model better generalizes, having trained on data from separate locations. In this paper, we implement multi-site learning with disparate datasets from the National Institutes of Health (NIH) and Vanderbilt University Medical Center (VUMC) without compromising PHI. Three neural networks are trained to convergence on a computed tomography (CT) brain hematoma segmentation task: one only with NIH data, one only with VUMC data, and one multi-site model alternating between NIH and VUMC data. Resultant lesion masks with the multi-site model attain an average Dice similarity coefficient of 0.64 and the automatically segmented hematoma volumes correlate to those done manually with a Pearson correlation coefficient of 0.87, corresponding to an 8% and 5% improvement, respectively, over the single-site model counterparts.
Chordoma is a rare type of tumor that usually appears in the bone near the spinal cord and skull base. Due to their location in the skull base and diverse appearance in size and shape, automatic segmentation of chordoma tumors from magnetic resonance images (MRI) is a challenging task. In addition, similar MR intensity distributions of different anatomical regions, specifically sinuses, make the segmentation task from MRI more challenging. In comparison, most of the state-of-the-art lesion segmentation methods are designed to segment pathologies inside the brain. In this work, we propose an automatic chordoma segmentation framework using two cascaded 3D convolutional neural networks (CNN) via an auto-context model. While the first network learns to detect all potential tumor voxels, the second network fine-tunes the classifier to distinguish true tumor voxels from the false positives detected by the first network. The proposed method is evaluated using multi-contrast MR images of 22 longitudinal scans from 8 patients. Preliminary results showed a linear correlation of 0.71 between the detected and manually outlined tumor volumes, compared to 0.40 for a random forest (RF) based method. Furthermore, the response of tumor growth over time, i.e. increasing, decreasing, or stable, is evaluated according to the response evaluation criteria in solid tumors with an outcome of 0.26 kappa coefficient, compared to 0.13 for the RF based method.
KEYWORDS: Image segmentation, Magnetic resonance imaging, Tissues, Image processing, Image processing algorithms and systems, Brain, Data acquisition, Neuroimaging, Medical imaging
A key feature of magnetic resonance (MR) imaging is its ability to manipulate how the intrinsic tissue parameters of the anatomy ultimately contribute to the contrast properties of the final, acquired image. This flexibility, however, can lead to substantial challenges for segmentation algorithms, particularly supervised methods. These methods require atlases or training data, which are composed of MR image and labeled image pairs. In most cases, the training data are obtained with a fixed acquisition protocol, leading to suboptimal performance when an input data set that requires segmentation has differing contrast properties. This drawback is increasingly significant with the recent movement towards multi-center research studies involving multiple scanners and acquisition protocols. In this work, we propose a new framework for supervised segmentation approaches that is robust to contrast differences between the training MR image and the input image. Our approach uses a generative simulation model within the segmentation process to compensate for the contrast differences. We allow the contrast of the MR image in the training data to vary by simulating a new contrast from the corresponding label image. The model parameters are optimized by a cost function measuring the consistency between the input MR image and its simulation based on a current estimate of the segmentation labels. We provide a proof of concept of this approach by combining a supervised classifier with a simple simulation model, and apply the resulting algorithm to synthetic images and actual MR images.
The subarachnoid space is a layer in the meninges that surrounds the brain and is filled with trabeculae and cerebrospinal fluid. Quantifying the volume and thickness of the subarachnoid space is of interest in order to study the pathogenesis of neurodegenerative diseases and compare with healthy subjects. We present an automatic method to reconstruct the subarachnoid space with subvoxel accuracy using a nested deformable model. The method initializes the deformable model using the convex hull of the union of the outer surfaces of the cerebrum, cerebellum and brainstem. A region force is derived from the subject’s T1-weighted and T2-weighted MRI to drive the deformable model to the outer surface of the subarachnoid space. The proposed method is compared to a semi-automatic delineation from the subject’s T2-weighted MRI and an existing multi-atlas-based method. A small pilot study comparing the volume and thickness measurements in a set of age-matched subjects with normal pressure hydrocephalus and healthy controls is presented to show the efficacy of the proposed method.
Magnetic Resonance (MR) imaging allows the acquisition of images with different contrast properties depending on the acquisition protocol and the magnetic properties of tissues. Many MR brain image processing techniques, such as tissue segmentation, require multiple MR contrasts as inputs, and each contrast is treated differently. Thus it is advantageous to automate the identification of image contrasts for various purposes, such as facilitating image processing pipelines, and managing and maintaining large databases via content-based image retrieval (CBIR). Most automated CBIR techniques focus on a two-step process: extracting features from data and classifying the image based on these features. We present a novel 3D deep convolutional neural network (CNN)- based method for MR image contrast classification. The proposed CNN automatically identifies the MR contrast of an input brain image volume. Specifically, we explored three classification problems: (1) identify T1-weighted (T1-w), T2-weighted (T2-w), and fluid-attenuated inversion recovery (FLAIR) contrasts, (2) identify pre vs postcontrast T1, (3) identify pre vs post-contrast FLAIR. A total of 3418 image volumes acquired from multiple sites and multiple scanners were used. To evaluate each task, the proposed model was trained on 2137 images and tested on the remaining 1281 images. Results showed that image volumes were correctly classified with 97.57% accuracy.
Normal pressure hydrocephalus (NPH) affects older adults and is thought to be caused by obstruction of the normal flow of cerebrospinal fluid (CSF). NPH typically presents with cognitive impairment, gait dysfunction, and urinary incontinence, and may account for more than five percent of all cases of dementia. Unlike most other causes of dementia, NPH can potentially be treated and the neurological dysfunction reversed by shunt surgery or endoscopic third ventriculostomy (ETV), which drain excess CSF. However, a major diagnostic challenge remains to robustly identify shunt-responsive NPH patients from patients with enlarged ventricles due to other neurodegenerative diseases. Currently, radiologists grade the severity of NPH by detailed examination and measurement of the ventricles based on stacks of 2D magnetic resonance images (MRIs). Here we propose a new method to automatically segment and label different compartments of the ventricles in NPH patients from MRIs. While this task has been achieved in healthy subjects, the ventricles in NPH are both enlarged and deformed, causing current algorithms to fail. Here we combine a patch-based tissue classification method with a registration-based multi-atlas labeling method to generate a novel algorithm that labels the lateral, third, and fourth ventricles in subjects with ventriculomegaly. The method is also applicable to other neurodegenerative diseases such as Alzheimer's disease; a condition considered in the differential diagnosis of NPH. Comparison with state of the art segmentation techniques demonstrate substantial improvements in labeling the enlarged ventricles, indicating that this strategy may be a viable option for the diagnosis and characterization of NPH.
Cerebral microbleeds (CMB) are a common marker of traumatic brain injury. Accurate detection and quantification of the CMBs are important for better understanding the progression and prognosis of the injury. Previous microbleed detection methods have suffered from a high rate of false positives, which is time consuming to manually correct. In this paper, we propose a fully automatic, example-based method to segment CMBs from susceptibility-weighted (SWI) scans, where examples from an already segmented template SWI image are used to detect CMBs in a new image. First, multiple radial symmetry transforms (RST) are performed on the template SWI to detect small ellipsoidal structures, which serve as potential microbleed candidates. Then 3D patches from the SWI and its RSTs are combined to form a feature vector at each voxel of the image. A random forest regression is trained using the feature vectors, where the dependent variable is the binary segmentation voxel of the template. Once the regression is learnt, it is applied to a new SWI scan, whose feature vectors contain patches from SWI and its RSTs. Experiments on 26 subjects with mild to severe brain injury show a CMB detection sensitivity of 85:7%, specificity 99:5%, and a false positive to true positive ratio of 1:73, which is competitive with published methods while providing a significant reduction in computation time.
Quantification of hemorrhages in head computed tomography (CT) images from patients with traumatic brain injury (TBI) has potential applications in monitoring disease progression and better understanding of the patho-physiology of TBI. Although manual segmentations can provide accurate measures of hemorrhages, the processing time and inter-rater variability make it infeasible for large studies. In this paper, we propose a fully automatic novel pipeline for segmenting intraparenchymal hemorrhages (IPH) from clinical head CT images. Unlike previous methods of model based segmentation or active contour techniques, we rely on relevant and matching examples from already segmented images by trained raters. The CT images are first skull-stripped. Then example patches from an "atlas" CT and its manual segmentation are used to learn a two-class sparse dictionary for hemorrhage and normal tissue. Next, for a given "subject" CT, a subject patch is modeled as a sparse convex combination of a few atlas patches from the dictionary. The same convex combination is applied to the atlas segmentation patches to generate a membership for the hemorrhages at each voxel. Hemorrhages are segmented from 25 subjects with various degrees of TBI. Results are compared with segmentations obtained from an expert rater. A median Dice coefficient of 0.85 between automated and manual segmentations is achieved. A linear fit between automated and manual volumes show a slope of 1.0047, indicating a negligible bias in volume estimation.
The simulation of magnetic resonance (MR) images plays an important role in the validation of image analysis algorithms such as image segmentation, due to lack of sufficient ground truth in real MR images. Previous work on MRI simulation has focused on explicitly modeling the MR image formation process. However, because of the overwhelming complexity of MR acquisition these simulations must involve simplifications and approximations that can result in visually unrealistic simulated images. In this work, we describe an example-based simulation framework, which uses an “atlas” consisting of an MR image and its anatomical models derived from the hard segmentation. The relationships between the MR image intensities and its anatomical models are learned using a patch-based regression that implicitly models the physics of the MR image formation. Given the anatomical models of a new brain, a new MR image can be simulated using the learned regression. This approach has been extended to also simulate intensity inhomogeneity artifacts based on the statistical model of training data. Results show that the example based MRI simulation method is capable of simulating different image contrasts and is robust to different choices of atlas. The simulated images resemble real MR images more than simulations produced by a physics-based model.
Automatic and accurate detection of white matter lesions is a significant step toward understanding the progression of many diseases, like Alzheimer’s disease or multiple sclerosis. Multi-modal MR images are often used to segment T2 white matter lesions that can represent regions of demyelination or ischemia. Some automated lesion segmentation methods describe the lesion intensities using generative models, and then classify the lesions with some combination of heuristics and cost minimization. In contrast, we propose a patch-based method, in which lesions are found using examples from an atlas containing multi-modal MR images and corresponding manual delineations of lesions. Patches from subject MR images are matched to patches from the atlas and lesion memberships are found based on patch similarity weights. We experiment on 43 subjects with MS, whose scans show various levels of lesion-load. We demonstrate significant improvement in Dice coefficient and total lesion volume compared to a state of the art model-based lesion segmentation method, indicating more accurate delineation of lesions.
Computed tomography (CT) is the preferred imaging modality for patient dose calculation for radiation therapy. Magnetic resonance (MR) imaging (MRI) is used along with CT to identify brain structures due to its superior soft tissue contrast. Registration of MR and CT is necessary for accurate delineation of the tumor and other structures, and is critical in radiotherapy planning. Mutual information (MI) or its variants are typically used as a similarity metric to register MRI to CT. However, unlike CT, MRI intensity does not have an accepted calibrated intensity scale. Therefore, MI-based MR-CT registration may vary from scan to scan as MI depends on the joint histogram of the images. In this paper, we propose a fully automatic framework for MR-CT registration by synthesizing a synthetic CT image from MRI using a co-registered pair of MR and CT images as an atlas. Patches of the subject MRI are matched to the atlas and the synthetic CT patches are estimated in a probabilistic framework. The synthetic CT is registered to the original CT using a deformable registration and the computed deformation is applied to the MRI. In contrast to most existing methods, we do not need any manual intervention such as picking landmarks or regions of interests. The proposed method was validated on ten brain cancer patient cases, showing 25% improvement in MI and correlation between MR and CT images after registration compared to state-of-the-art registration methods.
This paper presents a patch based method to normalize temporal intensities from longitudinal brain magnetic
resonance (MR) images. Longitudinal intensity normalization is relevant for subsequent processing, such as
segmentation, so that rates of change of tissue volumes, cortical thickness, or shapes of brain structures becomes
stable and smooth over time. Instead of using intensities at each voxel, we use patches as image features as a
patch encodes neighborhood information of the center voxel. Once all the time-points of a longitudinal dataset
are registered, the longitudinal intensity change at each patch is assumed to follow an auto-regressive (AR(1))
process. An estimate of the normalized intensities of a patch at every time-point are generated from a hidden
Markov model, where the hidden states are the unobserved normalized patches and the outputs are the observed
patches. A validation study on a phantom dataset shows good segmentation overlap with the truth, and an
experiment with real data shows more stable rates of change for tissue volumes with the temporal normalization
than without.
Intensity normalization is an important preprocessing step in magnetic resonance (MR) image analysis. In MR
images (MRI), the observed intensities are primarily dependent on (1) intrinsic magnetic resonance properties of
the tissues such as proton density (PD), longitudinal and transverse relaxation times (T1 and T2 respectively),
and (2) the scanner imaging parameters like echo time (TE), repeat time (TR), and flip angle (α). We propose a
method which utilizes three co-registered images with different contrast mechanisms (PD-weighted, T2-weighted
and T1-weighted) to first estimate the imaging parameters and then estimate PD, T1, and T2 values. We then
normalize the subject intensities to a reference by simply applying the pulse sequence equation of the reference
image to the subject tissue parameters. Previous approaches to solve this problem have primarily focused on
matching the intensity histograms of the subject image to a reference histogram by different methods. The
fundamental drawback of these methods is their failure to respect the underlying imaging physics and tissue
biology. Our method is validated on phantoms and we show improvement of normalization on real images of
human brains.
Labeling of cerebral vasculature is important for characterization of anatomical variation, quantification of brain
morphology with respect to specific vessels, and inter-subject comparisons of vessel properties and abnormalities.
We propose an automated method to label the anterior portion of cerebral arteries using a statistical inference
method on the Bayesian network representation of the vessel tree. Our approach combines the likelihoods obtained
from a random forest classifier trained using vessel centerline features with a belief propagation method
integrating the connection probabilities of the cerebral artery network. We evaluate our method on 30 subjects
using a leave-one-out validation, and show that it achieves an average correct vessel labeling rate of over 92%.
Magnetic resonance (MR) images of the tongue have been used in both clinical medicine and scientific research
to reveal tongue structure and motion. In order to see different features of the tongue and its relation to the
vocal tract it is beneficial to acquire three orthogonal image stacks-e.g., axial, sagittal and coronal volumes. In
order to maintain both low noise and high visual detail, each set of images is typically acquired with in-plane
resolution that is much better than the through-plane resolution. As a result, any one data set, by itself, is
not ideal for automatic volumetric analyses such as segmentation and registration or even for visualization when
oblique slices are required. This paper presents a method of super-resolution reconstruction of the tongue that
generates an isotropic image volume using the three orthogonal image stacks. The method uses preprocessing
steps that include intensity matching and registration and a data combination approach carried out by Markov
random field optimization. The performance of the proposed method was demonstrated on five clinical datasets,
yielding superior results when compared with conventional reconstruction methods.
This paper presents a patch-based non-parametric approach to the correction of intensity inhomogeneity from
magnetic resonance (MR) images of the human brain. During image acquisition, the inhomogeneity present in
the radio-frequency coil, is usually manifested on the reconstructed MR image as a smooth shading effect. This
artifact can significantly deteriorate the performance of any kind of image processing algorithm that uses intensities
as a feature. Most of the current inhomogeneity correction techniques use explicit smoothness assumptions
on the inhomogeneity field, which sometimes limit their performance if the actual inhomogeneity is not smooth, a
problem that becomes prevalent in high fields. The proposed patch-based inhomogeneity correction method does
not assume any parametric smoothness model, instead, it uses patches from an atlas of an inhomogeneity-free
image to do the correction. Preliminary results show that the proposed method is comparable to N3, a current
state of the art method, when the inhomogeneity is smooth, and outperforms N3 when the inhomogeneity
contains non-smooth elements.
Tissue contrast and resolution of magnetic resonance neuroimaging data have strong impacts on the utility of the
data in clinical and neuroscience tasks such as registration and segmentation. Lengthy acquisition times typically
prevent routine acquisition of multiple MR tissue contrast images at high resolution, and the opportunity for
detailed analysis using these data would seem to be irrevocably lost. This paper describes an example based
approach using patch matching from a multiple resolution multiple contrast atlas in order to change an image's
resolution as well as its MR tissue contrast from one pulse-sequence to that of another. The use of this approach
to generate different tissue contrasts (T2/PD/FLAIR) from a single T1-weighted image is demonstrated on both
phantom and real images.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.