KEYWORDS: Tissues, Microscopy, Tumors, Standards development, In vivo imaging, Education and training, Data modeling, 3D imaging standards, Spatial resolution, Phase imaging
Quantitative oblique back-illumination microscopy (qOBM) is a novel technology for label-free imaging of thick (unsectioned) tissue specimens, demonstrating high spatial resolution and 3-D capabilities. The grayscale contrast however, is unfamiliar to pathologist and histotechnicians without familiarization, limiting its adoption. We used deep learning techniques to convert qOBM into virtual H&E, observing successful conversion of both healthy and tumor thick (unsectioned) specimens. Transfer learning was demonstrated on a second collection of qOBM and H&E images of human astrocytoma specimens. With some improvement in robustness and generalizability, we anticipate that this approach can find clinical application.
Quantitative Oblique Back-Illumination Microscopy (qOBM) enables 3D quantitative phase imaging (QPI) in thick scattering samples. Recently, we developed a deep learning assisted single-capture qOBM (SC-qOBM) method to achieve fast quantitative phase tomography in vivo. This novel approach paves the way for a number of in-vivo applications that were previously out of reach for QPI. Here, we showcase two applications of SC-qOBM: (1) in-vivo imaging of blood flow in mouse brains for studying blood diseases and (2) non-invasive imaging of blood flow in humans for hematological and in-vivo flow cytometry analysis. This work highlights SC-qOBM’s potential in various biomedical applications.
Deep Ultraviolet (UV) microscopy enables high-resolution, molecular imaging and typically yields 2D images representing the axial projection of a sample’s 3D absorption onto a plane. In this work, we present a tomographic imaging approach based on multispectral UV microscopy, to visualize complex 3D structural features in samples. We aim to employ through-focus intensity images captured with varying partially coherent and asymmetric illumination patterns to extract the 3D absorption and refractive index (RI) distributions of the sample at distinct UV wavelengths. The recovery procedure relies on solving the inverse scattering problem using the 3D optical transfer function of our microscope.
KEYWORDS: 3D image processing, Biological imaging, Tissues, Stereoscopy, Real time imaging, Light sources and illumination, Biomedical applications, Phase contrast, In vivo imaging, 3D applications
Quantitative oblique-back-illumination microscopy (qOBM) enables quantitative phase imaging (QPI) with epi-illumination, and thus permits the use of phase contrast in applications that were previously out-of-reach for QPI, including clinical medicine. Here, I will discuss our latest efforts to apply qOBM for clinical applications, specifically tissue imaging for non-invasive diagnostics and image guided therapy. Our approach uses an unsupervised cycle generative adversarial networks to translate 3D phase images of thick fresh tissues to appear like H&E-stained tissue sections. This work paves the way for non-invasive, label-free, real-time 3D H&E imaging which can be transformative for disease detection and guided therapy.
Quantitative oblique back illumination microscopy (qOBM) is a recently developed phase imaging modality that enables 3D quantitative phase imaging and refractive index (RI) tomography of thick scattering samples. The approach uses four oblique illumination images (acquired in epi-mode) at a given focal plane to obtain cross sectional quantitative information. In order to quantify the information, qOBM uses a deconvolution algorithm which requires an estimate of the angular distribution of light at the focal plane to obtain the system’s optical transfer function (OTF). This information is obtained using Monte Carlo numerical simulations which uses published scattering parameters of tissues. While this approach has shown robust results with high quantitative fidelity, the reliance on available published scattering parameters is not optimal. Here we present an experimental approach to measure the angular distribution of the back-scattered light at the focal plane. The approach simultaneously obtains information from the imaging plane and the Fourier plane to provide insight into the overall angular distribution of light at the focal plane. Together with the pupil function, given by the known numerical aperture of the system, this approach directly yields the OTF. A theoretical analysis and experimental results will be presented. This approach has the potential to widen the utility of qOBM to also include tissues and samples whose scattering properties are not well documented in the literature.
Quantitative phase imaging (QPI) has emerged as a valuable method in biomedical research by providing label-free, high-resolution phase distribution of transparent cells and tissues. While QPI is limited to transparent samples, quantitative oblique back-illumination microscopy (qOBM) is a novel imaging technology that enables epi-mode 3D quantitative phase imaging and refractive index (RI) tomography of thick scattering samples. This technology employs four oblique back illumination images taken at the same focal planes, along with a rapid 2D deconvolution reconstruction algorithm, to generate 2D phase cross-sections of thick samples. Alternatively, a through-focus z-stack of oblique back illumination images can be utilized to produce 3D RI tomograms, offering enhanced RI quantitative accuracy. However, 3D RI generation requires a more computationally intensive reconstruction process, preventing its potential of a real-time 3D RI tomography. In this paper, we propose a neural network-involved reconstruction technique that significantly reduces the processing time to a third while maintaining high fidelity compared to the deconvolution-based results.
KEYWORDS: Biological samples, Phase imaging, Light sources and illumination, In vivo imaging, Imaging systems, Biological imaging, Brain, Tumors, Real time imaging, Design
Quantitative phase imaging (QPI) offers label-free access to refractive index information of biological samples, which can achieve nanometer-level optical-path-length sensitivity with cellular/sub-cellular biophysical and histological details. Recently we introduced quantitative oblique back-illumination microscopy (qOBM) which works in epi-mode and uses multiply scattered photons within thick samples to yield quantitative phase in thick scattering tissues, thus overcoming QPI’s long-standing limitation to thin transparent samples. qOBM provides real-time quantitative phase in 3D, and can be configured in a compact form factor. Here we describe a handheld qOBM probe, suitable for in-vivo diagnostic applications such as brain tumor assessment, dermatology, and more.
Phase imaging and fluorescence microscopy provide valuable complementary information, and individually form the basis for a significant portion of the routing biological and biomedical optical imaging performed today. While multimodal phase and fluorescence microscopy has been explored for thin transparent samples to obtain structural information based on the refractive index distribution (with phase contrast) and molecular content (with fluorescence), combining these complementary technologies to study thick samples has been challenging and remains largely unexplored. This work presents the results of a study that combines quantitative phase imaging (QPI) and refractive index (RI) tomography in thick samples—using quantitative oblique back illumination—and bright field fluorescence deconvolution microscopy. The two technologies use a simple bright field microscope configuration with epi-illumination and through-focus z-stack acquisition, along with a deconvolution algorithm, to achieve 3D imaging. Phase and RI information is acquired nearly simultaneously with the fluorescence information with inherent co-registration of the two modalities. In this work, we will present the theoretical underpinning of this multimodal approach, describe the simple multimodal system, and show imaging results of thick tissues, such as labeled mice brains. This multimodal imaging approach could help biologists and clinicians gain a more comprehensive understanding of the tissue’s morphology and molecular composition, and can be widely applied across a number of biological and biomedical disciplines, including neuroscience, pathology, and oncology.
Quantitative oblique back-illumination microscopy (qOBM) is a label-free imaging technique that enables tomographic phase imaging of thick scattering samples with epi-illumination. Here, we propose the use of two forms of functional imaging with qOBM to study tissue and cell cultures. In doing so, we obtain the spatiotemporal and quantitative functional information associated with the phase values extrapolated from qOBM imaging. We have applied this process to study the efficacy of individual immune T cells to kill glioblastoma spheroid cultures in 3D spheroids. Data show that we can effectively distinguish between cell phenotypes and characterize the dynamic motion of these cells in 3D cultures. This work offers a distinct advantage in tracking 3D cellular dynamics in thick tissue as many function imaging modalities are limited to 2D samples. Further, this technology can be expanded to analyze a wide variety of cellular and subcellular dynamics non-invasively in thick tissue.
Quantitative Phase Imaging (QPI) has become a mainstay imaging technique in the biomedical sciences to study cells and other biological processes. Traditional QPI techniques are transmission-based and, thus, limited to thin samples. Over the past few years, multiple 3D QPI tools have emerged attempting to overcome this limitation and provide cross-sectional phase information of thicker samples. However, most of these techniques remain transmission-based, which constrains their ability to image samples thicker than a few mean free scattering lengths. Recently, we have developed quantitative oblique back-illumination microscopy (qOBM) as an epimode technique that enables label-free quantitative phase imaging of thick samples with tomographic crosssectioning. Like in most 3D QPI instances, qOBM requires multiple captures to render a quantitative phase image. Specifically, qOBM requires four raw captures, obtained by illuminating the sample obliquely from four different directions, to reconstruct the quantitative phase. This muti-capture scheme hinders qOBM’s ability to investigate valuable fast dynamic processes, such as blood flow, as well as its usability in some in-vivo applications. Here, we present a deep-learning enabled single-capture version of qOBM that quadruples the system’s imaging speed and prevents motion artifacts. To this end, we have trained a U-Net GAN to learn the qOBM reconstruction from a single capture obtained with oblique illumination. We show the capabilities and limitations of this approach, as well as some of the novel applications that this system enables, such as in-vivo high-resolution non-invasive blood flow quantitative phase imaging.
Quantitative oblique back-illumination microscopy (qOBM) enables quantitative phase imaging (QPI) in thick samples using epi-illumination. While qOBM offers unprecedented access to refractive index (RI) information in arbitrarily thick scattering samples, QPI-based (or RI index based) imaging still suffers from low cell nuclear contrast, which important for disease detection, including cancer. In this work, we use the acetowhitening effect of acetic acid to enhance the nuclear phase contrast of thick fresh tissue samples. Imaging results from brain samples are presented. Acetic acid phase staining may have important implications for in-vivo QPI-based disease detection
Quantitative phase imaging (QPI) enables label-free optical-path-length measurement of biological samples with nanometer-scale sensitivity, which offers unparalleled access to important histological and biophysical properties of cells and tissues. However, traditional QPI methods require a transmission-based optical geometry and are thus restricted to thin samples, which prevents the use of QPI for in-vivo applications. In this work, we present the design, characterization, and experimental validation of a handheld rigid probe for QPI with epi-illumination, using an optimized lighting configuration to achieve high phase-contrast sensitivity. The approach is based on a recently developed technology called quantitative oblique back illumination microscopy (qOBM). We demonstrate the real-time operation of our system with the future goal of applying it to help guide human brain tumor margin assessment intraoperatively in vivo, among many other potential applications.
Slide-free microscopy techniques have been proposed for accelerating standard histopathology and intraoperative guidance. One such technology is quantitative oblique back-illumination microscopy (qOBM), which enables real-time, label-free quantitative phase imaging of thick, unsectioned in-vivo and ex-vivo tissues. However, the grayscale phase contrast provided by qOBM differs from the colored histology images familiar to pathologists and clinicians, limiting its current potential for adoption. Here we demonstrate the application of unsupervised deep learning using a Cycleconsistent Generative Adversarial Network (CycleGAN) model to transform qOBM images into virtual hematoxylin and eosin (H&E)-stained images. The models were trained on a dataset of qOBM and H&E images of similar regions in excised brain tissue from a 9 L gliosarcoma rat tumor model. We observed successful qOBM-to-H&E conversion of both uninvolved and tumor-containing specimens, as demonstrated by a classifier test. We describe several crucial preprocessing steps that improve the quality of conversion, such as intensity inversion, pixel harmonization, and color normalization. This unsupervised deep learning framework does exhibit occasional subpar performance; for example, as with GANs in general, it can create so-called “hallucinations”, displaying features not actually present in the original qOBM images. We anticipate that this behavior can be minimized with more extensive training and deployment of advanced ML techniques, and that virtual-H&E-converted qOBM imaging will prove safe and appropriate for rapid tissue imaging applications.
Quantitative oblique back-illumination microscopy (qOBM) is a label-free imaging technique that enables tomographic phase imaging of thick scattering samples with epi-illumination. Here, we apply qOBM to image three-dimensional brain organoid cell cultures of tuberous sclerosis complex (TSC) disease. We identify quantitative differences that occur between the TSC organoids and a control cell line, and discuss the implications of these differences on our understanding the development of TSC organoid cultures. These differences include disruptions in the tubular processes in the organoid, a higher degree of folding and non-spherical cell growth, and differences in the proliferating cell structures between the two groups.
Significance: Quantitative oblique back-illumination microscopy (qOBM) is a recently developed label-free imaging technique that enables 3D quantitative phase imaging of thick scattering samples with epi-illumination. Here, we propose dynamic qOBM to achieve functional imaging based on subcellular dynamics, potentially indicative of metabolic activity. We show the potential utility of this novel technique by imaging adherent mesenchymal stromal cells (MSCs) grown in bioreactors, which can help address important unmet needs in cell manufacturing for therapeutics.
Aim: We aim to develop dynamic qOBM and demonstrate its potential for functional imaging based on cellular and subcellular dynamics.
Approach: To obtain functional images with dynamic qOBM, a sample is imaged over a period of time and its temporal signals are analyzed. The dynamic signals display an exponential frequency response that can be analyzed with phasor analysis. Functional images of the dynamic signatures are obtained by mapping the frequency dynamic response to phasor space and color-coding clustered signals.
Results: Functional imaging with dynamic qOBM provides unique information related to subcellular activity. The functional qOBM images of MSCs not only improve conspicuity of cells in complex environments (e.g., porous micro-carriers) but also reveal two distinct cell populations with different dynamic behavior.
Conclusions: In this work we present a label-free, fast, and scalable functional imaging approach to study and intuitively display cellular and subcellular dynamics. We further show the potential utility of this novel technique to help monitor adherent MSCs grown in bioreactors, which can help achieve quality-by-design of cell products, a significant unmet need in the field of cell therapeutics. This approach also has great potential for dynamic studies of other thick samples, such as organoids.
Quantitative oblique back-illumination microscopy (qOBM) is a novel microscopy technology that enables real-time, label-free quantitative phase imaging (QPI) of thick and intact tissue specimens. This approach has the potential to address a number of important biomedical challenges. In particular, qOBM could enable in-situ/in-vivo imaging of tissue during surgery for intraoperative guidance, as opposed to the technically challenging and often unsatisfactory ex-vivo approach of frozen-section-based histology. However, the greyscale phase contrast provided by qOBM differ from the colorized histological contrasts most familiar to pathologists and clinicians, limiting potential adoption in the medical field. Here, we demonstrate the use of a CycleGAN (generative adversarial network), an unsupervised deep learning framework, to transform qOBM images into virtual H&E. We trained CycleGAN models on a collection of qOBM and H&E images of excised brain tissue from a 9L gliosarcoma rat tumor model. We observed successful mode conversion of both healthy and tumor specimens, faithfully replicating features of the qOBM images in the style of traditional H&E. Some limitations were observed however, including attention-based constraints in the CycleGAN framework that occasionally allowed the model to ‘hallucinate’ features not actually present in the qOBM images used. Strategies for preventing these hallucinations, comprising both improved hardware capabilities and more stringent software constraints, will be discussed. Our results indicate that deep learning could potentially bridge the gap between qOBM and traditional histology, an outcome that could be transformative for image-guided therapy.
We demonstrate a fiber-based quantitative phase imaging (QPI) system with epi-illumination to acquire tissue and cellar level structure. Our approach is based on quantitative oblique back-illumination microscopy (qOBM), which keeps the advantages of QPI—label-free and non-destructive with nanometer-scale sensitivity—while also delivering tomographic sectioning capabilities in thick scattering samples using epi-illumination. The developed system uses a simple and robust configuration consisting of a flexible fiber bundle and a GRIN lens. Here data are presented with histopathological feature analysis. This technique, with its compact setup and real-time processing algorithm, can lead to in-vivo medical diagnosis, for clinical surgery and endoscopy.
Neutropenia is a condition where the hematopoietic system has a suppressed production of neutrophils, a type of white blood cell that is critical for fighting infections. This condition affects half to nearly eighty percent of cancer patients receiving chemotherapy, depending on the type of malignancy. Neutropenia can also be congenital or acquired from autoimmune disorders or nutritional deficits, in addition to cancer. Neutropenia, formally defined as <1500 neutrophils/µL in peripheral blood, puts patients at an increased risk of life-threatening infections. Thus, it is critical to constantly monitor neutrophil counts for many patients. Hematological analysis of neutropenia is performed by highly trained personnel at certified laboratories via complete blood count (CBC) and visual inspection which require complex, time-consuming, and expensive sample preparation and instrumentation. Thus, an easy-to-use, label- and reagent-free, and inexpensive hematology analysis device is highly desirable to circumvent these limitations and allow point-of-care disease monitoring and diagnosis. In this work, we demonstrate the application of deep-ultraviolet (UV) microscopy as label-free method for rapid and facile neutropenia detection. Our approach provides key hematological information and enables quantitative assessment of live blood cells based on their molecular and structural signatures in minutes. Here we show the ability of deep-UV microscopy to clearly identify patients with moderate and severe neutropenia based on an automated blood smear analysis. We also demonstrate a pseudo-colorization scheme which recapitulates the gold-standard Giemsa stains and allows visual inspection and enumeration of various blood cells types. This work has significant implications for developing a simple and low-cost point-of-care device that can ultimately improve the care and quality of life of many neutropenia and cancer patients.
The first-line treatment for brain cancer is surgery, which focuses on maximizing the percentage of the tumor removed during surgery (i.e., extent of resection) while minimizing damage to healthy brain tissue. Data show that extent of resection is one of the most critical factors associated with prolonged survival. However, differentiating between tumor and healthy tissue intraoperatively remains a significant clinical challenge, resulting in an exceedingly low 5-year survival rate of only ~35%. In this work, we show that quantitative oblique back illumination microscopy (qOBM), a novel label-free optical imaging technique that achieves tomographic quantitative phase imaging (QPI) in thick scattering samples, clearly differentiates between tumor and healthy tissue. Using a 9L gliosarcoma rat tumor model, we show that quantitative image features from qOBM provide a robust set of biomarkers for disease. In addition, tumor regions, including diffuse tumor, and healthy brain structures, show excellent structural agreement with H&E stained and sliced brightfield images, the gold standard for cancer detection. The unique attribute of qOBM—low-cost, easy-to-use, label-free, and real-time—make this technology ideally suited to help guide neurosurgery and address this important unmet need. Here we describe our free-space qOBM system and present quantitative results from the 9L gliosarcoma rat tumor model.
Deep ultraviolet microscopy (UV) enables high-resolution, label-free imaging of biological samples and yields diagnostically relevant quantitative molecular and structural information. We recently demonstrated that deep UV microscopy can serve as a simple, fast, and low-cost alternative to modern hematology analyzers that assess variations in the morphological, molecular, and cytogenetic properties of blood cells to monitor and diagnose blood disorders. We also introduced a pseudocolorization scheme that uses multi-spectral UV images (acquired at three different wavelengths) to generate images whose colors accurately recapitulate those produced by conventional Giemsa staining, and can thus be used for visual hematological analysis. Here, we present a deeplearning framework to virtually stain single-channel UV images acquired at 260 nm, providing a factor of three improvement in imaging speed without sacrificing accuracy. We train a generative adversarial network (GAN) using image pairs consisting of single-channel UV images of blood smears and their corresponding pseudocolorized images to generate realistic, virtually stained images. The virtual stained images are post-processed to improve contrast and yield consistent background colors. We quantify the performance of our framework in terms of the structural similarity index (SSIM) for each color channel. Our virtual staining scheme is the first step towards a completely automated hematological analysis pipeline that includes segmentation and classification of different blood cell types to compute metrics of diagnostic value. Our method eliminates the need to acquire images at different wavelengths and could potentially lead to the development of a faster and more compact label-free, point-of-care hematology analyzer.
Quantitative phase imaging (QPI) provides unique access to cellular and subcellular structures with nanometer-scale sensitivity, making it a valuable tool for non-destructive, label-free imaging of biological samples. However, implementation of QPI typically involves a transmission-based geometry and requires thin samples, preventing use of QPI in many important clinical settings, including endoscopy. In this work we demonstrate a fiber-optic device, with epi-illumination, capable of providing quantitative phase information that is well suited for clinical endoscopy, among other biomedical applications.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.