With the rise in minimally invasive surgery and machine learning, there are emerging opportunities to improve patient outcomes with endoscopic techniques that quantify tissue shape and optical properties. We introduce a speckle-illumination stereo endoscope (SSE) that utilizes structured illumination to enhance both depth and optical property mapping. An SSE prototype was constructed and applied to fresh pig colon samples. SSE-estimated depth and optical property maps compare favorably to gold standard techniques. Requiring only minor modifications to existing commercial stereoscopes, the SSE could provide surgeons with improved visual depth perception and maps of biomarkers in vivo.
Urinalysis is an essential diagnostic tool in evaluating health and disease of the genitourinary tract. A urinalysis typically consists of dipstick testing, which can detect red blood cells, white blood cells, and bacteria, and microscopic evaluation of urine sediment after centrifugation, which further reveals other biomarkers such as crystals and casts. In the in-patient hospital setting, urinalysis is typically ordered after disease is suspected, drawing urine from the collection bag of a foley catheter and sending the sample to a core laboratory for analysis. To improve access to urine biomarkers, we propose a holographic lens free imaging (LFI) system that could allow automated bedside urine screening. LFI is uniquely suited for this task due to its low-cost, compact nature, and its ability to reconstruct large volumes from a single hologram without the depth-of-field trade-off of conventional microscopy. Here, we build and demonstrate an LFI system capable of detecting important biomarkers such as E. Coli in PBS and red blood cells, casts, and crystals in urinalysis control phantoms. In the future, this compact system could be connected to the drainage tube of a patient's foley catheter to enable real-time screening of urine at the bedside.
Some tumor resection procedures, such as Mohs surgery, utilize intraoperative histology for tumor margin assessments. Gold standard rapid histology methods are time-consuming for patients under anesthetic and rapid freezing techniques are prone to artifacts. The recent development of microscopy with ultraviolet surface excitation (MUSE) introduces a new possibility for the rapid imaging of the cut tissue surface using fluorescent dyes. The high attenuation of ultraviolet light limits MUSE signals to thicknesses close to typical histology sections. To generate MUSE images with familiar H&E-like contrast, recent work has explored the transformation of MUSE images to “virtual” H&E-like images using unsupervised deep learning models trained on unpaired images of separate tissues treated with each stain. Here, we present a method for acquiring registered images of the same tissue with MUSE and real H&E imaging using sequential staining and dye removal. Tissue blocks are flash frozen and sectioned for mounting onto slides and staining with MUSE fluorescent dyes. After MUSE imaging, a sequential immersion of the slides in increasing concentrations of ethyl alcohol followed by rehydration, similar to steps in paraffin-based histology processing, is sufficient to remove all fluorescent dyes. Rinsed tissue slides are then subjected to traditional H&E staining and brightfield imaging. Data of registered image fields of skin and pancreas are presented along with initial machine learning-based transformations from MUSE to H&E contrast. This protocol will be useful to obtain paired images for training, testing, and quantitative validation of virtual H&E reconstructions from MUSE images.
Spatial Frequency Domain Imaging (SFDI) is a powerful technique for non-contact tissue optical property and chromophore mapping over a large field of view. However, a major challenge that limits the clinical adoption of SFDI is that it requires carefully-controlled imaging geometry and the projection of known spatial frequencies. We present speckle-illumination SFDI (si-SFDI), a projector-free technique that measures tissue optical properties from structured illumination formed by randomized speckle patterns. We compute the local power spectral density of images under speckle illumination, from which a high-frequency and a low-frequency tissueresponse parameter can be characterized for each pixel. A lookup table generated by Monte-Carlo simulations is subsequently used to accurately determine optical absorption and reduced scattering coefficients. Compared to conventional SFDI, si-SFDI may be particularly useful for endoscopic applications due to its utilization of simple coherent illumination, which makes it more easily incorporated into existing endoscopic systems. Moreover, speckle illumination offers a large depth of focus compared to projector-based illumination. In this study, we explore wide-field optical property mapping with an endoscope camera and fiber-coupled laser speckle illumination. We apply this technique to tissue-mimicking silicone phantoms and biological tissues. The accuracy of si-SFDI is evaluated by comparing to optical properties measured by conventional SFDI. Future work could accelerate si-SFDI reconstruction by using parallel computing or machine learning algorithms.
Tissue oxygenation (StO2), which is the fraction of oxygenated hemoglobin in biological tissues, is an important biomarker that can reveal information about tissue viability and underlying pathologies. The continuous monitoring of StO2 is also useful for surgical guidance and patient management. In recent years, Spatial Frequency Domain Imaging (SFDI) has emerged as an elegant solution for mapping wide-field StO2. However, conventional SFDI requires capturing a sequence of images at different spatial frequencies and wavelengths, resulting in slow acquisition times and challenges with moving objects. Model-based single-snapshot techniques have shortened the acquisition time but introduce image artifacts and decrease accuracy. Here we propose a deep-learning technique for real-time StO2 mapping from snapshot structured light images. We train content-aware generative adversarial networks (OxyGAN) on pairs of structured light input at 659nm and 851nm wavelengths and StO2 ground truth predicted by conventional SFDI. We demonstrate that OxyGAN is not only capable of rapid data acquisition and processing but is also more accurate than a model-based benchmark. We also compare OxyGAN to a hybrid model that uses separate networks to estimate optical absorption at two wavelengths followed by Beer-Lambert fitting. The end-to-end OxyGAN approach shows better performance in terms of both speed and accuracy. We additionally demonstrate real-time OxyGAN by applying it to videos of in vivo tissues. OxyGAN has the potential to enable wide-field, real-time, and accurate tissue oxygenation measurements in many clinical applications.
Significance: Spatial frequency-domain imaging (SFDI) is a powerful technique for mapping tissue oxygen saturation over a wide field of view. However, current SFDI methods either require a sequence of several images with different illumination patterns or, in the case of single-snapshot optical properties (SSOP), introduce artifacts and sacrifice accuracy.
Aim: We introduce OxyGAN, a data-driven, content-aware method to estimate tissue oxygenation directly from single structured-light images.
Approach: OxyGAN is an end-to-end approach that uses supervised generative adversarial networks. Conventional SFDI is used to obtain ground truth tissue oxygenation maps for ex vivo human esophagi, in vivo hands and feet, and an in vivo pig colon sample under 659- and 851-nm sinusoidal illumination. We benchmark OxyGAN by comparing it with SSOP and a two-step hybrid technique that uses a previously developed deep learning model to predict optical properties followed by a physical model to calculate tissue oxygenation.
Results: When tested on human feet, cross-validated OxyGAN maps tissue oxygenation with an accuracy of 96.5%. When applied to sample types not included in the training set, such as human hands and pig colon, OxyGAN achieves a 93% accuracy, demonstrating robustness to various tissue types. On average, OxyGAN outperforms SSOP and a hybrid model in estimating tissue oxygenation by 24.9% and 24.7%, respectively. Finally, we optimize OxyGAN inference so that oxygenation maps are computed ∼10 times faster than previous work, enabling video-rate, 25-Hz imaging.
Conclusions: Due to its rapid acquisition and processing speed, OxyGAN has the potential to enable real-time, high-fidelity tissue oxygenation mapping that may be useful for many clinical applications.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.