We present a novel approach to achieve high-speed depth-resolved two-photon imaging through the development of a deep-learning-based temporal-focusing two-photon microscope utilizing the De-scattering with Excitation Patterning (DEEP) method, referred to as DEEP-Line. DEEP-Line incorporates a line-scanning scheme, widefield detection utilizing a high-speed Silicon Photomultiplier array, and employs deep-learning-based image reconstruction. The performance of our system is validated using diverse biological samples. Our imaging method achieves orders of magnitude improvement in speed by reducing excitation patterns to several tens and employing MHz parallel detections. Furthermore, our approach can enable fluorescence lifetime imaging and enhances axial resolution.
With applications in quantitative metabolomics and label-free digital pathology, Quantitative Phase Microscopy (QPM) measures refractive index maps of thin transparent specimens like live cells or tissue sections. In QPM, refractive index maps are usually reconstructed from interference measurements of the object’s light field with respect to a reference field. To this end, many previous works focused on designing stable full-field interferometers from the bottom up. In this work, we present an alternative strategy to design a QPM system top-down, starting from the desired measurement outcomes, with no explicit knowledge about interferometry. We call our inverse design strategy Differentiable Microcopy. To this end, our Differentiable Microcopy approach designed a range of Fourier-filter-based QPM systems that do not require computational post-reconstruction. Our designs are superior to existing similar designs in numerical benchmarks. We also experimentally validated one design using a spatial light modulator. Finally, to fabricate these custom designs in the future, we also propose a new fabrication-aware Differentiable Microcopy pipeline.
Since its introduction, diffractive deep neural networks (D2NN) have been an emerging technology with many useful applications, such as 3D object recognition, saliency segmentation, and quantitative phase microscopy. However, fabricating D2NNs operating in the visible range has not been performed due to the complexities of fabricating nano-scale elements. Recent advancements such as Implosion Fabrication have made it possible to fabricate such networks using a discrete number of phase weights. We propose a quantization-aware training approach for D2NNs through modeling the quantization process in a differentiable manner using a sigmoid-based quantization function, facilitating the fabrication process. We also propose an efficient training schedule to guide the optimization process to converge to a better minima despite the limited number of quantization levels. Our method is simulated and validated for an all-optical quantitative phase microscopy task based on the phase D2NN.
With applications ranging from metabolomics to histopathology, quantitative phase microscopy (QPM) is a powerful label-free imaging modality. However, the speed of current QPM systems is limited by electronic hardware. To improve throughput further, here we propose differentiable optical-electronic quantitative phase microscopy (∂μ) that acquires images in a compressed form such that more information can be transferred beyond the electronic hardware bottleneck. The proposed microscopy uses optical feature extractors as image compressors. The resultant intensity distribution is then decompressed into QPM images by a reconstruction network. By optimizing optical-electronic parameters in an end-to-end manner, our method can improve the QPI throughput from Hours to Seconds (more than an order of magnitude).
The trade-off between throughput and image quality is an inherent challenge in microscopy. To improve throughput, compressive imaging under-samples image signals; the images are then computationally reconstructed. However, the information loss in the acquisition process sets the compression bounds. Here we propose differentiable compressive fluorescence microscopy (∂μ) that includes a realistic generalizable forward model with learnable-physical parameters (i.e. illumination patterns), and a novel physics-inspired inverse model. The cascaded model is end-to-end differentiable and can learn optimal compressive sampling schemes through training data. Proposed learned sampling outperforms widely used traditional compressive sampling schemes at higher compressions. We also demonstrate task-aware sampling (e.g. segmentation-aware) with the proposed framework.
Multiphoton microscopy is the gold standard for deep tissue fluorescence imaging. Long wavelengths enable hundreds of microns deep penetration of excitation light, but the emission fluorescence at shorter wavelengths encounters scattering before detection. While not being an issue for point scanning geometries, for wide-field geometries emission light scattering degrades the image quality. In this work, we use temporally focused pattered excitations to spatially encode image information before emission light scattering. Upon detection, images are reconstructed computationally by solving a linear inverse problem. We further improve our results by learning inverse solvers and optimal patterns through physics-based deep learning.
Imaging cytometry is a vital tool in many bio-imaging studies. Here we introduce, Differentiable Microscopy (δμ) a general end-to-end optimizable design framework to improve the throughput of imaging cytometry using content aware sampling and reconstruction.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.