Proceedings Article | 27 July 2019
KEYWORDS: Microscopy, Molecules, Neural networks, Super resolution, Biological research, Image classification, Image restoration, Machine learning, Image resolution, Image processing
Deep learning has become an extremely effective tool for image classification and image restoration problems. Here, we address two fundamental problems of localization microscopy using machine learning: emitter density, and color determination.
Modern microscopy can produce images of biological specimen at very high (super) resolution, by precisely determining the positions of numerous blinking light emitting molecules over time. To achieve fast acquisition time, a high density of molecules is required, which poses a significant challenge in terms of image processing. Existing approaches use elaborate algorithms with many parameters that require tuning and a long computation time. Here, we report an ultra-fast, precise, and parameter-free method for super-resolution microscopy that utilizes deep-learning: by feeding the computer images of dense molecules along with their correct positions, it is trained to automatically produce super-resolution images from blinking data.
Next, we demonstrate how neural networks can exploit the chromatic dependence of the point-spread function to classify the colors of single emitters imaged on a grayscale camera. While existing single-molecule methods for spectral classification require additional optical elements in the emission path, e.g. spectral filters, prisms, or phase masks, our neural net correctly identifies static as well as mobile emitters with high efficiency using a standard, unmodified single-channel configuration – based on inherent chromatic aberrations in a standard microscope.
Finally, we demonstrate how deep learning can be used to design phase-modulating elements that, when implemented into the imaging path, result in further improved color differentiation between species. While point-spread-function engineering for spectral differentiation has been implemented in various applications in recent years, the optimal way to design such a PSF remains unclear. Here, we use a neural net to perform such design automatically, directly optimizing the desired cost function, namely, simultaneous localization and color detection of point emitters.
