Since its introduction, diffractive deep neural networks (D2NN) have been an emerging technology with many useful applications, such as 3D object recognition, saliency segmentation, and quantitative phase microscopy. However, fabricating D2NNs operating in the visible range has not been performed due to the complexities of fabricating nano-scale elements. Recent advancements such as Implosion Fabrication have made it possible to fabricate such networks using a discrete number of phase weights. We propose a quantization-aware training approach for D2NNs through modeling the quantization process in a differentiable manner using a sigmoid-based quantization function, facilitating the fabrication process. We also propose an efficient training schedule to guide the optimization process to converge to a better minima despite the limited number of quantization levels. Our method is simulated and validated for an all-optical quantitative phase microscopy task based on the phase D2NN.
Understanding and repairing complex biological systems, such as the brain, requires new technologies that enable such systems to be observed and controlled with great precision, across extended spatial and temporal scales. We are discovering new molecular principles that are leading to such technologies. For example, we recently discovered that it was possible to physically magnify biological specimens manyfold, in an even way, by embedding them in dense swellable polymers, mechanically homogenizing the specimens, and then adding water to isotropically swell the specimens. In this method, which we call expansion microscopy (ExM), we enable scalable, inexpensive diffraction-limited microscopes to do large-volume nanoscopy, in a multiplexed fashion – important, for example, for brain mapping. As another example, we discovered that microbial opsins, genetically expressed in neurons, could enable their electrical activities to be precisely driven or silenced in response to millisecond timescale pulses of light. These tools, called optogenetic tools, are enabling causal assessment of the contribution of defined neurons to behaviors and pathologies in a wide variety of basic science settings. Finally, we have developed new methods of directed evolution, and discovered mutant forms of optogenetic tools that enable precision fluorescent imaging of the high-speed voltage of neurons in the living brain. We share all these tools freely, and aim to integrate the use of these tools so as to lead to comprehensive understandings of neural circuits.
Understanding and repairing complex biological systems, such as the
brain, requires new technologies that enable such systems to be
observed and controlled with great precision, across extended spatial
and temporal scales. We are discovering new molecular principles that
are leading to such technologies. For example, we recently discovered
that it was possible to physically magnify biological specimens
manyfold, in an even way, by embedding them in dense swellable
polymers, mechanically homogenizing the specimens, and then adding
water to isotropically swell the specimens. In this method, which we
call expansion microscopy (ExM), we enable scalable, inexpensive
diffraction-limited microscopes to do large-volume nanoscopy, in a
multiplexed fashion – important, for example, for brain mapping. As
another example, we discovered that microbial opsins, genetically
expressed in neurons, could enable their electrical activities to be
precisely driven or silenced in response to millisecond timescale
pulses of light. These tools, called optogenetic tools, are enabling
causal assessment of the contribution of defined neurons to behaviors
and pathologies in a wide variety of basic science settings. Finally,
we have developed new methods of directed evolution, and discovered
mutant forms of optogenetic tools that enable precision fluorescent
imaging of the high-speed voltage of neurons in the living brain. We
share all these tools freely, and aim to integrate the use of these
tools so as to lead to comprehensive understandings of neural
circuits.
Nanotechnology and more generally the ability to fabricate devices with nanoscale features has developed from a history rooted in the semiconductor industry; however, biology has long been able to programmatically assemble nanoscale structures with a vast array of functions. Inspired by this, we develop a set of volumetric deposition strategies related to the mechanisms of assembly employed in biological systems. Leveraging a technology for 3D nanofabrication, Implosion Fabrication, we have explored novel methods for depositing nanomaterials relevant to optics, photonics, and plasmonics using thiol-binding, hydrophobic interactions, protein binding, and hydrogen bonds into any 3D geometry with nanoscale resolution and gradient capabilities. Through the development of these novel deposition chemistries, we create a platform by which a large variety of functional nanomaterials can be directed to assemble for the future of device manufacture.
Optogenetics has revolutionized the study of circuit function in the brain, by allowing activation of specific ensembles of neurons by light. However, this technique has not yet been exploited extensively at the subcellular level. Here, we test the feasibility of a focal stimulation approach using stimulated emission depletion/reversible saturable optical fluorescence transitions-like illumination, whereby switchable light-gated channels are focally activated by a laser beam of one wavelength and deactivated by an overlapping donut-shaped beam of a different wavelength, confining activation to a center focal region. This method requires that activated channelrhodopsins are inactivated by overlapping illumination of a distinct wavelength and that photocurrents are large enough to be detected at the nanoscale. In tests of current optogenetic tools, we found that ChR2 C128A/H134R/T159C and CoChR C108S and C108S/D136A—activated with 405-nm light and inactivated by coillumination with 594-nm light—and C1V1 E122T/C167S—activated by 561-nm light and inactivated by 405-nm light—were most promising in terms of highest photocurrents and efficient inactivation with coillumination. Although further engineering of step-function channelrhodopsin variants with higher photoconductances will be required to employ this approach at the nanoscale, our findings provide a framework to guide future development of this technique.
To enable the understanding and repair of complex biological systems such as the brain, we are creating novel optical tools that enable molecular-resolution maps of large scale systems, as well as technologies for observing and controlling high-speed physiological dynamics in such systems. First, we have developed a method for imaging large 3-D specimens with nanoscale precision, by embedding them in a swellable polymer, homogenizing their mechanical properties, and exposing them to water - which causes them to expand isotropically manyfold. This method, which we call expansion microscopy (ExM), enables scalable, inexpensive diffraction-limited microscopes to do large-volume nanoscopy, in a multiplexed fashion. Second, we have developed a set of genetically-encoded reagents, known as optogenetic tools, that when expressed in specific neurons, enable their electrical activities to be precisely driven or silenced in response to millisecond timescale pulses of light. Finally, we are developing novel reagents and systems to enable the imaging of fast physiological processes in 3-D with millisecond precision. In this way we aim to enable the systematic mapping, control, and dynamical observation of complex biological systems like the brain.
We introduce the design and theoretical analysis of a fiber-optic architecture for neural recording without contrast agents, which transduces neural electrical signals into a multiplexed optical readout. Our sensor design is inspired by electro-optic modulators, which modulate the refractive index of a waveguide by applying a voltage across an electro-optic core material. We estimate that this design would allow recording of the activities of individual neurons located at points along a 10-cm length of optical fiber with 40-μm axial resolution and sensitivity down to 100 μV using commercially available optical reflectometers as readout devices. Neural recording sites detect a potential difference against a reference and apply this potential to a capacitor. The waveguide serves as one of the plates of the capacitor, so charge accumulation across the capacitor results in an optical effect. A key concept of the design is that the sensitivity can be improved by increasing the capacitance. To maximize the capacitance, we utilize a microscopic layer of material with high relative permittivity. If suitable materials can be found—possessing high capacitance per unit area as well as favorable properties with respect to toxicity, optical attenuation, ohmic junctions, and surface capacitance—then such sensing fibers could, in principle, be scaled down to few-micron cross-sections for minimally invasive neural interfacing. We study these material requirements and propose potential material choices. Custom-designed multimaterial optical fibers, probed using a reflectometric readout, may, therefore, provide a powerful platform for neural sensing.
Jacob Bernstein, Xue Han, Michael Henninger, Emily Ko, Xiaofeng Qian, Giovanni Talei Franzesi, Jackie McConnell, Patrick Stern, Robert Desimone, Edward Boyden
Many neural disorders are associated with aberrant activity in specific cell types or neural projection pathways
embedded within the densely-wired, heterogeneous matter of the brain. An ideal therapy would permit correction of
activity just in specific target neurons, while leaving other neurons unaltered. Recently our lab revealed that the
naturally-occurring light-activated proteins channelrhodopsin-2 (ChR2) and halorhodopsin (Halo/NpHR) can, when
genetically expressed in neurons, enable them to be safely, precisely, and reversibly activated and silenced by pulses
of blue and yellow light, respectively. We here describe the ability to make specific neurons in the brain light-sensitive,
using a viral approach. We also reveal the design and construction of a scalable, fully-implantable optical
prosthetic capable of delivering light of appropriate intensity and wavelength to targeted neurons at arbitrary 3-D
locations within the brain, enabling activation and silencing of specific neuron types at multiple locations. Finally,
we demonstrate control of neural activity in the cortex of the non-human primate, a key step in the translation of
such technology for human clinical use. Systems for optical targeting of specific neural circuit elements may enable
a new generation of high-precision therapies for brain disorders.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.