3.1.

Scanning the Sample Stage Along the Focus Axis

Sample stage scanning, shown schematically in Fig. 1(a), is the classic and the most widely used method to obtain TF images. It is simple and straightforward and nearly every optical microscope has some provision to adjust the sample along the focus axis: these methods range from manual focusing on basic instruments to more sophisticated motorized or piezoscans on high-end modern tools. The scanning range can be up to several tens of millimeters, and positional accuracy can be better than 10 nm. However, TF scan speeds are typically regarded as relatively slow. Depending on the focus scan range, the exposure time, and the number of focal steps required, the total scan time could extend up to several minutes. The focus accuracy of this TF technique can be among the best, with high reproducibility and low distortion. However, optical mechanical instability³² and illumination aberrations^49,50 have also been reported for such systems.

TF optical images collected by the stage-scanning method can be stacked at their respective focus positions to create a 3-D space filled with the optical intensities. Plotting the optical intensities in a vertical cross-section through this 3-D space results in a TSOM image [Fig. 1(e)]. Even though TSOM uses conventional optical images (i.e., not a resolution enhancement method), it provides subnanometer 3-D shape measurement resolution²⁴ as it uses additional information present in the out-of-focus optical images. TSOM images are sensitive to changes in the (i) 3-D shape of a target,^24,29 (ii) position of a target in a 3-D space,⁴¹ (iii) optical properties of a target, and (iv) illumination.²⁰ A differential TSOM image (pixel-by-pixel difference between two well-aligned TSOM images) highlights all these differences with much higher signal-to-noise ratio compared to a conventional best-focus, top-down optical image.^18,19 TSOM is a strong candidate to analyze the position, shape, and optical properties of soft nanoparticles (e.g., hydrogel nanoparticles) in their native liquid environment, in addition to hard nanoparticles.⁴¹

3.2.

Scanning the Objective Lens Along the Focus Axis

Another method of collecting TF images is by scanning the objective lens along the focal axis [Fig. 1(b)], generally using piezomotors.^14,41,51 A relatively high-speed TF image collection time of 200 ms has been reported⁴¹ using this method, which makes it suitable, e.g., for nanoparticle tracking in 3-D space. It is relatively easy to convert a conventional microscope into a TF image collecting tool by replacing its objective lens base with one of the several commercially available objective-scanning piezomotors. In this high-speed scanning mode, the image quality could be degraded. However, this approach has the advantage of not disturbing the specimen because the sample stage remains stationary during imaging. Mechanically, this method is more stable than the stage-scanning method.

3.3.

Scanning the Image Plane Along the Focus Axis

Through focus image collection can also be accomplished by scanning the image plane or camera,^12,32 as shown in Fig. 1(c). The same effect can also be achieved by scanning a replica of the image plane.⁵² This method is reported to have the advantage of avoiding the spherical aberration common to other optical refocusing systems; it also allows for fast TF scans, extending the working distance, and keeping the specimen on stage stationary and undisturbed.⁵²

3.4.

Multifocal Plane Microscopy

Instead of scanning the image plane, one alternative is to split the imaging beam into several fixed beams that are refocused simultaneously onto multiple cameras positioned at different focal distances,^12,13,32 as shown in Fig. 1(d). This is also called image plane sharing microscopy.³² It has the main advantage of avoiding scanning altogether (of stage, objective or imaging plane). In this configuration, the TF images are collected simultaneously that enables high-speed TF image collection. However, since the imaging beam is split into several beams, the available optical intensity at each camera is reduced, which may lower the signal-to-noise ratio of the optical data, unless the collection time is increased. The number of focal planes available is also limited to the number of times the imaging beam is split. Since there are no moving (scanning) parts, this is one of the most mechanically stable configurations.

3.5.

Wavelength Scanning Method

For microscope objectives with chromatic aberration, differing wavelengths will have different focus positions. Normally microscope objectives are designed to minimize chromatic aberration. However, chromatic aberration can also be used to obtain TF images.^32,34,53 In this method, both the chromatic aberrated objective and the sample stage are kept at fixed positions. Instead, the wavelength of illumination is varied (or wavelength-scanned) so each wavelength focuses at a different focus position on the sample, enabling TF image collection (Fig. 2). A variation of this approach, which utilizes the chromatic dependence of diffraction, has been developed using tricolored LED illumination and a spatial light modulator (SLM).⁵³ Since the position of sample stage is not scanned, the wavelength scanning approach generally has superior mechanical stability compared to conventional stage-scanning optical microscopes. It also allows high-throughput TF image collection due to the ability to quickly and precisely tune the wavelength³⁴ (faster than scanning a stage).

Fig. 2

TF image collection configuration by wavelength-scanning using an objective with chromatic aberration. In the setup shown here, the white light is split into different wavelengths using a rotationally scanned grating.

3.6.

Flexible-Membrane Liquid Lens

While there are many variations in design and implementation, the general concept of the approach shown in Fig. 3(a) is that the fluid pressure inside two flexible membranes is varied to make the membranes inflate or deflate. This process alters the radius of curvature of the membranes and thus changes the effective focal length of the lens, allowing TF image collection.^54–62 The membrane curvature can alternatively be varied by different mechanisms such as changing the aperture diameter thus squeezing the membranes.⁶³ These lenses are generally fast, but are prone to have optical aberrations.⁵⁶

Fig. 3

Focus-tunable lenses. (a) Flexible membrane liquid lens. The effective focal length changes due to changes in the curvature of the flexible membranes when the pressure inside is varied. (b) Liquid-tunable lens. The curvature at the interface of two immiscible liquids varies due to changes in the interfacial tension when the applied voltage across the insulating and conducting fluids is changed. The change in the curvature results in varying the focal length.

3.7.

Liquid-Tunable Lens

This method, shown in Fig. 3(b), is also known as a variable focus liquid lens. These lenses utilize refraction at a liquid–liquid boundary interface. Focal adjustment is achieved by using a variable voltage to tune the curvature at the boundary interface of two immiscible liquids. A spherical surface is formed at the boundary of a polar and an apolar liquids. The curvature of the interface can be controlled by adjusting the relative wettability through electro-wetting.¹⁵ This creates a lens with variable focal length to enable TF image collection.^15,55,63,64 Liquid lenses are relatively fast and very durable and exhibit a high degree of phase shift, i.e., focal length change. These lenses have a long functional life and low sensitivity to mechanical stress, and they are also advantageous for autofocus applications due to their relatively low power consumption. Liquid-tunable lenses have relatively low cost and are commercially available. However, the optical performance of liquid-tunable lenses suffers from significant spherical as well as higher-order, gravity-induced aberrations.⁵⁶

3.8.

Adaptive Optics

As shown in Fig. 4, adaptive optical elements such as a tip/tilt or deformable mirror can be inserted into the optical path of a microscope to enable TF image collection.^32,65,66 Common examples of adoptive optics elements are deformable mirror devices or liquid crystal SLMs. The adoptive optics is used to alter the phase of incident wave fronts to displace the focal spot. Tip/tilt applied to the adoptive optics shifts the focus laterally within the focal plane, while defocus translates along the optic axis.⁶⁵ In the implementation with a tip/tilt mirror, a basic closed-loop quad cell is used to control the mirror. A high-order deformable mirror with a Shack–Hartmann sensor is used in the deformable mirror implementation. This approach can also correct high-order residual aberrations as well as performing the TF scanning without $z$-axis movement. A key advantage of these methods is high-speed TF image collection ($\sim \mathrm{KHz}$). Adaptive optics technology and hardware are readily available and are most commonly used for biological applications.¹⁶

Fig. 4

Schematic showing a design of using adaptive optical elements for TF image collection using tip/tilt or deformable mirrors.

3.9.

Multifocus Microscopy

Multifocus microscopy (MFM) allows rapid and simultaneous acquisition of TF images using a single exposure (Fig. 5). Unlike a conventional microscope, there are no scanning parts.^38–40 This is achieved by placing a multifocus grating (MFG) at the Fourier plane followed by a chromatic correction grating (CCG) and prism [Fig. 5(a)]. The combination of these elements divides the primary image into several TF images and simultaneously projects them onto the plane of the CCD camera. Unlike a conventional microscope, the CCD camera in MFM is divided into several squares, each one of which is used to collect one of the focal plane images [Fig. 5(b)]. To date, MFM has been used primarily for biology applications, and this technology is not yet widely available.

Fig. 5

Simplified schematic of MFM. (a) Optical elements unique to MFM. MFG, multifocus grating located at the Fourier plane; CCG, chromatic correction grating. (b) Images at predetermined focal planes are collected simultaneously by dividing the camera area as shown here. Optical designs are available to collect $3\times 3$, $4\times 4$, or $5\times 5$ images in the camera. (c) The location of aperture for aperture-scanning Fourier ptychography method.

3.10.

Aperture-Scanning Fourier Ptychography

In this method, an aperture is placed at the Fourier plane as shown in Fig. 5(c), and it is scanned in a raster manner while simultaneously collecting intensity images of the object. The acquired images are then synthesized in the frequency domain to recover the complex hologram of extended objects. This can then be digitally propagated into different planes along the optical axis to extract TF images of the object.⁶⁷ An alternative method is to scan the camera instead of an aperture.⁶⁷ Because of the requirement to scan certain parts, this approach could be slower.

3.11.

Confocal Microscopy

In confocal microscopy, out-of-focus image information is selectively discarded by placing a pin hole in front of the camera (Fig. 6). This allows collection of image-slices that contain primarily in-focus image information. By scanning the sample along the focal axis, multiple such image-slices through the sample are collected, and this enables a 3-D reconstruction of the sample surface. Numerous alternative implementations and variations on this basic method of data acquisition have been published. This is one of the most widely used TF imaging methods, particularly in biology.

Fig. 6

Basic schematic of a confocal microscopy setup. The pin hole selectively discards out-of-focus images information.

3.12.

Light Sheet Microscopy

In light sheet microscopy (LSM), the sample (usually transparent or semitransparent) is illuminated from the side using a thin sheet of light. Typically, the light source is rigidly coupled to the objective and aligned with its focal plane (Fig. 7).^68–70 By scanning the sample stage along the focal axis, volumetric images of the specimen can be obtained. In a strict sense, this method is not a TF method because the imaging plane is always in focus. However, it can extract 3-D volumetric image information similarly to a confocal microscope. Unlike a confocal microscope, however, the out-of-focus image information is not discarded, it is simply not illuminated. Numerous variations of this basic technique have been published.^69,71 LSM is mainly used to study biological tissues. Because of the opacity of such tissues, 3-D imaging is typically limited to a depth of 500 to $1000\mu \mathrm{m}$.^72,73 However, in some studies, LSM appears to outperform confocal microscopy.¹⁰

Fig. 7

Simplified schematic of LSM. In the basic setup, the imaging and illuminating objectives are rigidly coupled. The sample is scanned along the focal axis to collect volumetric images.

3.13.

Light Field Microscopy

In this microscopy, a microlens array is inserted into the optical train of a conventional microscope between the main lens and sensor plane, and this enables the capture of light-fields of specimens in a single image (Fig. 8).^74,75 TF images can then be extracted through application of 3-D deconvolution to this single image. LFM also has the capability to extract different perspective views from the single image. In an LFM, spatial resolution is determined by the number of microlenses. A disadvantage of LFM is the trade-off between spatial resolution and angular resolution.⁷⁴

Fig. 8

Simplified schematic of LFM. The lenslet arrays are located at the intermediate image plane.

3.14.

Phase Retrieval Techniques

Phase retrieval techniques make use of the relationship between phase and propagation direction in an optical microscope. This method initially requires a few TF images that are inverted to recover phase and amplitude quantitatively. By making use of the retrieved phase and the amplitude, the entire set of TF images can then be calculated.^76–78 A variation of this method, which is sometimes called quantitative phase imaging, has found many useful applications in biology.⁷⁹

3.15.

Digital Holography Microscopy

Holographic microscopy (including digital holography) is a coherent imaging system, and its advantage lies in the instantaneous and quantitative acquisition of both amplitude and the phase information from the reconstruction of the wavefront.^80–86 Digital holography microscopy can numerically extract TF images from a single experimentally recorded hologram without the need to move the sample (Fig. 9). The digital holographic approach has been successfully implemented to speed up TSOM image acquisition.³¹ Since no mechanical scanning is involved, and a complete set of TF images can be extracted from a single holographic image, it is a high-speed TF imaging method.

Fig. 9

Simplified schematic of DHM.

3.16.

Other Methods

Acousto-optic lens scanning¹⁷ can achieve a pure focal scan at a very high speed ($\approx 400\mathrm{kHz}$). This is accomplished using two adjacent counterpropagating acoustic waves with a synchronized frequency chirp so the transverse scans subtract to cancel each other, whereas the focal scans add. Acousto-optic modulators and acousto-optic scanners are used along with a laser beam, and there is a 180-deg phase shift between the two frequency-modulating signals. The light is focused on a CCD camera that is mounted on a translation stage and used to measure the focal distance. Temporal and spatial focusing⁸⁷ achieves high TF speeds with no moving parts. Full-field optical coherence microscopy produces volumetric imaging with all in-focus images similar to confocal microscopy.^88,89 It is based on the spatial coherence gate principle and generates in parallel complete two-dimensional TF type images without scanning.

Many variations of the preceding general TF methods have also been published. Some are a combination of two (or more) TF methods with some other methods. For example, a combination of high speed sinusoidal vibration of the microscope objective along with “smart” movements of galvanometric $x-y$ scanners (to repeatedly scan the laser focus along a closed 3-D trajectory) enables high-speed acquisition of TF images for two-photon microscopy that permits fast fluorescence measurements.¹⁴

Optical images such as the TSOM images [Fig. 1(e)] can be constructed using any one of the TF method presented here. However, for the TF methods such as confocal and light-sheet microscopy, all the images are in-focus creating a different type of TSOM image.

4.1.

Optical Data Type

TF optical images can be broadly divided into three types depending on the type of optical data

4.2.

Scanning

Depending on the presence or absence of scanning, TF image collection methods can be divided into two groups.

4.3.

Image Extraction

Depending on the image extraction method, two groups of TF approaches can be formed:

A complete categorization of all the TF methods is presented in Table 1.

Table 1

Complete categorization of the TF scanning methods.

5.1.

Speed

The speed of TF image collection and image quality will usually trend in opposite directions. In this review, the image quality mostly refers to signal-to-noise ratio. Typically, higher collection speeds reduce image quality and vice versa. In the scanning-based image collection methods, TF image collection time can vary considerably depending on the microscope conditions such as exposure time, the number of TF images needed, illumination source intensity, sample type, and method of scanning. All such factors must be optimized to achieve a suitable, low-noise image in the acceptable condition. Increasing the exposure time and the number of TF images will usually increase collection time, while increasing illumination source intensity decreases it due to lesser required exposure time. A sample that returns a large scattered intensity is beneficial in reducing the exposure time and hence TF image collection time. TF collection times of the scanning methods span from a relatively slow (in the order of minutes) to a relatively high-speed (in the order of less than a millisecond).

Scanning can either be performed continuously or using repeated, short, step-scans for the scanning-based TF image collection methods. In continuous-scanning approaches, the scanning part and the camera are synchronized that enables relatively high-speed TF image collections. For example, in the objective scanning method (Sec. 3.2), it is possible to continuously scan the objective back-and-forth and simultaneously record images at a relatively high-speed (200 ms⁴¹). The focus step size depends on the time gap between each exposure and the scanning speed of the objective. Even though the adaptive optics method (Sec. 3.8) is a type of continuous-scanning method, it achieves a very high-speed collection ($\approx 1\mathrm{kHz}$³²). Acousto-optic lens ($\approx 400\mathrm{kHz}$),¹⁷ liquid lenses ($2\mathrm{ms}/500\mathrm{Hz}$),^11,93 and adaptive optics ($10\mathrm{ms}/100\mathrm{Hz}$)⁶⁶ can also achieve a high-speed TF image collection.

Step-scan methods are relatively slow as scanning must be stopped briefly for image collection at every step, and this increases the overall TF image collection time. For example, in the stage-scanning method (Sec. 3.1), each time the stage is axially translated at a given step size, it is stopped for image collection, repeating the process until all the TF images are collected.

Usually, nonscanning methods are relatively fast. For example, in MFM, TF images are collected simultaneously in one exposure (or in one image). This results in TF image collection as fast as any single image collection using a conventional microscope. Multifocal plane microscopy also can collect TF images simultaneously at a high-speed. However, the number of images is restricted to the number of times imaging beam is split. High-speed TF imaging can also be achieved using DHM. A summary of the best speeds possible to collect a set of TF images for different TF methods is presented in Table 1.

5.2.

Image Quality

Not all TF imaging methods produce the same quality images. In fact, a wide range of image qualities can be expected depending on the TF method and the experimental conditions. It is the opinion of the author that the best quality images can be collected relatively easily using step-scan type of TF methods (e.g., stage-scanning method), but this is at the expense of collection speed. The presence of optical tool mechanical instability and other mechanical vibrations can deteriorate the image quality. In reality, scanning-based methods are usually prone to image deterioration of this kind.³⁴ Consequently, if careful attention is not given to these factors, the continuous-scanning methods will usually result in lower quality images. Liquid-tunable lenses and flexible membrane liquid lenses suffer from significant aberrations⁵⁶ which reduce image quality. Image quality in MFM can suffer because of splitting of the primary beam into several images.

As presented above, every TF method has certain advantages and disadvantages. The selection of a TF method depends on several factors, including type of data needed, image quality, applicability, speed of acquisition, cost, mechanical stability, accuracy, noise, simplicity, ease of use, and availability of the technology. Several aspects of optical microscopes have been identified to minimize degradation of optical images (e.g., laser stability, flat-field correction, camera performance, optical aberrations, noise, spectral reproducibility, lateral resolution, lens cleanliness, lens characteristics, temporal variability of signal and noise, absolute intensity calibration, and correcting field-dependent aberrations^90,94–102). However, to the best of our knowledge, there is no published method to test the fidelity of a set of TF images. Two aspects are uniquely associated with TF type of data collection: (i) focus step size and (ii) sample/stage vibration (or lateral movement) during mechanical scanning. Either imprecise focus step size (including in the extracted TF images) or lateral displacement of sample is highly likely to deteriorate TF optical data. A combination of (i) and (ii) could also be presented. For example, scanning methods could have mechanical instabilities and vibration issues³⁴ resulting in increased overall noise. If there is a mismatch between the sample stage scan axis with the optical axis along the focus direction, TF optical images appear to shift laterally³² creating a large error. Similar lateral image shift can also be observed if the aperture diaphragm is not correctly aligned with the optical axis.²⁰ For these reasons, it is important to evaluate fidelity of the TF data. We are actively working to develop a method to test TF data fidelity.