1.

Introduction

As a classical technique for microsample observation and measurement, microscopic imaging is widely used in both biological research and medical diagnosis. Realization of accurate focus is often important to capture the details of a sample with high resolution and contrast. Though the optimal focal position can be found manually, this method is inefficient and imprecise, and as a result, an autofocus system is often equipped in commercial microscopes to increase focusing speed and accuracy.^1,2 There are two main types of autofocus methods, based on laser reflection and image analysis, respectively. Autofocusing via laser reflection can quickly locate the sample slice to a fixed plane without massive computation, such as the image read-in and processing; however, it is invalid when a sample varies its location from the reference surface (such as the thickness of the coverslip deviates its standard value) since its focal plane is maintained at a certain distance above the reference surface.³ To overcome this shortcoming, an image-analysis-based autofocus technique is proposed; with massive intensities recorded by sample (or micro-objective) stage scanning along the optical axis, the focal position can be determined according to the contrast, resolution, frequency components, or entropy extracted from the captured images.^4–6 This technique avoids focusing error due to surface variations and additionally, determines the focal position directly from captured intensities; as a result, the image-analysis-based autofocus method has been widely adopted in a majority of commercial microscopes because of its accuracy and robustness. However, it still suffers from disadvantages: first, the autofocus efficiency is limited because of the substantial image acquisition requirements; next, the effective autofocus range is narrow often within several micrometers, indicating that the method is invalid for large defocus cases. To realize autofocusing with high speed and accuracy, as well as to extend the effective range, Zheng’s group designed rapid autofocus methods similar to those adopted in professional photography.^7,8 Unfortunately, in these methods, microscopes must be modified by introducing extra pinhole-modulated cameras, and their incorrect installation often causes error, which reduces the focusing accuracy. According to Brenner gradient, Yazdanfar et al.⁹ designed a three-shot autofocus method that can localize the focal plane with high accuracy and fast speed; therefore, this method is a rather promising method for autofocusing, especially in microscopy. When the sample is located far away from the focal plane, the sensitivity of the Brenner gradient is remarkably reduced, limiting its applications in large defocus cases. Ferraro’s group proposed an autofocus method in digital holography to retrieve high-quality in-focus image from holograms,^10–12 and based on these techniques, they also realized high-accurate three-dimensional image reconstructions and particle tracking.^13–15 However, limited by the demands of coherent source and extra reference beam, the method can hardly be integrated in commercial microscopes. To obtain high focusing speed and accuracy, while avoiding imaging system modifications, in this paper, we propose a wavefront-sensing-based autofocus technique relying on wavefront retrieval, propagation, and analysis.^16–22 In addition, the proposed autofocus method can be directly applied in commercial microscopes only with an extra illumination filter. Verified by both numerical simulations and experimental verification, the newly designed approach can precisely determine the focal plane using relatively few image collections (often $<10$) with a rather large effective range (over $100\mu \mathrm{m}$). Considering its fast speed, high accuracy, and large effective range, we believe that the proposed method can be adopted for rapid autofocusing in commercial microscopes.

2.

Principle

Figure 1 provides a flowchart of the wavefront-sensing-based autofocus method. First, three defocus intensities equidistant along the optical axis are captured by sample stage shifting; note that these images are blurred since the sample is not located at the focal plane. Next, the phase distribution is retrieved by solving the transport-of-intensity equation (TIE)^23–30 as shown in Eq. (1), in which $\partial I/\partial z$ indicates the intensity gradient (which can be computed from captured images at different planes), $k$ is the wave number, and $\phi$ is the phase distribution, which can be retrieved with a widely used fast Fourier transform-based solver³¹

Fig. 1

Flowchart of the wavefront-sensing-based autofocus method.

As the phase is obtained, through combination with the amplitude directly extracted from the captured image, intensities at different planes along the optical axis can be numerically computed according to the wavefront propagation using the angular spectrum method. Then, the focal plane can be determined by evaluating these numerically propagated intensities using focus criteria, such as derivative-based, statistical, intuitive algorithms, etc.³² Here, we choose the Tamura-coefficient-based focus criterion (TC), which is robust in focus determination, and it has a good noise suppression capability.³³ This focus criterion is based on the contrast between the gray-level variation and the average of an intensity distribution $I$, as shown in Eq. (2), in which $\sigma (I)$ and $⟨I⟩$ indicate the standard deviation and mean of a gray-level image, respectively. The focal plane can be determined by tracking the TC peak

Finally, after the sample stage is shifted to the determined focal plane, one more processing loop is needed for verification; if the location of the focal plane is the same as that determined by the previous loop, the focal plane has been identified, otherwise, another processing loop is required until the focal positions determined by two successive processing loops coincide. Compared to the widely used image-analysis-based autofocus technique, which requires massive numbers of image captures, the newly designed method only needs 2 to 3 loops with $<10$ intensity captures. Moreover, considering the time saved in sample stage scanning, it is believed that the proposed method can realize a comparatively rapid processing speed for autofocusing.

3.

Numerical Simulations and Experiments

To prove the feasibility of the proposed autofocus technique, a numerical simulation was first implemented as shown in Fig. 2. To mimic the practical microscopic configuration, a lens system with $10\times$ magnification representing both the micro-objective and tube lens in a commercial microscope was introduced in the simulation model. Preset intensity and phase distributions are listed in Fig. 2(b). Random Gaussian noise was added to generate a signal-to-noise ratio of 30 dB, close to that estimated from experiments. The wavelength was set to 532 nm and pixel size to $8.3\mu \mathrm{m}$, both in accordance with experimental devices. When the sample was located $40.0\mu \mathrm{m}$ away from the focal plane, only the blurred image in Fig. 2(c) could be obtained. Using the TIE algorithm, the phase at the defocus plane shown in Fig. 2(c) was extracted through use of another two symmetric defocus images with a separation of $4.0\mu \mathrm{m}$. Then, the intensities at different planes along the optical axis were numerically computed according to the extracted defocus wavefront information. Figure 2(d) shows quantitative evaluations of these numerically propagated intensities using the TC. These results indicate a defocus distance of around $40.0\mu \mathrm{m}$, proving the feasibility of the wavefront-sensing-based autofocus technique. Moreover, to quantitatively analyze the accuracy of the proposed wavefront-sensing-based autofocus method, various patterns with different defocus conditions from 10.0 to $80.0\mu \mathrm{m}$ were used as examples, the error (interval between the determined focal plane and the real one) was statistically computed as $0.16\pm 0.05\mu \mathrm{m}$ from all the simulated data, indicating the high accuracy of the proposed method.^3,5

Fig. 2

Numerical verification of the wavefront-sensing-based autofocus technique. (a) Optical system in numerical simulations, (b) preset intensity and phase distributions, (c) defocus intensity and phase distributions, and (d) numerical quantitative evaluation of intensities of the propagated wavefront using TC.

Next, the proposed method was adopted for autofocusing in a commercial upright microscope (Mshot ML-32, China), in which sample position can be scanned along the optical axis using a motorized sample stage (Mshot MS-300, China) with step of $0.5\mu \mathrm{m}$. A CCD camera (AVT Prosilica GC780, Germany) with pixel size of $8.3\mu \mathrm{m}$ was used for image recording. Narrowband illumination at 532 nm with FWHM of 10 nm was generated with an interference filter (Daheng Optics GCC-202003, China) to guarantee high-accuracy phase retrieval. In addition, Kohler illumination was implemented, and the condenser aperture was set as $\sim 40\%$ of the objective aperture to ensure partially spatial coherent illumination.^34–37 Plant rhizome cross cuttings and red blood cell smears (Keda Biological Sample Company, China) were used as samples. Before experimental verification of the wavefront-sensing-based autofocusing, the real focal plane was first determined using the classical image-analysis-based technique; the defocus distance could then be precisely set by moving the sample to a defocus plane. For the plant rhizome cross cutting, a $10\times$ micro-objective was used, and the defocus distance was set to $40.0\mu \mathrm{m}$ in the sample plane.

Figure 3(a1) shows the recorded defocus intensity. The defocus phase in Fig. 3(b1) was computed combining another two symmetric defocus image recorded next to the central one with a separation of $4.0\mu \mathrm{m}$. Then, using wavefront propagation, the intensities at different positions along the optical axis were numerically computed according to the extracted complex amplitude. Because the minimum shifting step of the used motorized sample stage was $0.5\mu \mathrm{m}$, besides, using the classical image-analysis-based autofocus method, the scanning step could only reach $0.5\mu \mathrm{m}$ limited by the hardware; therefore, the numerical wavefront propagation step in our proposed method was set as $0.5\mu \mathrm{m}$. After evaluation by the TC, the focal plane could be determined as shown in Fig. 3(c1), which shows a defocus interval of $40.0\mu \mathrm{m}$ coinciding with the preset value. Though the numerically retrieved in-focus intensity in Fig. 3(d1) improved the image quality, it was still limited due to error and information loss from wavefront retrieval and propagation. Finally, after the sample stage was shifted to the determined focal plane, the real in-focus intensity was captured as shown in Fig. 3(e1). The proposed autofocus method was also tested using a red blood cell smear, representing a discrete sample, different from the continuous plant rhizome cross cutting. In this case, a $40\times$ micro-objective was used, and the defocus distance was set to $10.0\mu \mathrm{m}$ in the sample plane. The defocus intensity and phase are shown in Figs. 3(a2) and 3(b2), respectively. Using the proposed approach, the real focal plane could be accurately localized, as shown in Fig. 3(c2), by searching the numerically propagated in-focus image shown in Fig. 3(d2). Finally, the real in-focus image in Fig. 3(e2) was recorded after the sample stage was shifted to the determined focal plane. Though another loop was required for focal plane confirmation, only images captured at six planes were needed. Considering the discrete step setting in wavefront propagation, the Tamura peaks always located at the correct focal plane, indicating the error of the proposed method was always within [$-0.25$, $0.25\mu \mathrm{m}$], which also fits well with the numerical simulation results. According to the practical application shown in Figs. 3(a1)–3(e1) with a defocus distance of $40\mu \mathrm{m}$, when using the classical image-analysis-based autofocus technique, multifocal images were captured with a step of $0.5\mu \mathrm{m}$ during the sample scanning along the imaging axis, besides, to certificate the focal plane, extra distance scanning passing the focal position ($\sim 10\mu \mathrm{m}$) was still required. Therefore, in this case, $\sim 100$ multifocal images were captured along a scanning distance of $50\mu \mathrm{m}$ in sample plane. However, only six image captures were enough to localize the focal plane with the same accuracy when using our proposed wavefront-sensing-based autofocus technique. Compared to the traditional image-analysis-based technique, this method considerably reduces image captures and time consumption. Using traditional image-analysis-based autofocusing, a total of 4.88 s was needed for autofocusing with our self-built autofocus system, including 1.25 s for sample stage translation, 1.40 s for image recording, and 2.23 s for digital image processing (with a desktop with Intel Core i5-3470 CPU at 3.20 GHz and a 4-GB RAM). In contrast, the time consumption of the proposed autofocus method was 3.45 s, with 0.40 s for stage moving, 0.14 s for image recording, and 2.91 s for digital image processing. In traditional autofocusing, as the necessary scanning distance for stage translation was longer than that in the proposed method, additionally, multiple sample stage translation was required for image recording; it required more time compared to that in proposed approach. Though the proposed autofocusing did not require massive image read-in, phase retrieval and wavefront propagation were indispensable and consumed excess time. Compared to image-analysis-based technique, the proposed autofocusing method is more time-consuming in digital image processing, including a large amount of numerical wavefront propagation and their in-focus evaluations to maintain wide focus searching range and high focus determination accuracy. Since often the time-consuming sequential calculation was adopted to numerically compute the intensities as well as to execute the in-focus evaluation, obviously decreasing the computation efficiency. However, in the future work, introducing graphics processing unit computation for both numerical wavefront propagation and in-focus evaluation will obviously accelerate the image processing, and the superiority of the proposed autofocusing technique in terms of processing time will be more significant.

Fig. 3

Experimental verification of the wavefront-sensing-based autofocusing in (1) a plant rhizome cross cutting and (2) a red blood cell smear. (a) Defocus intensity, (b) phase distribution, (c) numerical quantitative evaluation of intensities of the propagated wavefront, (d) numerically propagated in-focus image, and (e) real in-focus image obtained via sample stage shifting.

Figures 4(a) and 4(b) list the quantitative comparisons of different focus criteria in the condition of plant rhizome cross cutting shown in Fig. 3. In addition to the adopted TC, Brenner gradient,³⁸ image power,³⁹ and autocorrelation^40,41 methods were also implemented, which represent derivative-based, intuitive and statistical algorithms, respectively. All these focus criteria, including the Tamura-coefficient-based one, can precisely determine the focal plane not only in classical image-analysis-based autofocus method [Fig. 4(a)] but also in proposed wavefront-sensing-based autofocus technique [Fig. 4(b)], proving the capability of the TC.

Fig. 4

Experimental verification of TC by comparing with other criteria in both the (a) image-analysis-based autofocusing and (b) wavefront-sensing-based autofocusing.

If there is a large defocus distance in the sample location, a single processing loop cannot localize the focal plane due to the detail loss in wavefront extraction and the error in numerical wavefront propagation. However, the focal plane can still be determined using additional processing loops. Figure 5 provides two examples of autofocusing in large defocus distance cases. For the plant rhizome cross cutting, the defocus distance was set to $120.0\mu \mathrm{m}$; both defocus intensity and phase are shown in Fig. 5(a1). After the first processing loop, Fig. 5(b1) shows the determined focal position, which was only $31.0\mu \mathrm{m}$ away from the real focal plane. Next, the second processing loop was implemented as shown in Figs. 5(c1) and 5(d1); the focal plane could be precisely determined after the confirmation using an additional processing loop. Finally, the real in-focus image was captured, as shown in Fig. 5(e1). For the red blood cell smear, there was a large defocus distance of $30.0\mu \mathrm{m}$; the focal plane could still be determined using extra processing loops as shown in Figs. 5(a2) to 5(e2), proving that the wavefront-sensing-based autofocus method can precisely localize the focal plane even for large defocus cases. Additionally, both the autofocus effective ranges for $40\times$ and $10\times$ micro-objective cases were $\pm 50\mu \mathrm{m}$ and $\pm 170\mu \mathrm{m}$ away from the focal plane estimated by practical experiments, respectively. When the sample location is out of the effective range, the proposed autofocus method cannot provide a location that close to the actual focal plane due to high-frequency component lose and low signal-to-noise ratio, therefore preventing the convergent focal position searching. Though the autofocus method designed by Yazdanfar et al.⁹ can rapidly and precisely localize the focal plane only with three multifocal intensity captures, due to the low sensitivity in large defocus conditions, its effective ranges for $40\times$ and $10\times$ micro-objective cases were around $\pm 20\mu \mathrm{m}$ and $\pm 50\mu \mathrm{m}$ away from the focal plane also estimated by practical experiments, respectively. Compared to this three-shot autofocus technique, the effective ranges of the proposed autofocus method were relatively large, proving the proposed method can be successfully used in wide applications.

Fig. 5

Experimental verification of wavefront-sensing-based autofocusing with a large defocus distance in (1) a plant rhizome cross cutting and (2) a red blood cell smear. (a) Defocus intensity/phase distributions and (b) numerical quantitative evaluation of intensities of the propagated wavefront from the first processing loop, (c) defocus intensity/phase distributions and (d) numerical quantitative evaluation of intensities of the propagated wavefront from the second processing loop, and (e) real in-focus image.

4.

Conclusion

We combine wavefront retrieval, propagation, and analysis to propose wavefront-sensing-based autofocusing in microscopy. Since much less image recording is required compared to the widely used image-analysis-based technique, the proposed method typically demonstrates faster autofocus speed. Moreover, even if the initial sample location is far from the focal plane, this method can still effectively localize the focus position, which is proven by both numerical simulations and practical measurements. Additionally, the newly designed autofocus method can be applied in commercial microscopes only with an extra illumination filter. Considering its high processing efficiency and large effective range as well as its simple design and operation, the proposed approach is a good candidate for future adoption as the primary method of rapid autofocusing in microscopy.