Deep learning based on convolutional neural network (CNN) has attracted more and more attention in phase unwrapping of fringe projection three-dimensional (3D) measurement. However, due to the inherent limitations of convolutional operator, it is difficult to accurately determine the fringe order in wrapped phase patterns that rely on continuity and globality. To attack this problem, in this paper we develop a hybrid CNN-transformer model (Hformer) dedicated to phase unwrapping via fringe order prediction. The proposed Hformer model has a hybrid CNN-transformer architecture that is mainly composed of backbone, encoder, and decoder to take advantage of both CNN and transformer. Backbone is used as a wrapped phase pattern feature extractor. Encoder and decoder with cross attention are designed to enhance global dependency for the fringe order prediction. Experimental results show that the proposed Hformer model achieves better performance in fringe order prediction compared with the CNN models such as U-Net and DCNN. Our work opens an alternative way to the CNN-dominated deep learning phase unwrapping of fringe projection 3D measurement.
Phase encoding and phase-shift profilometry are two commonly used 3D measurement techniques. However, the acquired phases in the techniques are subject to jump errors due to phase ambiguity and phase errors caused by multiple heterodyne. The phase-shifting profilometry also makes the selection of fringe period difficult. To overcome this problem and achieve high-precision measurement, a phase unwrapping method that combines dual-frequency heterodyne with double complementary phase encoding is proposed. First, two wrapped phases are obtained by two groups of sinusoidal fringes; the heterodyne phase is obtained after heterodyne processing, and the high-frequency phase is expanded by heterodyne phase. Second, the fringe levels are obtained using the complementary phase encoding fringes that are shifted by half an order, and then the absolute phase is obtained by selecting different phase coding levels according to different regions for the first phase unwrapping; Finally, the phase noise is removed by exploiting the difference between the phase slopes of adjacent pixels. Experimental results show that a system with the proposed method achieves an RMS error of 0.015 mm. In addition, the period of dual-frequency heterodyne synthesis does not need to cover the whole field of view, which breaks the limitation of frequency selection of the traditional dual-frequency heterodyne method and triple frequency heterodyne method, enabling high-precision measurement with higher frequency fringes. This method overcomes the limitations of the phase principal value error when using higher frequency fringes for high-precision measurement, improves the measurement effect of reflective objects, and effectively avoids the error caused by phase jump.
Phase-shifting profilometry (PSP) is one of the mainstream fringe projection techniques for object surface reconstruction. Thanks to its multi-shot nature, PSP is less sensitive to ambient light and reflectivity on the object surface. For static objects, the performance of PSP can be improved by projecting and capturing more phase-shifted fringe patterns. However, when applied to dynamic objects, PSP suffers from motion-induced errors due to the loss of correspondence. This paper proposes a new approach to improving the performance of measuring objects with a general three-dimensional (3D) movement. Firstly, instead of employing a large number of fringe patterns for PSP, we apply three-step PSP, which suffers less from motion errors, to obtain multiple coarse measurements of the object. Then the moving object is segmented, and the iterative closest point (ICP) algorithm is applied to estimate the motion parameters. Finally, the multiple measurements of the object are fused using adaptive weights. The proposed scheme alleviates the motion errors of PSP with a large number of fringe patterns and enhances the accuracy of three-step PSP. Simulations and experiments verify the feasibility of the proposed scheme.
Fringe projection profilometry (FPP) is a non-contact, high-precision technique for measuring three-dimensional (3D) shapes. An essential step of FPP is to recover the phase distribution from the deformed fringe patterns. In real applications, the captured fringe patterns often suffer from noises, which results in degradation of the performance of phase retrieval and shape reconstruction. Fringe denoising can be applied to suppress the influence of noise in FPP. This paper introduces a novel fringe denoising method based on robust principal component analysis (RPCA). The proposed method makes use of the low-rankness of the clean fringe patterns and the sparsity of the strong impulsive fringe noise. RPCA is then applied to effectively mitigate the strong impulsive fringe noise and suppress the random additive noise. The proposed method features 2D processing of the fringe patterns and is easy to implement. Its effectiveness is demonstrated via numerical simulations.
Phase shifting profilometry (PSP) is considered as an effective method for 3D shape measurement based on fringe projection. However, PSP is not suitable for dynamic measurement, as it requires that the object be kept still. Movement of the object during the cause of projection of multiple fringe patterns may lead to significant error in the measurement of the 3D shape. A number of approaches were proposed to combat this problem consisting of two steps: Capturing of the movement and then compensation (or correction) of fringe patterns. However, such compensation is only valid for the cases where the object moves or rotates in the way that all points on the object surface change by the same amount. In other words, there is still not a method effective for measuring objects moving in a free 3D space. In this paper, a new method is proposed to combat the problem. Firstly, movement of the object is capturing by means of existing methods, yielding rotation matrix and translation vector, able to characterize arbitrary movement in a 3D space. Secondly, variation of the fringe patterns by the movement is analyzed and formulated, leading to the expressions of phase maps. Based on these expressions, a new method is proposed to compensate the variance on height map, with which PSP can be used to yield improved measurement performance. Computer simulations is carried out to verify the effectiveness of the proposed method.
In this work, we consider the problem of 3D shape reconstruction with a system with two cameras and one projector, which can be regarded as a composite system with two fringe projection profilometry (FPP) systems and one stereo vision (SV) system. Different from the active SV systems in the literature where FPP is used to assist SV to address the issue of corresponding matching, a new system is proposed in this paper by constructing a fusion cost function with the consideration of both FPP and SV triangulations, so that the measurements of the two cameras can be fused to achieve better measurement performance than that of the active SV and FPP. In addition, a message passing algorithm for 3D shape reconstruction with the system is developed by using the new cost function and exploiting the unknown correlation of object surfaces. Simulation results demonstrate that the proposed system can achieve considerable performance gain.
Phase unwrapping is an essential step for 3D shape measurement based on fringe projection. Temporal phase unwrapping methods can be implemented by analyzing the multiple patterns that encode the fringe order information. They can retrieve the fringe orders on a pixel-by-pixel basis and are less prone to error propagation compared with spatial methods. However, fringe orders errors may still occur due to noise, reflectivity fluctuation and discontinuity of the object surface. Such errors may exhibit an impulsive nature and result in significant error to the recovered absolute phase map. This has been exploited by several methods to correct the fringe order errors, e.g., by filtering the fringe order sequences in a line-by-line manner. In this paper, a new method is proposed to correct the errors associated with fringe orders for the temporal phase unwrapping. The scheme first makes use of the low-rankness property of the fringe order map and sparse nature of the impulsive fringe order errors to more effectively remove the impulsive errors by applying robust principal component analysis (RPCA) algorithm. Then the smoothness of the two-dimensional unwrapped phase map is examined and the residual fringe order errors are detected based on a discontinuity measure of the phase map and corrected by comparing phase difference between two adjacent pixels in the unwrapped phase. The effectiveness of the proposed method is demonstrated via numerical experiments.
The laser diode (LD) is modulated by the injection current of triangular waveform and a photo diode (PD) is packaged in the rear of the LD. The laser reflected by the target re-enters the cavity of LD and contains the target’s displacement information. The information is carried within the laser intensity and can be pickup by the PD. We call this laser intensity as self-mixing interferometry (SMI) signal. While processing the sensing SMI signals, we should carefully determine the windowing function and reduce the effect of windowing in the FFT and IFFT process while applying the mathematical model. Simulation results show that the proposed design is able to accomplish the measurement of micro-displacement with high resolution and great accuracy.
When a part of light emitted by a laser is back-reflected or back-scattered from an external target and re-enters the laser cavity, both the laser intensity and its wavelength can be modulated. This is so-called self-mixing effect (SME), the optical feedback interferometry (OFI) utilizes such effect in an LD developed various applications. In this paper, we use a dualcavity OFI system that operating in period one state, the laser intensity from this system exhibits an oscillation with its amplitude modulated by a traditional single cavity OFI signal. The dual-cavity OFI system has the same measurement resolution as the single cavity which is half laser wavelength. This paper developed a method to improve the resolution by using fringe subdivision. Our simulation result shows that this method can achieve subnanometer resolution.
KEYWORDS: Fringe analysis, Speckle pattern, Speckle, Principal component analysis, 3D metrology, Composites, Error analysis, Shape analysis, Superposition, Signal to noise ratio
Phase unwrapping is one of the key steps for fringe projection profilometry (FPP)-based 3D shape measurements. Conventional spatial phase unwrapping schemes are sensitive to noise and discontinuities, which may suffer from low accuracies. Temporal phase unwrapping is able to improve the reliability but often requires the acquisition of additional patterns, increasing the measurement time or hardware costs. This paper introduces a novel phase unwrapping scheme that utilizes composite patterns consisting of the superposition of standard sinusoidal patterns and randomly generated speckles. The low-rankness of the deformed sinusoidal patterns is studied. This is exploited together with the sparse nature of the speckle patterns and a robust principal component analysis (RPCA) framework is then deployed to separate the deformed fringe and speckle patterns. The cleaned fringe patterns are used for generating the wrapped phase maps using the standard procedures of phase shift profilometry (PSP) or Fourier Transform profilometry (FTP). Phase unwrapping is then achieved by matching the deformed speckle patterns that encode the phase order information. In order to correct the impulsive fringe order errors, a recently proposed postprocessing step is integrated into the proposed scheme to refine the phase unwrapping results. The analysis and simulation results demonstrate that the proposed scheme can improve the accuracy of FPP-based 3D shape measurements by effectively separating the fringe and speckle patterns.
Fringe projection profilometry (FPP) has attracted considerable interests for addressing the challenge of measuring three-dimension (3D) shapes of moving objects. Compared with phase shift profilometry (PSP) which requires the capture of multiple fringe patterns and is thus only suitable for static objects, Fourier transform profilometry (FTP) is less sensitive to motion-induced errors. However, FTP is prone to the influence of background lights and variations of the surface reflectivity, which may result in less accurate measurements. There are studies aimed to reduce the measurement errors with FTP using more sophisticated processing of the fringe patterns. However, existing works focus on schemes based on single images and the correlation of the dynamic 3D shapes is largely unexplored. In this work, we present a new method that refines FTP-based dynamic shape measurements. Assuming 3D rigid movements of the targets, we propose to utilize knowledge of the motion parameters and combine the multiple height maps obtained from several FTP measurements after compensating the motion effect. Approaches for automatically combining the height information are studied. It is observed that the measurement accuracy can be improved using the proposed method and the influence due to ambient lights and reflectivity variations can be suppressed. Computer simulations are performed to verify the effectiveness of the proposed method. The proposed method can also be integrated into other FPP systems to improve the performance for dynamic object measurements.
KEYWORDS: Image fusion, 3D metrology, 3D image processing, Calibration, Cameras, Clouds, High dynamic range imaging, Reflection, 3D image reconstruction, Image information entropy
To deal with the three-dimensional (3-D) point cloud loss caused by object reflection in the active fringe projection 3-D measurement, an active reflection suppression method for 3-D measurement is proposed. The method employs high-dynamic range images obtained by multiple exposure image fusion and a three-wavelength phase-shift profilometry method to achieve high-precision 3-D measurement of reflective objects. Experimental results show that, compared to traditional 3-D measurement methods, the proposed one can more effectively handle reflections thereby avoiding 3-D point cloud loss in the measurement of reflective objects.
When a fraction of external optical feedback re-enters inside cavity of a laser diode (LD), the laser intensity and its wavelength will thus be altered. The LD in this case is often called as a self-mixing laser diode (SMLD). This paper presents an SMLD for profile measurement. The LD is modulated by the injection current in triangular waveform and a target to be measured is installed on a mechanic scanning device. The reflection light by the target contains its surface profile. The profile information is then carried in the laser intensity and can be pickup by a photodiode packaged in the rear of the LD. We call this modulated laser intensity as self-mixing interferometric (SMI) signal. In this paper, a new algorithm is developed to retrieve the profile from the SMI signal. Results show that the proposed design is able to achieve the measurement of profile with high resolution.
Self-mixing interferometry (SMI) is a well-developed sensing technology. An SMI system can be described using a model derived from the well-known Lang and Kobayashi equations by setting the system operating in stable region. The features of an SMI signal are determined by the external optical feedback factor (denoted by C). Our recent work shows that when the factor C increases to a certain value, e.g. in moderate feedback regime with 1<C<4.6, the SMI system might enter unstable region and the existing SMI model is invalid. In this case, the SMI signals exhibit some novel features and contain higher-frequency components. To detect an SMI signal without distortion or take suitable correction on the SMI signal, it is must to make an analysis on the system bandwidth and its influence on SMI signals. The results in this paper provide useful guidance for developing an SMI sensing system.
Material parameters such as Young’s modulus and internal friction are important for estimation of material performance. This paper presents an experimental study for measuring material related parameters using a selfmixing interferometric (SMI) configuration. An SMI system consists of a laser diode (LD), a lens and an external target to be measured. When a part of the lasing light back-reflected or back-scattered by the external target re-enters the LD internal cavity, both optical frequency and intensity of the lasing light can be modulated. This modulated laser intensity is often referred as SMI signal. Generally, the target related movement or its surface information can be retrieved from this SMI signal. In this paper, an SMI system is implemented. A tested sample is used as the target to form the external cavity of the LD. The tested sample is stimulated in vibration. Continuous wavelet transform (CWT) is utilized to retrieve the vibration information of the tested sample from an SMI signal. We are able to obtain both Young’s modulus and internal friction from a piece of an experimental SMI signal. This work provides a novel, simple non-destructive solution for simultaneous measurement of Young’s modulus and internal friction.
KEYWORDS: 3D metrology, Calibration, Cameras, Clouds, 3D imaging standards, 3D modeling, 3D acquisition, Image registration, Optical engineering, Phase shift keying
A full-view three-dimensional (3-D) measurement method for complex surfaces is proposed, where 3-D data for standard balls with different angles are used to calibrate the rotation axes of a turntable and obtain transformation matrices of 3-D data of adjacent views. It can achieve accurate registration of 3-D data of views with different angles and obtains full-view 3-D data for complex surfaces in conjunction with the method for principal point calibration of cameras and modified triple-frequency six-step phase-shifting phase demodulation methods. Experiments show that the developed system based on the proposed method can achieve automatic registration of 3-D data of views with different angles, and good full-view 3-D measurement precision for complex surfaces.
An integrated method is proposed for the real-time measurement of filament lamp dimension based on machine vision (FLDMV). First, an online detection platform is built, and the image is acquired by telecentric lenses and charge-coupled diode (CCD). Second, a series of image processing, including filter, edge extraction, ellipse fitting, recursive minimum bounding rectangle, and curvature restrict estimation. Finally, the actual size of lamp is obtained by system calibration. The experimental analysis and comparison show that the maximum measurement error of this method is 0.21mm, which meets the requirements of filament lamp dimension measurement. The curvature restrict estimation based on ellipse fitting are proposed to guarantee the accuracy and real time. Compared with the traditional measurement method, our method has the advantages of fast measurement speed, high accuracy, and real time. It also can be widely used in other parts of the measurement.
Transient oscillations occur in a self-mixing interferometry (SMI) due to moderate or strong feedback. Such transient dynamics appear in the form of impulsive noises and degrade the sensing performance of an SMI-based sensing system. However, due to the complexity of the dynamics of the system, it is difficult to eliminate such impulse-like transients using conventional filtering techniques. A signal processing technique based on myriad filter is proposed to eliminate the transients. By numerically investigating the influence of the parameters of myriad filter on the transients, an adaptive algorithm is developed to ensure that the parameters are suitable for effectively eliminating the transients. The proposed technique is verified by experiments, and the results show that the transients can be effectively removed, leading to improved sensing performance for the SMI-based sensing system.
The existing Gaussian Mixture Model(GMM) which is widely used in vehicle detection suffers inefficiency in detecting foreground image during the model phase, because it needs quite a long time to blend the shadows in the background. In order to overcome this problem, an improved method is proposed in this paper. First of all, each frame is divided into several areas(A, B, C and D), Where area A, B, C and D are decided by the frequency and the scale of the vehicle access. For each area, different new learning rate including weight, mean and variance is applied to accelerate the elimination of shadows. At the same time, the measure of adaptive change for Gaussian distribution is taken to decrease the total number of distributions and save memory space effectively. With this method, different threshold value and different number of Gaussian distribution are adopted for different areas. The results show that the speed of learning and the accuracy of the model using our proposed algorithm surpass the traditional GMM. Probably to the 50th frame, interference with the vehicle has been eliminated basically, and the model number only 35% to 43% of the standard, the processing speed for every frame approximately has a 20% increase than the standard. The proposed algorithm has good performance in terms of elimination of shadow and processing speed for vehicle detection, it can promote the development of intelligent transportation, which is very meaningful to the other Background modeling methods.
Impulsive noise is a major problem that seriously degrades the performance of self-mixing interferometry (SMI). A new method to rectify this issue is proposed. First, an outlier detection approach is employed to detect the data samples corrupted by the impulsive noise, and then the SMI signal waveform is rectified by means of least square (LS) curve fitting. The results show that the proposed method can effectively remove the impulsive noise without introducing distortion to the original waveform and thus lead to improvement in the performance of an SMI system.
Self-mixing interferometry (SMI) is considered both efficient and accurate for alpha factor measurement. In this work, a
high-performance filtering method and effective data processing algorithms are combined to optimise the measurement
accuracy on the frequency-domain based alpha measurement method. In order to achieve fast real-time measurement,
FPGA (Field Programmable Gate Arrays) is employed for the implementation of the proposed algorithms. The FPGA
design includes noise reduction, SMI signal normalisation, phase signal detection, spectrum calculation and alpha
estimation. The results show that the FPGA based design can achieve fast and reliable alpha factor measurement.
Fringe projection profilometry (FPP) has been widely used for three dimensional (3D) imaging and measurement. The fringe acquisition of FPP mainly depends on the diffuse light from the surface of objects, thus the characteristics of object surface have significant influence on phase calculation. One of the essential factors related to phase precision is modulation index, which has a direct relationship with the surface reflectivity. This paper presents a comparative study which focuses on the modulation index of different materials. The distribution of modulation index for different samples is statistical analyzed, which leads to the conclusion that the modulation index is determined by the diffuse reflectivity rather than the type of materials. This work is helpful to the development of effective de-noising algorithms to improve the measurement accuracy.
An approach is presented to solve the problem of spatial shift wrapping associated with spatial shift estimation-based fringe pattern profilometry (FPP). This problem arises as the result of fringe reuses (that is, use of fringes with periodic light intensity variance), and the spatial shift can only be identified without ambiguity within the range of a fringe width. It is demonstrated that the problem is similar to the phase unwrapping problem associated with the phase-detection-based FPP, and the proposed method is inspired by the existing ideas of using multiple images with different wavelengths proposed for phase unwrapping. The effectiveness of the proposed method is verified by comparing experimental results against several objects, with the last object consisting of more complex surface features. We conclude by showing that our method is successful in reconstructing the fine details of the more complex object.
In this paper, we present a new approach for the 3D measurement using digital fringe projection. Instead of sinusoidal
fringe patterns and the traditional phase shift detection, the proposed technique makes use of triangular patterns and the
spatial shift estimation for extract the 3D shape. The proposed technique is advantageous not only by improved
immunization to nonlinear distortion associated with digital projections, but also reduced computational burden for its
implementation. Theoretical analysis and experimental results are also presented to confirm the effectiveness of the
proposed technique.
Fringe pattern profilometry using triangular patterns and intensity ratios is a robust and computationally efficient method in three-dimensional shape measurement technique. However, similar to other multiple-shot techniques, the object must be kept static during the process of measurement, which is a challenging requirement for the case of fast-moving objects. Errors will be introduced if the traditional multiple-shot techniques are used directly in the measurement of a moving object. A new method is proposed to address this issue. First, the movement of the object is measured in real time and described by the rotation matrix and translation vector. Then, the expressions are derived for the fringe patterns under the influence of the two-dimensional movement of the object, based on which the normalized fringe patterns from the object without movement are estimated. Finally, the object is reconstructed using the existing intensity ratio algorithm incorporating the fringe patterns estimated, leading to improved measurement accuracy. The performance of the proposed method is verified by experiments.
In this paper, a new approach is presented for solving the problem of spatial shift wrapping associated with Spatial Shift Estimation (SSE)-based Fringe Pattern Profilometry (FPP). The problem arises as the result of fringe reuse (that is, fringes periodic light intensity variance), and the spatial shift can only be identified without ambiguity with the range of a fringe width. It is demonstrated that the problem is similar to the phase unwrapping problem associated with the phase detection based FPP, and the proposed method is inspired by the existing ideas of using multiple images with different wavelengths proposed for phase unwrapping. The effectiveness of the proposed method is verified by experimental results on an object with complex surface shape.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.