1.

Introduction

Multiple video cameras of different spectral bands are widely used in the advanced vision systems to capture more information in biomedical imaging, remote sensing and other imaging applications. In many cases, images from multiple sensors need to be integrated into a single synthetic representation for human observers, who are in general under time constraints, stress and workload for interpretation, detection and decision making. Image registration is a key step before fusion. Image registration is also needed for interpreting time evolution of the images in remote sensing and medial diagnostics.

Image registration applied to a single video sequence is image video sequence stabilization. The primary means available for image stabilization is an electromechanical stabilizing platform, which is bulky and expensive. Its performance is degraded with vibration in the critical 0 - 20 Hz range. An automatic electronic image stabilizer should be able to first estimate components of the scene motion due to the camera movement and then eliminate those components by warping each frame into precise alignment with the next frame. Then, if a temporal filter is applied to compute the frame difference, then the background scene would be eliminated and the moving targets would be highlighted.

Multiple sensors can act in a synergistic manner. For instance, in two broad band visible/infrared battle field image sequences, soldiers and trucks can be hidden behind the smoke in the visible image, but they appear clearly as high contrast hot objects in the IR images. However, the contrast is extremely low in background of the IR images, so that one can not locate the hot objects within the background, and one needs to fuse the IR images with the corresponding visible images.

Challenge in the multiple sensor image registration is to align two images in spite of feature inconsistency. Some features in one image can do not show up in the image from another sensor. This is the feature inconsistency problem. Multiple sensors capture distinguished signatures from the input scene in different spectral bands. Multisensor imaging system should maximize independence of the acquired data. This is natural, since if one sensor captures images that are similar or correlated to the images already obtained by other sensors, then this sensor provides no additional information and should be removed from the system. The signature independence implies the features inconsistency.

Image registration and fusion are implemented in the practical image sensing systems. These are a potential application area of optical image processing. Optical correlators at the operating rate of 1000 correlations per second have been now built. Automatic target recognition and tracking using the optical correlators and optimally designed correlation filters have been demonstrated. However, many practical systems should accomplish complex tasks using a sequence of image processing algorithms such as detection, recognition, identification and tracking, which can be entirely automatic or assisted with human operator. In general, the tasks of practical and complex image processing systems can not be accomplished with a single short of correlation with an optical correlator. Thus, the high-speed optical correlator must be applied and integrated into the multi-step effective numerical image processing systems.

In this paper we present problematic of image registration, review the existing methods and show the algorithms that we developed to accomplish IR/visible battle field image registration. We show that the correlation is one of the basic operations very useful in many practical image processing systems, and the invariance problem is still the major issue associated in the correlation. Therefore, the algorithms developed for invariant optical pattern recognition filters could be useful in those systems. We choose the feature-based approach. We use multi-scale hierarchical edge detection, edge focusing and edge salience measure to extract salient edges from the low contrast and noisy IR image background. We use the Hausdorff distance measure for matching between the curves from two different modalities. We introduce the image partitioning technique in the Hausdorff distance matching, so that the global affine transformation is approximated by local translations. This approach speeds up significantly the computation of the Hausdorff distance.

2.

3-D projection

It is well known in the computer vision that 3-D space information may be obtained from two or more 2-D projection images of the same scene. When the camera is modeled as a pinhole camera, a point M in the 3-D space is projected into m₁ and m₂ in the two image planes, respectively, as shown in Fig.1, where c₁ and c₂ are the camera centers. One camera is rotated and translated in the 3-D space with respect to another. From the image point m₁ and camera center c₁ we know that the object point M should be on the projection line m₁c₁, but its position on m₁c₁ is undetermined. If the image point m₂ of the same point M in image 2 is known, then the position of the point M in the 3-D space can be uniquely determined¹. When M is move on the line m₁c₁ its image point m₂ would move along an epipolar line l_m2, in image 2. Reciprocally, when M moves along the projection line m₂c₂, its onto image 1 would move along an epipolar line l_m1 in image 1. The epipolar lines l_m1 and l_m2 are the intersections between the plane Π formed by the object point M and two camera centers c₁ and c₂ and the image plans I₁ and I₂. The key is to establish correspondence between points m₁ and m₂ in images 1 and 2, such that they are the images of the same object point M.

Under the pinhole camera model a 3-D scene is projected into a 2-D image through the full perspective projection. Let be the world coordinate system and be the camera geometric coordinate system, whose origin is on the center and the axis z_c is on the optical axis of the camera, then the relation between the world system and the camera system is

Fig. 1

Epipolar Geometry

where R is the rotation matrix and T is the translations vector in the 3-D space of the world system with respect to the camera system. In the camera system an object point (x_c,y_c, z_c) is projected onto the image plane at (x, y) by

where f is the focal length of the camera. The combination of Eqs. 1 and 2 describes the perspective projection from a 3-D space point (x_s,y_s,z_s) into its image point (x,y). For the sake of the simplicity we do not introduce the pixel coordinates of the camera here.

If the camera’s position and orientation in the 3-D space and the camera intrinsic parameters, such as the focal length, aspect ratio and optical center position, are known (calibrated cameras), according to that shown in Fig. 1 the object point can be reconstructed in the 3-D space from the two corresponding image points in the two images. This is a 3-D stereo vision problem. On the other hand, from a number of pairs of corresponding points, one can determine the camera rotation and translation matrices R and T. The is a motion estimation problem.

Image registration needs to determine the deformation of images due to the camera motion and view angle changes in order to realign the images. The registration function describes point (x₂,y₂) in image 2 as a function of point (x₁,y₁) in image 1, which can be derived by removing the object point M(x_s, y_s, z_s) from Eqs. (1) and (2), that results in

where and F is known as the fundamental matrix of the two images, which depends on the position, orientation and intrinsic parameters of the camera. The fundamental matrix F has 9 unknown coefficients. If we can determine 8 matches between points in Images 1 and 2 we would be able, in general, to determine a unique solution for F, defined up to a scale factor, and then the registration function.

In the pinhole camera model, the perspective projection described in Eq. 2 is non-linear. The coordinates (x,y) of the image point depend on the depth z_c of the object point. When the average distance from the camera to the center of mass of object L is much larger than the depth of the 3-D scene, z_c = L + z_s and L ≫ z_s, one can replace z_c in the right-hand side of Eq. (2) by L, and then Eqs. (1) and (2) become linear, describing an approximation to the perspective projection. In this case, the registration function can be easily obtained as^2,3

where p_j with j = 1, 2, …6 are the affine transformation coefficients relative to the 3-D rotation and translation of the camera, e is the epipolar vector and h(x,y) is the height function of the scene, which introduces displacements of the image points (x₂, y₂) along the epipolar lines and proportional to the heights in the scene. The solutions of Eq. (4) for the six unknown coefficients p_j with j = 1, 2, …6, h(x,y) and e are not unique. Any linear polynomial I(x,y) = ax + by + c can be added to h(x,y), as long as I(x,y)e is subtracted from the first term in the right-hand side of Eq. (4). The uniqueness of the solutions may be obtained if the solutions for p, h, and e satisfy a normalization constraints² that takes a planar approximation of the height function h(x,y) and subtracts the planar approximate plane from Eq. (4). Thus, one first puts h(x,y) = 0 in Eq. (4) and then looks for the least mean square solution (LMS) to estimate the affine coefficients p_j. After the determination of p_j, the epipolar vector e and the scene height function h(x,y) can be recovered from the residuals in the LMS fitting. The LMS solution for Eq. (4) needs to know a set of corresponding point pairs-i with i = I, …, k with k > 3. When more correspondences k are available, it is advantageous to use all the points to improve the accuracy of the solution and detect outliers which are false correspondences in the set.

3.

Image matching

To estimate the registration function Eqs. (3) or (4) one needs to determine a set of point matches between two images. The determination of point correspondences is the key step for 3-D stereo vision, motion analysis, image registration and model-based 3-D object recognition. When there are some characteristic points in the image, such as corners of 3-D objects or the landmarks in the remote sensing images, the point pairs are easy to determine. However, in outdoor battle field images the determination of point matches is a difficult task, especially for IR image registration, where one needs to detect and determine point matches in the static image background, where IR images have typically very low contrast due to the thermal equilibrium in the background.

3.1

Block matching

One approach is to partition the image into a number of sub-images, located in a regular grid as shown in Fig.2, and then define a central window in each sub-image as a template and correlate those block templates with their corresponding sub-images in another image⁴. These correlations can be implemented using the optical correlator.

Fig. 2

Block matching

The block matching results in a lattice of displacement vectors, which are evenly distributed over the image and are then useful to determine the transformation parameters for the registration function. However, the correlation can account only for translations. Other deformations, such as similarity or affine transform or local distortions¹ may be only approximated by the local translations of the blocks.

The area-based approach for image registration utilizes full image information and can be applied to any images with rich or poor structure. Cross-correlation-based matched filter approach is optimal for the robustness against random noise. The drawback is that this approach can account only for small translations. The method can fail if the displacement of the bloc exceeds the size of the sub-image. Moreover, the higher the number of sub-images the higher precision of that approximation, however, the smaller translations that can be accounted for, and the less image features contained in the sub-images. The bloc matching is widely used in the video image processing and compression. With the small bloc size and small translations, the cross-correlation may be computed by electronic hardware in real-time of video rate. However, when the block size is large and its translation is large such that the search area is large, the computation of cross-correlation becomes very expensive. Optical correlator, such as the joint transform correlator, can be useful for the implementation of the bloc matching.

4.

Image stabilization

Image stabilization is the registration of video sequence from a single camera. We choose the feature-based method which is more powerful to extract features in the IR image background and to account for image rotations and affine transformation⁵.

We use the Harris-Stephens comer detector. The operation is to find the local maxima of the principal curvatures of the image local autocorrelation. The Harris-Stephens comer detector is efficient to compute and has been shown to be one of the best corner detectors in terms of repeatability with scale, illumination and view point changes. It is effective to extract comer points in the texture of the IR outdoor images. Typically, this comer detector returns hundreds comer points in an outdoor image.

Points are 2-D features, which are invariant to rotation and scaling and useful for fitting image transformations. However, points themselves do not carry image structural information. The comer points detected by Harris-Stephens comers detector can be the maximum curvature point on object edges, it can also be comers from noisy and textures. To establish point match, each comer point must be characterized by its local support and the cross-correlation of the local supports can be used for establishing initial matches.

5.

Multisensor image registration

Challenge in the multiple sensor image registration is the feature inconsistency, especially, for the images from two well separated spectral bands (visible and 8–12 µm IR bands). The radiometric data from IR passive sensors consist of 1) energy emitted by thermal radiation from the object bodies; 2) atmospheric emission reflected from object surfaces. In general, the gray-scale level of IR images depend on differences in body temperature, emissivity and reflectivity of the objects in the scene. The IR images have high contrast for hot objects in the scene, which are in most cases moving targets and are therefore not landmarks useful for registration. Image registration should rely on the static objects on the background of the scene, where, unfortunately, the IR outdoor images have very low contrast because the background objects have uniform temperature in the thermal equilibrium state. The background in the outdoor IR image is usually of very low contrast and noisy, or simply a dark region. Feature extraction and image registration based on the background are difficult.

There exist significant gray-level disparities between the IR and visible image. The thermal emitters are not necessarily good visual reflectors. A surface of high visual reflectivity (white surface) in visible band usually has low emissivity, so that the bright objects in the visible image may be dark in the thermal scene and vice versa. The sky is usually the brightest region in the visible image. It is, however, a dark region in the IR image because of the low temperature and the lack of reflectance. This is the reversal of contrast polarity between the visible and IR images.

The gray level disparity between the IR and visible images of real-world natural outdoor scene is much more complex than the simple contrast polarity reversal.

Figure 3 shows a contrast reversed IR image compared with the corresponding visible image of the same scene. The gray level distributions in most regions in the two images become similar, although there are still important gray-level disparities. Clouds are brighter than the sky in the visible image, owing to its higher reflectivity. They are also brighter than the sky in the IR image because of its higher reflectivity and emissivity. As a result, in the contrast polarity reversed IR images, the clouds are darker than the sky, whereas they are brighter than the sky in the visible image. Hence, a simple reversal of contrast polarity can not remove all the contrast reversal and gray level disparities. Also, shadows in the visible images are absent in the IR images.

Fig. 3

Contrast reversed IR image (left) and visible image (right)

Fig. 4

Two step edges with opposite polarities, smoothed by Gaussian filter, their Laplacian pyramid and absolute coefficients of the Laplacian pyramid.

Because of the gray scale level disparity and contrast polarity reversal the area-based block matching methods and the feature-based methods using the cross correlation or GDI feature matching of the gray scale levels in the local area supports, as described in Sections 4, failed to determine point matches. Several approaches have been proposed to bypass the feature inconsistency problem for multiple sensor image registration.

5.1

Laplacian pyramid

In some applications, one can transform two dissimilar multisensor images into similar. The intensities of the Laplacian pyramid images are insensitive to polarity reversals of contrast in the visible and IR images⁷. Figure 5 shows two step edges with opposite polarities of contrast. The edges are smoothed by the Gaussian filter. The Laplacian pyramid coefficient is the difference between the original edge and the smoothed edge. This is the Laplacian pyramid image at a resolution level determined by the size of the Gaussian smoothing function. When we take the absolute values of the Laplacian pyramid coefficients, the two Laplacian pyramid images become the same for the two contrast reversed step edges. Then, the area-based image registration can be applied to the Laplacian pyramid image intensities⁷.

Fig. 5

Results of Hierarchical Edge Detection

The high frequency details in the image are lost in the Gaussian pyramid by the low-pass filtering and down sampling. One has to compute the difference between the averaged image and the original image in two successive pyramid levels. These difference images contain the detail information of the image. All the difference images form a new set of sequences that constitute another pyramid called the Laplacian pyramid. The original signal can be reconstructed exactly by summing the Laplacian pyramid. The Laplacian pyramid is a multiresolution representation and the Laplacian pyramid images represent detail information of the image. Because image gray-scale level disparities due to the sensor spectral responses are mostly low spatial frequencies, so that the Laplacian pyramid images contain also less gray level disparity. Hence the area-based image registration can be applied to images preprocessed by the Laplacian pyramid.

6.

Multiscale edge detection

The feature-based multisensor image registration utilizes the fact that whatever the spectral responses of multiple sensors, two real world objects in the scene would always appear still differently. The boundaries of the objects may be then used as matching entities for multi-sensor image registration. In the 3-D real world scene, objects are separated from the background by depth discontinuities, which are usually manifest as intensity discontinuities in the 2-D images. Those edges and boundaries represent structures in the image, that are common for multiple image types and can be used for multiple sensor image registration.

Edges are defined as points where the modulus of gradient is a maximum in the gradient direction. Along an edge the image intensity can be singular in one direction while varying smoothly in the perpendicular direction. Edges can be created by occlusions, shadows, sharp changes of surface orientation, changes in reflectance properties, or illumination. In IR images of a 3-D scene, most edges represent occlusions and depth discontinuities between objects in the scene, which represent structural information in the image.

6.3

Edge Focusing

Edge focusing is a coarse-to-fine edge tracking algorithm for recovering the edge points at the finest scale¹⁰. The scale-space tracking is implemented in a continuous manner. With continuous scaling, the edges are gradually focused by varying the resolution continuously, and moving in the scale space with sufficiently small steps, such that the edge element do not jump farther away than one pixel between successive steps. Our implementation of edge focusing is as following:

• 1. Detect edge using Canny Detector with the Gaussian smoothing σ₀ sufficiently large so that horizon curve is detected;

• 2. Extract the horizon using a threshold on the curve length; The horizon curve is denoted as E(i, j, σ₀). If (i, j) is an edge point, then E(i, j, σ) = 1.

• 3. Detect edges E(i, j,σ_k) in a window centered at each edge point E (i, j,σ_k-1), using the Canny edge detector of size σ_k = σ_k-1 - Δσ with k = 1,2,3… and Δσ =0.5. The window size is 7×7, when σ_k >2.0, and is 5×5 when 1.0 ≤ σ_k ≤ 2.0, and is3×3 when σ_k < 1.0.

• 4. Go on step 3) until a weak Gaussian smoothing of size σ_κ.

In the successive Canny edge detection, after application of the first derivative of Gaussian filter the non-maximum suppression process is applied which keeps only the local maximum in the gradient direction. There is no threshold at finer resolution. The only threshold is on the curve length applied at the coarsest scale σ₀.

Bergholm¹⁰ investigated the deformation of four elementary contour structures: step edge, comer, double edges and edge box. During the edge detection, those contours are generally deformed in four ways: rounding-off, expansion, transformation into circles, or merger, owing to the large Gaussian average operator which blurs the image. In each of the four cases, Bergholm showed that the displacement vector, describing the deformation of the edge contour, is normally of length within the range from 0 to 2 |Δσ|, where σ is the width of the Canny edge detector, Δσ is the increment of size of the successive Canny filters. Therefore, if |Δσ| = 0.5, the displacement of the edge points would be normally less than one pixels, so that comers and junctions may be recovered with a precision less than one pixel.

Real world images contain mostly ramp edges instead of ideal step edges. It is easy to show that the Gaussian blurring operating on a ramp edge always yields smaller displacement than that yielded on a step edge as affirmed by Bergholm. A ramp edge may be modeled as a step edge smoothed by a Gaussian G whose size σ₁ depends on the imaging condition and on the camera. Let r(x, y) denote the step edge and f (x, y) the ramp edge in gray level image, then

where ⊗ denotes the convolution. When we use Canny Edge Detector, the image is blurred again with a Gaussian smoothing whose size σ₂ depends on the scale of the edge detector. Let g(x, y) denote the blurred ramp edge before computing the first derivative, then

therefore

which is equal to

Therefore, the length of rounding-off displacement ρ from the corner of ideal step edges to the detected corner is equal to , where c is a constant. However, the displacement from the center of the ramp comer to the detected corner would be proportional to , as illustrated in Fig. 6 and would be less than σ₂. Therefore, if |Δσ₂| = 0.5 in the edge focusing, the displacement of the ramp edge corner would be less than one pixels.

Fig. 6

Rounding-off displacement for a ramp edge

In our IR images the ramp edges of trees can be very slow of more than 20 pixels wide, corresponding to a large σ₁ more than 10. The edges around the trees were cut completely when a Canny edge detector of σ₂ = 7 was applied. This is because the large displacement of the corner . However, using the edge focusing we were able to recover the edges and tops of the trees, which would be important for the image registration.

In our IR images the ramp edges of trees can be very slow of more than 20 pixels wide, corresponding to a large σ₁ more than 10. The edges around the trees were cut completely when a Canny edge detector of σ₂ = 7 was applied. This is because the large displacement of the corner . However, using the edge focusing we were able to recover the edges and tops of the trees, which would be important for the image registration.

We implement the edge focusing algorithm with the filter size increment Δσ = 0.5 and varying size windows. We chose to use the window size larger than the usually used, 3 × 3, so that the gradient magnitude values can be evaluated at the two neighboring pixels, because in the non-maximum suppression the determination of an edge pixel requires to compare with at least two neighboring pixels. We believe that the length of rounding-off displacement ρ can be larger than one pixel, because the real ramp edges in our IR images were noisy and do not follow the theoretical model described in the precedent.

For images shown in Fig.6, we first detected the coarse horizon with σ₀ = 4.5 for visible image and σ₀ = 7.0 for IR image using Canny Edge Detector. Then we applied the edge focusing with the scale step Δσ = 0.5 and the varying size windows. The final scale was σ = 0.7 for visible image and σ = 1.5 for IR image. Figure 6 shows the extracted edges which follow nicely the silhouette of the hill with some flat tops of trees recovered in both visible image and infrared image.

7.

Hausdorff distance

Given a set of salient structural edges from each image, the next step is to determine the image transformation parameters that are useful for aligning those features. The search for the optimal image transformation can be implemented in several ways. Most feature matching methods determine the correspondences between the elements of the feature sets, and then determine the transformation parameters, as the two approaches described in Section 2. Once a set of correct matches is found the fitting to transformation is, in general, quick to compute. The drawback is the prohibitive cost of detecting and eliminating false matches. Its advantage is that once a set of correct matches is found the image transformation is, in general, quick to compute.

Transformation space methods is the direct search in the transformation space to match two given sets of features, where no explicit correspondences between features are given. The approach can be prohibitively expensive because the search space is generally very large. However, outliers are easily handled by using rank order statistics. A strategy for efficiently searching the parameter space is given by Huttenlocher et al.¹¹ In view of the large proportion of outliers in feature based multi-modal image alignment a transformation space method based on the directed Hausdorff distance was implemented. The size of the search space is reduced by partitioning the image into blocks and searching for translations that minimize the Hausdorff distance between corresponding blocks. The assumptions are that the motion can be locally approximated by simple translations of blocks, and the percentage of outliers and an error bound for the feature alignment are known approximately.

We are given by two binary feature sets consisting of points and edges. There are no explicit feature correspondences. The optimal match between the two sets may be found by cross-correlation. However, the points and edges have no geometric size in mathematical sense. This correlation would very sensitive to noise in the features sets, so that instead the distances between the two feature sets may be computed. The Hausdorff distance is defined by

where A and B are point sets, H is the generalized Hausdorff distance and h is the directed Hausdorff distance. When the set B is aligned with the set A, the minimum distances between B and A result, and there is still one point in B whose distance to any point in A remains as the maximum among those minimum distances, that is defined as the directed Hausdorff distance from B to A. Inversely, when the set A is aligned with the set B, there is one point in A whose distance to any point in B (not necessarily to the point in B that gives the directed Hausdorff distance h from B to A) remains as the maximum among the minimum distances, that is defined as the directed Hausdorff distance from A to B. The generalized Hausdorff distance is the maximum among the directed and inverse Hausdorff distances.

To compute the Hausdorff distance between the image A and model B, we first build a distance transform map of A in which the value at every pixel represents the distance from that pixel to a nearest point in the set A. Then we superimpose the model point set B with its all possible distorted versions on the distance transform map of A. The minimum distances from every point in B to the point set A can be then easily sorted in a decreasing order.

In the presence of outliers the Hausdorff distance will return the greatest distance which is likely due to an outlier. To handle outliers, the partial directed Hausdorff distance is introduced as

which allows to accept (100-k)% of outliers and evaluates the k% ranked distances for determining the Hausdorff distance.

The Hausdorff distance can be prohibitively expensive because the search space is generally very large if all the possible distortions of the model must be considered. We reduce the size of the search space by partitioning the image into blocks and searching for translations that minimize the Hausdorff distance between corresponding blocks. The idea is similar to that of the block matching described in Section 3 for estimation of displacements of the block centers based on the image areas. The assumptions are that the motion can be locally approximated by simple translations of blocks, and the percentage of outliers and an error bound for the feature alignment are known approximately.

The alignment method using Hausdorff distances proceeds as follows for a pair of images after extraction of the salient edges. We found the horizons extracted by multiscale edge detection do not suffice to align two images, because the horizon features are not evenly distributed over the image. Thus, extracting structural features in the other parts of image is necessary. Figure 7 shows the edge features used for the Hausdorff distance matching

Fig. 6.

Experiment results of Edge Focusing. Right Visible image Left Infrared image.

The process steps are as following

Figure 7 shows a scene taken simultaneously by a daylight and IR camera at Defense Research Establishment Valcartier. The viewpoints of the two cameras are displaced slightly and there is a slight relative rotation about the optical axis which would yield a very poor fused image if no alignment is made. The quadtree decomposition stops at the first level, i.e., there are 4 blocks. The salient edges include the silhouette of the hill and some ground structure, which are overlayed on the images.

Fig. 7

Edges extracted in IR and Visible battle field images.

Finally, Figure 8 shows the fused aligned images. The salient edges are registered in each of 3 blocks, the fourth block contains no edge points. The Hausdorff distance is used to find the optimal displacement assuming 5 percent outliers for the blocks covering the hill edge and 10 percent outliers for the edge in the lower right block. The specified search strategy finds the translation for each block such that 90 percent of the visible image edge points are no more than 5 pixels from some IR image edge point for the corresponding block. The local displacements are then used to determine the global affine transformation to register the two images. The estimated (x,y) displacements for the blocks upper left, upper right and lower right that are supplied to the global affine estimator for aligning the visible image to the IR image are (33,-3), (-11,-7) and (-8,-11) respectively.

Fig. 8.

Aligned and fused IR and visible images. Fusion is by weighted combination of image brightness values after alignment

The estimated affine transformation parameters that map point p in the visible image to the point p’ in the IR image such that p′ = Mp+t are

Note that the image coordinate system origin is top left with positive x to the right and positive y down.

8.

Conclusion

We have analyzed the problematic in the real world visible/IR image registration with the image sequences from well separated spectral bands. After a preprocessing with the Laplacian pyramid the area-based approaches may be still applied. However, multiscale extraction of structural edges followed by feature matching using the Hausdorff distance measures are more powerful to process very poor quality IR images.

Because the common features extracted from images of two modalities can be still different in detail, the transformation space match methods with the Hausdorff distance measures were used which are more suitable than the direct feature matching methods for dealing with outliers in the extracted fature sets. We have introduced image quadtree partition technique to the Hausdorff distance matching, that dramatically reduces the size of the search space into that of the search for translations which minimize the Hausdorff distance between corresponding blocks.

We have shown image registration of visible/IR real world images of battle fields. The key point is to extract salient features from the real world images using local, regional and global information. Mulitsensor image registration and fusion is one realization of advanced computer vision systems. Multiple sensors, multiple spectra and color cameras, 3-D perspective projection image formation and time video image sequences are widely used in the advanced computer vision systems.

Cross correlation is one of the basic operations used in image registration algorithms for determining candidate point matches. The invariance to image rotation, scale and view angle changes is an important issue for feature matching. Table 1 gives a list and comparison of four algorithms described and implemented in this paper. High speed optical correlator could be useful as an efficient hardware for implementation of image registration algorithms, and the optimal correlation filter designs could enhance the performance of the image processing systems. However, most optical processing hardware have the lack of flexibility. The optical correlator should be integrated into the numerical systems, such that powerful robust algorithms can be used to processing the correlation output data and to remove the ambiguity of the correlation and thresholding outputs.

Table 1

Algorithms for image matching