We address two critical issues in the design of a finger multibiometric system, i.e., fusion strategy and template security. First, three fusion strategies (feature-level, score-level, and decision-level fusions) with the corresponding template protection technique are proposed as the finger multibiometric cryptosystems to protect multiple finger biometric templates of fingerprint, finger vein, finger knuckle print, and finger shape modalities. Second, we theoretically analyze different fusion strategies for finger multibiometric cryptosystems with respect to their impact on security and recognition accuracy. Finally, the performance of finger multibiometric cryptosystems at different fusion levels is investigated on a merged finger multimodal biometric database. The comparative results suggest that the proposed finger multibiometric cryptosystem at feature-level fusion outperforms other approaches in terms of verification performance and template security.

Super resolution is a technique that obtains high-resolution (HR) images from the corresponding low-resolution (LR) ones and moreover, makes these HR images as natural as possible. There are usually some artifacts of these HR images caused by the lack of the information of the detailed textures (high-frequency information). In our research, an algorithm based on weighted vector quantization is proposed to predict the high-frequency information. First of all, the classified vector quantization is adopted to establish a pair of codebooks of high-frequency information for LR and HR patches. Second, three code vectors are selected from the LR codebooks based on the correlation coefficients to establish a multilinear regression model with the input LR patch and meanwhile the weights are decided adaptively. Then, three corresponding HR code vectors are used with the weights to reconstruct the high-frequency information of the HR patch. Finally, the reconstructed high-frequency information is used to refine the preliminarily upscaled HR image. The experimental results show that the performance of the proposed algorithm is good not only in objective but also in subjective measurement.

We propose the integration of color and texture cues as an improvement of a rough set–based segmentation approach, previously implemented using only color features. Whereas other methods ignore the information of neighboring pixels, the rough set–based approximations associate pixels locally. Additionally, our method takes into account pixel similarity in both color and texture features. Moreover, our approach does not require cluster initialization because the number of segments is determined automatically. The color cues correspond to the a and b channels of the CIELab color space. The texture features are computed using a standard deviation map. Experiments show that the synergistic integration of features in this framework results in better segmentation outcomes, in comparison with those obtained by other related and state-of-the-art methods.

We proposed a technique to detect the global addition of noise to a digital image. As an anti-forensics tool, noise addition is typically used to disguise the visual traces of image tampering or to remove the statistical artifacts left behind by other operations. As such, the blind detection of noise addition has become imperative as well as beneficial to authenticate the image content and recover the image processing history, which is the goal of general forensics techniques. Specifically, the special image blocks, including constant and strip ones, are used to construct the features for identifying noise addition manipulation. The influence of noising on blockwise pixel value distribution is formulated and analyzed formally. The methodology of detectability recognition followed by binary decision is proposed to ensure the applicability and reliability of noising detection. Extensive experimental results demonstrate the efficacy of our proposed noising detector.

A new registration-based scene-based nonuniformity correction (SBNUC) technique called the feedback-integrated scene cancellation (FiSC) method is introduced, which demonstrates an ability to correct both high- and low-spatial frequency nonuniformity (NU) in infrared focal plane arrays. The theory of scene-cancellation is further developed to include a referencing mechanism that allows spatially correlated NU to be corrected, and a practical method of application is developed. The algorithm is suitable for implementation in a real-time processing environment such as a digital signal processor. A new metric called normalized root mean-squared error for quantifying SBNUC performance is introduced and applied. When applied to real data from a cooled HgTeCd focal plane, the FiSC algorithm outperforms other SBNUC algorithms considered when provided with accurate frame-to-frame image registration. An SBNUC simulation is described and applied to several SBNUC algorithms. When the most realistic case including both high- and low-spatial frequency NU is simulated, the FiSC algorithm outperforms all others tested.

The main contribution of this article is to evaluate the utility of different state-of-the-art deformable contour models for segmenting carotid lumen walls from computed tomography angiography images. We have also proposed and tested a new tracking-based lumen segmentation method based on our evaluation results. The deformable contour algorithm (snake) is used to detect the outer wall of the vessel. We have examined four different snakes: with a balloon, distance, and a gradient vector flow force and the method of active contours without edges. The algorithms were evaluated on a set of 32 artery lumens—16 from the common carotid artery (CCA)-the internal carotid artery section and 16 from the CCA-the external carotid artery section—in order to find the optimum deformable contour model for this task. Later, we evaluated different values of energy terms in the method of active contours without edges, which turned out to be the best for our dataset, in order to find the optimal values for this particular segmentation task. The choice of particular weights in the energy term was evaluated statistically. The final Dice’s coefficient at the level of 0.939±0.049 puts our algorithm among the best state-of-the-art methods for these solutions.

The multiscale morphological approaches for segmenting directional structures are proposed. First, the use of the composition of connections to extract the directional structures of the image is investigated. We show that even though the composition of connectivities enables the correct determination of the main directional structures, the requirement of the scales for segmenting the image makes this algorithm more or less complex to apply. Then, a morphological image segmentation approach is proposed based on the concept of connectivity in a viscous lattice sense. Two functions are computed to characterize the directional structures: viscosity and orientation. The viscosity function codifies the different scales of the structure and is computed from the supremum of directional erosions. This function contains the sizes of the longest lines that can be included in the structure. To determine the directions of the line segments, the orientation function is employed. By combining both images—viscosity and orientation functions— an orientation partition function is created. This last function contains information of the maxima of the viscosity function and their orientation. Finally, the elements of the orientation partition function are merged according to some criteria, using a histogram-based segmentation approach to compute an optimal partition.

Underwater robots equipped with a single forward-looking camera usually have a very limited visual range or field-of-view (FOV) due to the light absorption and scattering effects in the underwater environment, which greatly limit their applications for underwater video-based inspection, navigation, and so on. Although underwater robots using multicamera imaging systems can achieve wide FOV of surroundings, parallax distortion and time-consuming stitching computation are encountered, especially for short-distance observation. To overcome these problems, we present a fast multicamera video-stitching algorithm based on adaptive adjustment of image transformation matrix between adjacent images. The proposed method uses a fast and adaptive optimization algorithm to search the optimal parameters of transformation matrix that can minimize the parallax distortion due to short-distance imaging and maximize the matching degree between adjacent overlapping image areas. The advantage of the proposed stitching method lies in that it avoids the complex and time-consuming computations for feature-point extraction and matching. The experimental results show that the proposed method can construct multicamera-based wide FOV video effectively and meets the real-time requirement of wide FOV video observation for both indoor and underwater scenes.

The depth of field (DoF) effect is a useful tool in photography and cinematography because of its aesthetic value. However, capturing and displaying dynamic DoF effect were until recently a quality unique to expensive and bulky movie cameras. A computational approach to generate realistic DoF effects for mobile devices such as tablets is proposed. We first calibrate the rear-facing stereo cameras and rectify the stereo image pairs through FCam API, then generate a low-res disparity map using graph cuts stereo matching and subsequently upsample it via joint bilateral upsampling. Next, we generate a synthetic light field by warping the raw color image to nearby viewpoints, according to the corresponding values in the upsampled high-resolution disparity map. Finally, we render dynamic DoF effect on the tablet screen with light field rendering. The user can easily capture and generate desired DoF effects with arbitrary aperture sizes or focal depths using the tablet only, with no additional hardware or software required. The system has been examined in a variety of environments with satisfactory results, according to the subjective evaluation tests.

We present a method that can handle the color correction of multiple photographs with blind image restoration simultaneously and automatically. We prove that the local colors of a set of images of the same scene exhibit the low-rank property locally both before and after a color-correction operation. This property allows us to correct all kinds of errors in an image under a low-rank matrix model without particular priors or assumptions. The possible errors may be caused by changes of viewpoint, large illumination variations, gross pixel corruptions, partial occlusions, etc. Furthermore, a new iterative soft-segmentation method is proposed for local color transfer using color influence maps. Due to the fact that the correct color information and the spatial information of images can be recovered using the low-rank model, more precise color correction and many other image-restoration tasks—including image denoising, image deblurring, and gray-scale image colorizing—can be performed simultaneously. Experiments have verified that our method can achieve consistent and promising results on uncontrolled real photographs acquired from the Internet and that it outperforms current state-of-the-art methods.

An effective illuminant-estimation approach for color constancy is proposed. Bright and near-neutral pixels are selected to jointly represent the illuminant color and utilized for illuminant estimation. To assess the representing capability of pixels, bright-neutral strength (BNS) is proposed by combining pixel chroma and brightness. Accordingly, a certain percentage of pixels with the largest BNS is selected to be the representative set. For every input image, a proper percentage value is determined via an iterative strategy by seeking the optimal color-corrected image. To compare various color-corrected images of an input image, image color-cast degree (ICCD) is devised using means and standard deviations of RGB channels. Experimental evaluation on standard real-world datasets validates the effectiveness of the proposed approach.

Recently, structure-preserved projections (SPP) were proposed as a local matching-based algorithm for face recognition. Compared with other methods, the main advantage of SPP is that it can preserve the configural structure of subpatterns in each face image. However, the SPP algorithm ignores the information among samples from different classes, which may weaken its recognition performances. Moreover, the relationships of nearby pixels in the subpattern are also neglected in SPP. In order to address these limitations, a new algorithm termed spatially smoothed discriminant structure-preserved projections (SS-DSPP) is proposed. SS-DSPP takes advantage of the class information to characterize the discrimination structure of subpatterns from different classes, and a new spatially smooth constraint is also derived to preserve the intrinsic two-dimensional structure of each subpattern. The feasibility and effectiveness of the proposed algorithm are evaluated on four standard face databases (Yale, extended YaleB, CMU PIE, and AR). Experimental results demonstrate that our SS-DSPP outperforms the original SPP and several state-of-the-art algorithms.

A multimodal image registration framework based on searching the best matched keypoints and the incorporation of global information is proposed. It comprises two key elements: keypoint detection and an iterative process. Keypoints are detected from both the reference and test images. For each test keypoint, a number of reference keypoints are chosen as mapping candidates. A triplet of keypoint mappings determine an affine transformation that is evaluated using a similarity metric between the reference image and the transformed test image by the determined transformation. An iterative process is conducted on triplets of keypoint mappings, keeping track of the best matched reference keypoint. Random sample consensus and mutual information are applied to eliminate outlier keypoint mappings. The similarity metric is defined to be the number of overlapped edge pixels over the entire images, allowing for global information to be incorporated in the evaluation of triplets of mappings. The performance of the framework is investigated with keypoints extracted by scale invariant feature transform and partial intensity invariant feature descriptor. Experimental results show that the proposed framework can provide more accurate registration than existing methods.

Radar coincidence imaging (RCI) is a new instantaneous imaging technique that does not depend on Doppler frequency for resolution. Such an imaging method does not require target relative motion and has an imaging interval that is even shorter than a pulse width. The potential advantages in processing both the relatively stationary and maneuvering targets make RCI provide a supplementary imaging approach for the conventional range Doppler imaging methods. The simulation experiments have preliminarily demonstrated the feasibility of the RCI technique. However, further investigations show that the imaging error arises for moving targets, and moreover, it is particularly related to target scattering maps. The paper analyzes the target-motion-induced error and points out that three factors are involved: target velocity, target scattering map, and the time-space independence of detecting signals. The current image-reconstruction algorithms of RCI, which are based on the least-square (LS) principle, are found to be seriously sensitive to the motion-induced errors and will be limited in practical imaging scenarios. Accordingly, the compressive sensing (CS) recovery algorithm is employed, which can utilize sparsity restriction to diminish the effect of the motion-induced error on image reconstruction. Simulations are designed to illustrate the three factors of the target-motion-induced error. The imaging performance of the LS and the CS methods in RCI image recovery are compared as well.

A large dataset of Persian license plate characters is introduced. These extracted characters are provided by Bani Nick Pardazesh Company, which is a pioneer in the intelligent transportation systems field, and its license plate recognition system has been used for many applications in Iran. Natural scene vehicle images delivered from this company were in various conditions. Most of them were taken with visible light during day and a few of them by infrared light with 850 nm wavelength during night. The vehicle images were achieved by color, black and white, or infrared cameras from front view and back view of automobiles in ∼20 different indoor and outdoor locations, such as streets, roads, and parking lots, for different purposes, such as traffic control, issuing fines, etc. The images are different in size and angle and were taken in light and dark backgrounds, where the direction and intensity of the light varied. Also, some of the license plates were muddy and had parts that were shadowed. Out of all the available images, ∼20,145 Persian characters were extracted by an intelligent system and verified by human observers. The extracted images are different in size, and some of them suffer from elimination, distortion, rotation, and noise.

An approach of rapid hologram generation for the realistic three-dimensional (3-D) image reconstruction based on the angular tiling concept is proposed, using a new graphic rendering approach integrated with a previously developed layer-based method for hologram calculation. A 3-D object is simplified as layered cross-sectional images perpendicular to a chosen viewing direction, and our graphics rendering approach allows the incorporation of clear depth cues, occlusion, and shading in the generated holograms for angular tiling. The combination of these techniques together with parallel computing reduces the computation time of a single-view hologram for a 3-D image of extended graphics array resolution to 176 ms using a single consumer graphics processing unit card.

Human activity recognition based on RGB-D data has received more attention in recent years. We propose a spatiotemporal feature named three-dimensional (3D) sparse motion scale-invariant feature transform (SIFT) from RGB-D data for activity recognition. First, we build pyramids as scale space for each RGB and depth frame, and then use Shi-Tomasi corner detector and sparse optical flow to quickly detect and track robust keypoints around the motion pattern in the scale space. Subsequently, local patches around keypoints, which are extracted from RGB-D data, are used to build 3D gradient and motion spaces. Then SIFT-like descriptors are calculated on both 3D spaces, respectively. The proposed feature is invariant to scale, transition, and partial occlusions. More importantly, the running time of the proposed feature is fast so that it is well-suited for real-time applications. We have evaluated the proposed feature under a bag of words model on three public RGB-D datasets: one-shot learning Chalearn Gesture Dataset, Cornell Activity Dataset-60, and MSR Daily Activity 3D dataset. Experimental results show that the proposed feature outperforms other spatiotemporal features and are comparative to other state-of-the-art approaches, even though there is only one training sample for each class.

The cosegmentation problem is referred to as segmenting the same or similar objects simultaneously from a group of images. However, designing a robust and efficient cosegmentation algorithm is a challenging work because of the variety and complexity of the object and the background. We proposed a new seeded image cosegmentation method based on a local spectral method, which combines bottom-up information and seeds’ knowledge effectively for segmentation. Multiple images are connected into a weighted undirected graph so the cosegmentation problem is converted into a graph partitioning problem that is solved by biased normalized cuts. The results of the cosegmentation experiment reveal that the proposed method performs well even in the presence of some noise images (images not containing a common object) or in the condition of the image set containing more than one object.

We propose a method for solving the depth map super-resolution problem. Given a low-resolution depth map as input, we recover a high-resolution depth map using a registered and potentially high-resolution color image. We formulate the super-resolution process as a regularization and optimization framework. Specifically, taking into account that discontinuities in depth and color tend to coalign, we obtain the local and nonlocal regularization terms based on the raw depth map and also the features of high-resolution color images. Different from conventional methods, we model the relationship with two constraint terms of local and nonlocal priors to sufficiently explore the complementary nature. Experimental results demonstrate the effectiveness of our approach, which produces sharper edges and more faithful details compared with other state-of-the-art strategies.

We propose an algorithm to recover the latent image from the blurred and compressed input. In recent years, although many image deblurring algorithms have been proposed, most of the previous methods do not consider the compression effect in blurry images. Actually, it is unavoidable in practice that most of the real-world images are compressed. This compression will introduce a typical kind of noise, blocking artifacts, which do not meet the Gaussian distribution assumed in most existing algorithms. Without properly handling this non-Gaussian noise, the recovered image will suffer severe artifacts. Inspired by the statistic property of compression error, we model the non-Gaussian noise as hyper-Laplacian distribution. Based on this model, an efficient nonblind image deblurring algorithm based on variable splitting technique is proposed to solve the resulting nonconvex minimization problem. Finally, we also address an effective blind image deblurring algorithm which can deal with the compressed and blurred images efficiently. Extensive experiments compared with state-of-the-art nonblind and blind deblurring methods demonstrate the effectiveness of the proposed method.

Fusion of images from different imaging modalities, obtained by conventional fusion methods, may cause artifacts, including destructive superposition and brightness irregularities, in certain cases. This paper proposes two methods for improving image multimodal fusion quality. Based on the finding that a better fusion can be achieved when the images have a more positive correlation, the first method is a decision algorithm that runs at the preprocessing fusion stage and determines whether a complementary gray level of one of the input images should be used instead of the original one. The second method is suitable for multiresolution fusion, and it suggests choosing only one image from the lowest-frequency sub-bands in the pyramids, instead of combining values from both sub-bands. Experimental results indicate that the proposed fusion enhancement can reduce fusion artifacts. Quantitative fusion quality measures that support this conclusion are shown.

We propose a new watermarking algorithm for stereoscopic image tamper detection and self-recovery in three-dimensional multimedia services. Initially, left and right views of stereoscopic image are divided into nonoverlapping 2×2 blocks in order to improve the accuracy of tamper localization in an image. As the left and right views of a stereoscopic image are not independent from each other but have an inter-view relationship, every block of a stereoscopic image is classified into matching block or nonmatching block and then block disparities are obtained. Both matching blocks in the left and right views have similar pixel values, so that fewer bits are allocated for recovery watermark generation, which can increase the quality of watermarked stereoscopic images. A hierarchical tamper-detection strategy with a four-level checkup is presented to improve the accuracy of tamper localization. Additionally, two copies of block (matching block and nonmatching block) information are embedded into the stereoscopic image, and it assures the quality of tampered recovery. For the nonmatching block recovery, two copies of the partner block are embedded into their chaotic mapping blocks, which supply the second chance for tamper recovery. For the matching block recovery, the inter-view relationship between tampers of left and right views supplies the third chance for tamper recovery. Experimental results show that the proposed algorithm can not only detect and locate tampers in stereoscopic image more accurately but also recover the tampered regions better, compared with other algorithms.

We show that the so-called circularity measures based on radial moments, are a particular case of the family of circularity measures introduced recently by the authors of this note.

Color science is a multidisciplinary field with broad applications in industries such as digital imaging, coatings and textiles, food, lighting, archiving, art, and fashion. Accurate definition and measurement of color appearance is a challenging task that directly affects color reproduction in such applications. Color Appearance Models addresses those challenges and offers insight into the preferred solutions. Extensive research on the human visual system (HVS) and color vision has been performed in the last century, and this book contains a good overview of the most important and relevant literature regarding color appearance models.