Hole filling of depth maps is a core technology of the Kinect based visual system. In this paper, we
propose a hole filling algorithm for Kinect depth maps based on separately repairing of the foreground
and background. There are two-part processing in the proposed algorithm. Firstly, a fast pre-processing
to the Kinect depth map holes is performed. In this part, we fill the background holes of Kinect depth
maps with the deepest depth image which is constructed by combining the spatio-temporal information
of the pixels in Kinect depth map with the corresponding color information in the Kinect color image.
The second step is the enhancement for the pre-processing depth maps. We propose a depth
enhancement algorithm based on the joint information of geometry and color. Since the geometry
information is more robust than the color, we correct the depth by affine transform in prior to utilizing
the color cues. Then we determine the filter parameters adaptively based on the local features of the
color image which solves the texture copy problem and protects the fine structures. Since L1 norm
optimization is more robust to data outliers than L2 norm optimization, we force the filtered value to be
the solution for L1 norm optimization. Experimental results show that the proposed algorithm can
protect the intact foreground depth, improve the accuracy of depth at object edges, and eliminate the
flashing phenomenon of depth at objects edges. In addition, the proposed algorithm can effectively fill
the big depth map holes generated by optical reflection.
KEYWORDS: Distortion, Volume rendering, Image compression, Video coding, Quantization, Video, 3D video compression, 3D image processing, Video compression, Communication engineering
In multi-view plus depth (MVD) 3D video coding, texture maps and depth maps are coded jointly. The depth maps
provide the scene geometry information and are used to render the virtual view at the terminal through a
Depth-Image-Based-Rendering (DIBR) technique. The distortion of the coded texture maps and depth maps will induce
synthesized virtual view distortion. Besides the coding efficiency of texture maps and depth maps, bit allocation between
texture maps and depth maps also has a great effect on the virtual view quality. In this paper, the virtual view distortion
is divided into texture maps induced distortion and depth maps induced distortion separately, models of texture maps
induced virtual view distortion and depth maps induced virtual view distortion are derived respectively. Based on the
depth maps induced virtual view distortion model, depth maps coding Rate Distortion Optimization (RDO) is modified
and the depth maps coding efficiency is increased. Meanwhile, we also propose a Rate-distortion (R-D) model to solve
the joint bit allocation problem. Experimental results demonstrate the high accuracy of the proposed virtual view
distortion model. The R-D performance of the proposed algorithm is close to the full search algorithm that can give the
best R-D performance, while the coding complexity of the proposed algorithm is lower. Compared with fixed texture and
depth bits ratio (5:1), an average 0.3 dB gains can be achieved by the proposed algorithm. The proposed algorithm has
high rate control accuracy with the average error less than 1%.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.