Browse > Article
http://dx.doi.org/10.5370/JEET.2015.10.5.2189

Disparity Refinement near the Object Boundaries for Virtual-View Quality Enhancement  

Lee, Gyu-cheol (Dept. of Electronic Engineering, Kwangwoon University)
Yoo, Jisang (Dept. of Electronic Engineering, Kwangwoon University)
Publication Information
Journal of Electrical Engineering and Technology / v.10, no.5, 2015 , pp. 2189-2196 More about this Journal
Abstract
Stereo matching algorithm is usually used to obtain a disparity map from a pair of images. However, the disparity map obtained by using stereo matching contains lots of noise and error regions. In this paper, we propose a virtual-view synthesis algorithm using disparity refinement in order to improve the quality of the synthesized image. First, the error region is detected by examining the consistency of the disparity maps. Then, motion information is acquired by applying optical flow to texture component of the image in order to improve the performance. Then, the occlusion region is found using optical flow on the texture component of the image in order to improve the performance of the optical flow. The refined disparity map is finally used for the synthesis of the virtual view image. The experimental results show that the proposed algorithm improves the quality of the generated virtual-view.
Keywords
Stereo matching; Optical flow; Occlusion regions; Texture component;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Retrevo Corporation, Could low interest in 3DTV hurt the TV business?, Retrieved Nov. 2011, from http://www.retrevo.com/content/node/1915
2 G. M. Um, G. H. Cheong, W. S. Cheong and N. H. Hur, “Technical development and standardization trends of multi-view 3D and free-viewpoint video,” The Magazine of the IEEK, vol. 38, no. 2, pp. 18-23, Feb. 2011.
3 C. Zitnick and T. Kanade, “A cooperative algorithm for stereo matching and occlusion detection,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 7, pp. 675-684, Jul. 2000.   DOI
4 S. Y. Cho, I. S. Sun, J. M. Ha and H. Jeong, “Occlusion detection and filling in disparity map for multiple view synthesis,” 8th International Conference on Computing and Networking Technology (ICCNT), pp. 425-432, Aug. 2012.
5 Y. S. Kang and S. Y. Ho, “Generation of high-resolution disparity map using multiple cameras and low-resolution depth camera,” The Conference of Korea Institute of Communications and Information Sciences, pp. 287-288, Nov. 2012.
6 J. Aujol, G. Gilboa, T. Chan and S. Osher, “Structure-texture image decomposition - modeling, algorithms, and parameter selection,” International Journal of Computer Vision, vol. 67, no. 1, pp. 111-136, Apr. 2006.   DOI
7 B.D. Lucas and T. Kanade, “An iterative image registration technique with an application to stereo vision (IJCAI),” 7th International Joint Conference on Artificial Intelligence (IJCAI), pp. 674-679, Apr. 1981.
8 W. Sun, O. Au, L. Xu, Y.Li and W. Hu, “Novel temporal domain hole filling based on bakground modeling for view synthesis,” IEEE International Conference on Image Processing 2012, Florida, USA, Oct. 2012.
9 K. H. Oh, S. Y. Lim and H. I. Hahn, “Estimating the regularizing parameters for belief propagation based stereo matching algorithm,” The Journal of the Institute of Electronics Engineers of Korea, vol. 47, no. 1, pp. 112-119, Jan. 2011.
10 A. Wedel, T. Pock, C. Zach, H. Bischof, and D. Cremers, “An improved algorithm for TV-L1 optical flow,” In Statistical and Geometrical Approaches to Visual Motion Analysis, vol. 5604, pp. 23-45, Jul. 2008.
11 M. S. Ko and J. S. Yoo, “Boundary noises removal and hole filling algorithm for virtual viewpoint image generation,” J. KICS, vol. 37, no. 8, pp. 679-688, Aug. 2012.   DOI
12 K. J. Oh, S. Yea, and Y.S. Ho, “Hole filling method using depth based inpainting for view synthesis in free viewpoint television and 3-d video,” Proc. of the 27th conference on Picture Coding Symposium (PCS’09), pp. 233-236, May. 2009.
13 F. Yasutaka, J. Ponce, “Accurate camera calibration from multi-view stereo and bundle adjustment,” International Journal of Computer Vision, vol. 84, pp. 257-268, Sep. 2009.   DOI
14 T.J. Kim and J. S. Yoo, “Hierarchical stereo matching with color information,” The Journal of Korea Institute of Communications and Information Sciences, vol. 34, no. 3, pp. 279-287, Mar. 2009.
15 J. M. Lim, G. M. Um, H. C. Shin, G. S. Lee, N. H. Hur and J. S. Yoo, “Multi-view image generation using grid-mesh based image domain warping and occlusion region information,” The Journal of Korean Society of Broadcast Engineers, vol. 18, no. 6, pp. 859-871, Nov. 2013.
16 C. J. Park, J. H. Ko and E. S. Kim, “A new intermediate view reconstruction scheme based-on stereo image rectification algorithm,” The Journal of Korea Institute of Communications and Information Sciences, vol. 29, no. 5C, pp. 632-641, May. 2004.
17 L. I. Rudin, S. Osher and E. Fatemi, “Nonlinear total variation based noise removal algorithm,” physica D: Nonlinear Phenomena, vol. 60, pp. 259-268, Nov. 1992   DOI
18 G. C. Lee, Y. H. Seo and J. S. Yoo, “GPGPU-based multiview synthesis using kinect depth image,” The Conference of Korea Institute of Communications and Information Sciences, Yongpyong, Korea, Jan. 2012.
19 "http://vision.middlebury.edu/stero/data/"
20 L. Zhao and H. Wang, “Image denoising using trivariate shrinkage filter in the wavelet domain and joint bilateral filter in the spatial Domain,” IEEE Trans. Image Process., vol. 18, no. 10, pp. 2364-2369, Oct. 2009.   DOI
21 J. H. Park and C. G. Song, “Effective shadow removal from aerial image of golf course to extract components,” The Journal of Korean Institute of Information Scientists and Engineers, vol. 39, no. 7, pp. 577-582, Jul. 2012.
22 G. C. Lee and J. S. Yoo, “Real-time virtual-view image synthesis algorithm using Kinect camera,” The Journal of Korea Institute of Communications and Information Sciences, vol. 38, no. 5, pp. 409-419, May. 2013.
23 N. E. Yang, Y. G. Kim and R. H. Hong, “Depth hole filling using the depth distribution of neighboring regions of depth holes in the Kinect sensor,” 2012 IEEE International Conference on Signal Processing, Communication and Computing (ICSPCC), Aug. 2012