KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지

Hole-filling Based on Disparity Map for DIBR

  • Liu, Ran (College of Communication Engineering, Chongqing University) ;
  • Xie, Hui (College of Communication Engineering, Chongqing University) ;
  • Tian, Fengchun (College of Communication Engineering, Chongqing University) ;
  • Wu, Yingjian (Homwee Technology Co., Ltd, Changhong Group) ;
  • Tai, Guoqin (College of Communication Engineering, Chongqing University) ;
  • Tan, Yingchun (College of Communication Engineering, Chongqing University) ;
  • Tan, Weimin (College of Communication Engineering, Chongqing University) ;
  • Li, Bole (College of Communication Engineering, Chongqing University) ;
  • Chen, Hengxin (College of Computer Science, Chongqing University) ;
  • Ge, Liang (Chongqing Key Laboratory of Software Theory & Technology)
  • Received : 2012.06.23
  • Accepted : 2012.10.02
  • Published : 2012.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

Due to sharp depth transition, big holes may be found in the novel view that is synthesized by depth-image-based rendering (DIBR). A hole-filling method based on disparity map is proposed. One important aspect of the method is that the disparity map of destination image is used for hole-filling, instead of the depth image of reference image. Firstly, the big hole detection based on disparity map is conducted, and the start point and the end point of the hole are recorded. Then foreground pixels and background pixels are distinguished for hole-dilating according to disparity map, so that areas with matching errors can be determined and eliminated. In addition, parallaxes of pixels in the area with holes and matching errors are changed to new values. Finally, holes are filled with background pixels from reference image according to these new parallaxes. Experimental results show that the quality of the new view after hole-filling is quite well; and geometric distortions are avoided in destination image, in contrast to the virtual view generated by depth-smoothing methods and image inpainting methods. Moreover, this method is easy for hardware implementation.

Keywords

References

  1. A. Smolic, K. Mueller, P. Merkle, A. Vetro, "Development of a new MPEG standard for advanced 3D video applications," in 6th International Symposium on Image and Signal Processing and Analysi, pp. 400-407, Sep 2009.
  2. R. Liu, Q. Zhu, X. Xu, L. Zhi, H. Xie, J. Yang, X. Zhang, "Stereo effect of image converted from planar,".Information Sciences, vol. 178, no. 8, pp. 2079-2090, Apr 2008. https://doi.org/10.1016/j.ins.2007.12.008
  3. L. Pei-Jun, Effendi, "Nongeometric Distortion Smoothing Approach for Depth Map Preprocessing," IEEE Transactions on Multimedia, vol. 13, no. 2, pp. 246-254, Apr 2011. https://doi.org/10.1109/TMM.2010.2100372
  4. X. Yang, J. Liu, J. Sun, X. Li, W. Liu, Y. Gao, "DIBR based view synthesis for free-viewpoint television," in 5th 3DTV Conference, pp. 1-4, May 2011.
  5. L.-H. Wang, X.-J. Huang, M Xi, D-X Li, M Zhang, "An asymmetric edge adaptive filter for depth generation and hole filling in 3DTV," IEEE Transactions on Broadcasting, vol. 56, no. 3, pp. 425-431, Sep 2010. https://doi.org/10.1109/TBC.2010.2053971
  6. P.-J. Lee, Effendi, "Adaptive edge-oriented depth image smoothing approach for depth image based rendering," in 2010 IEEE International Symposium on Broadband Multimedia Systems and Broadcasting, pp. 1-5, Mar 2010.
  7. I. Daribo, B. Pesquet-Popescu, "Depth-aided image inpainting for novel view synthesis," in 2010 IEEE International Workshop on Multimedia Signal Processing (MMSP), pp. 167-170, Oct 2010.
  8. L. Azzari, F. Battisti, A. Gotchev, M. Carli, K. Egiazarian, "A modified non-local mean inpainting technique for occlusion filling in depth-image-based rendering," in Stereoscopic Displays and Applications XXII, Jan 2011.
  9. H. Ying-Rung, T. Yu-Cheng, C. Tian-Sheuan, "Stereoscopic images generation with directional Gaussian filter," in Proceedings of 2010 IEEE International Symposium on Circuits and Systems (ISCAS), pp. 2650-2653, May 30-June 2, 2010.
  10. S. Zinger, L. Do, P. H. N. De With, "Free-viewpoint depth image based rendering,". Journal of Visual Communication and Image Representation, vol. 21, no. 5-6, pp. 533-541, Jul 2010. https://doi.org/10.1016/j.jvcir.2010.01.004
  11. J. Gautier, O. Le Meur, C. Guillemot, "Depth-based image completion for view synthesis," in 5th 3DTV Conferenc , pp. 1-4, May 2011.
  12. C. Fehn, "Depth-image-based rendering (DIBR), compression and transmission for a new approach on 3D-TV," in Stereoscopic Displays and Virtual Reality Systems XI, pp. 93-104, Jan 2004.
  13. R. Liu, Y.-C. Tan, F.-C. Tian, H Xie, G.-Q. Tai, W.-M. Tan, J.-L. Liu, X.-Y. Xu, C. Kadri, N. Abakah, "Visual Fatigue Reduction Based on Depth Adjustment for DIBR System," Ksii Transactions on Internet and Information Systems, vol. 6, no. 4, pp. 1171-1187, 2012.
  14. R. Liu, H. Xie, G.-Q. Tai, Y.-C. Tan, R.-L. Guo, W.-Y. Luo, X.-Y. Xu, J.-L. Liu, "Depth adjustment for depth-image-based rendering in 3D TV system,". Journal of Information and Computational Science, vol. 8, no. 16, pp. 4233-4240, 2011.
  15. L Zhang, G.-Q. Tai, R. Liu, H. Xie, X.-Y. Xu, "Correction algorithm of matching error for DIBR.," Huanan Ligong Daxue Xuebao/Journal of South China University of Technology (Natural Science), vol. 39, no. 12, pp. 51-55, 2011.
  16. G. Zhang, J. Jia, T.-T. Wong, H. Bao, "Consistent depth maps recovery from a video sequence," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 6, pp. 974-988,2009. https://doi.org/10.1109/TPAMI.2009.52