ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Reduced Reference Quality Metric for Synthesized Virtual Views in 3DTV

  • Le, Thanh Ha (Faculty of Information Technology, University of Engineering and Technology, Vietnam National University) ;
  • Long, Vuong Tung (Faculty of Information Technology, University of Engineering and Technology, Vietnam National University) ;
  • Duong, Dinh Trieu (Faculty of Information Technology, University of Engineering and Technology, Vietnam National University) ;
  • Jung, Seung-Won (Department of Multimedia Engineering, Dongguk University)
  • Received : 2016.04.15
  • Accepted : 2016.11.03
  • Published : 2016.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

Multi-view video plus depth (MVD) has been widely used owing to its effectiveness in three-dimensional data representation. Using MVD, color videos with only a limited number of real viewpoints are compressed and transmitted along with captured or estimated depth videos. Because the synthesized views are generated from decoded real views, their original reference views do not exist at either the transmitter or receiver. Therefore, it is challenging to define an efficient metric to evaluate the quality of synthesized images. We propose a novel metric-the reduced-reference quality metric. First, the effects of depth distortion on the quality of synthesized images are analyzed. We then employ the high correlation between the local depth distortions and local color characteristics of the decoded depth and color images, respectively, to achieve an efficient depth quality metric for each real view. Finally, the objective quality metric of the synthesized views is obtained by combining all the depth quality metrics obtained from the decoded real views. The experimental results show that the proposed quality metric correlates very well with full reference image and video quality metrics.

Keywords

References

  1. A. Smolic et al., "3-D Video and Free Viewpoint Video-Technologies, Applications and MPEG Standards," Proc. IEEE Int. Conf. Multimedia Expo, Toronto, Canada, July 9-12, 2006, pp. 2161-2164.
  2. A. Smolic et al., "Multi-view Video Plus Depth (MVD) Format for Advanced 3D Video Systems," Joint Video Team, document JVT-W100, 2007.
  3. C. Fehn, "Depth-Image-based Rendering (DIBR), Compression and Transmission for a New Approach on 3DTV," Proc. SPIE Stereoscopic Displays Virtual Reality Syst. XI, vol. 5291, 2004.
  4. C. Zhu et al., 3D-TV System with Depth-Image-Based Rendering: Architectures, Techniques and Challenges, New York, USA: Springer, 2013.
  5. A. Benoit et al., "Quality Assessment of Stereoscopic Images," EURASIP Journal on Image and Video Processing, New York, USA: Springer, 2008, pp. 1-13.
  6. P. Joveluro et al., "Perceptual Video Quality Metric for 3D Video Quality Assessment," 3DTV-Conf.: True Vision-Capture, Transmission Display 3D Video, Tampere, Finland, June 7-9, 2010.
  7. E. Bosc et al., "Towards a New Quality Metric for 3D Synthesized View Assessment," IEEE J. Sel. Topics Signal Process., vol. 5, no. 7, Nov. 2011, pp. 1332-1343. https://doi.org/10.1109/JSTSP.2011.2166245
  8. J. You et al., "Perceptual Quality Assessment for Stereoscopic Images based on 2D Image Quality Metrics and Disparity Analysis," Proc. VPQM, Scottsdale, AZ, USA, Jan. 2010.
  9. H. Shao, X. Cao, and G. Er, "Objective Quality Assessment of Depth Image based Rendering in 3DTV System," 3DTV Conf.: True Vision-Capture, Transmission Display 3D Video, Potsdam, Germanay, May 4-6, 2009.
  10. P.H. Conze, P. Robert, and L. Morin, "Objective View Synthesis Quality Assessment," Stereoscopic Displays and Appl. XXIII, vol. 8288, Feb. 2012, pp. 1-14.
  11. Y. Zhao and L. Yu, "Aperceptual Metric for Evaluating Quality of Synthesized Sequences in 3DV System," Proc. SPIE, vol. 7744, Aug. 2010, p. 77440X.
  12. M.S. Farid, M. Lucenteforte, and M. Grangetto, "Objective Quality Metric for 3D Virtual Views," IEEE Int. Conf. Image Process., Quebec, Canada, Sept. 27-30, 2015, pp. 3720-3724.
  13. T.H. Le, S.W. Jung, and C.S. Won, "A New Depth Image Quality Metric Using a Pair of Color and Depth Images," Multimedia Tools and Application, New York, USA: Springer, 2016, pp. 1-19.
  14. H. Yuan et al., "Coding Distortion Elimination of Virtual View Synthesis for 3D Video System: Theoretical Analyses and Implementation," IEEE Trans. Broadcast., vol. 58, no. 4, Dec. 2012, pp. 558-568,. https://doi.org/10.1109/TBC.2012.2187612
  15. Y. Zhang et al., "Regional Bit Allocation and Rate Distortion Optimization for Multiview Depth Video Coding with View Synthesis Distortion Model," IEEE Trans. Image Process., vol. 22, no. 9, Sept. 2013, pp. 3497-3512. https://doi.org/10.1109/TIP.2013.2265883
  16. W.R. Mark, L. McMillan, and G. Bishop, "Post-rendering 3D Warping," Proc. Symp. Interactive 3D Graph., Providence, RI, USA, Apr. 27-30, 1997, pp. 7-16.
  17. C. Fehn, "A 3D-TV Approach Using Depth-Image-based Rendering (DIBR)," Proc. Vis., Imag. Image Process., Benalmadena, Spain, Sept. 8-10, 2003, pp. 482-487.
  18. D. Tian et al., "View Synthesis Techniques for 3D Video," Appl. Digital Image Process. XXXII, vol. 7443, Sept. 2009, pp. 74430T-1-74430T-11.
  19. M. Domanki, M. Gotfryd, and K. Wegner, "View Synthesis for Multiview Video Transmission," Int. Conf. Image Process., Comput. Vision, Pattern Recogn., Las Vegas, NV, USA, July 13-16, 2009, pp. 13-16.
  20. C. Grigorescu, N. Petkov, and M.A. Westenberg, "Contour Detection based on Nonclassical Receptive Field Inhibition," IEEE Trans. Image Process., vol. 12, no. 6, July 2003, pp. 729-739. https://doi.org/10.1109/TIP.2003.814250
  21. A. D'Angelo, L. Zhaoping, and M. Barni, "A Full-Reference Quality Metric for Geometrically Distorted Images," IEEE Trans. Image Process., vol. 19, no. 4, Apr. 2010, pp. 867-881. https://doi.org/10.1109/TIP.2009.2035869
  22. Nagoya Sequences, Accessed Sept. 20, 2015. http://www.fujii.nuee.nagoya-u.ac.jp/multiview-data/
  23. Joint Collaborative Team for 3DV, 3D-HTM Software Platform, Accessed Sept. 20, 2015. https://hevc.hhi.fraunhofer.de/3dhevc
  24. D. Tian et al., "View Synthesis Techniques for 3D Video," Proc. Appl. Digital Image Process. XXXII, vol. 7443, Sept. 2009, pp. 74430T-1-74430T-11.
  25. Z. Wang, A.C. Bovik, and E.P. Simoncelli, "Image Quality Assessment: from Error Visibility to Structural Similarity," IEEE Trans. Image Process., vol. 13, no. 4, Apr. 2004, pp. 600-612. https://doi.org/10.1109/TIP.2003.819861
  26. Z. Wang, E.P. Simoncelli, and A.C. Bovik, "Multiscale Structural Similarity for Image Quality Assessment," Proc. IEEE Asilomar Conf. Signals, Syst. Comput., Pacific Grove, CA, USA, Nov. 3-9, 2003, pp. 1398-1402.
  27. L. Zhang et al., "FSIM: A Feature Similarity Index for Image Quality Assessment," IEEE Trans. Image Process., vol. 20, no. 8, Aug. 2011, pp. 2378-2386. https://doi.org/10.1109/TIP.2011.2109730
  28. M.H. Pinson and S. Wolf, "A New Standardized Method for Objectively Measuring Video Quality," IEEE Trans. Broadcast., vol. 50, no. 3, Sept. 2004, pp. 312-322. https://doi.org/10.1109/TBC.2004.834028
  29. VQEG - Video Quality Expert Group, "Final Report from the Video Quality Experts Group on the Validation of Objective Models of Video Quality Assessment," 2003.
  30. ITU-R BT.500-11, "Methodology for the Subjective Assessment of the Quality of Television Pictures," 2002.
  31. E. Boscet al., "Perceived Quality of DIBR-Based Synthesized Views," Appl. Digital Image Process. XXXIV, vol. 81350I, 2011.

Cited by

  1. Quality Assessment of DIBR-Synthesized Images by Measuring Local Geometric Distortions and Global Sharpness vol.20, pp.4, 2016, https://doi.org/10.1109/tmm.2017.2760062