Browse > Article
http://dx.doi.org/10.5909/JBE.2019.24.6.1035

Toward 6 Degree-of-Freedom Video Coding Technique and Performance Analysis  

Park, Hyeonsu (Department of Electronic and Electrical Engineering, Ewha W. University)
Park, Sang-hyo (Department of Electronic and Electrical Engineering, Ewha W. University)
Kang, Je-Won (Department of Electronic and Electrical Engineering, Ewha W. University)
Publication Information
Journal of Broadcast Engineering / v.24, no.6, 2019 , pp. 1035-1052 More about this Journal
Abstract
Recently, as the demand for immersive videos increases, efficient video processing techniques for omnidirectional immersive video is actively developed by MPEG-I. While the omnidirectional video provides a larger degree of freedom for a free viewpoint, the size of the video increases significantly. Furthermore, in order to compress 6 degree-of-freedom (6 DoF) videos that support motion parallax, it is required to develop a codec to yield better coding efficiency. In this paper, we develop a 6 DoF codec using Versatile Video Coding (VVC) as the next generation video coding standard. To the authors' best knowledge, this is the first VVC-based 6 DoF video codec toward the future ISO/IEC 23090 Part 7 (Metadata for Immersive Media (Video)) MPEG-I standardization. The experiments were conducted on the seven test video sequences specified in Common Test Condition (CTC) in two operation modes of TMIV (Test Model for Immersive Media) software. It is demonstrated that the proposed codec improves coding performance around 33.8% BD-rate reduction in the MIV (Metadata for Immersive Video) mode and 30.2% BD-rate reduction in the MIV view mode as compared to the state-of-the-art TMIV reference software. We also show the performance comparisons using Immersive Video PSNR (IV-PSNR) and Mean Structural Similarity (MSSIM).
Keywords
6DoF video; MPEG-I; TMIV software; HEVC; VVC;
Citations & Related Records
연도 인용수 순위
  • Reference
1 ISO/IEC JTC1/SC29/WG11/M47372, "Description of Nokia's response to CFP for 3DOF+ visual", MPEG 126, Mar. 2019.
2 ISO/IEC JTC1/SC29/WG11/M47684, "Description of Zhejiang University's response to 3DoF+ Visual CfP", MPEG 126, Mar. 2019.
3 ISO/IEC JTC1/SC29/WG11/M47179, "Philips response to CfP on 3DoF", MPEG 126, Mar. 2019.
4 JVET of ITU-T and ISO/IEC, "Meeting Report of the 15th Meeting of the Joint Video Experts Team (JVET)", Document JVET-O2000, July 2019.
5 ISO/IEC JTC1/SC29/WG11/N18563, "Common Test Conditions for Immersive Video", MPEG 127, July 2019.
6 ISO/IEC JTC1/SC29/WG11/N18576, "Working Draft2 of Metadata for Immersive Video", MPEG 127, July 2019.
7 JCT-VC of ITU-T and ISO/IEC, "Common test conditions", Document JCTVC-Z1100, January 2017
8 JVET of ITU-T and ISO/IEC, "JVET common test conditions and software reference configurations for SDR video", Document JVETN1010, March 2019.
9 JVET of ITU-T and ISO/IEC, "Algorithm description for Versatile Video Coding and Test Model 6 (VTM 6)", Document JVET-O2002-v2, July 2019.
10 ISO/IEC JTC1/SC29/WG11/N18068, "Reference View Synthesizer (RVS) manual", MPEG 124, Oct. 2018.
11 ISO/IEC JTC1/SC29/WG11/M42415, "3DoF+ test sequence ClassroomVideo", MPEG 122, April. 2018.
12 ISO/IEC JTC1/SC29/WG11/M43748, "Kermit test sequence for Windowed 6DoF Activities", MPEG 123, July. 2018.
13 Y. Sun, A. Lu, L. Yu, "Weighted-to-spherically-uniform quality evaluation for omnidirectional video", IEEE Signal Process. Lett., 24(9), pp.1408-1412, 2017.   DOI
14 G. Choi, G. Bang, "MPEG-I Video Standardization Trends", Broadcast and Media, 23(4), pp.41-57, 2018.
15 ISO/IEC JTC1/SC29/WG11/N18709, "Software manual of IV-PSNR for Immersive Video", MPEG 127, July 2019.
16 Yu-Chuan su, Kristen Grauman, "Learning Compressible 360◦Video Isomers", In CVPR, 2018.
17 J. Lee, "Immersive Video Format Standardization Trends", Broadcast and Media, 23(4), pp.31-40, 2018.
18 ISO/IEC JTC1/SC29/WG11/M41555, "Proposed architectures for supporting Windowed 6DoF, Omnidirectional 6DoF and 6DoF media", MPEG 120, Oct. 2017.
19 https://www.theverge.com/2017/4/19/15345738/facebook-surround-360-video-cameras-f8-conference-2017
20 ISO/IEC JTC1/SC29/WG11/M48278, "[MPEG-I Visual] Proposal of Depth Improvement for Depth Estimation Reference Software (DERS8.0)", MPEG 127, July. 2019.
21 ISO/IEC JTC1/SC29/WG11/M47170, "[MPEG-I Visual] Proposal of Trimming and Color Matching of Multi-View Sequences", MPEG 126, Mar. 2019.
22 ISO/IEC JTC1/SC29/WG11/N18564, "Exploration Experiments for MPEG-I: 6DoF", MPEG 127, July. 2019.
23 ISO/IEC JTC1/SC29/WG11/N18450, "Manual of Depth Estimation Reference Software (DERS 8.0)", MPEG 126, Mar. 2019.
24 ISO/IEC JTC1/SC29/WG11/N18172, "Versatile View Synthesizer (VVS) 2.0 manual", MPEG 125, Jan. 2019.
25 ISO/IEC JTC1/SC29/WG11/N18145, "Call for Proposals on 3DoF+ Visual", MPEG 125, Jan 2019.
26 ISO/IEC JTC1/SC29/WG11/N18577, "Test Model 2 for Immersive Video", MPEG 127, July 2019.
27 ISO/IEC JTC1/SC29/WG11/M47544, "Description of Technicolor Intel response to MPEG-I 3DoF+ Call for Proposal", MPEG 126, Mar. 2019.
28 ISO/IEC JTC1/SC29/WG11/M45995, "[MPEG-I Visual] Results of Exploration Experiments for MPEG-I: Omnidirectional 6DoF - EEsynthesis", MPEG 125, Jan. 2019.
29 ISO/IEC JTC1/SC29/WG11/M47705, "[MPEG-I Visual] 6DoF EE Proposal for ClassroomImage Pose e", MPEG 126, Mar. 2019.
30 ISO/IEC JTC1/SC29/WG11/M47407, "Technical description of proposal for Call for Proposals on 3DoF+ Visual prepared by Poznan University of Technology (PUT) and Electronics and Telecommunications Research Institute (ETRI)", MPEG 126, Mar. 2019.