Browse > Article
http://dx.doi.org/10.5909/JBE.2019.24.5.743

Implementing 3DoF+ 360 Video Compression System for Immersive Media  

Jeong, Jong-Beom (Sungkyunkwan University (SKKU), Department of Computer Education)
Lee, Soonbin (Gachon University, Department of Computer Engineering)
Jang, Dongmin (Sungkyunkwan University (SKKU), Department of Computer Education)
Lee, Sangsoon (Gachon University, Department of Computer Engineering)
Ryu, Eun-Seok (Sungkyunkwan University (SKKU), Department of Computer Education)
Publication Information
Journal of Broadcast Engineering / v.24, no.5, 2019 , pp. 743-754 More about this Journal
Abstract
System for three degrees of freedom plus (3DoF+) and 6DoF requires multi-view high resolution 360 video transmission to provide user viewport adaptive 360 video streaming. In this paper, we implement 3DoF+ 360 video compression system which removes the redundancy between multi-view videos and merges the residual into one video to provide high quality 360 video corresponding to an user's head movement efficiently. Implementations about 3D warping based redundancy removal method between 3DoF+ 360 videos and residual extraction and merger are explained in this paper. With the proposed system, 20.14% of BD-rate reduction in maximum is shown compared to traditional high-efficiency video coding (HEVC) based system.
Keywords
360 Video; 3DoF+; HEVC; View Synthesis; multi-view;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Hyun-Joon Roh, Bok-Gi Lee, and Eun-Seok Ryu. 2018. Tile Partitioning and Allocation for HEVC Parallel Decoding on Asymmetric Multicores. The Journal of Korean Institute of Communications and Information Sciences (J-KICS), Vol.43, No.05, pp. 791-800   DOI
2 Seehwan Yoo and Eun-Seok Ryu. 2017. Parallel HEVC decoding with asymmetric mobile multicores. Multimedia Tools and Applications 76, 16 (2017), 17337-17352.   DOI
3 Dien Nguyen, Tuan Le, Sangsoon Lee, and Eun-Seok Ryu. 2018. SHVC Tile-Based 360-Degree Video Streaming for Mobile VR: PC Offloading Over mmWave. Sensors 18, 11 (2018), 3728.   DOI
4 Julien Fleureau, Franck Thudor, Renaud Doré, Max Dmitrichenko, Basel Salahieh, Jill Boyce. 2019. Description of Technicolor Intel response to MPEG-I 3DoF+ Call for Proposal. 126th MPEG meeting of ISO/IEC JTC1/SC29/ WG11, MPEG2019/m47445.
5 Jang-Woo Son, Eun-Seok Ryu. 2018. Tile-Based 360-Degree Video Streaming for Mobile Virtual Reality in Cyber Physical System. Elsevier, Computers and Electrical Engineering.
6 Tuan Thanh Le, Dien Van Nguyen, and Eun-Seok Ryu. 2018. Computing Offloading Over mmWave for Mobile VR: Make 360 Video Streaming Alive. IEEE Access (2018).
7 Robert Skupin, Yago Sanchez, Karsten SAijhring, Thomas Schierl, Eun-Seok Ryu, and Jangwoo Son. 2018. Temporal MCTS Coding Constraints Implementation. 122th MPEG meeting of ISO/IEC JTC1/SC29/WG11, MPEG 122/m42423.
8 Jang-Woo Son, Dongmin Jang, Eun-Seok Ryu. 2018. Implementing Motion-Constrained Tile and Viewport Extraction for VR Streaming. ACM Network and Operating System Support for Digital Audio and Video 2018 (NOSSDAV2018).
9 Xin Wang, Lulin Chen, Shuai Zhao, and Shawmin Lei. 2017. From OMAF for 3DoF VR to MPEG-I Media Format for 3DoF+, Windowed 6DoF and 6DoF VR. 119th MPEG meeting of ISO/IECJTC1/SC29/WG11, MPEG119/m44197.
10 Jong-Beom Jeong, Dongmin Jang, Jangwoo Son, and Eun-Seok Ryu. 2018. 3DoF+ 360 Video Location-Based Asymmetric Down-Sampling for View Synthesis to Immersive VR Video Streaming. Sensors 18, 9(2018), 3148.   DOI
11 Basel Salahieh, Bart Kroon, Joel Jung, Marek Domanski. 2019. Test Model for Immersive Video. 126th MPEG meeting of ISO/IEC JTC1/SC29/ WG11, MPEG2019/n18470.
12 Marek Domański, Adrian Dziembowski, Dawid Mieloch, Olgierd Stankiewicz, Jakub Stankowski, Adam Grzelka, Gwangsoon Lee, Jun Young Jeong, Jeongil Seo. 2019. Technical description of proposal for Call for Proposals on 3DoF+ Visual prepared by Poznań University of Technology (PUT) and Electronics and Telecommunications Research Institute (ETRI). 126th MPEG meeting of ISO/IEC JTC1/SC29/WG11, MPEG2019/m47407.
13 Vinod Kumar Malamal Vadakital, Kimmo Roimela, Lauri Ilola, Jaakko Keränen, Mika Pesonen, Sebastian Schwarz, Jani lainema, Miska Hannuksela. 2019. Description of Nokia's response to CFP for 3DOF+ visual. 126th MPEG meeting of ISO/IEC JTC1/SC29/ WG11, MPEG2019/m47372.
14 Bin Wang, Yule Sun, Wenyi Wang, Jiafeng Chen, Lu Yu, Huiming Yu. 2019. Description of Zhejiang University's response to 3DoF+ Visual CfP. 126th MPEG meeting of ISO/IEC JTC1/SC29/ WG11, MPEG2019/m47684.
15 Ying Chen, Xin Zhao, Li Zhang, Je-Won Kang. 2016. Multiview and 3D Video Compression Using Neighboring Block Based Disparity Vectors. IEEE Transactions on Multimedia 18.4 (2016): 576-589.   DOI
16 Je-Won Kang, Ying Chen, Li Zhang, Marta Karczewicz. 2014. Low Complexity Neighboring Block based Disparity Vector Derivation in 3D-HEVC. IEEE International Symposium on Circuits and Systems (ISCAS). IEEE, 2014. p. 1921-1924.
17 M Mary Synthuja Jain Preetha, L Padma Suresh, and M John Bosco. 2012. Image segmentation using seeded region growing. In 2012 International Conference on Computing, Electronics and Electrical Technologies (ICCEET). IEEE, 576-583.
18 ISO/IEC JTC1/SC29/WG11. 2018. Reference View Synthesizer(RVS) manual. 124th MPEG meeting of ISO/IECJTC1/SC29/WG11, MPEG124/n18068.
19 ISO/IEC JTC1/SC29/WG11. 2019. Call for Proposals on 3DoF+ Visual. 125th MPEG meeting of ISO/IECJTC1/SC29/WG11, MPEG/n18145.
20 ISO/IEC JTC1/SC29/WG11. 2018. 3DoF+ Software Platform description. 125th MPEG meeting of ISO/IECJTC1/SC29/WG11, MPEG2018/n18070.
21 Patrick Boissonade, Joel Jung, and P.Nikitin. 2018. [MPEG-IVisual] View synthesis algorithm for windowed-6DoF. 124th MPEG meeting of ISO/IECJTC1/SC29/ WG11, MPEG2018/m44788.
22 ISO/IECJTC1/SC29/WG11.2008. Overview of 3D Video Coding. 84th MPEG meeting of ISO/IECJTC1/SC29/WG11, MPEG2008/n9784.
23 Gerhard Tech, Ying Chen, Karsten Muller, Jens-Rainer Ohm, Anthony Vetro, Ye-Kui Wang. 2016. Overview of the Multiview and 3D Extensions of High Efficiency Video Coding. IEEE Transactions on Circuits and Systems for Video Technology, 2015, 26.1: 35-49.   DOI
24 Ying Chen, Gerhard Tech, Krzysztof Wegner and Sehoon Yea. 2014. Test Model 10 of 3D-HEVC and MV-HEVC. 110th MPEG meeting of ISO/IEC JTC1/SC29/ WG11, MPEG2014/n14975.
25 Joel Jung, P. Nikitin. 2017. [MPEG-I Visual] List of issues with MVand 3D-HEVC when coding 1D- and 2D-multi-view contents. 120th MPEG meeting of ISO/IEC JTC1/SC29/ WG11, MPEG2017/m41701.
26 Renaud Dore, Franck Thudor. 2018. Outperforming 3DoF+ Anchors: first evidence. 123th MPEG meeting of ISO/IEC JTC1/SC29/ WG11, MPEG2018/m43504.
27 Franck Thudor, Renaud Dore. RVS based 3DoF+ evidence results. 124th MPEG meeting of ISO/IEC JTC1/SC29/ WG11, MPEG2018/m44588.
28 Bart Kroon, Bart Sonneveldt. 2019. Philips response to CfP on 3DoF+. 126th MPEG meeting of ISO/IEC JTC1/SC29/ WG11, MPEG2019/m47179.
29 2018. Open computer vision. [Online]. https://github.com/opencv/opencv/releases/tag/3.4.2
30 Renaud Dore. 2018. Technicolor 3DoF+ test materials. 122th MPEG meeting of ISO/IEC JTC1/SC29/ WG11, MPEG2018/m42349.
31 Bart Kroon. 2018. 3DoF+ test sequence ClassroomVideo. 122th MPEG meeting of ISO/IEC JTC1/SC29/ WG11, MPEG2018/m42415.
32 Hyun-Joon Roh, SungWon Han, and Eun-Seok Ryu. 2017. Prediction complexitybased HEVC parallel processing for asymmetric multicores. Multimedia Tools and Applications 76, 23 (2017), 25271-25284.   DOI
33 Mary-Luc Champel, Thomas Stockhammer, Thierry Fautier, Emmanuel Thomas, Rob Koenen. 2016. Quality Requirements for VR. 116th MPEG meeting of ISO/IEC JTC1/SC29/ WG11, MPEG 116/m39532.