Browse > Article
http://dx.doi.org/10.5909/JBE.2022.27.5.773

Performance Analysis of 6DoF Video Streaming Based on MPEG Immersive Video  

Jeong, Jong-Beom (Department of Computer Science Education, Sungkyunkwan University)
Lee, Soonbin (Department of Computer Science Education, Sungkyunkwan University)
Kim, Inae (Department of Computer Science Education, Sungkyunkwan University)
Ryu, Eun-Seok (Department of Computer Science Education, Sungkyunkwan University)
Publication Information
Journal of Broadcast Engineering / v.27, no.5, 2022 , pp. 773-793 More about this Journal
Abstract
The moving picture experts group (MPEG) immersive video (MIV) coding standard has been established to support six degrees of freedom (6DoF) in virtual reality (VR) by transmitting high-quality multiple immersive videos. The MIV exploits two approaches considering tradeoff between bandwidth and computational complexity: 1) eliminating correlation between multi-view videos or 2) selecting representative videos. This paper presents performance analysis on intermediate synthesized views on source view positions and synthesized pose traces using high-efficiency video coding (HEVC) and versatile video coding (VVC) for above-mentioned two approaches.
Keywords
Virtual reality; MIV; 6DoF; HEVC; VVC; Immersive video; Metaverse;
Citations & Related Records
Times Cited By KSCI : 7  (Citation Analysis)
연도 인용수 순위
1 T. T. Le, J. -B. Jeong, S. S. Lee, J. Kim, E. -S. Ryu. 2021. An Efficient Viewport-Dependent 360 VR System Based on Adaptive Tiled Streaming. CMC-Computers, Materials & Continua, Vol. 66, No. 3, pp. 2627-2643. doi: https://doi.org/10.32604/cmc.2021.013399   DOI
2 T. T. Le, J. B. Jeong, E. -S. Ryu. 2019. Efficient transcoding and encryption for live 360 CCTV system. Applied Sciences, Vol. 9, No. 4. doi: https://doi.org/10.3390/app9040760   DOI
3 J. -B. Jeong, S. Lee, I. Kim, E. -S. Ryu. 2021. Implementing Viewport Tile Extractor for Viewport-Adaptive 360-Degree Video Tiled Streaming. International Conference on Information Networking 2022 (ICOIN2022), pp. 8-12. doi: https://doi.org/10.1109/ICOIN50884.2021.9333964   DOI
4 M. M. Hannuksela, Y. Yan, X. Huang, H. Li. 2015. Overview of the multiview high efficiency video coding (MV-HEVC) standard. IEEE International Conference on Image Processing 2015 (ICIP2015), pp. 2154-2158. doi: https://doi.org/10.1109/ICIP.2015.7351182   DOI
5 C. G. Bampis, A. C. Bovik, and Z. Li. 2018. A simple prediction fusion improves data-driven full-reference video quality assessment models. Picture Coding Symposium 2018 (PCS2018, pp. 298-302. doi: https://doi.org/10.1109/PCS.2018.8456293   DOI
6 V. K. M. Vadakital, K. Roimela, L. Ilola, J. Keranen, M. Pesonen, S. Schwarz, J. lainema, M. Hannuksela. 2019. Description of Nokia's response to CFP for 3DOF+ visual. 126th MPEG meeting of ISO/IEC JTC1/SC29/WG11, MPEG2019/m47372.
7 J. -B. Jeong, S. Lee, E. -S. Ryu. 2021. DWS-BEAM: Decoder-Wise Subpicture Bitstream Extracting and Merging for MPEG Immersive Video. International Conference on Visual Communications and Image Processing 2021 (VCIP2021), pp. 1-5. doi: https://doi.org/10.1109/VCIP53242.2021.9675419   DOI
8 HM reference software, [Online]. Available at: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/.
9 VTM reference software, [Online]. Available at: https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftwareVTM/.
10 Y. Sun, A. Lu, L. Yu. 2017. Weighted-to-spherically-uniform quality evaluation for omnidirectional video. IEEE signal processing letters, Vol. 24, No. 9, pp. 1408-1412. doi: https://doi.org/10.1109/LSP.2017.2720693   DOI
11 J. -B. Jeong, S. Kim, S. Lee, I. Kim, E. -S. Ryu. 2021. Super Metric: Quality Assessment Methods for Immersive Video. Journal of Internet Computing and Services (JICS), Vol. 22, No. 2, pp. 51-58. doi: https://doi.org/10.7472/jksii.2021.22.2.51   DOI
12 J. Jung, B. Kroon. 2021. Common Test Conditions for MPEG Immersive Video. 134th MPEG meeting of ISO/IEC JTC 1/SC 29/WG4, MPEG/n00085.
13 Versatile video encoder (VVenC), [Online]. Available at: https://github.com/fraunhoferhhi/vvenc/.
14 A. Vetro, T. Wiegand, G. J. Sullivan. 2011. Overview of the stereo and multiview video coding extensions of the H. 264/MPEG-4 AVC standard. Proceedings of the IEEE, Vol. 99, No. 4, pp. 626-642. doi: https://doi.org/10.1109/JPROC.2010.2098830   DOI
15 F. Bossen, K. Suhring, A. Wieckowski, S. Liu. 2021. VVC Complexity and Software Implementation Analysis. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 31, No. 10, pp. 3765-3778. doi: https://doi.org/10.1109/TCSVT.2021.3072204   DOI
16 S. Lee, D. Jang, J. B. Jeong, E. -S. Ryu. 2019. Motion-constrained tile set based 360-degree video streaming using saliency map prediction. ACM Network and Operating System Support for Digital Audio and Video 2019 (NOSSDAV2019), pp. 20-24. doi: https://doi.org/10.1145/3304112.3325614   DOI
17 J. -B. Jeong, S. Lee, I. Kim, S. Lee, E. -S. Ryu. 2020. Implementing VVC Tile Extractor for 360-degree Video Streaming Using Motion-Constrained Tile Set. Journal of Broadcast Engineering (JBE), Vol. 25, No. 7, pp. 1073-1080. doi: https://doi.org/10.5909/JBE.2020.25.7.1073   DOI
18 ISO/IEC JTC1/SC29/WG11. 2019. Call for Proposals on 3DoF+ Visual. 125th MPEG meeting of ISO/IEC JTC1/SC29/WG11, MPEG/n18145.
19 R. Dore, F. Thudor. 2018. Outperforming 3DoF+ Anchors: first evidence. 123th MPEG meeting of ISO/IEC JTC1/SC29/WG11, MPEG2018/m43504.
20 J. -B. Jeong, S. Lee, D. Jang, E. -S. Ryu. 2019. Towards 3DoF+ 360 Video Streaming System for Immersive Media. IEEE Access, Vol. 7, pp. 136399-136408. doi: https://doi.org/10.1109/ACCESS.2019.2942771   DOI
21 M. Domanski, A. Dziembowski, D. Mieloch, O. Stankiewicz, J. Stankowski, A. Grzelka, G. Lee, J. Y. Jeong, J. Seo. 2019. Technical description of proposal for Call for Proposals on 3DoF+ Visual prepared by Poznan University of Technology (PUT) and Electronics and Telecommunications Research Institute (ETRI). 126th MPEG meeting of ISO/IEC JTC1/SC29/WG11, MPEG2019/m47407.
22 B. Kroon, B. Sonneveldt. 2019. Philips response to CfP on 3DoF+. 126th MPEG meeting of ISO/IEC JTC1/SC29/WG11, MPEG2019/m47179.
23 B. Wang, Y. Sun, L. Yu. 2019. Description of Zhejiang University's response to 3DoF+ Visual CfP. 126th MPEG meeting of ISO/IEC JTC1/SC29/WG11, MPEG2019/m47684.
24 ISO/IEC JTC 1/SC 29/WG 4. 2021. Text of ISO/IEC FDIS 23090-12 MPEG Immersive Video. 135th MPEG meeting of ISO/IEC JTC 1/SC 29/WG 4, MPEG/n00111.
25 B. Bross, Y. -K. Wang, Y. Ye, S. Liu, J. Chen, G. J. Sullivan, J. -R. Ohm. 2021. Overview of the versatile video coding (VVC) standard and its applications. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 31, No. 10, pp. 3736-3764. doi: https://doi.org/10.1109/TCSVT.2021.3101953   DOI
26 M.-L. Champel, T. Stockhammer, T. Fautier, E. Thomas, R. Koenen. 2016. Quality Requirements for VR. 116th MPEG meeting of ISO/IEC JTC1/SC29/WG11, MPEG 116/m39532.\
27 Jongseok Lee, Juntaek Park, Hansol Choi, Joohyung Byeon, Donggyu Sim, "Overview of VVC", Broadcast and Media Magazine, pp.10-25. Vol. 24, No. 3, Oct. 2019.
28 A. Dziembowski, D. Mieloch, J. Stankowski. A. Grzelka. 2022. IV-PSNR-the objective quality metric for immersive video applications. IEEE Transactions on Circuits and Systems for Video Technology. doi: https://doi.org/10.1109/TCSVT.2022.3179575   DOI
29 J. Fleureau, F. Thudor, R. Dore, M. Dmitrichenko, B. Salahieh, J. Boyce. 2019. Description of Technicolor Intel response to MPEG-I 3DoF+ Call for Proposal. 126th MPEG meeting of ISO/IEC JTC1/SC29/WG11, MPEG2019/m47445.
30 ISO/IEC JTC 1/SC 29/WG 7. 2021. Text of ISO/IEC DIS 23090-5 Visual Volumetric Video-based Coding and Video-based Point Cloud Compression 2nd Edition. 135th MPEG meeting of ISO/IEC JTC 1/SC 29/WG 7, MPEG/n00188.
31 E. S. Jang, M. Preda, K. Mammou, A. M. Tourapis, J. Kim, D. B. Graziosi, S. Rhyu, M. Budagavi. 2019. Video-based point-cloud-compression standard in MPEG: From evidence collection to committee draft [standards in a nutshell]. IEEE Signal Processing Magazine, Vol. 36, No. 3, pp. 118-123. doi: https://doi.org/10.1109/MSP.2019.2900721   DOI
32 D. B. Graziosi, O. Nakagami, S. Kuma, A. Zaghetto, T. Suzuki, A. Tabatabai. 2020. An overview of ongoing point cloud compression standardization activities: Video-based (V-PCC) and geometry-based (G-PCC). APSIPA Transactions on Signal and Information Processing, Vol. 9, pp. 1-17. doi: https://doi.org/10.1017/ATSIP.2020.12   DOI
33 DaeHyeok Gwon, Heeji Han, Haechul Choi , "Video based Point Cloud Compression with Versatile Video Coding" 2020 The Korean Institute of Broadcast and Media Engineers Summer Conference, pp. 375-377, July, 2020.
34 J. -B. Jeong, S. Lee, E. -S. Ryu. 2021. [MIV] Extraction and Merging on Frame Packed Video. 134th MPEG meeting of ISO/IEC JTC 1/SC 29/WG 4, MPEG/m56591.
35 S. Lee, J. -B. Jeong, I. Kim, S. Lee, E. -S. Ryu. 2020. Screen Content Coding Analysis to Improve Coding Efficiency for Immersive Video. Journal of Broadcast Engineering (JBE), Vol. 25, No. 6, pp. 911-921. doi: https://doi.org/10.5909/JBE.2020.25.6.911   DOI
36 L. Kondrad, V. K. M. Vadakital, L. Ilola. 2020. .CE-1.3: frame packed video sub-bitstream type. 131th MPEG meeting of ISO/IEC JTC 1/SC 29/WG 11, MPEG/m54274.
37 B. Salahieh, G. Naf, J. Boyce. 2021. Frame Packing Implementation in TMIV. 134th MPEG meeting of ISO/IEC JTC 1/SC 29/WG 4, MPEG/m56827.
38 J. -B. Jeong, S. Lee, I. -W. Ryu, T. T. Le, E. -S. Ryu. 2020. Towards Viewport-dependent 6dof 360 Video Tiled Streaming for Virtual Reality Systems. ACM International Conference on Multimedia 2020 (MM 2020), pp. 3687-3695. doi: https://doi.org/10.1145/3394171.3413712   DOI
39 B. Salahieh, J. Jung, A. Dziembowski, C. Bachhuber. 2021. Test Model 8 for MPEG Immersive Video. 133th MPEG meeting of ISO/IEC JTC 1/SC 29/WG 4, MPEG/n00050.
40 J. -B. Jeong, S. Lee, E. -S. Ryu. 2021. Sub-bitstream packing based lightweight tiled streaming for 6 degree of freedom immersive video. Electronics Letters, Vol. 57, No. 25, pp. 973-976. doi: https://doi.org/10.1049/ell2.12329   DOI
41 ISO/IEC JTC 1/SC 29/WG 11. 2018. Reference View Synthesizer (RVS) manual. 124th MPEG meeting of ISO/IEC JTC 1/SC 29/WG 11, MPEG/n18068.
42 S. Facheada, D. Bonatto, A. Schenkel, G. Lafruit. 2018. Depth image based view synthesis with multiple reference views for virtual reality. The True Vision-Capture, Transmission and Display of 3D Video 2018 (3DTV-CON 2018), pp. 1-4. doi: https://doi.org/10.1109/3DTV.2018.8478484   DOI
43 S. Kwak, J. Yun, J. Jeong, W. -S. Cheong, J. Seo. 2020. [MPEG-I Visual] Ray-based blending weight for 6DoF view synthesis. 131th MPEG meeting of ISO/IEC JTC 1/SC 29/WG 11, MPEG/m54409.
44 J. Jung, B. Kroon. 2021. Common Test Conditions for MPEG Immersive Video. 133th MPEG meeting of ISO/IEC JTC 1/SC 29/WG 4, MPEG/n00051.
45 D. Bonatto, S. Fachada, G. Lafruit. 2020. RaViS: Real-time accelerated View Synthesizer for immersive video 6DoF VR. Electronic Imaging, pp. 382-1-382-9. doi: https://doi.org/10.2352/ISSN.2470-1173.2020.13.ERVR-381   DOI
46 H. Ahn, M. -j. Lee. 2020. MPEG-I RVS Software Speed-up for Real-time Application. Journal of Broadcast Engineering (JBE), Vol. 25, No. 5, pp. 655-664. doi: https://doi.org/10.5909/JBE.2020.25.5.655   DOI
47 S. Kwak, J. Yun, J. -Y. Jeong, Y. Kim, I. Ihm, W. -S. Cheong, J. Seo. 2021. View synthesis with sparse light field for 6DoF immersive video. ETRI Journal, Vol. 44, No. 1, pp. 24-37. doi: https://doi.org/10.4218/etrij.2021-0205   DOI
48 J. -B. Jeong, S. Lee, E. -S. Ryu. 2021. On OpenGL-based View Synthesizer Optimization for Real-time 6DoF Immersive Video Streaming. IEEE P3079.1 Contribution (DCN: 3079-21-0057-00-0001).