Browse > Article
http://dx.doi.org/10.5909/JBE.2022.27.2.216

Group-based Adaptive Rendering for 6DoF Immersive Video Streaming  

Lee, Soonbin (Department of Computer Science Education, Sungkyunkwan University)
Jeong, Jong-Beom (Department of Computer Science Education, Sungkyunkwan University)
Ryu, Eun-Seok (Department of Computer Science Education, Sungkyunkwan University)
Publication Information
Journal of Broadcast Engineering / v.27, no.2, 2022 , pp. 216-227 More about this Journal
Abstract
The MPEG-I (Immersive) group is working on a standardization project for immersive video that provides 6 degrees of freedom (6DoF). The MPEG Immersion Video (MIV) standard technology is intended to provide limited 6DoF based on depth map-based image rendering (DIBR) technique. Many efficient coding methods have been suggested for MIV, but efficient transmission strategies have received little attention in MPEG-I. This paper proposes group-based adaptive rendering method for immersive video streaming. Each group can be transmitted independently using group-based encoding, enabling adaptive transmission depending on the user's viewport. In the rendering process, the proposed method derives weights of group for view synthesis and allocate high quality bitstream according to a given viewport. The proposed method is implemented through the Test Model for Immersive Video (TMIV) test model. The proposed method demonstrates 17.0% Bjontegaard-delta rate (BD-rate) savings on the peak signalto-noise ratio (PSNR) and 14.6% on the Immersive Video PSNR(IV-PSNR) in terms of various end-to-end evaluation metrics in the experiment.
Keywords
Immersive Video; MPEG-I; Adaptive Streaming; TMIV; 6DoF;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 S. Lee, J. -B. Jeong and E. -S. Ryu, "Efficient Group-Based Packing Strategy for 6DoF Immersive Video Streaming," 2022 International Conference on Information Networking (ICOIN), 2022, pp. 310-314. doi: https://doi.org/10.1109/ICOIN53446.2022.9687139.   DOI
2 Jeong, J.-B., Lee, S. and Ryu, E.-S. (2021), Sub-bitstream packing based lightweight tiled streaming for 6 degree of freedom immersive video. Electron. Lett., 57: 973-976. doi: https://doi.org/10.1049/ell2.12329   DOI
3 J. Son, D. Jang, E-S. Ryu. "Implementing Motion-Constrained Tile and Viewport Extraction for VR Streaming." In Proceedings of the 28th ACM Network and Operating System Support for Digital Audio and Video (NOSSDAV'18), pp. 61-66, 2018. doi: https://doi.org/10.1145/3210445.3210455   DOI
4 ISO/IEC JTC1/SC29/WG11/N19485, "Description of MPEG Immersive Video Core Experiments 1", MPEG 131, Jul. 2020.
5 S. Lee, D. Jang, J-B. Jeong, E-S. Ryu. "Motion-constrained tile set based 360-degree video streaming using saliency map prediction." In Proceedings of the 29th ACM Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV'19), pp. 20-24, 2019. doi: https://doi.org/10.1145/3304112.3325614   DOI
6 ISO/IEC JTC1/SC29/WG11/M41555, "Proposed architectures for supporting Windowed 6DoF, Omnidirectional 6DoF and 6DoF media", MPEG 120, Oct. 2017.
7 ISO/IEC JTC1/SC29/WG11/N19491, "Exploration Experiments on Coding for Future MPEG Immersive Video", MPEG 130, Apr. 2020.
8 ISO/IEC JTC1/SC29/WG11/M54151, "Grouping and anchor study on MIV content", MPEG 131, Jun, 2020.
9 J. Jeong, S. Lee, I. Ryu, T. T. Le, and E. Ryu. "Towards Viewportdependent 6DoF 360 Video Tiled Streaming for Virtual Reality Systems." In Proceedings of the 28th ACM International Conference on Multimedia (MM '20), pp. 3687-3695, 2020. doi: https://doi.org/10.1145/3394171.3413712   DOI
10 ISO/IEC JTC1/SC29/WG11/N19484, "Common Test Conditions for MPEG Immersive Video," MPEG 131, Jul. 2020.
11 ISO/IEC JTC1/SC29/WG4/N0013, "Software manual of IV-PSNR for Immersive Video," MPEG 132, Oct. 2020.
12 ISO/IEC JTC1/SC29/WG11/M49406, "Group-Based TMIV", MPEG 127, Jul. 2019.
13 Kwak, S., Yun, J., Jeong, J.-Y., Kim, Y., Ihm, I., Cheong, W.-S., and Seo, J., View synthesis with sparse light field for 6DoF immersive video, ETRI Journal 44 (2022), 24-37. doi: https://doi.org/10.4218/etrij.2021-0205   DOI
14 ISO/IEC JTC1/SC29/WG11, "New Work Item Proposal on Coded Representation of Immersive Media," MPEG 117, N16541, Jan.2017.
15 ISO/IEC JTC1/SC29/WG11/N18145, "Call for Proposals on 3DoF+Visual", MPEG 125, Jan. 2019.
16 ISO/IEC JTC1/SC29/WG11/N19492, "Exploration Experiments on Processing for Future MPEG Immersive Video", MPEG 131, Jul. 2020.
17 ISO/IEC JTC1/SC29/WG11/M49590, Video-based Point Cloud Coding High Level Syntax: Updates and Unification with the Working Draft on Metadata for Immersive Video", MPEG 127, Jul. 2019.
18 X. Corbillon, G. Simon, A. Devlic and J. Chakareski, "Viewportadaptive navigable 360-degree video delivery," 2017 IEEE International Conference on Communications (ICC), 2017, pp. 1-7. doi: https://doi.org10.1109/ICC.2017.7996611.   DOI
19 ISO/IEC JTC1/SC29/WG11/M47749, "[V-PCC] A tile group design for V-PCC", A. M. Tourapis, J. Kim, K. Mammou, MPEG 126, Mar. 2019.
20 ISO/IEC JTC1/SC29/WG11/N18654, ,"Requirements for Immersive Media Access and Delivery", MPEG 127, Jul. 2019.
21 ISO/IEC JTC1/SC29/WG11/M54409, "[MPEG-I Visual] Ray-based blending weight for 6DoF view synthesis," MPEG 131, 2020.
22 ISO/IEC JTC1/SC29/WG11/N19483, "Test Model 6 for MPEG Immersive Video," MPEG 131, Jul. 2020.
23 ISO/IEC JTC1/SC29/WG4/M57726, "AhG on MPEG Immersive Video", MPEG 136, Oct. 2021.
24 ISO/IEC JTC1/SC29/WG4/N0145, "Exploration Experiments on Future MPEG Immersive video", MPEG 136, Oct. 2021.
25 J. Jeong, S. Lee, D. Jang and E. Ryu, "Towards 3DoF+ 360 Video Streaming System for Immersive Media," in IEEE Access, vol. 7, pp. 136399-136408, 2019. doi: https://doi.org/10.1109/ACCESS.2019.2942771.   DOI