Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Towards Group-based Adaptive Streaming for MPEG Immersive Video

MPEG Immersive Video를 위한 그룹 기반 적응적 스트리밍

  • Jong-Beom Jeong (Department of Computer Science Education, Sungkyunkwan University) ;
  • Soonbin Lee (Department of Computer Science Education, Sungkyunkwan University) ;
  • Jaeyeol Choi (Department of Computer Science Education, Sungkyunkwan University) ;
  • Gwangsoon Lee (Electronics and Telecommunications Research Institute) ;
  • Sangwoon Kwak (Electronics and Telecommunications Research Institute) ;
  • Won-Sik Cheong (Electronics and Telecommunications Research Institute) ;
  • Bongho Lee (Electronics and Telecommunications Research Institute) ;
  • Eun-Seok Ryu (Department of Computer Science Education, Sungkyunkwan University)
  • Received : 2023.02.03
  • Accepted : 2023.03.03
  • Published : 2023.03.30
Download PDF
⟨ Previous Next ⟩

Abstract

The MPEG immersive video (MIV) coding standard achieved high compression efficiency by removing inter-view redundancy and merging the residuals of immersive video which consists of multiple texture (color) and geometry (depth) pairs. Grouping of views that represent similar spaces enables quality improvement and implementation of selective streaming, but this has not been actively discussed recently. This paper introduces an implementation of group-based encoding into the recent version of MIV reference software, provides experimental results on optimal views and videos per group, and proposes a decision method for optimal number of videos for global immersive video representation by using portion of residual videos.

다수의 색상 및 거리 순서쌍으로 구성된 몰입형 영상 압축을 위한 MPEG immersive video (MIV) 표준은 시점 간 중복 영역 제거 후 잔여 영상을 병합하여 높은 압축률을 확보하였다. 비슷한 영역을 표현하는 시점 간 그룹화를 통해 품질 향상 및 선택적 스트리밍 구현이 가능하나, 최근 그룹 기반 MIV 부호화 기술은 활발히 논의되고 있지 않다. 본 논문은 최신 MIV 참조 소프트웨어에서 그룹 기반 부호화 기술을 이식하고, 최적의 그룹 별 시점 및 영상 개수 산출을 위한 실험을 진행하였으며, 출력 영상 내 잔여 영상의 비율을 기반으로 전역적 영상 표현을 위한 최적의 출력 영상 수를 결정하는 기법을 제안한다.

Keywords

Acknowledgement

This work was supported by Electronics and Telecommunications Research Institute (ETRI) grant funded by ICT R&D program of MSIT/IITP[2022-0-00022-001, Development of immersive video spatial computing technology for ultra-realistic metaverse services]

References

  1. E. S. Jang, M. Preda, K. Mammou, A. M. Tourapis, J. Kim, D. B. Graziosi, S. Rhyu, M. Budagavi. 2019. Video-based point-cloud-compression standard in mpeg: From evidence collection to committee draft [standards in a nutshell]. IEEE Signal Processing Magazine, vol. 36, no. 3, pp. 118-123. doi: https://doi.org/10.1109/MSP.2019.2900721
  2. S. Schwarz, M. Pesonen. 2019. Real-time decoding and AR playback of the emerging MPEG video-based point cloud compression standard. Nokia Technologies; IBC: Helsinki, Finland.
  3. J. M. Boyce, R. Dore, A. Dziembowski, J. Fleureau, J. Jung, B. Kroon, B. Salahieh, V. K. M. Vadakital, L. Yu. 2021. MPEG immersive video coding standard. Proceedings of the IEEE, vol. 109, no. 9, pp. 1521-1536. doi: https://doi.org/10.1109/JPROC.2021.3062590
  4. J. -B. Jeong, S. Lee, E. -S. Ryu. 2022. Rethinking Fatigue-Aware 6DoF Video Streaming: Focusing on MPEG Immersive Video. International Conference on Information Networking 2022 (ICOIN2022), pp. 304-309. doi: https://doi.org/10.1109/ICOIN53446.2022.9687247
  5. Sangho Lee, Hongchang Shin, Gwangsoon Lee, Jeongil Seo. 2022. GPU Implementation of TMIV Decoder for Real-time Playback. The Korean Institute of Broadcast and Media Engineers (KIBME) Summer Conference, pp. 93-96.
  6. G. J. Sullivan, J. -R. Ohm, W. -J. Han, T. Wiegand. 2012. Overview of the High Efficiency Video Coding (HEVC) Standard. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1649-1668. doi: https://doi.org/10.1109/TCSVT.2012.2221191
  7. B. Bross, Y. -K. Wang, Y. Ye, S. Liu, J. Chen, G. J. Sullivan, J. -R. Ohm. 2021. Overview of the versatile video coding (VVC) standard and its applications. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 31, No. 10, pp. 3736-3764. doi: https://doi.org/10.1109/TCSVT.2021.3101953
  8. Y. Bai, X. Sheng, S. Li, C. Wang, L. Yu. 2021. [MIV]Undistorted CBA Basketball Test Sequence for MPEG-I Visual, Standard ISO/IEC JTC1/SC29/WG4, m58500.
  9. ISO/IEC JTC 1/SC 29/WG 7. 2021. Text of ISO/IEC DIS 23090-5 Visual Volumetric Video-based Coding and Video-based Point Cloud Compression 2nd Edition. Standard ISO/IEC JTC 1/SC 29 WG 7, n00188.
  10. ISO/IEC JTC 1/SC 29/WG 4. 2022. Preliminary WD1 of ISO/IEC 23090-12 MPEG Immersive video Ed. 2. Standard ISO/IEC JTC 1/SC 29/WG 4, n00176.
  11. D. Park, S. -G. Lim, K. -J. Oh, G. Lee, J. -G. Kim. 2022. Nonlinear Depth Quantization Using Piecewise Linear Scaling for Immersive Video Coding. IEEE Access, Vol. 10, pp. 4483-4494. doi: https://doi.org/10.1109/ACCESS.2022.3140537
  12. J. -B. Jeong, S. Lee, E. -S. Ryu. 2021. Sub-bitstream packing based lightweight tiled streaming for 6 degree of freedom immersive video. Electronics Letters, Vol. 57, No. 25, pp. 973-976. doi: https://doi.org/10.1049/ell2.12329
  13. J. -B. Jeong, S. Lee, E. -S. Ryu. 2021. DWS-BEAM: Decoder-Wise Subpicture Bitstream Extracting and Merging for MPEG Immersive Video. International Conference on Visual Communications and Image Processing 2021 (VCIP2021), pp. 1-5. doi: https://doi.org/10.1109/VCIP53242.2021.9675419
  14. H. -C. Shin, J. -Y. Jeong, G. Lee, M. U. Kakli, J. Yun, J. Seo. 2022. Enhanced pruning algorithm for improving visual quality in MPEG immersive video. ETRI Journal, Vol. 44, No. 1, pp. 73-84. doi: https://doi.org/10.4218/etrij.2021-0211
  15. S. Kwak, J. Yun, J. -Y. Jeong, Y. Kim, I. Ihm, W. -S. Cheong, J. Seo. 2022. View synthesis with sparse light field for 6DoF immersive video. ETRI Journal, Vol. 44, No. 1, pp. 24-37. doi: https://doi.org/10.4218/etrij.2021-0205
  16. B. Salahieh, S. Bhatia, J. Boyce. 2019. Group-based TMIV. Standard ISO/IEC JTC1/SC29/WG11, MPEG/m49406.
  17. S. Lee, J. -B. Jeong, E. -S. Ryu. 2022. Group-based Adaptive Rendering for 6DoF Immersive Video Streaming. Journal of Broadcast Engineering, Vol. 27, No. 2, pp. 216-227. doi: https://doi.org/10.5909/JBE.2022.27.2.216
  18. S. Lee, J. -B. Jeong, E. -S. Ryu. 2022. Efficient Group-Based Packing Strategy for 6DoF Immersive Video Streaming. International Conference on Information Networking 2022 (ICOIN2022), pp. 310-314. doi: https://doi.org/10.1109/ICOIN53446.2022.9687139
  19. B. Salahieh, B. Kroon, J. Jung, M. Domanski. 2019. Test Model for Immersive Video. Standard ISO/IEC JTC 1/SC 29 WG 11, MPEG/n18470.
  20. V. K. M. Vadakital. 2019. Description of Immersive Video Core Experiments 1. Standard ISO/IEC JTC1/SC29/WG11, MPEG/n18465.
  21. B. Salahieh, J. Fleureau, R. Dore, J. Boyce. 2020. Interactions Between Grouping, Pruning, and Synthesizing. Standard ISO/IEC JTC 1/SC 29 WG 11, MPEG2020/m52824.
  22. T. T. Le, J. -B. jeong, S. S. Lee, J. Kim, E. -S. Ryu. 2021. An Efficient Viewport-Dependent 360 VR System Based on Adaptive Tiled Streaming. Computers, Materials & Continua, Vol. 66, No. 3, pp. 2627-2643. doi: https://doi.org/10.32604/cmc.2021.013399
  23. J. -B. Jeong, S. Lee, I. Kim, E. -S. Ryu. 2021. Implementing Viewport Tile Extractor for Viewport-Adaptive 360-Degree Video Tiled Streaming. International Conference on Information Networking 2021 (ICOIN2021), pp. 8-12. doi: https://doi.org/10.1109/ICOIN50884.2021.9333964
  24. B. Sonneveldt, B. Kroon. 2021. Philips CE-2.10 response "Server-side Inpainting 2". Standard ISO/IEC JTC 1/SC 29 WG 4, MPEG/m55736.
  25. J. Jung, B. Kroon. 2022. Common Test Conditions for MPEG Immersive Video. Standard ISO/IEC JTC 1/SC 29/WG 4, n00232.
  26. K. -J. Oh, J. Y. Oh, T. Dong, X. Li, G. Lee, E. S. Jang. 2022. [EE-7 related] Atlas cropping-based TMIV encoding results. Standard ISO/IEC JTC 1/SC 29/WG 4, m60274.
  27. A. Dziembowski, D. Mieloch, J. Stankowski, A. Grzelka. 2022. IV-PSNR-The objective quality metric for immersive video applications. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 32, No. 11, pp. 7575-7591. doi: https://doi.org/10.1109/TCSVT.2022.3179575