방송공학회논문지 (Journal of Broadcast Engineering)

  • 제27권6호
  • /
  • Pages.861-871
  • /
  • 2022
  • /
  • 1226-7953(pISSN)
  • /
  • 2287-9137(eISSN)
한국방송∙미디어공학회 (The Korean Institute of Broadcast and Media Engineers)
권호 표지
DOI QR코드

DOI QR Code

MPEG 몰입형 비디오를 위한 Geometry Packing 구현

Implementing Geometry Packing for MPEG Immersive Video

  • 정종범 (성균관대학교 컴퓨터교육학과) ;
  • 이순빈 (성균관대학교 컴퓨터교육학과) ;
  • 류은석 (성균관대학교 컴퓨터교육학과)
  • Jong-Beom, Jeong (Department of Computer Science Education, Sungkyunkwan University) ;
  • Soonbin, Lee (Department of Computer Science Education, Sungkyunkwan University) ;
  • Eun-Seok, Ryu (Department of Computer Science Education, Sungkyunkwan University)
  • 투고 : 2022.09.07
  • 심사 : 2022.11.04
  • 발행 : 2022.11.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

실사 및 컴퓨터 그래픽을 표현하는 다수의 몰입형 영상을 효율적으로 부호화하는 표준으로 moving picture experts group (MPEG)에서는 MPEG immersive video (MIV) 표준을 개발하였다. MIV 표준은 다수의 몰입형 영상을 압축하여 다수의 출력 영상인 아틀라스를 생성하나, 다수의 아틀라스를 부호화 후 복호화 시 저사양 장비에서 복호기 간 동기화 문제가 발생할 수 있다. 본 논문은 저사양 및 고사양 장비에 모두 대응하여 적응적 복호기 개수 조절을 위한 geometry packing 기법을 제안하고 구현한다. 제안하는 방법은 MIV의 최신 참조 소프트웨어에서 문제없이 동작함을 확인하였다.

The moving picture experts group (MPEG) developed the MPEG immersive video (MIV) standard for efficient compression of multiple immersive videos representing natural contents and computer graphics. The MIV compresses multiple immersive videos and generates multiple output videos which are defined as atlases. However, there is a synchronization issue of multiple decoders in a legacy device when decoding multiple encoded atlases. This paper proposes and implements the geometry packing method for adaptive control of decoder instances for low-end and high-end devices. The proposed method on the recent version of the MIV reference software worked correctly.

키워드

과제정보

This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No.RS-2022-00167169, Development of Moving Robot-based Immersive Video Acquisition and Processing System in Metaverse)

참고문헌

  1. Oculus Quest 2 Specification, [Online]. Available at: https://developer.oculus.com/resources/oculus-device-specs/.
  2. G. J. Sullivan, J. -R. Ohm, W. -J. Han, T. Wiegand. 2012. Overview of the High Efficiency Video Coding (HEVC) Standard. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp. 1649-1668. doi: https://doi.org/10.1109/TCSVT.2012.2221191
  3. B. Bross, Y. -K. Wang, Y. Ye, S. Liu, J. Chen, G. J. Sullivan, J. -R. Ohm. 2021. Overview of the versatile video coding (VVC) standard and its applications. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 31, No. 10, pp. 3736-3764. doi: https://doi.org/10.1109/TCSVT.2021.3101953
  4. J. M. Boyce, R. Dore, A. Dziembowski, J. Fleureau, J. Jung, B. Kroon, B. Salahieh, V. K. M. Vadakital, L. Yu. 2021. MPEG immersive video coding standard. Proceedings of the IEEE, Vol. 109, No. 9, pp. 1521-1536. doi: https://doi.org/10.1109/JPROC.2021.3062590
  5. ISO/IEC JTC 1/SC 29/WG 7. 2021. Text of ISO/IEC DIS 23090-5 Visual Volumetric Video-based Coding and Video-based Point Cloud Compression 2nd Edition. Standard ISO/IEC JTC 1/SC 29 WG 7, MPEG/n0188.
  6. P. Hanhart, X. Xiu, Y. He, Y. Ye. 2019. 360 video coding based on projection format adaptation and spherical neighboring relationship. IEEE Journal on Emerging and Selected Topics in Circuits and Systems, Vol. 9, No. 1, pp. 71-83. doi: https://doi.org/10.1109/JETCAS.2018.2888960
  7. B. Salahieh, G. Naf, J. Boyce. 2021. "Frame Packing Implementation in TMIV." Standard ISO/IEC JTC 1/SC 29/WG 4, MPEG/m56827.
  8. J. -B. Jeong, S. Lee, E. -S. Ryu. 2021. "Sub-bitstream packing based lightweight tiled streaming for 6 degree of freedom immersive video". Electronics Letters, Vol. 57, No. 25, pp. 973-976. doi: https://doi.org/10.1049/ell2.12329
  9. J. -B. Jeong, S. Lee, E. -S. Ryu. 2021. "DWS-BEAM: Decoder-Wise Subpicture Bitstream Extracting and Merging for MPEG Immersive Video". International Conference on Visual Communications and Image Processing 2021 (VCIP2021), pp. 1-5. doi: https://doi.org/10.1109/VCIP53242.2021.9675419
  10. J. -B. Jeong, S. Lee, E. -S. Ryu. 2022. "[MIV] Geometry Packing Implementation in TMIV for Frame Packed Video". Standard ISO/IEC JTC 1/SC 29 WG 4, MPEG/m59442.
  11. B. Salahieh, J. Jung, A. Dziembowski. 2022. Test Model 14 for MPEG immersive video. Standard ISO/IEC JTC 1/SC 29 WG 4, MPEG/n0242.
  12. M. M. Hannuksela, Y. Yan, X. Huang, H. Li. 2015. Overview of the multiview high efficiency video coding (MV-HEVC) standard. IEEE International Conference on Image Processing 2015 (ICIP2015), pp. 2154-2158. doi: https://doi.org/10.1109/ICIP.2015.7351182
  13. J. Jung, B. Kroon. 2022. Common Test Conditions for MPEG Immersive Video. Standard ISO/IEC JTC 1/SC 29/WG 4, MPEG/n0232.
  14. J. Fleureau, F. Thudor, T. Tapie, R. Gendrot, G. Briand, R. Dore. 2021. [MIV] MIV / MPI on Android. Standard ISO/IEC JTC 1/SC 29/WG 4, MPEG/m56727.
  15. J. Boyce, B. Salahieh. 2020. Frame Packing and Extended MIV Profile. Standard ISO/IEC JTC 1/SC 29/WG 11, MPEG/m54491.
  16. B. Wang, L. Yu, B. Kroon, and J. Jung. 2018. [MPEG-I Visual] Results on depth QPs in CTC of 3DoF+ Video. Standard ISO/IEC JTC 1/SC 29/WG 11, MPEG/m44688.
  17. HEVC test model (HM), [Online]. Available at: https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSoftware/.
  18. M. Viitanen, A. Koivula, A. Lemmetti, A. Y. Outinen, J. Vanne, T. D. Hamalainen. 2016. Kvazaar: open-source HEVC/H. 265 encoder. Proceedings of the 24th ACM international conference on Multimedia (ACM MM), pp. 1179-1182. doi: http://dx.doi.org/10.1145/2964284.2973796
  19. VVC test model (VTM), [Online]. Available at: https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM.
  20. A. Wieckowski, J. Brandenburg, T. Hinz, C. Bartnik, V. George, G. Hege, C. Helmrich, A. Henkel, C. Lehmann, C. Stoffers, I. Zupancic, B. Bross, D. Marpe. 2021. VVenC: An open and optimized VVC encoder implementation. Proceedings of 2021 IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp. 1-2. doi: https://doi.org/10.1109/ICMEW53276.2021.9455944
  21. B. Salahieh, S. Bhatia, J. Boyce. 2019. Grouping Implementation in TMIV. Standard ISO/IEC JTC 1/SC 29/WG 11, MPEG/m49859.
  22. S. Lee, J. -B. Jeong, E. -S. Ryu. 2022. Group-based Adaptive Rendering for 6DoF Immersive Video Streaming. Journal of Broadcast Engineering, Vol. 27, No. 2, pp. 216-227. doi: https://doi.org/10.5909/JBE.2022.27.2.216
  23. A. Hallapuro, M. M. Hannuksela. 2020. AHG3/AHG12: Subpicture merging software. Standard ITU-T SG 16 WP3 ISO/IEC JTC 1/SC 29/WG 11, JVETS0162.
  24. HEVC Tile Merger, [Online]. Available at: https://gitlab.com/MCSLJeong/hevctilemerger.
  25. Kvazaar-compliant HEVC Tile Extractor, [Online]. Available at: https://gitlab.com/MCSLJeong/hevctileextractor.
  26. J. -B. Jeong, S. Lee, E. -S. Ryu. 2022. Performance Analysis of 6DoF Video Streaming Based on MPEG Immersive Video. Journal of Broadcast Engineering (JBE), Vol. 27, No. 5, pp. 773-793. doi: https://doi.org/10.5909/JBE.2022.27.5.773
  27. A. Dziembowski, D. Mieloch, J. Stankowski, A. Grzelka. 2022. IV-PSNR-The objective quality metric for immersive video applications. IEEE Transactions on Circuits and Systems for Video Technology, Vol. 32, No. 11, pp. 7575-7591. doi: https://doi.org/10.1109/TCSVT.2022.3179575