Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Screen Content Coding Analysis to Improve Coding Efficiency for Immersive Video

몰입형 비디오 압축을 위한 스크린 콘텐츠 코딩 성능 분석

  • Lee, Soonbin (Department of Computer Education, Sungkyunkwan University) ;
  • Jeong, Jong-Beom (Department of Computer Education, Sungkyunkwan University) ;
  • Kim, Inae (Department of Computer Education, Sungkyunkwan University) ;
  • Lee, Sangsoon (Department of Computer Engineering, Gachon University) ;
  • Ryu, Eun-Seok (Department of Computer Education, Sungkyunkwan University)
  • 이순빈 (성균관대학교 컴퓨터교육과) ;
  • 정종범 (성균관대학교 컴퓨터교육과) ;
  • 김인애 (성균관대학교 컴퓨터교육과) ;
  • 이상순 (가천대학교 컴퓨터공학과) ;
  • 류은석 (성균관대학교 컴퓨터교육과)
  • Received : 2020.09.21
  • Accepted : 2020.11.12
  • Published : 2020.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, MPEG-I (Immersive) has been exploring compression performance through standardization projects for immersive video. The MPEG Immersion Video (MIV) standard technology is intended to provide limited 6DoF based on depth map-based image rendering (DIBR). MIV is a model that processes the Basic View and the residual information into an Additional View, which is a collection of patches. Atlases have the unique characteristics depending on the kind of the view they are included, requiring consideration of the compression efficiency. In this paper, the performance comparison analysis of screen content coding tools such as intra block copy (IBC) is conducted, based on the pattern of various views and patches repetition. It is demonstrated that the proposed method improves coding performance around -15.74% BD-rate reduction in the MIV.

최근 MPEG-I (Immersive) 그룹에서는 몰입형 비디오(Immersive Video)에 대한 표준화 프로젝트를 통해 압축 성능 탐색을 진행하고 있다. MIV(MPEG Immersive Video) 표준 기술은 다수의 시점 영상과 깊이 맵을 통한 깊이 맵 기반 이미지 렌더링(DIBR)을 바탕으로 제한적인 6DoF을 제공하고자 하는 기술이다. 현재 MIV에서는 바탕 시점(Basic View)과 각 시점의 고유한 영상 정보를 패치 단위로 모아둔 추가 시점(Additional View)으로 처리하는 모델을 채택하고 있다. MIV에서 생성된 아틀라스는 포함되는 시점의 성격에 따라 다른 영상의 특성을 나타내어 비디오 코덱의 압축 효율에 대한 고찰이 필요하다. 따라서 본 논문에서는 다양한 시점과 패치들이 반복되는 패턴에 착안하여 화면 내 블록 카피(IBC: intra block copy) 등의 압축 기법이 포함된 스크린 콘텐츠 코딩 툴에 대한 성능 비교 분석을 진행하여 복원 영상에서 최대 -15.74% Peak Signal-to-Noise Ratio (PSNR) 관점에서의 부호화 성능 향상을 제공하였다.

Keywords

References

  1. ISO/IEC JTC1/SC29/WG11/M41555, "Proposed architectures for supporting Windowed 6DoF, Omnidirectional 6DoF and 6DoF media", MPEG 120, Oct. 2017.
  2. ISO/IEC JTC1/SC29/WG11/N19491, "Exploration Experiments on Coding for Future MPEG Immersive Video", MPEG 130, Apr. 2020.
  3. ISO/IEC JTC1/SC29/WG11/N19492, "Exploration Experiments on Processing for Future MPEG Immersive Video", MPEG 131, Jul. 2020.
  4. ISO/IEC JTC1/SC29/WG11/N19485, "Description of MPEG Immersive Video Core Experiments 1", MPEG 131, Jul. 2020.
  5. ISO/IEC JTC1/SC29/WG11/N18145, "Call for Proposals on 3DoF+ Visual", MPEG 125, Jan. 2019.
  6. ISO/IEC JTC1/SC29/WG11/M52994, "Proposed simplifications of MIV", MPEG 130, Apr. 2020.
  7. ISO/IEC JTC1/SC29/WG11/M49590, Video-based Point Cloud Coding High Level Syntax: Updates and Unification with the Working Draft on Metadata for Immersive Video", MPEG 127, Jul. 2019.
  8. ISO/IEC JTC1/SC29/WG11/M47544, "Description of Technicolor Intel response to MPEG-I 3DoF+ Call for Proposal", MPEG 126, Mar. 2019.
  9. ISO/IEC JTC1/SC29/WG11/M47407, "Technical description of proposal for Call for Proposals on 3DoF+ Visual prepared by PUT and ETRI", MPEG 126, Mar. 2019.
  10. ISO/IEC JTC1/SC29/WG11/M47372, "Description of Nokia's response to CFP for 3DOF+ visual", MPEG 126, Mar. 2019.
  11. ISO/IEC JTC1/SC29/WG11/M47684, "Description of Zhejiang University's response to 3DoF+ Visual CfP", MPEG 126, Mar. 2019.
  12. ISO/IEC JTC1/SC29/WG11/M47179, "Philips response to CfP on 3DoF", MPEG 126, Mar. 2019.
  13. ISO/IEC JTC1/SC29/WG11/N19484, "Test Model 6 for MPEG Immersive Video", MPEG 131, Jul. 2020.
  14. ISO/IEC JTC1/SC29/WG11/M49406, "Group-Based TMIV", MPEG 127, Jul. 2019.
  15. ISO/IEC JTC1/SC29/WG11/M52475, "Object-Based Implementation in TMIV", MPEG 129, Jan. 2020.
  16. ISO/IEC JTC1/SC29/WG11/M51604, "Immersive Video CE3-related: Spatio-temporal patch redundancy removal", MPEG 129, Jan. 2020.
  17. ISO/IEC JTC1/SC29/WG11/M53701, "CE-3: additional patch dilation in temporal patch redundancy removal", MPEG 130, Apr. 2020.
  18. ISO/IEC JTC1/SC29/WG11/M54145, "[MIV] Philips CE-2 related response "basic view allocator"", MPEG 131, Jun. 2020.
  19. "Overview of the Emerging HEVC Screen Content Coding Extension," in IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 1, pp. 50-62, Jan. 2016. https://doi.org/10.1109/TCSVT.2015.2478706
  20. ISO/IEC JTC1/SC29/WG11/N19484, "Common Test Conditions for MPEG Immersive Video", MPEG 131, Jul. 2020.
  21. ISO/IEC JTC1/SC29/WG11/M53427, "[MPEG-I Visual] HEVC-SCC in TMIV", MPEG 130, Apr. 2020.
  22. ISO/IEC JTC1/SC29/WG11/N19495, "Software manual of IV-PSNR for Immersive Video", MPEG 131, Jul. 2020.
  23. X. Xu, X. Li and S. Liu, "Intra block copy in Versatile Video Coding with Reference Sample Memory Reuse," Picture Coding Symposium (PCS), Ningbo, China, 2019.
  24. J. Jeong, S. Lee, D. Jang and E. Ryu, "Towards 3DoF+ 360 Video Streaming System for Immersive Media," in IEEE Access, vol. 7, pp. 136399-136408, 2019. https://doi.org/10.1109/ACCESS.2019.2942771
  25. J. Jeong, S. Lee, I. Ryu, T. T. Le, and E. Ryu. "Towards Viewport-dependent 6DoF 360 Video Tiled Streaming for Virtual Reality Systems." In Proceedings of the 28th ACM International Conference on Multimedia (MM '20), pp. 3687-3695, 2020.