Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Intra Prediction Using Multiple Models Based on Fully Connected Neural Network

다중 모델을 이용한 완전연결 신경망 기반 화면내 예측

  • Moon, Gihwa (Korea Aerospace University, School of Electronics and Information Engineering) ;
  • Park, Dohyeon (Korea Aerospace University, School of Electronics and Information Engineering) ;
  • Kim, Minjae (Korea Aerospace University, School of Electronics and Information Engineering) ;
  • Kwon, Hyoungjin (Electronics and Telecommunications Research Institute) ;
  • Kim, Jae-Gon (Korea Aerospace University, School of Electronics and Information Engineering)
  • 문기화 (한국항공대학교 항공전자정보공학부) ;
  • 박도현 (한국항공대학교 항공전자정보공학부) ;
  • 김민재 (한국항공대학교 항공전자정보공학부) ;
  • 권형진 (한국전자통신연구원) ;
  • 김재곤 (한국항공대학교 항공전자정보공학부)
  • Received : 2021.09.17
  • Accepted : 2021.11.05
  • Published : 2021.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, various research on the application of deep learning to video encoding for enhancing coding efficiency are being actively studied. This paper proposes a deep learning based intra prediction which uses multiple models by extending Matrix-based Intra Prediction(MIP) that is a neural network-based technology adopted in VVC. It also presents an efficient learning method for the multi-model intra prediction. To evaluate the performance of the proposed method, we integrated the VVC MIP and the proposed fully connected layer based multi-model intra prediction into HEVC reference software, HM16.19 as an additional intra prediction mode. As a result of the experiments, the proposed method can obtain bit-saving coding gain up to 0.47% and 0.19% BD-rate, respectively, compared to HM16.19 and VVC MIP.

최근 딥러닝 기술을 비디오 부호화에 적용하는 다양한 연구가 진행되고 있다. 본 논문은 차세대 비디오 코덱인 VVC(Versatile Video Coding)에 채택된 신경망 기반의 기술인 MIP(Matrix-based Intra Prediction)를 확장한 완전연결계층(Fully Connected Layer) 기반의 다중 모델을 이용하는 화면내 예측 부호화 기법을 제시한다. 또한 다중 화면내 예측 모델을 위한 효율적인 학습기법을 제안한다. HEVC(High Efficiency Video Coding)에서의 성능검증을 위해 VVC의 MIP와 제안하는 완전연결계층 기반 다중 화면내 예측 모델을 HEVC의 참조 소프트웨어인 HM16.19에 추가적인 화면내 예측모드로 구현하였다. 실험결과 제안하는 방법이 HM16.19와 VVC MIP 대비 각각 0.47%과 0.19% BD-rate 성능향상이 있음을 확인하였다.

Keywords

Acknowledgement

본 논문은 2021년도 정부(과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원을 받아 수행된 연구임(No. 2017-0-00072, 초실감 테라미디어를 위한 AV부호화 및 LF미디어 원천기술 개발).

References

  1. High Efficiency Video Coding, Version 1, Rec. ITU-T H.265, ISO/IEC 23008-2, Jan. 2013.
  2. Versatile Video Coding, ISO/IEC FDIS 23090-3, Jul. 2020.
  3. J. Chen, Y. Ye, S. Kim, "Algorithm description for Versatile Video Coding and Test Model 13 (VTM 13)," Joint Video Experts Team of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, JVET-V2002, Apr. 2021.
  4. S. Liu, E. Alshina, J. Pfaff, M. Wien, P. Wu and Y. Ye, "JVET AHG report: Neural-network-based video coding," Joint Video Experts Team of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, JVET-V0011, Apr. 2021.
  5. Alshina, S. Lui, W. Chen, F. Galpin, Y. Li, Z. Ma, H. Wang, "EE1: Summary of Exploration Experiments on Neural Network-based Video Coding," Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29, JVET-W0023, Jul. 2021.
  6. "Use cases and requirements for Deep Neural Networks based Video Coding," ISO/IEC JTC 1/SC 29/WG 2, N22, Oct. 2020.
  7. T. Dumas, A. Roumy and C. Guillemot, "Context Adaptive Neural Network Based Prediction for Image Compression," IEEE Trans. Image Proc., vol. 29, Aug. 2019.
  8. J. Li, B. Li, J. Xu and R. Xiong, "Intra prediction using fully connected network for video coding," In Proc. IEEE International Conference on Image Processing (ICIP) 2017, IEEE, Sept. 2017.
  9. P. Helle, J. Pfaff, M. Schafer, R. Rischke, H.Schwarz, D. Marpe, and T. Wiegand, "Intra Picture Prediction for Video Coding with Neural Networks," In Proc. DCC 2019, IEEE, Mar. 2019.
  10. T. Lin, M. Maire, S. Belongie, L. Bourdev, R. Girshick, J. Hays, P. Perona, D. Ramanan, C. Zitnick, and P. Dollar, "Microsoft COCO: Common Objects in Context," 2015, arXiv:1405.0312.
  11. J. Boyce, K. Suehring, X. Li, and V. Seregin, "JVET common test conditions and software reference configurations," Joint Video Experts Team (JVET) of ITU-T SG 16 WP 3 and ISO/IEC JTC 1/SC 29/WG 11, JVET-J1010, Apr. 2018.
  12. M. Kim, G. Moon, D. Park, H. Kwon and J. Kim, "Intra Prediction Using Multiple Models Based on Fully Connected Layer," In Proc. KIBME Annual Summer Conf. June. 2021.