Journal of Broadcast Engineering (방송공학회논문지)

The Korean Institute of Broadcast and Media Engineers (한국방송∙미디어공학회)
권호 표지
DOI QR코드

DOI QR Code

Survey on Deep learning-based Content-adaptive Video Compression Techniques

딥러닝 기반 컨텐츠 적응적 영상 압축 기술 동향

  • Han, Changwoo (Department of Electrical Engineering, Korea University) ;
  • Kim, Hongil (School of Electronics and Electrical Engineering, Dongguk University) ;
  • Kang, Hyun-ku (Department of Electrical Engineering, Korea University) ;
  • Kwon, Hyoungjin (Telecommunications & Media Research Laboratory, Electronics and Telecommunications Research Institute) ;
  • Lim, Sung-Chang (Telecommunications & Media Research Laboratory, Electronics and Telecommunications Research Institute) ;
  • Jung, Seung-Won (Department of Electrical Engineering, Korea University)
  • 한창우 (고려대학교 전기전자공학과) ;
  • 김홍일 (동국대학교 전기전자공학부) ;
  • 강현구 (고려대학교 전기전자공학과) ;
  • 권형진 (한국전자통신연구원 통신미디어연구소) ;
  • 임성창 (한국전자통신연구원 통신미디어연구소) ;
  • 정승원 (고려대학교 전기전자공학과)
  • Received : 2022.05.19
  • Accepted : 2022.07.04
  • Published : 2022.07.30
Download PDF
⟨ Previous Next ⟩

Abstract

As multimedia contents demand and supply increase, internet traffic around the world increases. Several standardization groups are striving to establish more efficient compression standards to mitigate the problem. In particular, research to introduce deep learning technology into compression standards is actively underway. Despite the fact that deep learning-based technologies show high performance, they suffer from the domain gap problem when test video sequences have different characteristics of training video sequences. To this end, several methods have been made to introduce content-adaptive deep video compression. In this paper, we will look into these methods by three aspects: codec information-aware methods, model selection methods, and information signaling methods.

멀티미디어 컨텐츠의 수요와 공급이 증가함에 따라 전 세계의 인터넷 트래픽이 증가하는 가운데 이를 완화하기 위해 여러 표준화 그룹에서는 더 효율적인 압축 표준을 제정하는데 노력을 기울이고 있다. 이러한 노력 중 압축 표준에 딥러닝 기술을 도입하고자 하는 연구들이 활발히 진행되고 있다. 그러나 딥러닝 기반 압축 기술은 학습 데이터와 특성이 다른 영상을 압축할 때 압축 효율이 저하되는 문제를 갖는다. 이를 해결하기 위해 컨텐츠에 적응적으로 딥러닝 기술을 도입하는 시도들이 있었다. 본 논문에서는 이들을 크게 코덱 정보 사용, 모델 선택, 추가 정보 전송의 세 가지로 나누어 살펴보고자 한다.

Keywords

Acknowledgement

이 논문은 2022년도 정부(과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원을 받아 수행된 연구임 (No. 2017-0-00072, 초실감 테라미디어를 위한 AV 부호화 및 LF 미디어 원천기술 개발).

References

  1. Cisco, Cisco Annual Internet Report (2018-2023) White Paper, Mar. 2020.
  2. A. Skodras, C. Christopoulos and T. Ebrahimi, "The JPEG 2000 still image compression standard," Signal Processing Magazine, Vol.18, No.5, pp 36-58, 2001. doi: https://doi.org/10.1109/79.952804
  3. B. Bross, J. Chen, S. Liu and Y.-K. Wang, "Versatile video coding (Draft 10)," JVET-S2001, Jul. 2020.
  4. Wei Jia, et al "Residual-guided In-loop Filter Using Convolution Neural Network," ACM Trans. Multimedia Comput. Communications, and Applications, 2021 doi: https://doi.org/10.1145/3460820
  5. Li, Daowen, and Lu Yu. "An in-loop filter based on low-complexity CNN using residuals in intra video coding," IEEE International Symposium on Circuits And Systems 2019. doi: https://doi.org/10.1109/ISCAS.2019.8702443
  6. Dai, Yuanying, Dong Liu, and Feng Wu. "A convolutional neural network approach for post-processing in HEVC intra coding," International Conference on Multimedia Modeling, Springer, Cham, pp. 28-39, 2017. doi: https://doi.org/10.1007/978-3-319-51811-4_3
  7. Huang, Zhijie, et al. "An efficient QP variable convolutional neural network based in-loop filter for intra coding." IEEE Data Compression Conference, pp. 33-42, 2021. doi: https://doi.org/10.1109/dcc50243.2021.00011
  8. Y. Li, L. Zhang, K. Zhang, "Conditional in-loop filter with parameter selection", JVET-V0101, Apr. 2021.
  9. Wang, Ming-Ze, et al. "Attention-based dual-scale CNN in-loop filter for versatile video coding," IEEE Access, Vol.7, pp. 145214-145226, 2019. doi: https://doi.org/10.1109/access.2019.2944473
  10. Xu, Xiaoyu, et al. "Dense inception attention neural network for in-loop filter," IEEE Picture Coding Symposium, pp. 1-5, 2019. doi: https://doi.org/10.1109/pcs48520.2019.8954499
  11. Z. Dai, et al, "AHG11: Neural network-nased adaptive model selection for CNN in-loop filtering", JVET-X0126, Oct. 2021.
  12. Jia, Chuanmin, et al. "Content-aware convolutional neural network for in-loop filtering in high efficiency Video coding," IEEE Transactions on Image Processing, Vol.28, No.7, 2019. doi: https://doi.org/10.1109/tip.2019.2896489
  13. Li, Yue, Li Zhang, and Kai Zhang. "IDAM: Iteratively trained deep in-loop filter with adaptive model selection," ACM Transaction on Multimedia Computing, Communications, and Application, 2022. doi: https://doi.org/10.1145/3529107
  14. Y. Li, K. Zhang, and L. Zhang. "EE1-1.2: Test on deep in-loop filter with adaptive model selection and external attention," JVET-X0065, Oct, 2021.
  15. L. Wang, X. Xu, and S. Liu, "AHG11: Neural network based in-loop filter with adaptive model selection," JVET-X0054, Oct. 2021.
  16. W. Lin, et al. "Partition-aware adaptive switching neural networks for post-processing In HEVC," IEEE Transactions on Multimedia, Vol.22, No.11, pp. 2749-2763, 2019. doi: https://doi.org/10.1109/tmm.2019.2962310
  17. L. van Der Maaten, and G. Hinton. "Visualizing data using t-SNE," Journal of Machine Learning Research, Vol.9, No.11, 2008.
  18. Lam, Yat-Hong, et al. "Efficient adaptation of neural network filter for video compression." Adaptive Model Selection," ACM International Conference on Multimedia, pp. 358-366, 2020. doi: https://doi.org/10.1145/3394171.3413536
  19. M. Santamaria, et al. "AHG11: Hannuksela, Content-adaptive post-processing filter," JVET-Y0059, Jan. 2022.
  20. M. Santamaria, et al. "AHG11: Content-adaptive neural network post-filte," JVET-Z0082, Apr. 2022.
  21. Lee. So Yoon, et al. "Offset-based in-loop filtering with a deep network in HEVC," IEEE Access, Vol.8, pp. 213958-213967, 2020. doi: https://doi.org/10.1109/access.2020.3040751
  22. Kong, Lingyi, et al. "Guided CNN restoration with explicitly signaled linear combination," IEEE International Conference on Image Processing, pp. 3379-3383, 2020. doi: https://doi.org/10.1109/icip40778.2020.9190807
  23. Bordes, Philippe, et al. "Revisiting the sample adaptive offset post-filter of VVC with neural-networks," IEEE Picture Coding Symposium, pp. 1-5, 2021. doi: https://doi.org/10.1109/pcs50896.2021.9477457