Browse > Article
http://dx.doi.org/10.5909/JBE.2022.27.4.527

Survey on Deep learning-based Content-adaptive Video Compression Techniques  

Han, Changwoo (Department of Electrical Engineering, Korea University)
Kim, Hongil (School of Electronics and Electrical Engineering, Dongguk University)
Kang, Hyun-ku (Department of Electrical Engineering, Korea University)
Kwon, Hyoungjin (Telecommunications & Media Research Laboratory, Electronics and Telecommunications Research Institute)
Lim, Sung-Chang (Telecommunications & Media Research Laboratory, Electronics and Telecommunications Research Institute)
Jung, Seung-Won (Department of Electrical Engineering, Korea University)
Publication Information
Journal of Broadcast Engineering / v.27, no.4, 2022 , pp. 527-537 More about this Journal
Abstract
As multimedia contents demand and supply increase, internet traffic around the world increases. Several standardization groups are striving to establish more efficient compression standards to mitigate the problem. In particular, research to introduce deep learning technology into compression standards is actively underway. Despite the fact that deep learning-based technologies show high performance, they suffer from the domain gap problem when test video sequences have different characteristics of training video sequences. To this end, several methods have been made to introduce content-adaptive deep video compression. In this paper, we will look into these methods by three aspects: codec information-aware methods, model selection methods, and information signaling methods.
Keywords
Contents adaptive filtering; Deep-learning; In-loop filtering; Post-processing; Video compression;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 M. Santamaria, et al. "AHG11: Hannuksela, Content-adaptive post-processing filter," JVET-Y0059, Jan. 2022.
2 Jia, Chuanmin, et al. "Content-aware convolutional neural network for in-loop filtering in high efficiency Video coding," IEEE Transactions on Image Processing, Vol.28, No.7, 2019. doi: https://doi.org/10.1109/tip.2019.2896489   DOI
3 Z. Dai, et al, "AHG11: Neural network-nased adaptive model selection for CNN in-loop filtering", JVET-X0126, Oct. 2021.
4 Cisco, Cisco Annual Internet Report (2018-2023) White Paper, Mar. 2020.
5 A. Skodras, C. Christopoulos and T. Ebrahimi, "The JPEG 2000 still image compression standard," Signal Processing Magazine, Vol.18, No.5, pp 36-58, 2001. doi: https://doi.org/10.1109/79.952804   DOI
6 Li, Daowen, and Lu Yu. "An in-loop filter based on low-complexity CNN using residuals in intra video coding," IEEE International Symposium on Circuits And Systems 2019. doi: https://doi.org/10.1109/ISCAS.2019.8702443   DOI
7 Y. Li, L. Zhang, K. Zhang, "Conditional in-loop filter with parameter selection", JVET-V0101, Apr. 2021.
8 Y. Li, K. Zhang, and L. Zhang. "EE1-1.2: Test on deep in-loop filter with adaptive model selection and external attention," JVET-X0065, Oct, 2021.
9 L. van Der Maaten, and G. Hinton. "Visualizing data using t-SNE," Journal of Machine Learning Research, Vol.9, No.11, 2008.
10 Bordes, Philippe, et al. "Revisiting the sample adaptive offset post-filter of VVC with neural-networks," IEEE Picture Coding Symposium, pp. 1-5, 2021. doi: https://doi.org/10.1109/pcs50896.2021.9477457   DOI
11 Huang, Zhijie, et al. "An efficient QP variable convolutional neural network based in-loop filter for intra coding." IEEE Data Compression Conference, pp. 33-42, 2021. doi: https://doi.org/10.1109/dcc50243.2021.00011   DOI
12 M. Santamaria, et al. "AHG11: Content-adaptive neural network post-filte," JVET-Z0082, Apr. 2022.
13 B. Bross, J. Chen, S. Liu and Y.-K. Wang, "Versatile video coding (Draft 10)," JVET-S2001, Jul. 2020.
14 Wei Jia, et al "Residual-guided In-loop Filter Using Convolution Neural Network," ACM Trans. Multimedia Comput. Communications, and Applications, 2021 doi: https://doi.org/10.1145/3460820   DOI
15 L. Wang, X. Xu, and S. Liu, "AHG11: Neural network based in-loop filter with adaptive model selection," JVET-X0054, Oct. 2021.
16 Wang, Ming-Ze, et al. "Attention-based dual-scale CNN in-loop filter for versatile video coding," IEEE Access, Vol.7, pp. 145214-145226, 2019. doi: https://doi.org/10.1109/access.2019.2944473   DOI
17 Xu, Xiaoyu, et al. "Dense inception attention neural network for in-loop filter," IEEE Picture Coding Symposium, pp. 1-5, 2019. doi: https://doi.org/10.1109/pcs48520.2019.8954499   DOI
18 Li, Yue, Li Zhang, and Kai Zhang. "IDAM: Iteratively trained deep in-loop filter with adaptive model selection," ACM Transaction on Multimedia Computing, Communications, and Application, 2022. doi: https://doi.org/10.1145/3529107   DOI
19 W. Lin, et al. "Partition-aware adaptive switching neural networks for post-processing In HEVC," IEEE Transactions on Multimedia, Vol.22, No.11, pp. 2749-2763, 2019. doi: https://doi.org/10.1109/tmm.2019.2962310   DOI
20 Dai, Yuanying, Dong Liu, and Feng Wu. "A convolutional neural network approach for post-processing in HEVC intra coding," International Conference on Multimedia Modeling, Springer, Cham, pp. 28-39, 2017. doi: https://doi.org/10.1007/978-3-319-51811-4_3   DOI
21 Lam, Yat-Hong, et al. "Efficient adaptation of neural network filter for video compression." Adaptive Model Selection," ACM International Conference on Multimedia, pp. 358-366, 2020. doi: https://doi.org/10.1145/3394171.3413536   DOI
22 Lee. So Yoon, et al. "Offset-based in-loop filtering with a deep network in HEVC," IEEE Access, Vol.8, pp. 213958-213967, 2020. doi: https://doi.org/10.1109/access.2020.3040751   DOI
23 Kong, Lingyi, et al. "Guided CNN restoration with explicitly signaled linear combination," IEEE International Conference on Image Processing, pp. 3379-3383, 2020. doi: https://doi.org/10.1109/icip40778.2020.9190807   DOI