Browse > Article
http://dx.doi.org/10.5909/JBE.2022.27.4.477

Accurate Prediction of VVC Intra-coded Block using Convolutional Neural Network  

Jeong, Hye-Sun (Department of Electronic and Electrical Engineering and Graduate Program in Smart Factory, Ewha Womans University)
Kang, Je-Won (Department of Electronic and Electrical Engineering and Graduate Program in Smart Factory, Ewha Womans University)
Publication Information
Journal of Broadcast Engineering / v.27, no.4, 2022 , pp. 477-486 More about this Journal
Abstract
In this paper, we propose a novel intra-prediction method using convolutional neural network (CNN) to improve a quality of a predicted block in VVC. The proposed algorithm goes through a two-step procedure. First, an input prediction block is generated using one of the VVC intra-prediction modes. Second, the prediction block is further refined through a CNN model, by inputting the prediction block itself and reconstructed reference samples in the boundary. The proposed algorithm outputs a refined block to reduce residual signals and enhance coding efficiency, which is enabled by a CU-level flag. Experimental results demonstrate that the proposed method achieves improved rate-distortion performance as compared a VVC reference software, I.e., VTM version 10.0.
Keywords
Intra Prediction; Convolutional Neural Network; Versatile Video Coding;
Citations & Related Records
Times Cited By KSCI : 5  (Citation Analysis)
연도 인용수 순위
1 JVET of ITU-T and ISO/IEC, "Versatile Video Coding (Draft 10)", Documents JEVT-S2001, June 2020.
2 J. Li, et al. "Intra prediction using fully connected network for video coding." 2017 IEEE International Conference on Image Processing (ICIP). IEEE, 2017. doi: https://doi.org/10.1109/ICIP.2017.8296231   DOI
3 Y. Hu, et al. "Optimized spatial recurrent network for intra prediction in video coding." 2018 IEEE Visual Communications and Image Processing (VCIP). IEEE, 2018. doi: https://doi.org/10.1109/VCIP.2018.8698658   DOI
4 L. Zhu, et al. "Generative adversarial network-based intra prediction for video coding." IEEE transactions on multimedia 22.1 (2019): 45-58. doi: https://doi.org/10.1109/TMM.2019.2924591   DOI
5 JVET of ITU-T and ISO/IEC, "AHG11: Neural Network based cross-component Prediction model", Deocument JVET-W0111, July 2021
6 Sookyung Ryu and Je-Won Kang, "Machine Learning-Based Fast Angular Prediction Mode Decision Technique in Video Coding," IEEE Transaction on Image Processing, Nov. 2018. doi: https://doi.org/10.1109/TIP.2018.2857404   DOI
7 B. Bross et al., "Overview of the Versatile Video Coding (VVC) Standard and its Applications"IEEE Transactions on Cicuits and Systems on Video Technologies, vol. 31, no. 10, 2021. doi: https://doi.org/10.1109/TCSVT.2021.3101953   DOI
8 Je-Won Kang, Soo-Kyung Ryu, Na-Young Kim, Minjoo Kang, "Efficient Residual DPCM using an L-1 Robust Linear Prediction in Screen Content Video Coding," IEEE Transaction on Multimedia, vol. 18, no. 10, pp.2054-2065, Oct. 2016. doi: https://doi.org/10.1109/TMM.2016.2595259   DOI
9 Versatile Video Coding Test Model (VTM) 10.0 : https://vcgit.hhi.fraunhofer.de/jvet/VVCSoftware_VTM/-/tree/VTM-10.0
10 Moving Picture Experts Group (MPEG) : https://mpeg.chiariglione.org
11 J. Pfaff et al., "Intra Prediction and Mode Coding in VVC", IEEE Transactions on Cicuits and Systems on Video Technologies, vol. 31, no. 10, 2021. doi: https://doi.org/10.1109/TCSVT.2021.3072430   DOI
12 M. G. Blanch, et al. "Chroma intra prediction with attention-based CNN architectures." 2020 IEEE International Conference on Image Processing (ICIP). IEEE, 2020. doi: https://doi.org//ICIP40778.2020.9191050
13 Y. Hu, et al. "Enhanced intra prediction with recurrent neural network in video coding." 2018 Data Compression Conference. IEEE, 2018. doi: https://doi.org/https://doi.org/10.1109/DCC.2018.00066   DOI
14 Y. Hu, et al. "Progressive spatial recurrent neural network for intra prediction." IEEE Transactions on Multimedia 21.12 (2019): 3024-3037. doi: https://doi.org/10.1109/TMM.2019.2920603   DOI
15 Y. Wang, et al. "Multi-scale convolutional neural network-based intra prediction for video coding." IEEE Transactions on Circuits and Systems for Video Technology 30.7 (2019): 1803-1815. doi: https://doi.org/10.1109/TCSVT.2019.2934681   DOI
16 Sang-hyo Park and Je-Won Kang, "Fast Multi-type Tree Partitioning for Versatile Video Coding Using a Lightweight Neural Network", IEEE Transactions on Multimedia, 2021. doi: https://doi.org/10.1109/TMM.2020.3042062   DOI
17 Jyung-Kyung Lee, Nayoung Kim, Seunghyun Cho, and Je-Won Kang, "Deep Video Prediction Network Based Inter-Frame Coding in HEVC," IEEE Access, 2020. doi: https://doi.org/10.1109/ACCESS.2020.2993566   DOI
18 JVET of ITU-T and ISO/IEC, "AHG11 Neural Network-based Intra Prediction with Transform selection in VVC", Document JVET-T0073, Oct. 2020.
19 F. Brand, S. Jurgen, and K. Andre. "Intra frame prediction for video coding using a conditional autoencoder approach." 2019 Picture Coding Symposium (PCS). IEEE, 2019. doi: https://doi.org/10.1109/PCS48520.2019.8954546   DOI
20 T. Barnett, et al. "Cisco visual networking index (vni) complete forecast update, 2017-2022." Americas/EMEAR Cisco Knowledge Network (CKN) Presentation (2018).
21 Jyung-Kyung Lee, Nayoung Kim, and Je-Won Kang, "Rate-distortion optimized temporal segmentation using reinforcement leaning for video coding," APSIPA, 2021.
22 Radu Timofte, Eirikur Agustsson, Luc Van Gool, MingHsuan Yang, and Lei Zhang, "Ntire 2017 challenge on single image super-resolution: Methods and results," in Proceedings of the IEEE conference on computer vision and pattern recognition workshops, 2017, pp. 114-125. doi: https://doi.org/10.1109/cvprw.2017.150   DOI
23 H. Jeong, Je-Won Kang, "Intra prediction through block refinement", 34th Image processing and understanding, 2022.
24 Je-Won Kang, Gabbouj, M., and Jay Kuo, C. C. "Sparse/DCT (S/DCT) two-layered representation of prediction residuals for video coding", IEEE Transactions on Image Processing, 22(7), 2711-2722. doi: https://doi.org/10.1109/TIP.2013.2256917   DOI
25 H. Jeong, Je-Won Kang "Improvements of intra-prediction in VVC", Summer conference in the Korean Society of Broadcast and Media Engineers, 2022