KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Bayesian-theory-based Fast CU Size and Mode Decision Algorithm for 3D-HEVC Depth Video Inter-coding

  • Chen, Fen (Faculty of Information Science and Engineering, Ningbo University) ;
  • Liu, Sheng (Faculty of Information Science and Engineering, Ningbo University) ;
  • Peng, Zongju (Faculty of Information Science and Engineering, Ningbo University) ;
  • Hu, Qingqing (Faculty of Information Science and Engineering, Ningbo University) ;
  • Jiang, Gangyi (Faculty of Information Science and Engineering, Ningbo University) ;
  • Yu, Mei (Faculty of Information Science and Engineering, Ningbo University)
  • Received : 2016.11.28
  • Accepted : 2017.12.10
  • Published : 2018.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

Multi-view video plus depth (MVD) is a mainstream format of 3D scene representation in free viewpoint video systems. The advanced 3D extension of the high efficiency video coding (3D-HEVC) standard introduces new prediction tools to improve the coding performance of depth video. However, the depth video in 3D-HEVC is time consuming. To reduce the complexity of the depth video inter coding, we propose a fast coding unit (CU) size and mode decision algorithm. First, an off-line trained Bayesian model is built which the feature vector contains the depth levels of the corresponding spatial, temporal, and inter-component (texture-depth) neighboring largest CUs (LCUs). Then, the model is used to predict the depth level of the current LCU, and terminate the CU recursive splitting process. Finally, the CU mode search process is early terminated by making use of the mode correlation of spatial, inter-component (texture-depth), and inter-view neighboring CUs. Compared to the 3D-HEVC reference software HTM-10.0, the proposed algorithm reduces the encoding time of depth video and the total encoding time by 65.03% and 41.04% on average, respectively, with negligible quality degradation of the synthesized virtual view.

Keywords

References

  1. Tanimoto M., "FTV: Free-viewpoint television," Signal Processing: Image Commutation, vol. 27, no.6, pp. 555-570, 2012. https://doi.org/10.1016/j.image.2012.02.016
  2. Chen Y., Hannuksela M.M., Suzuki T., et al., "Overview of the MVC+ D 3D video coding standard," Journal of Visual Communication and Image Representation, vol. 25, no. 4, pp. 679-688, 2014. https://doi.org/10.1016/j.jvcir.2013.03.013
  3. Fehn C., "Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV," Proceedings of SPIE - The International Society for Optical Engineering, pp. 93-104, 2004.
  4. Sullivan G.J., Ohm J.R., Han W.J., et al., "Overview of the high efficiency video coding (HEVC) standard," IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no.12, pp.1649-1668, 2012. https://doi.org/10.1109/TCSVT.2012.2221191
  5. Hannuksela M.M., Rusanovskyy D., Su W., et al., "Multi view-video-plus-depth coding based on the advanced video coding standard," IEEE Transactions on Image Processing, vol. 22, no.9, pp.3449-3458, 2013. https://doi.org/10.1109/TIP.2013.2269274
  6. Chen Y., Tech G., Wegner K., et al., "Test model 10 of 3D-HEVC and MV-HEVC," Document of Joint Collaborative Team on 3D Video Coding Extension Development, JCT3V JI003, 2014.
  7. Helle P., Oudin S., Bross B., et al., "Block merging for quadtree-based partitioning in HEVC," IEEE Transactions on Circuits and Systems for Video Technology, vol. 22, no.12, pp.1720-1731, 2012. https://doi.org/10.1109/TCSVT.2012.2223051
  8. Bross B., Han W.J., Sullivan G.J., et al., "High efficiency video coding (HEVC) text specification draft 10 (JCTVCL1003)," JCT-VC Meeting (Joint Collaborative Team of ISO/IEC MPEG & ITU-T VCEG), 2013.
  9. Zhang Q., Wang X., Huang X., et al., "Fast mode decision algorithm for 3D-HEVC encoding optimization based on depth information," Digital Signal Processing, vol. 44, no. 1, pp.37-46, 2015. https://doi.org/10.1016/j.dsp.2015.06.005
  10. Gweon R., LEE Y.-L., "Early termination of CU encoding to reduce HEVC complexity," IEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences, vol. 95, no. 7, 1215-1218, 2012.
  11. Choi K., Park S.H., Jang E.S., "Coding tree pruning based CU early termination," Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11 6th Meeting, JCTVC, Torino, Italy 14-22 , 2011.
  12. Chung K. L., Huang Y. H., Lin C.H., et al., "Novel bitrate-saving and fast coding for depth videos in 3D-HEVC," IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no.10, pp.1859-1869, 2016. https://doi.org/10.1109/TCSVT.2015.2473296
  13. Zhang Q., Chen M., Huang X., et al., "Low-complexity depth map compression in HEVC-based 3D video coding," EURASIP Journal on Image and Video Processing, vol. 2015, no. 2, pp. 1-14, 2015. https://doi.org/10.1186/1687-5281-2015-1
  14. Lei H., Wang A., Liu W., et al., "Early termination algorithm for the depth modeling mode in three-dimensional extension of high efficiency video coding," Journal of Electronic Imaging, vol. 25, no. 5, pp. 050503-1 - 050503-4, 2016.
  15. Zhao J., Zhao X., Zhang W., et al., "An efficient depth modeling mode decision algorithm for 3D-HEVC depth map coding," Optik, vol. 127, no. 24, pp. 12048-12055, 2016. https://doi.org/10.1016/j.ijleo.2016.10.008
  16. Yang C., An P., Shen L., et al., "Fast depth map coding based on virtual view quality," Signal Processing: Image Communication, vol. 47, no. c, pp. 183-192, 2016. https://doi.org/10.1016/j.image.2016.05.015
  17. Mora E.G., Jung J., Cagnazzo M., et al., "Initialization, limitation, and predictive coding of the depth and texture quadtree in 3D-HEVC," IEEE Transactions on Circuits and Systems for Video Technology, vol. 24, no.9, pp.1554-1565, 2014. https://doi.org/10.1109/TCSVT.2013.2283110
  18. Lei J., Li S., Zhu C., et al., "Depth coding based on depth-texture motion and structure similarities," IEEE Transactions on Circuits and Systems for Video Technology, vol. 25, no.2, pp.275-286, 2015. https://doi.org/10.1109/TCSVT.2014.2335471
  19. Shen L., An P., Zhang Z., et al., "A 3D-HEVC Fast Mode Decision Algorithm for Real-Time Applications," ACM Transactions on Multimedia Computing, Communications, and Applications, vol. 11, no.3, pp.1-23, 2015.
  20. Zhang Q., Zhang N., Wei T., et al., "Fast depth map mode decision based on depth-texture correlation and edge classification for 3D-HEVC," Journal of Visual Communication and Image Representation, vol. 45, no.c, pp.170-180, 2017. https://doi.org/10.1016/j.jvcir.2017.03.004
  21. Zhang Y., Kwong S., Wang X., et al., "Machine learning-based coding unit depth decisions for flexible complexity allocation in high efficiency video coding," IEEE Transactions on Image Processing, vol. 24, no.7, pp.2225-2238, 2015. https://doi.org/10.1109/TIP.2015.2417498
  22. Shen L., Zhang Z., Zhang X., et al., "Fast TU size decision algorithm for HEVC encoders using Bayesian theory detection," Signal Process:Image Communication, vol. 32, pp.121-128, 2015. https://doi.org/10.1016/j.image.2015.01.008
  23. Tohidypour H. R., Pourazad M. T., Nasiopoulos P., "Online learning-based complexity reduction scheme for 3D-HEVC," IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 10, pp.1870-1883, 2016. https://doi.org/10.1109/TCSVT.2015.2477955
  24. Zhu L., Zhang Y., Pan Z., et al., "Binary and Multi-Class Learning Based Low Complexity Optimization for HEVC Encoding," IEEE Transactions on Broadcasting, vol. 63, no. 3, pp.547-561, 2017. https://doi.org/10.1109/TBC.2017.2711142
  25. Kim. H.-S., Park R.-H., "Fast CU Partitioning Algorithm for HEVC Using an Online-Learning-Based Bayesian Decision Rule," IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 1, pp.130-138, 2016. https://doi.org/10.1109/TCSVT.2015.2444672
  26. Muller K., Schwarz H., Marpe D., et al., "3D high-efficiency video coding for multi-view video and depth data," IEEE Transactions on Image Processing, vol. 22, no. 9, pp.3366-3378, 2013. https://doi.org/10.1109/TIP.2013.2264820
  27. Tech G., Chen Y., Muller K., et al., "Overview of the multiview and 3D extensions of high efficiency video coding," IEEE Transactions on Circuits and Systems for Video Technology, vol. 26, no. 1, pp.35-49, 2016. https://doi.org/10.1109/TCSVT.2015.2477935
  28. Muller K., Vetro A., "Common test conditions of 3DV core experiments," JCT3V meeting, JCT3VG1100, 2014.
  29. Bjontegaard G., "Calculation of Average PSNR Differences between RD-Curves," in Proc. of Austin: 13th Video Coding Experts Group (VCEG) Meeting, VCEG-M33, 2001.