Browse > Article
http://dx.doi.org/10.5909/JBE.2014.19.5.640

HEVC Encoder Optimization using Depth Information  

Lee, Yoon Jin (Dept. of Computer Engineering, Kyung Hee University)
Bae, Dong In (Dept. of Computer Engineering, Kyung Hee University)
Park, Gwang Hoon (Dept. of Computer Engineering, Kyung Hee University)
Publication Information
Journal of Broadcast Engineering / v.19, no.5, 2014 , pp. 640-655 More about this Journal
Abstract
Many of today's video systems have additional depth camera to provide extra features such as 3D support. Thanks to these changes made in multimedia system, it is now much easier to obtain depth information of the video. Depth information can be used in various areas such as object classification, background area recognition, and so on. With depth information, we can achieve even higher coding efficiency compared to only using conventional method. Thus, in this paper, we propose the 2D video coding algorithm which uses depth information on top of the next generation 2D video codec HEVC. Background area can be recognized with depth information and by performing HEVC with it, coding complexity can be reduced. If current CU is background area, we propose the following three methods, 1) Earlier stop split structure of CU with PU SKIP mode, 2) Limiting split structure of CU with CU information in temporal position, 3) Limiting the range of motion searching. We implement our proposal using HEVC HM 12.0 reference software. With these methods results shows that encoding complexity is reduced more than 40% with only 0.5% BD-Bitrate loss. Especially, in case of video acquired through the Kinect developed by Microsoft Corp., encoding complexity is reduced by max 53% without a loss of quality. So, it is expected that these techniques can apply real-time online communication, mobile or handheld video service and so on.
Keywords
HEVC(High Efficiency Video Coding); Kinect; Depth Information; Encoder Optimization;
Citations & Related Records
Times Cited By KSCI : 4  (Citation Analysis)
연도 인용수 순위
1 JCT-VC, http://www.itu.int/en/ITU-T/studygroups/com16/video/Pages/ jctvc.aspx
2 JCT-3V, http://www.itu.int/en/ITU-T/studygroups/2013-2016/16/Pages/ video/jct3v.aspx
3 Sullivan, G.J., Ohm, J., Woo-Jin Han, Wiegand, T, "Overview of the High Efficiency Video Coding (HEVC) Standard", IEEE Transactions on CSVT, vol. 22, no. 12, pp.1649-1668, Dec. 2012
4 JCT-3V of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, "CE1.h: Backward View Synthesis Prediction using Neighbouring Blocks", JCT3V-C0152, Geneva, CH, Jan. 2013.
5 JCT-3V of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, "CE3: Results on Depth-based Block Partitioning (DBBP)", JCT3V-G0106, San Jose, US, Jan. 2014.
6 Jorn Ostermann, Euee S. Jang, Jae-Seob Shin, Tsuhan Chen, "Coding of Arbitrarily Shaped Video Objects in MPEG-4", International Conference of Image Processing, pp.496-499, 1997.
7 Kaup, A., "Object-based texture coding of moving video in MPEG-4", IEEE Transactions on CSVT, vol. 9, no. 1, pp.5-15, 1999.
8 Il-Koo Kim, Junghye Min, Tammy Lee, Woo-Jin Han, JeongHoon Park, "Block Partitioning Structure int the HEVC Standard", IEEE Transactions on CSVT, vol. 22, no. 12, pp.1697-2012, Dec. 2012
9 ISO/IEC 14496-2 (MPEG-4 Visual Version 1), "Coding of audio-visual objects-Part 2: Visual", Apr. 1999.
10 Rong Ding, Qionghai Dai, Wenli Xu, Dongdong Zhu, Hao Yin, "Background-frame based motion compensation for video compression", 2004 IEEE ICME, vol. 2, pp.1487-1490, 2014.
11 Ohm, J., Sullivan, G.J., Schwarz, H., Thiow Keng Tan, Wiegand, T., "Comparison of the Coding Efficiency of Video Coding Standards-Including High Efficiency Video Coding (HEVC)", IEEE Transactions on CSVT, vol. 22, no. 12, pp.1669-1684, Dec. 2012.
12 Woo-Jin Han, "Conditional Probability Based Early Termination of Recursive Coding Unit Structures in HEVC", Journal of Broadcast Engineering, vol. 17, no. 2, pp.354-362, Mar. 2012.   DOI   ScienceOn
13 Chansub Park, Byeonggyu Kim, Dongsan Jeon, Sunheung Jung, Yeonhui Kim, Jinuk Suk, Jinsun Choi, "Early Termination Algorithm of Merge Mode Search for Fast High Efficiency Video Coding (HEVC) Encoder", Journal of Broadcast Engineering, vol. 18, no. 5, pp.691-701, Sep. 2012.   DOI   ScienceOn
14 JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, "Common HM test conditions and software reference configurations", JCTVC-I1100, Geneva, Switzerland, May. 2012.
15 Hansu Lee, Kyeongyong Kim, Taeryong Kim, Gwanghoon Park, Hwiyong Kim, Seongchang Lim, Jinho Lee, "Improving Encoder Complexity and Coding Method of the Split Information in HEVC", Journal of Broadcast Engineering, vol. 17, no. 2, pp.325-343, Mar. 2012.   DOI   ScienceOn
16 JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, "Early Termination of CU Encoding to Reduce HEVC Complexity", JCTVC-F045, Torino, IT, Jul. 2011.
17 JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, "Coding tree pruning based CU early termination", JCTVC-F092, Torino, IT, Jul. 2011.
18 Gary J. Sullvan and Thowas Wiegand, "Rate-Distortion Optimization for Video Compression", IEEE Signal Processing Magazine, vol. 15, no. 6, pp.74-90, Nov. 1998.   DOI   ScienceOn
19 JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, "Samsung Response to the Call for Proposals on Video Compression Technology", JCTVC-A124, Dresden, German, Apr. 2011.
20 JCT-VC of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, "Description of video coding technology proposal by Fraunhofer HHI", JCTVC-A116, Dresden, German, Apr. 2011.
21 Jens-Rainer Ohm, G. J. Sullivan, Heiko Schwarz, T. K. Tan, T. Wiegand, "Comparison of the Coding Efficiency of Video Coding Standards-Including High Efficiency Video Coding (HEVC)", IEEE Transactions on CSVT, vol. 22, no. 12, pp.1669-1684, Dec. 2012.
22 Zoran Zivkovic, "Improved adaptive Gaussian mixture model for background subtraction", International Conference Pattern Recognition, vol. 2, pp.28-31, 2004.
23 M. Piccardi, "Background subtraction techniques: a review", IEEE International Conference on SMC, vol. 4, pp. 3099-3100, 2004.
24 Yongjo Ahn, Taejin Hwang, Sungeun Yoo, Woo-jin Han, Donggyu Sim, "Statistical Characteristics and Complexity Analysis of HEVC Encoder Software", Journal of Broadcast Engineering, vol. 17, no. 6, pp.1091-1105, Nov. 2012.   DOI   ScienceOn
25 Frank Bossen, Benjamin Bross, Karsten Suhring, David Flynn, "HEVC Complexity and Implementation Analysis", IEEE Transactions on CSVT, vol. 22, no. 12, pp.1685-1696, Dec. 2012.
26 G. Bjontegaard, "Calculation of average PSNR differences between RD-curves," ITU-T SG16 Q.6, VCEG-M33, Texas, USA, Apr. 2001.
27 Intel RealSense 3D Camera, http://www.intel.com/content/www/us/ en/architecture-and-technology/realsense-depth-technologies.html
28 Francois, E., Chupeau, B., "Depth-based segmentation", IEEE Transactions On CSVT, vol. 7, no. 1, pp.237-240, Feb. 1997.
29 Kinect camera, http://www.xbox.com/en-US/kinect/default.htm
30 Project Tango, http://www.google.com/atap/projecttango/
31 Jungong Han, Ling Shao, Dong Xu, Shotton. J., "Enhanced Computer Vision With Microsoft Kinect Sensor: A Review", IEEE Transactions on Cybernetices, vol. 43, no. 5, pp.1318-1334, Oct. 2013.   DOI   ScienceOn
32 Suarez, J., Murphy, R.R., "Hand gesture recognition with depth images: A review", 2012 IEEE Conference on RO-MAIN, pp.411-417, Sep. 2012.
33 Shahram Izadi, David Kim, Otmar Hilliges, David Molyneaux, Richard Newcombe, Pushmeet Kohli, Jamie Shotton, Steve Hodges, Dustin Freeman, Andrew Davison, Andrew Fitzgibbon, "KinectFusion: Real-time 3D Reconstruction and Interaction Using a Moving Depth Camera", UIST'11, pp.559-568, Oct. 2011.
34 Yan Cui, Schuon, S., Chan, D., Thrun, S., Theobalt, C., "3D Shape Scanning with a Time-of-Flight Camera", 2010 IEEE Conference on CVPR, pp.1173-1180, Jun. 2010.
35 MPEG, http://mpeg,chiaiglione.org
36 VCEG, http://www.itu.int/en/ITU-T/studygroups/com16/video/Pages/ default.aspx
37 T. Wiegand, G. J. Sullivan, G. Bjontegaard, A. Luthra, "Overview of the H.264/AVC video coding standard", IEEE Transactions on CSVT, vol. 13, no. 7, pp.560-576, Jul. 2003.