Browse > Article
http://dx.doi.org/10.7472/jksii.2018.19.6.41

A Method for 3D Human Pose Estimation based on 2D Keypoint Detection using RGB-D information  

Park, Seohee (Human Care System Research Center, Korea Electronics Technology Institute(KETI))
Ji, Myunggeun (Department of Computer Science, Kyonggi University)
Chun, Junchul (Department of Computer Science, Kyonggi University)
Publication Information
Journal of Internet Computing and Services / v.19, no.6, 2018 , pp. 41-51 More about this Journal
Abstract
Recently, in the field of video surveillance, deep learning based learning method is applied to intelligent video surveillance system, and various events such as crime, fire, and abnormal phenomenon can be robustly detected. However, since occlusion occurs due to the loss of 3d information generated by projecting the 3d real-world in 2d image, it is need to consider the occlusion problem in order to accurately detect the object and to estimate the pose. Therefore, in this paper, we detect moving objects by solving the occlusion problem of object detection process by adding depth information to existing RGB information. Then, using the convolution neural network in the detected region, the positions of the 14 keypoints of the human joint region can be predicted. Finally, in order to solve the self-occlusion problem occurring in the pose estimation process, the method for 3d human pose estimation is described by extending the range of estimation to the 3d space using the predicted result of 2d keypoint and the deep neural network. In the future, the result of 2d and 3d pose estimation of this research can be used as easy data for future human behavior recognition and contribute to the development of industrial technology.
Keywords
Video Surveillance; Object Detection; Keypoint Detection; Human Pose Estimation; Deep Learning;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Seohee Park, Myunggeun Ji, and Junchul Chun, "2D Human Pose Estimation based on Object Detection using RGB-D information", KSII Transactions on Internet & Information Systems, Vol. 12, No. 2, pp. 800-816, 2018. https://doi.org/10.3837/tiis.2018.02.015   DOI
2 Ramakrishna, Varun, Takeo Kanade, and Yaser Sheikh, "Reconstructing 3d human pose from 2d image landmarks", European conference on computer vision. Springer, Berlin, Heidelberg, pp. 573-586, 2012. https://doi.org/10.1007/978-3-642-33765-9_41
3 Parekh, Himani S., Darshak G. Thakore, and Udesang K. Jaliya, "A survey on object detection and tracking methods", International Journal of Innovative Research in Computer and Communication Engineering, Vol. 2, No. 2, pp. 2970-2978, 2014. http://www.ijircce.com/upload/2014/february/7J_A%20S urvey.pdf
4 Zivkovic, Zoran, "Improved adaptive Gaussian mixture model for background subtraction", Pattern Recognition, 2004. https://doi.org/10.1109/icpr.2004.1333992   DOI
5 Hirschmuller, Heiko, "Stereo processing by semiglobal matching and mutual information", IEEE Transactions on pattern analysis and machine intelligence, Vol. 30, No. 2, pp. 328-341, 2008. https://doi.org/10.1109/tpami.2007.1166   DOI
6 Ionescu, Catalin, et al, "Human3.6m: Large scale datasets and predictive methods for 3d human sensing in natural environments", IEEE transactions on pattern analysis and machine intelligence, Vol. 36, No. 7, pp. 1325-1339, 2014. https://doi.org/10.1109/tpami.2013.248   DOI
7 Tekin, Bugra, et al, "Direct prediction of 3d body poses from motion compensated sequences", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. https://doi.org/10.1109/cvpr.2016.113
8 Chen, Ching-Hang, and Deva Ramanan, "3d human pose estimation = 2d pose estimation + matching", CVPR, Vol. 2, No. 5, 2017. https://doi.org/10.1109/cvpr.2017.610
9 Zhou, Xiaowei, et al, "Sparseness meets deepness: 3D human pose estimation from monocular video", Proceedings of the IEEE conference on computer vision and pattern recognition, 2016. https://doi.org/10.1109/cvpr.2016.537
10 Du, Yu, et al, "Marker-less 3d human motion capture with monocular image sequence and height-maps", European Conference on Computer Vision. Springer, Cham, 2016. https://doi.org/10.1007/978-3-319-46493-0_2
11 Park, Sungheon, Jihye Hwang, and Nojun Kwak, "3D human pose estimation using convolutional neural networks with 2D pose information", European Conference on Computer Vision. Springer, Cham, 2016. https://arxiv.org/abs/1608.03075
12 Zhou, et al, "Deep kinematic pose regression", European Conference on Computer Vision. Springer, Cham, 2016. https://arxiv.org/abs/1609.05317
13 Tome, Denis, Christopher Russell, and Lourdes Agapito, "Lifting from the deep: Convolutional 3d pose estimation from a single image", CVPR 2017 Proceedings, pp. 2500-2509, 2017. https://doi.org/10.1109/cvpr.2017.603
14 Martinez, et al, "A simple yet effective baseline for 3d human pose estimation", International Conference on Computer Vision, Vol. 1, No. 2. 2017. https://doi.org/10.1109/iccv.2017.288
15 OpenPose: A Real-Time Multi-Person Keypoint Detection and Multi-Threading C++ Library, 2017.
16 Wei, Shih-En, et al, "Convolutional pose machines", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016. https://doi.org/10.1109/cvpr.2016.511
17 Ramakrishna, Varun, et al, "Pose machines: Articulated pose estimation via inference machines", European Conference on Computer Vision. Springer, Cham, 2014. https://doi.org/10.1007/978-3-319-10605-2_3
18 Sigal, Leonid, et al, "Humaneva: Synchronized video and motion capture dataset and baseline algorithm for evaluation of articulated human motion", International journal of computer vision, 2010. https://doi.org/10.1007/s11263-009-0273-6   DOI
19 Newell, Alejandro, Kaiyu Yang, and Jia Deng, "Stacked hourglass networks for human pose estimation", European Conference on Computer Vision. Springer, Cham, 2016. https://doi.org/10.1007/978-3-319-46484-8_29