Browse > Article
http://dx.doi.org/10.7472/jksii.2017.18.6.85

A Method for Body Keypoint Localization based on Object Detection using the RGB-D information  

Park, Seohee (Department of Computer Science, Kyonggi University)
Chun, Junchul (Department of Computer Science, Kyonggi University)
Publication Information
Journal of Internet Computing and Services / v.18, no.6, 2017 , pp. 85-92 More about this Journal
Abstract
Recently, in the field of video surveillance, a Deep Learning based learning method has been applied to a method of detecting a moving person in a video and analyzing the behavior of a detected person. The human activity recognition, which is one of the fields this intelligent image analysis technology, detects the object and goes through the process of detecting the body keypoint to recognize the behavior of the detected object. In this paper, we propose a method for Body Keypoint Localization based on Object Detection using RGB-D information. First, the moving object is segmented and detected from the background using color information and depth information generated by the two cameras. The input image generated by rescaling the detected object region using RGB-D information is applied to Convolutional Pose Machines for one person's pose estimation. CPM are used to generate Belief Maps for 14 body parts per person and to detect body keypoints based on Belief Maps. This method provides an accurate region for objects to detect keypoints an can be extended from single Body Keypoint Localization to multiple Body Keypoint Localization through the integration of individual Body Keypoint Localization. In the future, it is possible to generate a model for human pose estimation using the detected keypoints and contribute to the field of human activity recognition.
Keywords
Video Surveillance; Object Detection; Body Keypoint Localization; Convolutional Pose Machines; Belief Map; Human Activity Recognition;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Zhang, Shugang, et al., "Vision-Based Human Activity Recognition: A Review," Journal of Healthcare Engineering, Vol 2017, pp. 1-31, 2017. https://doi.org/10.1155/2017/3090343
2 Vrigkas, Michalis, Christophoros Nikou, and Ioannis A. Kakadiaris, "A review of human activity recognition methods," Frontiers in Robotics and AI, Vol 2, article 28, 2015. https://doi.org/10.3389/frobt.2015.00028
3 Paul, Manoranjan, Shah ME Haque, and Subrata Chakraborty., "Human detection in surveillance videos and its applications-a review," EURASIP Journal on Advances in Signal Processing, Vol 176, No. 1, pp.1-16, 2013. https://doi.org/10.1186/1687-6180-2013-176
4 Toshev, Alexander, and Christian Szegedy., "Deeppose: Human pose estimation via deep neural networks," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1653-1660, 2014. https://doi.org/10.1109/cvpr.2014.214
5 Pishchulin, Leonid, et al., "Deepcut: Joint subset partition and labeling for multi person pose estimation," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4929-4937. 2016. https://doi.org/10.1109/cvpr.2016.533
6 OpenPose: A Real-Time Multi-Person Keypoint Detection And Multi-Threading C++ Library, 2017. https://github.com/CMU-Perceptual-Computing-Lab/openpose
7 Cao, Zhe, et al., "Realtime multi-person 2d pose estimation using part affinity fields," arXiv preprint arXiv:1611.08050, 2016. https://arxiv.org/abs/1611.08050
8 Simon, Tomas, et al., "Hand Keypoint Detection in Single Images using Multiview Bootstrapping," arXiv preprint arXiv:1704.07809, 2017. https://arxiv.org/abs/1704.07809
9 Wei, Shih-En, et al., "Convolutional pose machines," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4724-4732, 2016. https://doi.org/10.1109/cvpr.2016.511
10 Seohee Park, Junchul Chun., "A Robust Object Detection and Tracking Method using RGB-D Model", Journal of Internet Computing and Services (JICS), Vol 18, No. 4, pp. 61-67, 2017. http://dx.doi.org/10.7472/jksii.2017.18.4.61   DOI
11 Papandreou, George, et al. "Towards Accurate Multi-person Pose Estimation in the Wild." arXiv preprint arXiv:1701.01779, 2017. https://arxiv.org/abs/1701.01779
12 Bulat, Adrian, and Georgios Tzimiropoulos., "Human pose estimation via convolutional part heatmap regression," European Conference on Computer Vision, pp. 717-732, 2016. https://doi.org/10.1007/978-3-319-46478-7_44
13 Linna, Marko, Juho Kannala, and Esa Rahtu., "Real-time human pose estimation from video with convolutional neural networks," arXiv preprint arXiv:1609.07420, 2016. https://arxiv.org/abs/1609.07420
14 Ramakrishna, Varun, et al., "Pose machines: Articulated pose estimation via inference machines," European Conference on Computer Vision, pp. 33-47, 2014. https://doi.org/10.1007/978-3-319-10605-2_3
15 Andriluka, Mykhaylo, et al., "2d human pose estimation: New benchmark and state of the art analysis," Proceedings of the IEEE Conference on computer Vision and Pattern Recognition, pp. 3686-3693, 2014. https://doi.org/10.1109/cvpr.2014.471
16 Belagiannis, Vasileios, and Andrew Zisserman., "Recurrent human pose estimation," Automatic Face & Gesture Recognition (FG 2017), 2017 12th IEEE International Conference on. IEEE, pp. 468-475, 2017. https://doi.org/10.1109/fg.2017.64
17 Google, "MNIST For ML Beginners,". https://www.tensorflow.org
18 Mehta, Dushyant, et al., "VNect: Real-time 3D Human Pose Estimation with a Single RGB Camera," arXiv preprint arXiv:1705.01583, 2017. https://arxiv.org/abs/1705.01583
19 Ramakrishna, Varun, Takeo Kanade, and Yaser Sheikh., "Reconstructing 3d human pose from 2d image landmarks," Computer Vision-ECCV 2012, pp. 573-586, 2012. https://doi.org/10.1007/978-3-642-33765-9_41
20 Grant, Jason M., and Patrick J. Flynn., "Crowd Scene Understanding from Video: A Survey," ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM), Vol 13, No. 2, pp. 19, 2017. https://doi.org/10.1145/3052930