Browse > Article
http://dx.doi.org/10.6109/jkiice.2021.25.10.1323

Hierarchical Hand Pose Model for Hand Expression Recognition  

Heo, Gyeongyong (Department of Electronic Engineering, Dong-eui University)
Song, Bok Deuk (Intelligent Convergence Research Laboratory, ETRI)
Kim, Ji-Hong (Department of Electronic Engineering, Dong-eui University)
Publication Information
Journal of the Korea Institute of Information and Communication Engineering / v.25, no.10, 2021 , pp. 1323-1329 More about this Journal
Abstract
For hand expression recognition, hand pose recognition based on the static shape of the hand and hand gesture recognition based on the dynamic hand movement are used together. In this paper, we propose a hierarchical hand pose model based on finger position and shape for hand expression recognition. For hand pose recognition, a finger model representing the finger state and a hand pose model using the finger state are hierarchically constructed, which is based on the open source MediaPipe. The finger model is also hierarchically constructed using the bending of one finger and the touch of two fingers. The proposed model can be used for various applications of transmitting information through hands, and its usefulness was verified by applying it to number recognition in sign language. The proposed model is expected to have various applications in the user interface of computers other than sign language recognition.
Keywords
Hand expression; Hand pose recognition; Hierarchical model; Mediapipe; Sign language;
Citations & Related Records
연도 인용수 순위
  • Reference
1 T. Chatzis, A. Stergioulas, D. Konstantinidis, K. Dimitropoulos, and P. Daras, "A Comprehensive Study on Deep Learning-Based 3D Hand Pose Estimation Methods," Applied Sciences, vol. 10, no. 19, pp. 1-27, Sep. 2020
2 A. Singh, S. Agarwal, P. Nagrath, A. Saxena, and N. Thakur, "Human Pose Estimation Using Convolutional Neural Networks," in Proceedings of 2019 Amity International Conference on Artificial Intelligence, Dubai, United Arab Emirates, pp. 946-952, 2019.
3 M. Pandiya, S. Dassani, and P. Mangalraj, "Analysis of Deep Learning Architectures for Object Detection - A Critical Review," in Proceedings of 2020 IEEE-HYDCON International Conference on Engineering in the 4th Industrial Revolution, Hyderabad, India, pp. 1-6, 2020.
4 M. Patel and N. Kalani, "A Survey on Pose Estimation using Deep Convolutional Neural Networks," in Proceedings of 2nd International Conference on Machine Learning, Security and Cloud Computing, Hyderabad, India, pp. 1-9, 2020.
5 R. Berral-Soler, F. J. Madrid-Cuevas, R. Munoz-Salinas, and M. J. Marin-Jimwnez, "RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild," Neural Computing and Applications, vol. 33, no. 13, pp. 7673-7689, July. 2021.   DOI
6 MediaPipe [Internet]. Available: https://mediapipe.dev.
7 W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg, "SSD: Single Shot MultiBox Detector," in Proceedings of 2016 European Conference on Computer Vision, Amsterdam, Netherlands, pp. 21-37, 2016.
8 T. Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, "Feature Pyramid Networks for Object Detection," in Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, pp. 936-944, 2017.
9 M. M. Rahman, Y. Tan, J. Xue, and K. Lu, "Recent Advances in 3D Object Detection in the Era of Deep Neural Networks: A Survey," IEEE Transactions on Image Processing, vol. 29, pp. 2947-2962, 2020.   DOI
10 X. Ji, Q. Fang, J. Dong, Q Shuai, W. Jiang, and X. Zhou, "A survey on monocular 3D human pose estimation," Virtual Reality & Intelligent Hardware, vol. 2, no. 6, pp. 471-500, Dec. 2020.   DOI