Browse > Article
http://dx.doi.org/10.6109/jkiice.2021.25.12.1770

Hand Expression Recognition for Virtual Blackboard  

Heo, Gyeongyong (Department of Electronic Engineering, Dong-eui University)
Kim, Myungja (Department of Nursing, Dong-eui University)
Song, Bok Deuk (Intelligent Convergence Research Laboratory, ETRI)
Shin, Bumjoo (Department of Applied IT & Engineering, Pusan National University)
Publication Information
Journal of the Korea Institute of Information and Communication Engineering / v.25, no.12, 2021 , pp. 1770-1776 More about this Journal
Abstract
For hand expression recognition, hand pose recognition based on the static shape of the hand and hand gesture recognition based on hand movement are used together. In this paper, we proposed a hand expression recognition method that recognizes symbols based on the trajectory of a hand movement on a virtual blackboard. In order to recognize a sign drawn by hand on a virtual blackboard, not only a method of recognizing a sign from a hand movement, but also hand pose recognition for finding the start and end of data input is also required. In this paper, MediaPipe was used to recognize hand pose, and LSTM(Long Short Term Memory), a type of recurrent neural network, was used to recognize hand gesture from time series data. To verify the effectiveness of the proposed method, it was applied to the recognition of numbers written on a virtual blackboard, and a recognition rate of about 94% was obtained.
Keywords
Hand expression; Hand gesture recognition; MediaPipe; Recurrent neural network; LSTM;
Citations & Related Records
연도 인용수 순위
  • Reference
1 G. SantoshiEmail, P. Parwekar, G. G. Pushpa, and T. Kranthi, "Multiple Hand Gestures for Cursor Movement Using Convolution Neural Networks," in Intelligent System Design, Springer, pp. 813-825, 2020.
2 F. Karim, S. Majumdar, H. Darabi, and S. Chen, "LSTM Fully Convolutional Networks for Time Series Classification," IEEE Access, vol. 6, pp. 1662-1669, Dec. 2017.   DOI
3 G. Pala, J. B. Jethwani, S. S. Kumbhar, and S. D. Patil, "Machine Learning-based Hand Sign Recognition," in Proceeding of 2021 International Conference on Artificial Intelligence and Smart Systems, Coimbatore, India, pp. 356-363, 2021.
4 G. Heo, B. D. Song, and J. H. Kim, "Hierarchical Hand Pose Model for Hand Expression Recognition," Journal of the Korea Institute of Information and Communication Engineering, accepted for publication, vol. 25, no. 10, pp. 1323-1329, 2021.
5 MediaPipe [Internet]. Available: https://mediapipe.dev.
6 S. Smys, J. I. Z. Chen, and S. Shakya, "Survey on Neural Network Architectures with Deep Learning," Journal of Soft Computing Paradigm, vol. 2, no. 3, pp. 186-194, Sept. 2020.   DOI
7 T. Y. Lin, P. Dollar, R. Girshick, K. He, B. Hariharan, and S. Belongie, "Feature Pyramid Networks for Object Detection," in Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, USA, pp. 936-944, 2017.
8 P. C. Vashist, A. Pandey, and A. Tripathi, "A Comparative Study of Handwriting Recognition Techniques," in Proceeding of 2020 International Conference on Computation, Automation and Knowledge Management, Dubai, United Arab Emirates, pp. 456-461, 2020.
9 F. Karim, S. Majumdar, and H. Darabi, "Insights Into LSTM Fully Convolutional Networks for Time Series Classification," IEEE Access, vol. 7, pp. 67718-67725, May. 2019.   DOI
10 W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg, "SSD: Single Shot MultiBox Detector," in Proceedings of 2016 European Conference on Computer Vision, Amsterdam, Netherlands, pp. 21-37, 2016.
11 T. H. Tsai, C. C. Huang, and K. L. Zhang, "Design of hand gesture recognition system for human-computer interaction," Multimedia Tools and Applications, vol. 79, no. 9-10, pp. 5989-6007, Feb. 2020.   DOI
12 R. R. Itkarkar and A. V. Nandi, "A survey of 2D and 3D imaging used in hand gesture recognition for human-computer interaction (HCI)," in Proceeding of 2016 IEEE International WIE Conference on Electrical and Computer Engineering, Pune, India, pp. 188-193, 2016.