Browse > Article
http://dx.doi.org/10.3837/tiis.2020.05.006

An ANN-based gesture recognition algorithm for smart-home applications  

Huu, Phat Nguyen (School of Electronics and Telecommunications, Hanoi University of Science and Technology)
Minh, Quang Tran (Faculty of Computer Science and Engineering, Ho Chi Minh City University of Technology, VNU-HCM)
The, Hoang Lai (School of Electronics and Telecommunications, Hanoi University of Science and Technology)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.14, no.5, 2020 , pp. 1967-1983 More about this Journal
Abstract
The goal of this paper is to analyze and build an algorithm to recognize hand gestures applying to smart home applications. The proposed algorithm uses image processing techniques combing with artificial neural network (ANN) approaches to help users interact with computers by common gestures. We use five types of gestures, namely those for Stop, Forward, Backward, Turn Left, and Turn Right. Users will control devices through a camera connected to computers. The algorithm will analyze gestures and take actions to perform appropriate action according to users requests via their gestures. The results show that the average accuracy of proposal algorithm is 92.6 percent for images and more than 91 percent for video, which both satisfy performance requirements for real-world application, specifically for smart home services. The processing time is approximately 0.098 second with 10 frames/sec datasets. However, accuracy rate still depends on the number of training images (video) and their resolution.
Keywords
3-Dimensional Convolutional Network; Human-Computer Interaction; Smart-home; Machine Learning; IoT Applications;
Citations & Related Records
연도 인용수 순위
  • Reference
1 P. N. Huu, V. T. Quang, and T. Miyoshi, "Multi-hop Reed-Solomon encoding scheme for image transmission on wireless sensor networks," in Proc. of 2012 Fourth International Conference on Communications and Electronics (ICCE), Hue, Vietnam, 74-79, 2012.
2 P. N. Huu, V. T. Quang, and T. Miyoshi, "Low-Complexity and Energy-Efficient Algorithms on Image Compression for Wireless Sensor Networks," IEICE Transactions on Communications, E93-B (12), 3438-3447, 2010.   DOI
3 V. T. Quang, P. N. Huu and T. Miyoshi, "Adaptive transmission range assignment algorithm for in-routing image compression on wireless sensor networks," in Proc. of International Conference on Communications and Electronics 2010, Nha Trang, Vietnam, 18-23, 2010.
4 S. Bilal, R. Akmeliawati, and M. Salami, "A hybrid method using haar-like and skin-color algorithm for hand posture detection, recognition and tracking," in Proc. of IEEE Intl Conf. on Mechatronics and Automation, 4-7, 2010.
5 Z. Zhao, P. Zheng, S. Xu, and X. Wu, "Object detection with deep learning: a review," IEEE Transactions on Neural Networks and Learning Systems, 30(11), 3212-3232, 2019.   DOI
6 W. Liu, D. Anguelov, D. Erhan, C. Szegedy, S. Reed, C. Y. Fu, and A. C. Berg, "SSD: Single shot multiBox detector," Computer Vision and Pattern Recognition (ECCV), 21-37, 2016.
7 J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, "You only look once: unified, real-time object detection," in Proc. of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, 779-788, 2016.
8 M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. Chen, "MobileNetV2: inverted residuals and linear bottlenecks," in Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 4510-4520, 2018.
9 Y. Freund and R. E. Schapire, "A Short Introduction to Boosting," Journal of Japanese Society for Artificial Intelligence, 14(5), 771-780, 1999.
10 H. Hasan and S. Abdul-Kareem, "RETRACTED ARTICLE: Human-computer interaction using vision-based hand gesture recognition systems: a survey," Neural Comput & Applic, 25, 251-261, 2014.   DOI
11 M. V. Lamar, "Hand gesture recognition using T-Comb net a neural network model dedicated to temporal information processing," Ph.D. thesis, Nagoya Institute of Technology, Japan, 2002.
12 S. S. Rautaray and A. Agrawal, "Vision based hand gesture recognition for human computer interaction: a survey," Artif Intell Rev., 43, 1-54, 2015.   DOI
13 P. Viola and M. Jones, "Rapid object detection using a boosted cascade of simple features," in Proc. of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, Kauai, HI, USA, I-I, 2001.
14 C. C. Hsieh and D. H. Liou, "Novel Haar features for real-time hand gesture recognition using SVM," J Real-Time Image Proc, 10, 357-370, 2015.   DOI
15 P. Barros, N. T. M. Junior, B. J. T. Fernandes, B. L. D. Bezerra , and S. M. M. Fernandes, "A dynamic gesture recognition and prediction system using the convexity approach," Computer Vision and Image Understanding, 155, 139-149, 2017.   DOI
16 C. Huang and W. Huang, "Sign language recognition using model-based tracking and a 3D Hopfield neural network," Machine Vision and Applications, 10, 292-307, 1998.   DOI
17 S. Bilal, R. Akmeliawati, M. J. E. Salami, A. A. Shafie, and E. M. Bouhabba, "A hybrid method using haar-like and skin-color algorithm for hand posture detection, recognition and tracking," in Proc. of 2010 IEEE International Conference on Mechatronics and Automation, Xi'an, 934-939, 2010.
18 V. Dibia, "How to Build a Real-time Hand-Detector using Neural Networks (SSD) on Tensorflow," GitHub repository, 2017.
19 T. B. Neurons, "Gesture recognition using end-to-end learning from a large video database," Medium Corporation, 2017.
20 N. Dalal and B. Triggs, "Histograms of oriented gradients for human detection," in Proc. of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05), San Diego, CA, USA, 886-893, 2005.
21 M. Kounavis, "Fingertip detection without the use of depth data, color information, or large training data sets," in Proc. of 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Banff, AB, 2396-2401, 2017.
22 K. Feng and F. Yuan, "Static hand gesture recognition based on HOG characters and support vector machines," in Proc. of 2013 2nd International Symposium on Instrumentation and Measurement, Sensor Network and Automation (IMSNA), Toronto, 936-938, 2013.
23 P. N. Huu and H. L. The, "Low-Complexity Image Processing Algorithm for Estimating Distance and Actual Size of Subject," in Proc. of 7th International Conference on Communications and Electronics (ICCE 2018), Hue, Vietnam, 463-468, 2018.
24 G. Tofighi, A. Venetsanopoulos, K. Raahemifar, S. Beheshti, and H. Mohammadi, "Hand posture recognition using K-NN and Support Vector Machine classifiers evaluated on our proposed HandReader dataset." in Proc. of 18th International Conference on Digital Signal Processing (DSP), 1-5, 2013.
25 E. Stergiopoulou and N. Papamarkos, "Hand gesture recognition using a neural network shape fitting technique," Engineering Applications of Artificial Intelligence, 22 (8), 1141-1158, 2009.   DOI