Browse > Article
http://dx.doi.org/10.9718/JBER.2021.42.4.186

Hand Gesture Recognition with Convolution Neural Networks for Augmented Reality Cognitive Rehabilitation System Based on Leap Motion Controller  

Song, Keun San (Department of Biomedical Engineering & Science, Graduate School of Konyang University)
Lee, Hyun Ju (Department of Physical Therapy, Konyang University)
Tae, Ki Sik (Department of Biomedical Engineering & Science, Graduate School of Konyang University)
Publication Information
Journal of Biomedical Engineering Research / v.42, no.4, 2021 , pp. 186-192 More about this Journal
Abstract
In this paper, we evaluated prediction accuracy of Euler angle spectrograph classification method using a convolutional neural networks (CNN) for hand gesture recognition in augmented reality (AR) cognitive rehabilitation system based on Leap Motion Controller (LMC). Hand gesture recognition methods using a conventional support vector machine (SVM) show 91.3% accuracy in multiple motions. In this paper, five hand gestures ("Promise", "Bunny", "Close", "Victory", and "Thumb") are selected and measured 100 times for testing the utility of spectral classification techniques. Validation results for the five hand gestures were able to be correctly predicted 100% of the time, indicating superior recognition accuracy than those of conventional SVM methods. The hand motion recognition using CNN meant to be applied more useful to AR cognitive rehabilitation training systems based on LMC than sign language recognition using SVM.
Keywords
Euler angle spectrograph; Convolutional neural network (CNN); Hand gesture recognition; Augmented reality (AR); Cognitive rehabilitation system; Leap motion controller (LMC);
Citations & Related Records
연도 인용수 순위
  • Reference
1 Tran DS, Ho NH, Yang HJ, Baek ET. Real-Time Hand Gesture Spotting and Recognition Using RGB-D Camera and 3D Convolutional Neural Network. Applied Sciences. 2020;10(2):722.   DOI
2 Alankus G, Lazar A, May M, Kelleher C. Towards customizable games for stroke rehabilitation. Proceedings of the 28th Annual CHI Conference on Human Factors in Computing Systems (CHI '10). 2010;2113-2122.
3 Annema JH, Verstraete M, Abeele VV, Desmet S. Video games in therapy: a therapist's perspective. International Journal of Arts and Technology. 2010;6(1):94-98.
4 Doidge N. The Brain That Changes Itself: Stories of Personal Triumph from the Frontiers of Brain Science. Penguin Group. 2007.
5 Badi H. Recent methods in vision-based hand gesture recognition. International Journal of Data Science and Analytics. 2016;1:77-87.   DOI
6 Khelil B, Amiri H, Hand Gesture Recognition Using leap motion controller for recognition of arabic sign language. Computer Science. 2016.
7 Huang DY, Hu WC, Chang SH. Vision-Based Hand Gesture Recognition Using PCA+Gabor Filters and SVM. 15th International Conference on Intelligent Information Hiding and Multimedia Signal Processing, 2009.
8 Cote-Allard U, Fall CL, Drouin A, Campeau-Lecours A, Gosselin C, Glette K, Laviolette F, Gosselin B. Deep Learning for Electromyographic Hand Gesture Signal Classification Using Transfer Learning. IEEE Transactions on Neural Systems and Rehabilitation Engineering. 2019;27(4):760-771.   DOI
9 Radianti J, Majchrzak T, Fromm J, Wohlgenannt I. A systematic review of immersive virtual reality applications for higher education: Design elements, lessons learned, and research agenda. Computers & Education, 2020;147:103778.   DOI
10 Huang DY, Hu WC, Chang SH. Gabor filter-based handpose angle estimation for hand gesture recognition under varying illumination. Expert Systems with Applications, 2011;38(5): 6031-6042.   DOI
11 Burdea CG. Virtual Rehabilitation: benefits and Challenges. Methods of Information in Medicine. 2003;42(5):519-523.   DOI
12 Burke JW, McNeill M, Charles D. Morrow P. Crosbie J. McDonough S. Designing engaging, playable games for rehabilitation. Proceedings of the 8th International Conference On Disability, 2010.
13 Donker T, Cornelisz I, van Klaveren C, van Straten A, Carlbring P, Cuijpers P, van Gelder JL. Effectiveness of self-guided app-based virtual reality cognitive behavior therapy for acrophobia: A randomized clinical trial. JAMA Psychiatry. 2019;76:682-690.   DOI
14 Tsao YC, Shu CC, Lan TS. Development of a Reminiscence Therapy System forthe Elderly Using the Integration of Virtual Realityand Augmented Reality. Sustainability. 2019;11:2-10.   DOI
15 Sagayam KM, Hemanth DJ. Hand posture and gesture recognition techniques for virtual reality applications: a survey. Virtual Reality. 2017;21:91-107.   DOI
16 Andersson G, Carlbring P, Titov N, Lindefors N. Internet interventions for adults with anxiety and mood disorders: A narrative umbrella review of recent meta-analyses. Can. J. Psychiatry. 2019;64(7):465-470.   DOI
17 Stinghen I, Chen E, da Silva JM. da Silva R. Gesture recognition using leap motion: a comparison between machine learning algorithms. ACM SIGGRAPH 2018 Posters, 2018.
18 Panic AS. Addressing patient motivation in virtual reality based neurocognitive rehabilitation. Delft University of Technology, 2010.