KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

A Dynamic Hand Gesture Recognition System Incorporating Orientation-based Linear Extrapolation Predictor and Velocity-assisted Longest Common Subsequence Algorithm

  • Yuan, Min (Shanghai Key Lab of Modern Optical System, and Engineering Research Center of Optical Instrument and System, Ministry of Education, University of Shanghai for Science and Technology) ;
  • Yao, Heng (Shanghai Key Lab of Modern Optical System, and Engineering Research Center of Optical Instrument and System, Ministry of Education, University of Shanghai for Science and Technology) ;
  • Qin, Chuan (Shanghai Key Lab of Modern Optical System, and Engineering Research Center of Optical Instrument and System, Ministry of Education, University of Shanghai for Science and Technology) ;
  • Tian, Ying (Shanghai Key Lab of Modern Optical System, and Engineering Research Center of Optical Instrument and System, Ministry of Education, University of Shanghai for Science and Technology)
  • Received : 2016.06.15
  • Accepted : 2017.05.25
  • Published : 2017.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

The present paper proposes a novel dynamic system for hand gesture recognition. The approach involved is comprised of three main steps: detection, tracking and recognition. First, the gesture contour captured by a 2D-camera is detected by combining the three-frame difference method and skin-color elliptic boundary model. Then, the trajectory of the hand gesture is extracted via a gesture-tracking algorithm based on an occlusion-direction oriented linear extrapolation predictor, where the gesture coordinate in next frame is predicted by the judgment of current occlusion direction. Finally, to overcome the interference of insignificant trajectory segments, the longest common subsequence (LCS) is employed with the aid of velocity information. Besides, to tackle the subgesture problem, i.e., some gestures may also be a part of others, the most probable gesture category is identified through comparison of the relative LCS length of each gesture, i.e., the proportion between the LCS length and the total length of each template, rather than the length of LCS for each gesture. The gesture dataset for system performance test contains digits ranged from 0 to 9, and experimental results demonstrate the robustness and effectiveness of the proposed approach.

Keywords

References

  1. Siddharth S. Rautaray and Anupam Agrawal, "Vision based hand gesture recognition for human computer interaction: a survey," Artificial Intelligence Review, vol. 43, no. 1, pp. 1-54, January, 2015. https://doi.org/10.1007/s10462-012-9356-9
  2. S. Padam Priyal and Prabin Kumar Bora, "A robust static hand gesture recognition system using geometry based normalizations and Krawtchouk moments," Pattern Recognition, vol. 46, no. 8, pp. 2202-2219, August, 2013. https://doi.org/10.1016/j.patcog.2013.01.033
  3. Rein Lien Hsu, Mohamed Abdel Mottaleb and Anil K. Jain, "Face detection in color images," IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 24, no. 5, pp. 696-706, May, 2002. https://doi.org/10.1109/34.1000242
  4. Wei Ren Tan, Chee Seng Chan, Pratheepan Yogarajah and Joan Condell, "A fusion approach for efficient human skin detection," IEEE Transactions on Industrial Informatics, vol. 8, no. 1, pp. 138-147, February, 2012. https://doi.org/10.1109/TII.2011.2172451
  5. Marko Subasic, Sven Loncaric and Adam Hedi, "Segmentation and labeling of face images for electronic documents," Expert Systems with Applications, vol. 39, no. 5, pp. 5134-5143, April, 2012. https://doi.org/10.1016/j.eswa.2011.11.027
  6. Stan Z. Li and Zhenqiu Zhang, "FloatBoost learning and statistical face detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1112-1123, September, 2004. https://doi.org/10.1109/TPAMI.2004.68
  7. Zhou Ren, Junsong Yuan, Jingjing Meng and Zhengyou Zhang, "Robust part-based hand gesture recognition using Kinect sensor," IEEE Transactions on Multimedia, vol. 15, no. 5, pp. 1110-1120, August, 2013. https://doi.org/10.1109/TMM.2013.2246148
  8. Min Chun Hu, Ming Hsiu Chang, Ja Ling Wu and Lin Chi, "Robust camera calibration and player tracking in broadcast basketball video," IEEE Transactions on Multimedia, vol. 13, no. 2, pp. 266-279, April, 2011. https://doi.org/10.1109/TMM.2010.2100373
  9. Kuo Hsien Hsia, Shao Fan Lien and Juhng Perng Su, "Moving target tracking based on CamShift approach and Kalman filter," Applied Mathematics & Information Sciences, vol. 9, no. 1, pp. 395-401, 2015. https://doi.org/10.12785/amis/090146
  10. Peixun Liu,Wenhui Li, Ying Wang and Hongyin Ni, "On-road multi-vehicle tracking algorithm based on an improved particle filter," IET Intelligent Transport Systems, vol. 9, no. 4, pp. 429-441, May, 2014. https://doi.org/10.1049/iet-its.2014.0088
  11. Junghyun Kwon, Hee Seok Lee, Frank C. Park and Kyoung Mu Lee, "A Geometric Particle Filter for Template-Based Visual Tracking," IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 36, no. 4, pp. 625-643, September, 2013. https://doi.org/10.1109/TPAMI.2013.170
  12. Cheng Tse Chiang, Po Hsuan Tseng and Kai Ten Feng, "Hybrid unified Kalman tracking algorithms for heterogeneous wireless location systems," IEEE Transactions on Vehicular Technology, vol. 61, no. 2, pp. 702-715, Febrary, 2012. https://doi.org/10.1109/TVT.2011.2180939
  13. Emmanuel Marilly, Arnaud Gonguet, Olivier Martinot and Frederique Pain, "Gesture interactions with video: From algorithms to user evaluation," Bell Labs Technical Journal, vol. 17, no.4, pp. 103-118, March, 2013. https://doi.org/10.1002/bltj.21577
  14. Heung-ii Suk, Bong Kee Sin and Seong Whan Lee, "Hand gesture recognition based on dynamic Bayesian network framework," Pattern Recognition, vol. 43, no. 9, pp. 3059-3072, September, 2010. https://doi.org/10.1016/j.patcog.2010.03.016
  15. Antonis A. Argyros and Manolis I. A. Lourakis, "Real-time tracking of multiple skin-colored objects with a possibly moving camera," Lecture Notes in Computer Science, vol. 3023, no. 3, pp. 368-379, 2004.
  16. Hyeon Kyu Lee and Jin H. Kim, "An HMM-based threshold model approach for gesture recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, no. 10, pp. 961-973, October, 1999. https://doi.org/10.1109/34.799904
  17. K. M. Vamsikrishna, Debi Prosad Dogra and Maunendra Sankar Desarkar, "Computer-vision-assisted palm rehabilitation with supervised learning," IEEE Transactions on Biomedical Engineering, vol. 63, no. 5, pp. 991-1001, May, 2016. https://doi.org/10.1109/TBME.2015.2480881
  18. Cristian Sminchisescu, Atul Kanaujia and Dimitris Metaxas, "Conditional models for contextual human motion recognition," Computer Vision and Image Understanding, vol. 104, no. 2, pp. 210-220, November-December, 2006. https://doi.org/10.1016/j.cviu.2006.07.014
  19. Sotirios P. Chatzis, Dimitrios I. Kosmopoulos and Paul Doliotis, "A conditional random field-based model for joint sequence segmentation and classification," Pattern Recognition, vol. 46, no. 6, pp. 1569-1578, June, 2013. https://doi.org/10.1016/j.patcog.2012.11.028
  20. Darya Frolova, Helman Stern and Sigal Berman, "Most probable longest common subsequence for recognition of gesture character input," IEEE Transactions on Cybernetics, vol. 43, no. 3, pp. 871-880, June, 2013. https://doi.org/10.1109/TSMCB.2012.2217324
  21. Helman Stern, Merav Shmueli and Sigal Berman, "Most discriminating segment-Longest common subsequence (MDSLCS) algorithm for dynamic hand gesture classification," Pattern Recognition Letters, vol. 34, no. 15, pp. 1980-1989, November, 2013. https://doi.org/10.1016/j.patrec.2013.02.007
  22. Jinfu Yang, Wanlu Yang and Mingai Li, "An efficient moving object detection algorithm based on improved GMM and cropped frame technique," in Proc. of IEEE International Conf. on Mechatronics and Automation, pp. 658-663, August 5-8, 2012.
  23. Jinhui Lan, Min Guo and Xiaojie Liu, "Real-time detection algorithm for moving vehicles in dynamic traffic environment," in Proc. of IEEE International Conf. on Electro/Information Technology (EIT), pp. 1-6, May 9-11, 2013.
  24. Paul Viola and Michael J. Jones, "Robust real-time face detection," International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, May, 2004. https://doi.org/10.1023/B:VISI.0000013087.49260.fb

Cited by

  1. A Hand Gesture Recognition Method using Inertial Sensor for Rapid Operation on Embedded Device vol.14, pp.2, 2020, https://doi.org/10.3837/tiis.2020.02.016