Browse > Article

Real-Time Hand Pose Tracking and Finger Action Recognition Based on 3D Hand Modeling  

Suk, Heung-Il (고려대학교 컴퓨터학과)
Lee, Ji-Hong (고려대학교 컴퓨터학과)
Lee, Seong-Whan (고려대학교 컴퓨터.통신공학부)
Publication Information
Journal of KIISE:Software and Applications / v.35, no.12, 2008 , pp. 780-788 More about this Journal
Abstract
Modeling hand poses and tracking its movement are one of the challenging problems in computer vision. There are two typical approaches for the reconstruction of hand poses in 3D, depending on the number of cameras from which images are captured. One is to capture images from multiple cameras or a stereo camera. The other is to capture images from a single camera. The former approach is relatively limited, because of the environmental constraints for setting up multiple cameras. In this paper we propose a method of reconstructing 3D hand poses from a 2D input image sequence captured from a single camera by means of Belief Propagation in a graphical model and recognizing a finger clicking motion using a hidden Markov model. We define a graphical model with hidden nodes representing joints of a hand, and observable nodes with the features extracted from a 2D input image sequence. To track hand poses in 3D, we use a Belief Propagation algorithm, which provides a robust and unified framework for inference in a graphical model. From the estimated 3D hand pose we extract the information for each finger's motion, which is then fed into a hidden Markov model. To recognize natural finger actions, we consider the movements of all the fingers to recognize a single finger's action. We applied the proposed method to a virtual keypad system and the result showed a high recognition rate of 94.66% with 300 test data.
Keywords
3D Hand Pose Tracking; Belief Propagation; Probabilistic Graphical Model; Hidden Markov Model; Virtual Keypad; Human-Computer Interaction;
Citations & Related Records
연도 인용수 순위
  • Reference
1 A. Heap and D. Hogg, "Improving Specificity in PDMs using a Hierarchical Approach," Proc. British Machine Vision Conference, Essex, UK, Vol. 1, pp. 80-89, Sept. 1997
2 J. Kuch and T. Huang, "Vision based Hand Modeling and Tracking for Virtual Teleconferencing and Telecollaboration," Proc. 5th International Conference on Computer Vision, Cambridge, USA, pp. 666-671, June 1995
3 C. Bishop, Pattern Recognition and Machine Learning, Chapter 8, Springer, 2007
4 M. Isard and A. Blake, "CONDENSATION - Conditional Density Propagation for Visual Tracking," International Journal of Computer Vision, Vol. 29, No. 1, pp. 5-28, Aug. 1998   DOI   ScienceOn
5 M. Vittrup, M. Srensen, and B. McCane, "Pose Estimation by Applied Numerical Techniques," Proc. Image and Vision Computing New Zealand, Auckland, New Zealand, Vol. 2, pp. 35-38, Nov. 2002
6 H. Rijpkema and M. Girard, "Computer Animation of Knowledge-based Human Grasping," Proc. International Conference on Computer Graphics and Interactive Techniques, New York, USA, Vol. 25, No. 4, pp. 339-348, Aug. 1991
7 N. Shimada, Y. Shirai, Y. Kuno, and J. Miura. "Hand Gesture Estimation and Model Refinement using Monocular Camera Ambiguity Limitation by Inequality Constraints," Proc. 3rd IEEE International Conference on Automatic Face and Gesture Recognition, Nara, Japan, pp. 268-273, 1998
8 J. Deutscher, A. Blake, and I. Reid, "Articulated Body Motion Capture by Annealed Particle Filtering," Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, South California, USA, Vol. 2, pp. 126-133, June 2000
9 J. Lee and T. Knuii, "Model-based Analysis of Hand Posture," Proc. IEEE Computer Graphics and Application, New York, USA, Vol. 15, No. 5, pp. 77-86, 1995   DOI   ScienceOn
10 Y. Wu and T. Huang, "View-Independent Recognition of Hand Postures," Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, South California, USA, Vol. 2, pp. 88-94, June 2000
11 P. Viola and M. Jones, "Robust Real-Time Face Detection," International Journal of Computer Vision, Vol. 57, No. 2, pp. 137-154, 2004   DOI
12 T. Han, H. Ning, and T. Huang, "Efficient Nonparametric Belief Propagation with Application to Articulated Body Tracking," Proc. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, USA, Vol. 1, pp. 214-221, June 2006
13 O. Bernier and P. Cheung-Mon-Chan, "Real-Time 3D Articulated Pose Tracking using Particle Filters Interacting through Belief Propagation," Proc. 18th IAPR/IEEE International Conference on Pattern Recognition, Hong Kong, China, Vol. 1, pp. 90-93, Aug. 2006
14 M. Tosas and B. Li, "Virtual Touch Screen for Mixed Reality," Proc. European Conference on Computer Vision, Lecture Notes in Computer Science, Prague, Czech Republic, Vol. 3058, pp. 48-59, May 2004
15 B. Stenger, A. Thayananthan, P. Torr, and R. Cipolla, "Hand Pose Estimation Using Hierarchical Detection," Proc. European Conference on Computer Vision, Lecture Notes in Computer Science, Prague, Czech Republic, Vol. 3058, pp. 105-116, May 2004
16 L. Rabiner, "A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition," Proceedings of the IEEE, Vol. 77, No. 2, pp. 257-285, Feb. 1989   DOI   ScienceOn
17 J. Rehg and T. Kanade, "Model-based Tracking of Self-Occluding Articulated Object," Proc. 5th International Conference on Computer Vision, Cambridge, USA, pp. 612-617, June. 1995
18 R. Rosales, S. Sclaroff, and V. Athitsos, "3D Hand Pose Reconstruction using Specialized Mappings," Proc. 8th IEEE International Conference on Computer Vision, Vancouver, Canada, Vol. 1, pp. 378-385, July 2001
19 Y. Wu and T. Huang, "Capturing Articulated Human Hand Motion: A Divide-and-Conquer Approach," Proc. 7th IEEE International Conference on Computer Vision, Kerkyra, Greece, Vol. 1, pp. 606-611, 1999