Browse > Article
http://dx.doi.org/10.9708/jksci.2012.17.7.155

The Estimation of Hand Pose Based on Mean-Shift Tracking Using the Fusion of Color and Depth Information for Marker-less Augmented Reality  

Lee, Sun-Hyoung (Dept. of Electronic Engineering, Soongsil University)
Hahn, Hern-Soo (Dept. of Information Communication & Electronic Engineering, Soongsil University)
Han, Young-Joon (Dept. of Information Communication & Electronic Engineering, Soongsil University)
Publication Information
Journal of the Korea Society of Computer and Information / v.17, no.7, 2012 , pp. 155-166 More about this Journal
Abstract
This paper proposes a new method of estimating the hand pose through the Mean-Shift tracking algorithm using the fusion of color and depth information for marker-less augmented reality. On marker-less augmented reality, the most of previous studies detect the hand region using the skin color from simple experimental background. Because finger features should be detected on the hand, the hand pose that can be measured from cameras is restricted considerably. However, the proposed method can easily detect the hand pose from complex background through the new Mean-Shift tracking method using the fusion of the color and depth information from 3D sensor. The proposed method of estimating the hand pose uses the gravity point and two random points on the hand without largely constraints. The proposed Mean-Shift tracking method has about 50 pixels error less than general tracking method just using color value. The augmented reality experiment of the proposed method shows results of its performance being as good as marker based one on the complex background.
Keywords
Marker-less Augmented Reality; Mean Shift Tracking; 3D Sensor; Hand Pose; Hand Detection;
Citations & Related Records
Times Cited By KSCI : 7  (Citation Analysis)
연도 인용수 순위
1 OpenNI. (n.d.). OpenNI documentation. http://www.openni.org/ images/ stories/ pdf /OpenNI_UserGuide_v4.pdf
2 Rovelo, G. (n.d.). ARTOOLKIT II. from https://jira.ai2.upv.es/confluence/download/attachments/12222496/WGM18_ARToolKitII.pdf?version=1&modificationDate=1304095263000
3 Kwangsoo Kim, Sooyoun Hong, Sooyeong Kwak, Jungho Ahn, Hyeran Byun, "Multiple Human Tracking using Mean Shift and Depth Map with a Moving Stereo Camera," The korean Institute of Ingormation Scientists and Engineers, Vol.34, no.10, pp.937-944, 2007 October.   과학기술학회마을
4 Y. Cheng, "Mean shift, mode seeking, and clustering," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol.17, pp.790-799, 1995.   DOI   ScienceOn
5 D. Comaniciu, "Kernel-Based Object Tracking," IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, vol. 25, no. 5, May 2003.
6 Sang-Geol Lee, Cheol-Ki Kim, Eui-Young Cha, "Retouching Method for Watercolor Painting Effect Using Mean Shift Segmentation," The Korea Society of Computer Information, Vol.15, no.9, pp.25-33, 2010 September.   과학기술학회마을   DOI   ScienceOn
7 Heeman Lee, "Implementing Augmented Reality By Using Face Detection, Recognition And Motion Tracking," The Korea Society of Computer Information, Vol.17, no.1, pp.97-104, 2012 January.   과학기술학회마을   DOI
8 Ju-Hyun Lee, "A Development Strategy of Augmented Reality Contents in the Contextual Environments," Human Contents, no.19, pp.179-218, 2010 November.
9 Kyung-Hee Noh, Hyung-Keun Jee, Sukhyun Lim, "Effect of Augmented Reality Contents Based Instruction on Academic Achievement, Interest and Flow of Learning," Korea Contents, Vol.10, no.2, pp.1-13, 2010 February.   DOI
10 Keon-Hee Park, Guee-Sang Lee, "Hand Gesture Interface for Manipulating 3D Objects in Augmented Reality," Korea Contents, Vol.10, no.5, pp.20-28, 2010 May.   과학기술학회마을   DOI
11 Y. Shen, "Vision-Based Hand Interaction in Augmented Reality Environment," International Journal of Human-Computer Interaction, vol.27, no.6, pp.523-544, May 2011.   DOI   ScienceOn
12 ARTOOLKIT. Human Interface Technology Laboratory, http://www.hitl.Washin gton.edu/artoolkit
13 Junchul Chun, Byungsung Lee, "Dynamic Man ipulation of a Virtual Object in Marker-less AR system Based on Both Human Hands," KSII Transactions on Internet and Information Systems, vol.4, no.4, pp.618-632, August 2010.
14 Junyeong Choi, Hanhoon Park, Jungsik Park, and Jong-Il Park, "Implementation of Hand-Gesture Bas ed Augmented Reality Interface on Mobile Phone," The korea Society of Broadcast Engineers, Vol.16, no.6, pp.941-950, 2011 November.
15 Z. Zhang, "Flexible Camera Calibration by Viewing a Plane from Unknown Orientations," Proc. Int'l Conf. Computer Vision, pp. 666-673, 1999.