Browse > Article
http://dx.doi.org/10.3837/tiis.2010.08.000

Dynamic Manipulation of a Virtual Object in Marker-less AR system Based on Both Human Hands  

Chun, Jun-Chul (Department of Computer Science, Kyonggi University)
Lee, Byung-Sung (Nexteye Co.)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.4, no.4, 2010 , pp. 618-632 More about this Journal
Abstract
This paper presents a novel approach to control the augmented reality (AR) objects robustly in a marker-less AR system by fingertip tracking and hand pattern recognition. It is known that one of the promising ways to develop a marker-less AR system is using human's body such as hand or face for replacing traditional fiducial markers. This paper introduces a real-time method to manipulate the overlaid virtual objects dynamically in a marker-less AR system using both hands with a single camera. The left bare hand is considered as a virtual marker in the marker-less AR system and the right hand is used as a hand mouse. To build the marker-less system, we utilize a skin-color model for hand shape detection and curvature-based fingertip detection from an input video image. Using the detected fingertips the camera pose are estimated to overlay virtual objects on the hand coordinate system. In order to manipulate the virtual objects rendered on the marker-less AR system dynamically, a vision-based hand control interface, which exploits the fingertip tracking for the movement of the objects and pattern matching for the hand command initiation, is developed. From the experiments, we can prove that the proposed and developed system can control the objects dynamically in a convenient fashion.
Keywords
Augmented reality; marker-less AR; fingertip tracking; hand detection; skin-color model; hand pattern recognition;
Citations & Related Records

Times Cited By Web Of Science : 0  (Related Records In Web of Science)
Times Cited By SCOPUS : 1
연도 인용수 순위
  • Reference
1 T. Lee and T. Hollerer, "Hybrid Feature Tracking and User Interaction for Markerless Augment Reality," in IEEE VR'08 (10th Int'l Conference on Virtual Reality), pp. 145-152, 2008.
2 BS. Lee and JC Chun "Manipulation of virtual objects in marker-less AR system by fingertip tracking and hand gesture recognition," in Int'l Conference on Interaction Science, pp. 1110-1115, 2009.
3 H. Ishii and B. Ullmer, "Tangible bit: towards seamless interface between people, bits and atoms," in ACM CHI'97, pp. 234-241, 1977.
4 K. Min and J. Chun, "A nonparametric skin color model for face detection from color images," in PDCAT'04, pp. 115-119, 2004.
5 G. Borgefors, "Distance transformations in digital images," Computer Vision, Graphics and Image Processing, vol. 34, pp. 344-371, 1986.   DOI   ScienceOn
6 M. Black, "Robust incremental optical flow," Ph.D thesis, Yale University, 1992.
7 Z. Zhang, "A flexible new technique for camera calibration," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no.11, pp. 1330-1334, 2000.   DOI   ScienceOn
8 C.C Chen and H.T Chu, "Similarity Measurement Between Images," in Proc. of the 29th Annual International Computer Software and Applications Conference (COMPSAC'05), vol.1, pp. 41-41, 2005.
9 F. J. Anscombe, "Graphs in Statistical Analysis," American Statistician, vol. 27, no. 1, pp. 17-21, 1973.   DOI   ScienceOn
10 HJ. Park, DS. Kim, and JC Chun, "An Interactive Method for Volumetric Human Organ Model on Augmented Reality," in Proc. of APICT-IST and ICONI 2009, pp. 383-386, 2009.
11 H. Kato and M. Billinghurst, "Marker tracking and HMD calibration for a video-based augmented reality conferencing system" in IWAR'99, pp. 85-94, 1999.
12 D. Stricker, D. Klinker, and D. Reiners, "A fast and robust line-based optical tracker for augmented reality application," in IWAR'98, pp. 31-46, 1998.
13 Y. Cho, J. Lee, and U. Neumann, "A multi-ring fiducial system and an intensity-invariant detection method for scalable augment reality," in IWAR'98, pp. 147-156, 1998.
14 S. Vogt, A. Khamene, F. Sauer, and H. Niemann, "Single camera tracking of marker clusters: Multiparameter cluster optimization and experimental verification," in SIMAR'02, pp. 127-136, 2002.
15 A. Comport, E. Marchand, and F. Chaumette, "A real-time tracker for markerless augment reality," in ISMAR'03, pp. 36-45, 2003.
16 H. Kato, M. Billinghurst,I. Poupyrev, K. Imamoto, and K. Tachibana, "Virtual object manipulation on a table-top AR environments," in ISAR'00, pp. 111-119, 2000.
17 M. Kolsch and M. Turk, "Fast 2D hand tracking with flocks of features and multi-cue integration," in CVPRW'04, pp. 158, 2004.
18 T. Kurata, T. Okuma, M. Kourogi, and K. Sakaue, "The hand mouse: GMM hand-color classification and mean shift tracking," in 2nd Int'l Workshop on Recognition, Analysis and Tracking of Faces and Gestures in Real-time System, pp. 119-124, 2001.
19 W. Lee and J. Park, "Augmented foam: A tangible augmented reality for product design," in ISMAR'05, pp. 106-109, 2005.
20 J. Letessier and F. Berard, "Visual tracking of bare fingers for interactive surface," in Proc. ACM symposium on User interface software and technology, pp. 119-122, 2004.
21 W. Mayol, A. Davison, B. Tordoff, N. Molton, and D. Murray, "Interaction between hand and wearable camera in 2D and 3D environments," in BMVA'04, 2004.
22 J. Terrillon, M. Shirazi, H. Fukamachi, and S. Akamatsu, "Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images," in Proc. 9th Conference on Automatic Face and Gesture Recognition, pp. 54-61, 2000.
23 M. Pressigout and E. Marchand, " Hybrid tracking algorithm for planar and non-planar structures subject to illumination change," in ISMAR'06, pp. 52-56, 2006.
24 T. Lee and T. Hollerer, "Hand AR: Markerless Inspection of Augmented Reality Objects Using Fingertip Tracking," in Proc. IEEE ISWC'07, pp. 83-90, 2007.
25 T. Lee and T. Hollerer, "Initializing Markerless Tracking Using a Simple Hand Gesture," in Proc. ACM/IEEE ISMAR'07, pp. 1-2, 2007.