ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

A Long-Range Touch Interface for Interaction with Smart TVs

  • Lee, Jaeyeon (IT Convergence Technology Research Laboratory, ETRI) ;
  • Kim, DoHyung (IT Convergence Technology Research Laboratory, ETRI) ;
  • Kim, Jaehong (IT Convergence Technology Research Laboratory, ETRI) ;
  • Cho, Jae-Il (IT Convergence Technology Research Laboratory, ETRI) ;
  • Sohn, Joochan (IT Convergence Technology Research Laboratory, ETRI)
  • Received : 2011.10.19
  • Accepted : 2012.08.22
  • Published : 2012.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

A powerful interaction mechanism is one of the key elements for the success of smart TVs, which demand far more complex interactions than traditional TVs. This paper proposes a novel interface based on the famous touch interaction model but utilizes long-range bare hand tracking to emulate touch actions. To satisfy the essential requirements of high accuracy and immediate response, the proposed hand tracking algorithm adopts a fast color-based tracker but with modifications to avoid the problems inherent to those algorithms. By using online modeling and motion information, the sensitivity to the environment can be greatly decreased. Furthermore, several ideas to solve the problems often encountered by users interacting with smart TVs are proposed, resulting in a very robust hand tracking algorithm that works superbly, even for users with sleeveless clothing. In addition, the proposed algorithm runs at a very high speed of 82.73 Hz. The proposed interface is confirmed to comfortably support most touch operations, such as clicks, swipes, and drags, at a distance of three meters, which makes the proposed interface a good candidate for interaction with smart TVs.

Keywords

References

  1. W.T. Freeman et al., "Computer Vision for Interactive Computer Graphics," IEEE Comput. Graphics Appl., vol. 18, no. 3, 1998, pp. 42-53. https://doi.org/10.1109/38.674971
  2. C.C. Wang and K.-C. Wang, "Hand Posture Recognition Using Adaboost with SIFT for Human Robot Interaction," LNCIS, vol. 370, 2008, pp. 317-329.
  3. Y. Verdie, B. Fang, and F. Quek, "MirrorTrack-Tracking with Reflection-Comparison with Top-Down Approach," Proc. Int. Conf. Multimodal Interfaces Workshop Mach. Learning Multimodal Interfaces, 2009, pp. 347-350.
  4. K. Smith et al., "Real-Time 3D Hand Tracking in a Virtual Environment," Proc. SPIE: Int. Society Optical Eng., vol. 5006, 2003, pp. 529-543.
  5. C. Manresa et al., "Real-Time Hand Tracking and Gesture Recognition for Human-Compute Interaction," Electron. Lett. Comput. Vision Image Anal., 2000.
  6. L. Sun, U. Klank, and M. Beetz, "EYEWATCHME: 3D Hand and Object Tracking for Inside Out Activity Analysis," IEEE Conf. Comput. Vision Pattern Recog., 2009, pp. 9-16.
  7. C. Hardenberg and F. Berard, "Bare-Hand Human-Computer Interaction," Proc. ACM Workshop Perceptive User Interface, 2001.
  8. P. Garg, N. Aggarwal, and S. Sofat, "Vision Based Hand Gesture Recognition," World Academy Sci., Eng., Technol., vol. 49, 2009, pp. 972-977.
  9. M. Gorce, N. Paragios, and D.J. Fleet, "Model-Based Hand Tracking with Texture, Shading and Self-Occlusions," Proc. IEEE Conf. Comput. Vision Pattern Recog., 2008, pp. 1-8.
  10. M. Yuan et al., "Robust Hand Tracking Using a Simple Color Classification Technique," Int. J. Virtual Reality, vol. 8, no. 2, 2009, pp. 7-12.
  11. F. Mahmoudi and M. Parviz, "Visual Hand Tracking Algorithms," Proc. Geometric Modeling Imaging New Trends, 2006, pp. 228-232.
  12. C. Manders et al., "Robust Hand Tracking Using a Skin Tone and Depth Joint Probability Model," 8th IEEE Int. Conf Autom. Face Gesture Recog., 2008.
  13. J. Romero et al., "Dynamic Time Warping for Binocular Hand Tracking and Reconstruction," Proc. IEEE Int. Conf. Robotics Autom., 2008, pp. 2289-2294.
  14. D. Chik, J. Trumpf, and N.N. Schraudolph, "3D Hand Tracking in a Stochastic Approximation Setting," LNCS, vol. 4814, 2007, pp. 136-151.
  15. W.Y. Chang, C.S. Chen, and Y.P. Hung, "Appearance-Guided Particle Filtering for Articulated Hand Tracking," Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recog., vol. 1, 2005, pp. 235-242.
  16. A. Argyros and I. Lourakis, "Binocular Hand Tracking and Reconstruction Based on 2D Shape Matching," Proc. Int. Conf. Pattern Recog., 2006, pp. 207-210.
  17. V. Vezhnevets, V. Sazonov, and A. Andreeva, "A Survey on Pixel-Based Skin Color Detection Techniques," Proc. GraphiCon, 2003, pp. 85-92.
  18. R. Kjeldsen and J. Kender, "Finding Skin in Color Images," Proc. Int. Conf. Autom. Face Gesture Recog. , 1996, pp. 312-317.
  19. Y. Wu and T.S. Huang, "Color Tracking by Transductive Learning," Proc. IEEE Comput. Soc. Conf. Comput. Vision Pattern Recog., 2000, pp. 133-138.
  20. S.B. Gorturk, H. Yalcin, and C. Bamji, "A Time-Of-Flight Depth Sensor: System Description, Issues and Solutions," Comput. Vision Pattern Recog. Workshop, 2004.
  21. K.L. Boyer and A.C. Kak, "Color-Encoded Structured Light for Rapid Active Ranging," IEEE Trans. Pattern Anal. Mach. Intell., vol. 9, no. 1, 1987, pp. 14-28.
  22. S.T. Barnard and W.B. Thompson, "Disparity Analysis of Images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 2, no. 4, 1980, pp. 333-340.
  23. P. Viola and M. Jones, "Robust Real-Time Object Detection," IEEE ICCV Workshop Statistical Comput. Theories Vision, Vancouver, Canada, July 2001, pp. 1-25.
  24. N. Dalal and B. Triggs, "Histograms of Oriented Gradients for Human Detection," IEEE CVPR, June 2005, pp. 886-893.
  25. M. Hussein, F. Porikli, and L. Davis, "A Comprehensive Evaluation Framework and a Comparative Study for Human Detectors," IEEE Trans. Intell. Transp. Syst., vol. 10, no. 3, Sept. 2009, pp. 417-427. https://doi.org/10.1109/TITS.2009.2026670
  26. M. Enzweiler and D.M. Gavrila, "Monocular Pedestrian Detection: Survey and Experiments," IEEE Trans. Pattern Anal. Mach. Intell., vol. 31, no. 12, Dec. 2009, pp. 2179-2195. https://doi.org/10.1109/TPAMI.2008.260
  27. C. Zeng and H. Ma, "Robust Head-Shoulder Detection by PCA-Based Multilevel HOG-LBP Detector for People Counting," IEEE ICPR, Aug. 2010, pp. 2069-2072.
  28. B. Jun and D. Kim, "Robust Real-Time Face Detection Using Face Certainty Map," Int. Conf. Biometrics, vol. 4642, no. 125, Aug. 2007, pp. 29-38.
  29. L. Sigal, S. Sclaroff, and V. Athitsos, "Skin-Color Based Video Segmentation Under Time-Varying Illumination," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 7, 2004, pp. 862-877. https://doi.org/10.1109/TPAMI.2004.35
  30. M.J. Swain and D.H. Ballard, "Color Indexing," Int. J. Computer Vision, vol. 7, no. 1, 1991, pp. 11-32. https://doi.org/10.1007/BF00130487
  31. J.I. Agbinya and D. Ress, "Multi-Object Tracking in Video," Real-Time Imaging, vol. 5, no. 5, 1999, pp. 295-304. https://doi.org/10.1006/rtim.1998.0174

Cited by

  1. High-Quality Stereo Depth Map Generation Using Infrared Pattern Projection vol.35, pp.6, 2013, https://doi.org/10.4218/etrij.13.2013.0052