Journal of the Institute of Electronics and Information Engineers (전자공학회논문지)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지
DOI QR코드

DOI QR Code

A Hand Gesture Recognition System using 3D Tracking Volume Restriction Technique

3차원 추적영역 제한 기법을 이용한 손 동작 인식 시스템

  • Kim, Kyung-Ho (Graduate School of Advanced Imaging Science, Multimedia, and Film, Chung-Ang University) ;
  • Jung, Da-Un (Graduate School of Advanced Imaging Science, Multimedia, and Film, Chung-Ang University) ;
  • Lee, Seok-Han (Graduate School of Advanced Imaging Science, Multimedia, and Film, Chung-Ang University) ;
  • Choi, Jong-Soo (Graduate School of Advanced Imaging Science, Multimedia, and Film, Chung-Ang University)
  • 김경호 (중앙대학교 첨단영상대학원 영상공학과) ;
  • 정다운 (중앙대학교 첨단영상대학원 영상공학과) ;
  • 이석한 (중앙대학교 첨단영상대학원 영상공학과) ;
  • 최종수 (중앙대학교 첨단영상대학원 영상공학과)
  • Received : 2012.12.21
  • Published : 2013.06.25
Download PDF
⟨ Previous Next ⟩

Abstract

In this paper, we propose a hand tracking and gesture recognition system. Our system employs a depth capture device to obtain 3D geometric information of user's bare hand. In particular, we build a flexible tracking volume and restrict the hand tracking area, so that we can avoid diverse problems caused by conventional object detection/tracking systems. The proposed system computes running average of the hand position, and tracking volume is actively adjusted according to the statistical information that is computed on the basis of uncertainty of the user's hand motion in the 3D space. Once the position of user's hand is obtained, then the system attempts to detect stretched fingers to recognize finger gesture of the user's hand. In order to test the proposed framework, we built a NUI system using the proposed technique, and verified that our system presents very stable performance even in the case that multiple objects exist simultaneously in the crowded environment, as well as in the situation that the scene is occluded temporarily. We also verified that our system ensures running speed of 24-30 frames per second throughout the experiments.

본 논문에서는 손 추적과 제스처 인식 시스템을 제안한다. 제안한 시스템은 사용자 손의 3차원 기하학적 정보를 취득하기 위해 별도의 장비를 사용한다. 특히, 기존의 물체 검출 및 추적 시스템들에서 제기 되었던 추적 과정에서의 문제점을 피하기 위해 능동적인 타원체 영역을 만들고 손 추적을 위한 영역을 타원체 영역의 안으로 제한했다. 제안된 시스템은 미리 정의된 기간 동안에 손 위치의 이동평균을 계산한다. 그리고 추적영역은 3차원 공간에 편성된 공분산에 기반한 사용자 손 움직임의 불확실성을 추정하여 통계적인 데이터에 따라 능동적으로 제어하였다. 또한 손 위치가 획득되었을 때, 손 제스처를 인식하기 위해 펼쳐진 손가락을 검출한다. 사용자 인터페이스 체제 기반의 시스템을 구현하여 복잡한 환경에서 다중의 대상들이 동시에 존재하는 경우이거나 일시적인 가려짐이 발생하는 경우에도 정확성을 보여 매우 안정적으로 동작할 수 있음을 보여주며, 약 24-30fps의 프레임 비율로 사용할 수 있는 가능성을 보여주었다.

Keywords

References

  1. J. Aggarwal and Q. Cai, "Human motion analysis: a review, "Computer Vision and Image Understanding, vol. 73, no. 3, pp. 429-440, March 1999.
  2. M. M. Hasan and P. K. Mishra, "Hand gesture modeling and recognition using geometric features: a review," Canadian Journal on Image Processing and Computer Vision, vol. 33, no. 1, pp. 12-26, March 2012.
  3. S. S. Rautaray and A. Aggrawal, "Real time multiple hand gesture recognition system for human computer interaction," International Journal of Intelligent Systems and Application, vol. 4, no. 4, pp. 56-64, May 2012.
  4. J. P. Wachs, M Kolsch, H. Stern and Y. Edan, "Vision-based hand-gesture application," Comm. of ACM, vol. 54, no. 2, pp. 60-71, Feb. 2011
  5. Z. Ren, J. Yuan and Z. Zhang, "Robust hand gesture recognition based on finger-earth mover distance with a commodity depth camera," In Proceedings of the 19th ACM International Conference on Multimedia, pp. 1093-1096, 2011.
  6. S. Soutschek, J. Penne, J. Hornegger and J. Kornhuber, "3-D gesture-based scene navigation in medical imaging applications using time-of-flight cameras," IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops CVPRW' 08, pp. 1-6, June 2008.
  7. M. Van den Bergh and L. V. Gool, "Combining rgb and tof cameras for real-time 3d hand gesture interaction," 2011 IEEE Workshop on Applications of Computer Vision, pp.66-72, Jan. 2011.
  8. P. Viola and M. J. Jones, "Robust real-time face detection," International Journal of Computer Vision, vol. 57, no. 2, pp. 137-154, May 2004. https://doi.org/10.1023/B:VISI.0000013087.49260.fb
  9. D. Lee and Y. Park, "Vision-based remote control system by motion detection and open finger counting," IEEE Trans. on Consumer Electronics, vol. 55, no. 4, pp. 2308-2313, Nov. 2009. https://doi.org/10.1109/TCE.2009.5373803
  10. T. Lee and T. Hollerer, "Multithreaded hybrid feature tracking for markerless augmented reality," IEEE Trans. Visualization and Computer Graphics, vol. 15, no. 3, pp. 355-368, May/June 2009. https://doi.org/10.1109/TVCG.2008.190
  11. Y. Liu, G. Li and Z. Shi, "Covariance tracking via geometric particle filtering," The Eruopean Association for Signal Processing Journal on Advances in Signal Processing, vol. 2010, no. 22, pp. 1-9, July 2010.
  12. F. Porikli, O. Tuzel and P. Meer, "Covariance tracking using model update based on lie algebra," IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 728-735, June 2006.
  13. O. Tuzel, F. Porikli and P. Meer, "Region covariance: a fast descriptor for detection and classification," European Conference on Computer Vision, vol. 3952, pp. 589-600, 2006.
  14. Y. Wu, J. Cheng, J. Wang, H. Lu, J. Wang, H. Ling, E. Blasch and L. Bai, "Real-time probabilistic covariance tracking with efficient model update," IEEE Trans. on Image Processing, vol. 21, no. 5, pp. 2824-2837, May 2012. https://doi.org/10.1109/TIP.2011.2182521
  15. D. Comaniciu, V. Ramesh and P. Meer, "Real-time tracking of non-rigid objects using mean shift," In Proc. IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, pp.142-149, 2000.
  16. N. Bouaynaya, W. Qu and D. Schonfeld, "An online motion-based particle filter for head tracking applications," In Proc. of the IEEE International Conference on Acoustics, Speech, and Signal Processing, vol. 2, pp. 225-228, March 2005.
  17. M. Yin, J. Zhang, H. Sun and W. Gu, "Multi-cue-based camshift guided particle filter tracking," Journal Expert Systems with Applications: An International Journal, vol. 38, pp. 6313-6318, May 2011. https://doi.org/10.1016/j.eswa.2010.11.111
  18. M. Van den Bergh, E. Koller-Meirer and L. Van Gool, "Real-time body pose recognition using 2d or 3d haarlets," International Journal of Computer Vision, vol. 83, no. 1, pp. 77-84, June 2009.
  19. G. Borgefors, "Distance transformations in digital images," Indian Conference on Computer Vision Graphics and Image Processing, vol. 34, no. 3, pp.344-371, June 1986. https://doi.org/10.1016/S0734-189X(86)80047-0
  20. 김대환, 이승준, 고성제, "영역 기반 물체 추적에서 색상 배치를 고려한 표적 모델링," 전자공학회논문지, vol. 49, no. 1, pp. 1-10, Jan. 2012.
  21. 변기원, 주재흠, 남기곤, "Mean Shift 알고리즘 기반의 히스토그램 근사화를 이용한 피부 영역 검출," 전자공학회논문지, vol. 48, no. 4, pp. 465-473, July 2011.
  22. OpenNI. PrimeSense Sensor Module, 2011. URL https://github.com/PrimeSense/Sensor.
  23. Microsoft Corp. Kinect for Xbox 360