Journal of KIISE:Software and Applications (한국정보과학회논문지:소프트웨어및응용)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지

Real-time 3D Feature Extraction Combined with 3D Reconstruction

3차원 물체 재구성 과정이 통합된 실시간 3차원 특징값 추출 방법

  • Published : 2008.12.15
Download PDF
⟨ Previous Next ⟩

Abstract

For the communication between human and computer in an interactive computing environment, the gesture recognition has been studied vigorously. The algorithms which use the 2D features for the feature extraction and the feature comparison are faster, but there are some environmental limitations for the accurate recognition. The algorithms which use the 2.5D features provide higher accuracy than 2D features, but these are influenced by rotation of objects. And the algorithms which use the 3D features are slow for the recognition, because these algorithms need the 3d object reconstruction as the preprocessing for the feature extraction. In this paper, we propose a method to extract the 3D features combined with the 3D object reconstruction in real-time. This method generates three kinds of 3D projection maps using the modified GPU-based visual hull generation algorithm. This process only executes data generation parts only for the gesture recognition and calculates the Hu-moment which is corresponding to each projection map. In the section of experimental results, we compare the computational time of the proposed method with the previous methods. And the result shows that the proposed method can apply to real time gesture recognition environment.

상호작용이 가능한 컴퓨팅 환경에서 사람과 컴퓨터 사이의 자연스러운 정보 교환을 위해 동작 인식과 관련한 연구가 활발하게 이루어지고 있다. 기존의 2차원 특징값을 이용하는 인식 알고리즘은 특징값 추출과 인식 속도는 빠르지만, 정확한 인식을 위해서 많은 환경적인 제약이 따른다. 또한 2.5차원 특징값을 이용하는 알고리즘은 2차원 특징값에 비해 높은 인식률을 제공하지만 물체의 회전 변화에 취약하고, 3차원 특징값을 이용하는 인식 알고리즘은 특징값 추출을 위해 3차원 물체를 재구성하는 선행 과정이 필요하기 때문에 인식 속도가 느리다. 본 논문은 3차원 물체 재구성 단계와 특징값 추출 단계를 통합하여 실시간으로 3차원 정보를 가지는 특징값 추출 방법을 제안한다. 제안하는 방법은 기존의 GPU 기반 비주얼 헐 생성 방법의 세부 과정 중에서 동작 인식에 필요한 데이타 생성 부분만을 수행하여 임의의 시점에서 3차원 물체에 대한 3종류의 프로젝션 맵을 생성하고, 각각의 프로젝션 맵에 대한 후-모멘트(Hu-moment)를 계산한다. 실험에서 우리는 기존의 방법들과 단계별 수행 시간을 비교하고, 생성된 후-모멘트에 대한 혼동 행렬(confusion matrix)을 계산함으로써 제안하는 방법이 실시간 동작 인식 환경에 적용될 수 있음을 확인하였다.

Keywords

References

  1. D. Ballard, C. Brown, "Computer Vision," Prentice- Hall, pp. 24-30, 1982
  2. D. G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," The International Journal of Computer Vision, Vol. 60, No. 2, pp. 91-110, 2004 https://doi.org/10.1023/B:VISI.0000029664.99615.94
  3. P. O. Hoyer, A. Hyvarinen, "Feature Extraction from Color and Stereo Images using ICA," The International Joint Conference on Neural Networks, Vol. 3, pp. 369-374, 2000
  4. D. Torkar, N. Pavesic, "Feature Extraction from Aerial Images and Structural Stereo Matching," The 13th International Conference on Pattern Recognition, Vol. 3, pp. 880-884, 1996
  5. C.M. Cyr, B.B. Kimia, "3D Object Recognition Using Shape Similarity-based Aspect Graph," The 8th IEEE International Conference on Computer Vision, Vol. 1, pp. 254-261, 2001
  6. C. Chu, I. Cohen, "Posture and Gesture Recognition using 3D Body Shapes Decomposition," The IEEE Computer Society Conference on CVPR 2005, Vol. 3, pp. 69, 2005
  7. D. Kyoung, Y. Lee, W. Baek, E. Han, J. Yang, K. Jung, "Efficient 3D Voxel Reconstruction using Pre-computing Method for Gesture Recognition," Korea-Japan Joint Workshop 2006, pp. 67-73, 2006
  8. T. Funkhouser, P. Min, M. Kazhdan, J. Chen, A. Halderman, D. Dobkin, D. Jacobs, "A Search Engine for 3D Models," ACM Transactions on Graphics, Vol. 22, pp. 83-105, 2003 https://doi.org/10.1145/588272.588279
  9. M. Hilaga, Y. Shinagawa, T. Kohmura, T. Kunii, "Topology Matching for Fully Automatic Similarity Estimation of 3D Shapes," The 28th Annual Conference on Computer Graphics and Interactive Techniques, pp. 203-212, 2001
  10. H. Sundar, D. Silver, N. Gagvani, S. Dickinson, "Skeleton based Shape Matching and Retrieval," International Conference on Shape Modeling International, pp. 130-139, 2003
  11. N.D. Cornea, D. Silver, P. Min, "Curve-Skeleton Properties, Applications and Algorithms," IEEE Transactions on Visualization and Computer Graphics, Vol. 13, pp. 530-548, 2007 https://doi.org/10.1109/TVCG.2007.1002
  12. N.D. Cornea, D. Silver, X. Yuan, R. Balasubramanian, "Computing Hierarchical Curve-Skeletons of 3D Objects," The Visual Computer, Vol. 21, pp. 945-955, 2005 https://doi.org/10.1007/s00371-005-0308-0
  13. A. Brennecke, T. Isenberg, "3D Shape Matching using Skeleton Graphs," Simulation and Visualization 2004, Vol. 13, pp. 299-310, 2004
  14. M. Li, M. Magnor, H. Seidel, "Hardware-Accelerated Visual Hull Reconstruction and Rendering," Graphics Interface 2003, pp. 65-71, 2003
  15. M. Hu, "Visual Pattern Recognition by Moment Invariants," IRE Transaction on Information Theory, Vol. 8, No. 2, pp. 179-187, 1962
  16. A. Laurentini, "The Visual Hull Concept for Silhouette-based Image Understanding," IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 16, pp. 150-162, 1994 https://doi.org/10.1109/34.273735
  17. R. Szeliski, "Rapid Octree Construction from Image Sequences," CVGIP: Image Understanding, Vol. 58, pp. 23-32, 1993 https://doi.org/10.1006/ciun.1993.1029
  18. W. Matusik, C. Buehler, L. McMillan, "Polyhedral Visual Hulls for Real-time Rendering," The 12th Eurographics Workshop on Rendering Technique, pp. 115-126, 2001
  19. C. Everitt, A. Rege, C. Cebenoyan, "Hardware Shadow Mapping," Technical Report, NVIDIA, 2002