DOI QR코드

DOI QR Code

Visual Feedback System for Manipulating Objects Using Hand Motions in Virtual Reality Environment

가상 환경에서의 손동작을 사용한 물체 조작에 대한 시각적 피드백 시스템

  • Seo, Woong (Department of Computer Science and Engineering, Sogang University) ;
  • Kwon, Sangmo (Department of Computer Science and Engineering, Sogang University) ;
  • Ihm, Insung (Department of Computer Science and Engineering, Sogang University)
  • 서웅 (서강대학교 컴퓨터공학과) ;
  • 권상모 (서강대학교 컴퓨터공학과) ;
  • 임인성 (서강대학교 컴퓨터공학과)
  • Received : 2020.06.11
  • Accepted : 2020.06.25
  • Published : 2020.07.01

Abstract

With the recent development of various kinds of virtual reality devices, there has been an active research effort to increase the sense of reality by recognizing the physical behavior of users rather than by classical user input methods. Among such devices, the Leap Motion controller recognizes the user's hand gestures and can realistically trace the user's hand in a virtual reality environment. However, manipulating an object in virtual reality using a recognized user's hand often causes the hand to pass through the object, which should not occur in the real world. This study presents a way to build a visual feedback system for enhancing the user's sense of interaction between hands and objects in virtual reality. In virtual reality, the user's hands are examined precisely by using a ray tracing method to see if the virtual object collides with the user's hand, and when any collision occurs, visual feedback is given through the process of reconstructing the user's hand by moving the position of the end of the user's fingers that enter the object through sign distance field and reverse mechanics. This enables realistic interaction in virtual reality in real time.

최근 다양한 종류의 가상 현실 기기가 개발되면서 고전적인 사용자 입력 방식 이외에도 사용자의 신체 동작을 인식하여 현실감을 높이려는 기술에 관한 연구가 활발하게 진행되고 있다. 이러한 장치 중의 하나인 립모션 컨트롤러는 사용자의 손동작을 인식하여 가상 현실 환경에서 사용자의 손 모양을 실시간으로 나타낼 수 있다. 하지만 인식한 사용자의 손을 사용하여 가상 현실의 물체를 조작하면 종종 손이 물체를 통과하게 되는데, 이러한 상호작용은 실제 현실과는 거리감이 있다. 본 연구에서는 가상 현실에서 사용자의 손과 물체 간의 상호작용 현실감을 높이기 위한 시각적 피드백 시스템의 구축 방법을 제시한다. 가상의 현실에서 사용자의 손과 가상 물체가 충돌하는지 광선 추적법을 활용하여 정밀하게 검사하고, 충돌하였을 때 부호 거리 장과 역기구학을 통해 물체 내부로 들어간 사용자의 손가락 말단의 위치를 물체 표면 밖으로 이동시켜 사용자의 손을 재구성하는 과정을 통해 시각적 피드백을 준다. 이를 통해 실시간으로 가상 현실에서 현실감 있는 상호작용을 할 수 있다.

Keywords

References

  1. C. W. Borst and A. P. Indugula, "Realistic virtual grasping," in Proceedings of 2005 IEEE Conference on Virtual Reality(VR), pp. 91-98, March 2005.
  2. J. Blake and H. B. Gurocak, "Haptic glove with MR brakes for virtual reality," IEEE/ASME Transactions on Mechatronics, vol. 14, no. 5, pp. 606-615, 2009. https://doi.org/10.1109/TMECH.2008.2010934
  3. M. Prachyabrued and C. W. Borst, "Effects and optimization of visual-proprioceptive discrepancy reduction for virtual grasping," in Proceedings of 2013 IEEE Symposium on 3D User Interfaces (3DUI), pp. 11-14, 2013.
  4. I. Choi, H. Culbertson, M. R. Miller, A. Olwal and S. Follmer, "Gravity: A Wearable Haptic Interface for Simulating Weight and Grasping in Virtual Reality," in Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology(UIST '17), pp. 119-130, 2017.
  5. Ultraleap Ltd., "Leap Motion Controller," Available: https://developer.leapmotion.com [Online accessed 24-Apr.-2020].
  6. Valve Corporation, "Valve Index Controllers," Available: https://store.steampowered.com/valveindex [Online accessed 24-Apr.-2020].
  7. M. Chessa, G. Maiello, L. K. Klein, V. C. Paulun, and F. Solari, "Grasping objects in immersive virtual reality," in Proceedings of 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), pp. 1749-1754, March 2019.
  8. K. Nasim, and Y. J. Kim, "Physics-based interactive virtual grasping," in Proceedings of HCI Korea 2016, pp. 114-120, Jan. 2016.
  9. NVIDIA Corporation, "PhysX SDK," Available: https://developer.nvidia.com/physx-sdk [Online accessed 24-Apr.-2020].
  10. M. Holl, M. Oberweger, C. Arth, and V. Lepetit, "Efficient Physics-Based Implementation for Realistic Hand-Object Interaction in Virtual Reality," in Proceedings of 2018 IEEE Conference on Virtual Reality(VR), pp. 175-182, March 2018.
  11. I. Wald, W. Mark, J. Gunther, S. Boulos, T. Ize, W. Hunt, S. Parker, and P. Shirley, "State of the art in ray tracing animated scene," STAR - State of The Art Report, Eurographics 2007, 2007.
  12. B. Chang, D. Cha, and I. Ihm, "Computing Local Signed Distance Fields for Large Polygonal Models," Computer Graphics Forum (EuroVis 2008: Eurographics/IEEE Symposium on Visualization), Vol. 27, No. 3, pp. 799-806, May 2008.
  13. S. R. Buss, "Introduction to inverse kinematics with jacobian transpose, pseudoinverse and damped least squares methods," Technical report, Dept. Math. Univ. of California, San Diego, CA, USA, 2004.
  14. I. Wald, V. Havran, "On building fast kd-trees for ray tracing, and on doing that in O (N log N)," in Proceedings of the 2006 IEEE Symposium on Interactive Ray Tracing, pp. 61-69, 2006.
  15. I. Wald, Realtime Ray Tracing and Interactive Global Illumination, PhD thesis, Saarland University, 2004.