Korean Journal of Computational Design and Engineering (한국CDE학회논문집)

Society for Computational Design and Engineering (한국CDE학회)
권호 표지

Resolving Hand Region Occlusion in Tangible Augmented Reality Envrionments

감각형 증강현실 환경에서의 손 가림 현상 해결 방안

  • Received : 2011.01.18
  • Accepted : 2011.06.23
  • Published : 2011.08.01
Download PDF
⟨ Previous Next ⟩

Abstract

In tangible augmented reality (AR) environments for virtual prototyping, the user interacts with virtual products by manipulating tangible objects with his or her hands, but the user often encounter awkward situations in which his or her hands are occluded by augmented virtual objects, which reduces both immersion and ease of interaction. In this paper, we present how to resolve such hand region occlusion in order to enhance natural interaction and immersive visualization. In the AR environment considered, we use two types (product-type and pointer-type) of tangible objects for tangible user interaction with a virtual product of interest. Holding the tangible objects with his or her hands, the user can create input events by touching specified regions of the product-type tangible object with the pointer-type tangible object. We developed a method for resolving hand region occlusion frequently arising during such user interaction, It first detect hand region in a real image and refines the rendered image of the virtual object by subtracting the hand region from the rendered image, Then, it superimposes the refined image onto the real image to obtain an image in which the occlusion is resolved. Incorporated into tangible AR interaction for virtual prototyping of handheld products such as cellular phones and MP3 players, the method has been found by a preliminary user study that it is not only useful to improve natural interaction and immersive visualization of virtual products, but also helpful for making the users experience the products' shapes and functions better.

Keywords

References

  1. K. Lee Principles of CAD/CAM/CAE Systems, Addison Wesley, Berkeley, 1999.
  2. Burdea, G. C. and Coiffet, P., Virtual Reality Technology, John Wiley & Sons, USA, 2003.
  3. Bullinger, H. J., Breining, R. and Baucer, W., "Virtual Prototyping-State of the Art in Product Design," Proceedings of the 26th International Conference on Computers & Industrial Engineering, pp. 103-107, 1999.
  4. 박형준, 배채열, 이관행, "HMI 기능성 시뮬레이션과 VR 기법과의 연동을 통한 개인용 전자제품의 가상시작 방안," 한국CAD/CAM 학회논문집, Vol. 12, No. 2, pp. 87-94, 2007.
  5. Azuma, R. T., "A Survey of Augmented Reality," Presence: Teleoperators and Virtual Environments, Vol. 6, No. 4, pp. 355-385, 1997.
  6. Ishii, H. and Ullmer, B., "Tangible Bits: Towards Seamless Interfaces Between People, Bits and Atoms," Proceedings of Conference of Human Factors in Computing Systems (CHI), pp. 234-241, 1997.
  7. Kato, H., Billinghurst, M., Poupyrev, I., Imamoto, K. and Tachibana, K., "Virtual Object Manipulation on a Table-top AR Environment," Proceedings of the International Symposium on Augmented Reality (ISAR), pp. 111-119, 2000.
  8. Billinghurt, M., Kato, H. and, Poupyrev, I., "Collaboration with tangible augmented reality interfaces," Proceedings of HCI International, pp. 234-241, 2001.
  9. 박형준, 문희철, "증강현실 기반 상호작용과 시뮬레 이션을 이용한 휴대용 전자제품의 설계품평," 한국CAD/CAM 학회 논문집, Vol 13, No 3, pp. 209-216, 2008.
  10. Park, H., Moon, H. C. and Lee, J. Y., "Tangible Augmented Prototyping of Digital Handheld Products," Computers in Industry, Vol. 60, No. 2, pp. 114-125, 2009. https://doi.org/10.1016/j.compind.2008.09.001
  11. 문희철, 박형욱, 박형준, "증강현실 기반 감각형 상호작용의 유효영역 측정," 한국 CAD/CAM 학술발표회 논문집(poster), pp. 929-933, 2008.
  12. ARToolKit, http://www.hitl.washington.edu/ARTool-Kit
  13. Lee, W. and Park, J., "Augmented Foam: A Tangible Augmented Reality for Product Design," Fourth IEEE and ACM International Symposium on Mixed and Augmented Reality, pp. 106-109, 2005.
  14. Tan, W., Wu, C., Zhao, S. and Chen, S., "Hand Extraction Using Geometric Moments Based on Active Skin Color Model," Intelligent Computing and Intelligent Systems, pp. 468-471, 2009.
  15. Ruiz-del-Solar, J. and Verschae, R., "Skin Detection using Neighborhood Information," 6th IEEE Inter national Conference on Automatic Face and Gesture Recognition, 2004.
  16. Chai, D. and King, K. N., "Face Segmentation Using Skin-Color Map in Videophone Applications," IEEE Transactions on Circuits and System for Video Technology, Vol. 9, No. 4, pp. 551-564, 1999. https://doi.org/10.1109/76.767122
  17. Dadgostar, F. and Sarrafzadeh, A., "An adaptive real-time skin detector based on Hue thresholding," Pattern Recognition Letters, Vol. 27, pp. 1342-1352, 2006. https://doi.org/10.1016/j.patrec.2006.01.007
  18. GLUT, http://www.opengl.org
  19. OpenCV, http://www.intel.com/technology/computing/opencv/index.htm