ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Multimodal Interaction on Automultiscopic Content with Mobile Surface Haptics

  • Received : 2016.04.19
  • Accepted : 2016.09.05
  • Published : 2016.12.01
Download PDF
⟨ Previous Next ⟩

Abstract

In this work, we present interactive automultiscopic content with mobile surface haptics for multimodal interaction. Our system consists of a 40-view automultiscopic display and a tablet supporting surface haptics in an immersive room. Animated graphics are projected onto the walls of the room. The 40-view automultiscopic display is placed at the center of the front wall. The haptic tablet is installed at the mobile station to enable the user to interact with the tablet. The 40-view real-time rendering and multiplexing technology is applied by establishing virtual cameras in the convergence layout. Surface haptics rendering is synchronized with three-dimensional (3D) objects on the display for real-time haptic interaction. We conduct an experiment to evaluate user experiences of the proposed system. The results demonstrate that the system's multimodal interaction provides positive user experiences of immersion, control, user interface intuitiveness, and 3D effects.

Keywords

References

  1. Unity3D. http://www.unity3d.com
  2. Senseg. http://www.senseg.com
  3. A. Cellatoglu and K. Balasubramanian, "Autostereoscopic Imaging Techniques for 3D TV: Proposals for Improvements," J. Display Technol., vol. 9, no. 8, Aug. 2013, pp. 666-672. https://doi.org/10.1109/JDT.2013.2257983
  4. D. Li et al., "3D Synthesis and Crosstalk Reduction for Lenticular Autostereoscopic Displays," J. Display Technol., vol. 11, no. 11, Nov. 2015, pp. 939-946. https://doi.org/10.1109/JDT.2015.2405065
  5. L. Lipton and M. Feldman, "New Stereoscopic Display Technology: the SynthaGram," SPIE Stereoscopic Displays Virtual Reality Syst. IX, San Jose, CA, USA, May 24, 2002, pp. 229-235.
  6. A. Schmidt and A. Grasnick, "Multi-viewpoint Autostereoscopic Displays from 4D-Vision," SPIE Stereoscopic Displays Virtual Reality Syst. IX, San Jose, CA, USA, May 24, 2002, pp. 212-221.
  7. R.L. Kooima et al., "A GPU Sub-pixel Algorithm for Autostereoscopic Virtual Reality," IEEE Virtual Reality Conf., Charlotte, NC. USA, Mar. 10-14, 2007, pp. 131-137.
  8. D. Ruijters, "Integrating Autostereoscopic Multi-view Lenticular Displays in Minimally Invasive Angiography," MICCAI AMI-ARCS Workshop, New York, USA, Jan. 2008, pp. 87-94.
  9. T. Watanabe and S. Fukui, "A Method for Controlling Tactile Sensation of Surface Roughness Using Ultrasonic Vibration," IEEE Int. Conf. Robot. Autom., Nagoya, Japan, May 21-27, 1995, pp. 1134-1139.
  10. L. Winfield et al., "T-Pad: Tactile Pattern Display through Variable Friction Reduction," World Haptics Conf. Symp. Haptic Interfaces Virtual Environment Teleoperator Syst., Mar. 22-24, 2007, pp. 421-426.
  11. E. Mallinckrodt, A.L. Hughes, and W.S. Jr., "Perception by the Skin of Electrically Induced Vibrations," Sci., vol. 118, Sept. 1953, pp. 277-278. https://doi.org/10.1126/science.118.3062.277
  12. R.M. Strong and D.E. Troxel, "An Electrotactile Display," IEEE Trans. Man-Mach. Syst., vol. 11, no. 1, Mar. 1970, pp. 72-79. https://doi.org/10.1109/TMMS.1970.299965
  13. O. Bau et al., "Teslatouch: Electrovibration for Touch Surfaces," Proc. Annu. ACM Symp. User Interface Softw. Technol., New York, USA, Oct. 3-6, 2010, pp. 283-292.
  14. Tanvas. http://www.tanvas.co
  15. M. Mine et al., "Projection-Based Augmented Reality in Disney Theme Parks," IEEE Comput., vol. 45, no. 7, July 2012, pp. 32-40.
  16. B.R. Jones et al., "IllumiRoom: Peripheral Projected Illusions for Interactive Experiences," Proc. SIGCHI Conf. Human Factors Comput. Syst., Paris, Prance, Apr. 27-May 2, 2013, pp. 869-878.
  17. J. Beira, R. Carvalho, and S. Kox, "Mixed Reality Immersive Design: a Study in Interactive Dance," Proc. ACM Int. Workshop Immersive Media Experiences, Barocelona, Spain, Oct. 22, 2013, pp. 45-50.
  18. C. Berkel and J.A. Clarke, "Characterization and Optimization of 3D-LCD Module Design," SPIE Stereoscopic Displays Virtual Reality Syst. IV, San Jose, CA, USA, May 15, 1997, pp. 179-187.
  19. NVidia CUDA. https://developer.nvidia.com/cuda-zone
  20. FreeFrame. http://freeframe.sourceforge.net/