Journal of the Korean Society for Precision Engineering (한국정밀공학회지)

Korean Society for Precision Engineering (한국정밀공학회)
권호 표지

Feasibility Study on Audio-Tactile Display via Spectral Modulation

스펙트럼 변조를 이용한 청각정보의 촉감재현 가능성 연구

  • Kwak, Hyun-Koo (Department of Control and Instrumentation Engineering, Korea Vniv.) ;
  • Kim, Whee-Kuk (Department of Control and Instrumentation Engineering, Korea Vniv.) ;
  • Chung, Ju-No (Department of Control and Instrumentation Engineering, Korea Vniv.) ;
  • Kang, Dae-Im (Division of Physical Metrology, Korea Research Institute of Standards and Science) ;
  • Park, Yon-Kyu (Division of Physical Metrology, Korea Research Institute of Standards and Science) ;
  • Koo, Min-Mo (Wisdom Science Center, Korea Univ.)
  • 곽현구 (고려대학교 제어계측공학과) ;
  • 김희국 (고려대학교 제어계측공학과) ;
  • 정주노 (고려대학교 제어계측공학과) ;
  • 강대임 (한국표준과학연구원) ;
  • 박연규 (한국표준과학연구원) ;
  • 구민모 (고려대학교 지혜과학연구센터)
  • Received : 2010.03.23
  • Accepted : 2011.01.25
  • Published : 2011.05.01
Download PDF
⟨ Previous Next ⟩

Abstract

Various approaches directly using vibrations of speakers have been suggested to effectively display the aural information such as the music to the hearing-impaired or the deaf. However, in these approaches, the human can't sense the frequency information over the maximum perceivable vibro-tactile frequency (around 1kHz). Therefore, in this study, an approach via spectral modulation of compressing the high frequency audio information into perceivable vibro-tactile frequency domain and outputting the modulated signals through the designated speakers is proposed. Then it is shown, through simulations of using Short-Time Fourier Transform (STFT) with Hanning windows and through preliminary experiments of using the vibro-tactile display testbed which is built and interfaced with a notebook PC, that the modulated signal of a natural sound composing sounds of a frog, a bird, and a water stream could produce the noise-free signal suitable enough for vibro-tactile speakers without causing Significant interfering disturbances, Lastly, for three different combinations of information provided to the subject, that is, i) with only video image, ii) with video image along with the modulated vibro-tactile stimuli as proposed in this study to the forearm of the subject, and iii) with video image along with full audio information, the effects to the human sense of reality and his emotion to given audio-video clips including various sounds and images are investigated and compared. It is shown from results of those experiments that the proposed method of providing modulated vibro-tactile stimuli along with the video images to the human has very high feasibility to transmit pseudo-aural sense to the human.

Keywords

References

  1. Sohn, M. S., Shin, H. S., Park, J. S. and Han, D. W., "Trends of Haptic Interface for Wearable Computers," Report of the Electronics and Telecommunication Research Institute of the Korea, Vol. 20, No. 5, pp. 149-155, 2005.
  2. Lindeman, R. W., Yanagida, Y., Noma, H. and Hosaka, K., "Wearable vibrotactile systems for virtual contact and information display," Virtual Reality, Vol. 9, No. 2, pp. 203-213, 2006. https://doi.org/10.1007/s10055-005-0010-6
  3. Hayward, V. and Maclean, K. E., "Do It Yourself Haptics : Part I," IEEE Robotics and Automation Magazine, Vol. 14 No. 4, pp. 88-104, 2007.
  4. Chouvardas, V. G., Miliou, A. N. and Hatalis, M. K., "Tactile displays : Overview and recent advances," Displays, Vol. 29, No. 3, pp. 185-194, 2008. https://doi.org/10.1016/j.displa.2007.07.003
  5. Gunther, E., Davenport, G. and O'Modhrain, S., "Cutaneous Grooves: Composing for the Sense of Touch," Proceedings of Conf. on New Instruments for Musical Expression(NIME-02), pp.37-42, 2002.
  6. Park, Y.-K., Choi, I.-M., Kim, W.-K. and Kang, D.-I., "Eyesight and Hearing Assistant Technique using Multiple Sensory Display," J. of KSPE, Vol. 26, No. 11, pp. 7-11, 2009.
  7. Jones, L. A. and Sarter, N. B., "Tactile Displays : Guidance for Their Design and Application," Human Factors, Vol. 50, No. 1, pp. 90-111, 2008. https://doi.org/10.1518/001872008X250638
  8. Wada, C., Shoji, H. and Ifukube, T., "Development and evaluation of a tactile display for a tactile vocoder," Technology and Disability, Vol. 11. No. 3, pp. 151-159, 1999.
  9. Brown, L. M., Brewster, S. A. and Purchase, H. C., "A First Investigation into the Effectiveness of Tactons," Proceedings of the First Eurohaptics Conference and Symposium on Haptic Interfaces for virtual Environment and Teleoperator Systems, pp. 167-176, 2005.
  10. Ternes, T. and MacLean, K. E., "Designing Large Sets of Haptic Icons with Rhythm," Haptics: Perception, Devices and Scenarios, Vol. 5024, pp. 199-208, 2008.
  11. Karam, M., Russo, F., Branje, C., Price, E. and Fels, D. I., "Towards A Model Human Cochlea: Sensory substitution for crossmodal audio-tactile displays," Graphics Interface Conference, pp. 267-274, 2008.
  12. Nanayakkara, S., Taylor, E., Wyse, L. and Ong, S. H., "An Enhanced Musical Experience for the Deaf: Design and Evaluation of a Music Display and a Haptic Chair," Proceedings of the 27th International Conference on Human Factors in Computing Systems, pp. 337-346, 2009.
  13. Sethares, W. A., "Rhythm and Transforms," Springer, pp. 115-128, 2007.