Electronics and Telecommunications Trends (전자통신동향분석)

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Haptic-based Music Experience Technology Trends for the Hearing Impaired

청각장애인을 위한 햅틱 기반 음악 실감 기술 동향

  • Y.M. Song ;
  • S.Y. Shin ;
  • C.Y. Jeong ;
  • M.S. Kim
  • 송영미 (휴먼증강연구실) ;
  • 신승용 (휴먼증강연구실) ;
  • 정치윤 (휴먼증강연구실) ;
  • 김무섭 (휴먼증강연구실)
  • Published : 2024.02.01
Download PDF
⟨ Previous Next ⟩

Abstract

Music is a means of emotional expression and self-expression. In addition, it allows to interact with others and communicate with the world. Music may be considered as inaccessible for people with hearing impairment, who are sometimes discriminated from the music community. We explore trends in various technologies and research that enable everyone to access and enjoy music through experiences that leverage new and innovative technological approaches and bridge the gap between people with and without hearing impairment. Various aspects of haptic systems are being studied, but most of them are performance-oriented and focus only on technical functions. As research matures, more detailed and new studies that converge with various senses are being attempted. These studies will likely evolve into influential research areas that can positively affect the lives of people in terms of accessibility and inclusion by providing detailed functions and stimuli to specific users, including those with hearing impairment.

Keywords

Acknowledgement

본 연구는 2023년 정부(과학기술정보통신부)의 재원으로 정보통신기획평가원의 지원을 받아 수행된 연구임[no. RS-2023-00223440, 청각장애인을 위한 다감각 실감 시스템 및 교육 솔루션 개발].

References

  1. WTO, Deafness and Hearing Loss, 2023.
  2. Australia Department of Health and Aged Care, Roadmap for Hearing Health, 2019.
  3. N.T. Jiam, M.T. Caldwell, and C.J. Limb, "What does music sound like for a cochlear implant user?," Otol. Neurotol., vol. 38, no. 8, 2017, pp. 240-247.
  4. M.F. Dorman et al., "Sound source localization by normal-hearing listeners, hearing-impaired listeners and cochlear implant listeners," Audiol. Neurotol., vol. 21, no. 3, 2016, pp. 127-131.
  5. C. Bermejo and P. Hui, "A survey on haptic technologies for mobile augmented reality," ACM Comput. Surv., vol. 54, no. 9, 2021, pp. 1-35.
  6. B. Remache-Vinueza et al., "Audio-tactile rendering: A review on technology and methods to convey musical information through the sense of touch," Sensors, vol. 21, no. 19, 2021.
  7. M. Fleury et al., "A survey on the use of haptic feedback for brain-computer interfaces and neurofeedback," Front. Neurosci., vol. 14, 2020.
  8. C. Pacchierotti et al., "Wearable haptic systems for the fingertip and the hand: Taxonomy, review, and perspectives," IEEE Trans. Haptics., vol. 10, no. 4, 2017, pp. 580-600.
  9. N.T. Jiam and C.J. Limb, "Rhythm processing in cochlear implant-mediated music perception," Ann. N. Y. Acad. Sci., vol. 1453, 2019, pp. 22-28.
  10. B. Petry et al., "Scaffolding the music listening and music making experience for the deaf," in Cognitive Science and Technology, Springer, 2018, pp. 23-48.
  11. Z. Eitan and I. Rothschild, "How music touches: Musical parameters and listeners' audio-tactile metaphorical mappings," Psychol. Music., vol. 39, 2011, pp. 449-467.
  12. T.J. West et al., "The design of the body: suit: score, a full-body vibrotactile musical score," in Human Interface and the Management of Information. Information in Intelligent Systems, Springer, vol. 11570, 2019, pp. 70-89.
  13. F. Alves Araujo et al., "Auris system: Providing vibrotactile feedback for hearing impaired population," BioMed Res. Int., 2017, pp. 1-9.
  14. M.D. Fletcher, "Can haptic stimulation enhance music perception in hearing-impaired listeners?," Front. Neurosci., vol. 15, 2021.
  15. S. Shin et al., "Tactile tone system: A wearable device to assist accuracy of vocal pitch in cochlear implant users," in Proc, ASSETS '20, (Virtual only), Oct. 2020, pp. 1-3.
  16. R. Paisa et al., "Design and evaluation of a multisensory concert for cochlear implant users," Arts., vol. 12, no. 4, 2023.
  17. A. Haynes et al., "FeelMusic: Enriching our emotive experience of music through audio-tactile mappings," Multimodal Technol. Interact., vol. 5, no. 6, 2021.
  18. M.D. Fletcher et al., "Enhanced pitch discrimination for cochlear implant users with a new haptic neuroprosthetic," Sci. Rep., vol. 10, 2020, article no. 10354.
  19. B. Petry et al., "MuSS-bits: Sensor-display blocks for deaf people to explore musical sounds," in Proc. OzCHI '16, (Launceston, Australia), Nov. 2016, pp. 72-80.
  20. M. Mirzaei et al., "EarVR: Using ear haptics in virtual reality for deaf and hard-of-hearing people," IEEE Trans. Vis. Comput. Graph., vol. 26, 2020, pp. 2084-2093.
  21. K. Enriquez et al., "SENSE: Sensory component VR application for hearing impaired people to enhance the music experience," in Proc. CISTI '20, (Seville, Spain), 2020, pp. 1-6
  22. N.E. Kepp et al., "Virtual reality pitch ranking in children with cochlear implants, hearing Aids or normal hearing," Int. J. Pediatr. Otorhinolaryngol., vol. 161, 2022, article no. 111241.
  23. https://www.vodafone.co.uk/newscentre/news/5g-gives-deaf-festival-fans-a-first-of-its-kind-live-music-experience/
  24. https://www.classicfm.com/discover-music/live-music-deaf-audiences-vibrating-vests/
  25. https://www.lyricopera.org/shows/your-visit/accessibility/soundshirt/