Journal of Digital Contents Society (디지털콘텐츠학회 논문지)

Digital Contents Society (한국디지털콘텐츠학회)
권호 표지
DOI QR코드

DOI QR Code

Development of Smart Mirror System for Hearing Deaf's Pronunciation Training

청각 장애인을 위한 발음 교정 학습용 스마트 미러 시스템 개발

  • Jung, Ha-Yoon (Department of IT Engineering, Sookmyung Women's University) ;
  • Jeong, Da-Mi (Department of IT Engineering, Sookmyung Women's University) ;
  • Lee, Jong-Hyeok (Big Data Utilization Research Center, Sookmyung Women's University) ;
  • Kim, Byung-Gyu (Department of IT Engineering, Sookmyung Women's University)
  • 정하윤 (숙명여자대학교 IT 공학과) ;
  • 정다미 (숙명여자대학교 IT 공학과) ;
  • 이종혁 (숙명여자대학교 빅데이터활용 연구소) ;
  • 김병규 (숙명여자대학교 IT 공학과)
  • Received : 2017.04.19
  • Accepted : 2017.04.25
  • Published : 2017.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, there is a new trend about internet of things (IoT) such as shops with smart mirror around the fashion and beauty industry. Since smart mirror can display a content through a monitor which is attached to back of mirror system while looking through a mirror, it can be applied to various industries such as fashion, beauty and health care. This paper proposes an efficient learning system requiring no assistance from others for the hearing deaf who atrophy verbal skill and are inaccurate in pronunciation by using features of smart mirror. Also, this system proposes an efficient and simple lip reading method which can be applied to an embedded system and improves a learning efficiency by employing previously verified pronunciation training data.

최근 패션, 뷰티 업계를 중심으로 스마트 미러를 도입한 매장을 선보이는 등 IoT기술을 적용하고자 다양한 노력이 시도되고 있다. 스마트 미러는 거울을 통해 모습을 비춰볼 수 있으면서 거울 뒤에 부착된 화면을 통해 원하는 내용을 디스플레이 할 수 있어 패션, 뷰티, 헬스 케어 등 다양한 산업 분야에 적용이 가능하다. 본 논문에서는 이러한 스마트 미러의 특성을 이용하여 구술 능력이 퇴화되어 발음이 부정확한 청각 장애인이 타인의 도움 없이 스스로 발음 교정 학습을 할 수 있도록 하는 학습 시스템을 제안하고자 한다. 제안된 시스템에서는 임베디드 시스템에서 적용 가능한 효과적인 립 리딩 기법을 제안하여 기존에 검증된 발음 교정용 데이터와 연동하여 학습 효율을 높였다.

Keywords

References

  1. M. S. Lee, Experiences of Raising Children without Hearing Impairments among Hearing-Impaired Mothers, Thesis of master's degree, Department of Social Welfare, Chonbuk National University, 2010.
  2. S. H. Kim, Y. H. Lee, J. H. Hwang, M. Oh, M. G. Yee, N. H. Yee, D. Kang, S. J. Kwon, H. K. Oh ,S. Y. Yoon, S. W. Lee, Survey on the Actual Condition of Disabled Person at 2014, Korea Institute for Health and Social Affairs, Policy Report, 2014.
  3. Y. J. Lee, S. M. Lim, Y. J. Choi, B. H. Moon, "A Mobile App(See&Speech) of Correcting Pronunciation for Hearing-Impaired Persons," The Journal of Korean Association of Computer Education, Vol. 18, No. 4, pp.11-18, June 2015.
  4. J. W. Lee, J. W. Ryoo, J. D. Lee, J. H. Choi, "A Study on the Technical Trends of the IoT Home Assistant in Global Market," Journal of the Korea Society of Computer and Information, Vol. 25, No. 1, pp 109-110, January 2017.
  5. M. S. Oh, "Evaluation Research on Smart Mirror UX for Efficient Communication of the IoT Generation," Journal of the Korea Society of Digital Industry and Information Management, Vol. 11, No. 1, pp 121-133, March 2015. https://doi.org/10.17662/KSDIM.2015.11.1.121
  6. M. S. Oh, "A Study on Analysis of User Behavior and Needs for Efficient Use of a Home Smart Mirror," Journal of the Korea Society of Digital Industry and Information Management, Vol. 12, No. 1, pp 119-129, March 2016. https://doi.org/10.17662/ksdim.2016.12.1.119
  7. Mina Moon, Seonhee Kim, Hyejin Yun, "IoT Construction thorugh Smart Mirror and its Connection with Wearable Devices", Korean Institute of Information Scientists and Engineers, Pyeongchang, 2016.
  8. B. H. Choi, Study on the Values of Smart Mirror Products in the Smart Home Industry, Thesis of master's degree, Graduate School of Business Administration, Sungkyunkwan University, 2013.
  9. E. S. Lee, "Design & Implementation of Lipreading System using Robust Lip Area Extraction," The Journal of Korea Multimedia Society, pp. 524-527, May 2003.
  10. Y. U. Kim, "Real Time Lip Reading System Implementation in Embedded Environment," The Journal of Information Processing Society , Vol 17, No. B, pp. 227 -232, June 2010.
  11. J. S. Lee, "A New Temporal Filtering Method for Improved Automatic Lipreading," The Journal Korean Information Processing Society , Vol 15, No. B, pp. 123 -130, April 2008.
  12. H. M. Park, Articulation training supporting system for the hearing-impaired using a lip shape recognition method, Thesis of master's degree, Department of Computer Engineering, Dongguk University, 2009.
  13. J. S. Yang, K. S. Hong, "A Study on Lip Reading Based on Geometric Facial Feature," The Institute of Electronics and Information Engineers, Daegu, 2016.
  14. K. H. Lee, J. J. Kum, and S. B. Rhee, "Design & Implementation of Lipreading System using the Articulatory Controls Analysis of the Korean 5 Vowels," Journal of the Korea Computer Industry Society, vol. 8, no. 4, pp. 281-288, 2007.
  15. J. Y. Kim, S. H. Min, and S. H. Choi, "Robustness of Bimodal Speech Recognition on Degradation of Lip Parameter Estimation Performance," Journal of the Korean Society of Phonetic Science and Speech Technology, vol. 10, no. 2, pp. 29-33, 2003.
  16. Hwa-Jin Park, "Auditory and Language Training Service Model and Serious Game Contents Design for the hearing-impaired," Journal of Digital Contents Society, vol.12, no.4, pp.467-474, 2011 https://doi.org/10.9728/dcs.2011.12.4.467

Cited by

  1. 이종 공작기계 운용 관리를 위한 분산 스마트 데이터 모니터링 시스템 개발 vol.18, pp.6, 2017, https://doi.org/10.9728/dcs.2017.18.6.1175
  2. 광류와 조음 발성 특성을 이용한 립리딩 알고리즘 vol.21, pp.7, 2018, https://doi.org/10.9717/kmms.2018.21.7.745