International Journal of Computer Science & Network Security

국제컴퓨터통신보호논문지학회 (International Journal of Computer Science & Network Security)
권호 표지
DOI QR코드

DOI QR Code

A Low-Cost Speech to Sign Language Converter

  • 투고 : 2021.03.05
  • 발행 : 2021.03.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

This paper presents a design of a speech to sign language converter for deaf and hard of hearing people. The device is low-cost, low-power consumption, and it can be able to work entirely offline. The speech recognition is implemented using an open-source API, Pocketsphinx library. In this work, we proposed a context-oriented language model, which measures the similarity between the recognized speech and the predefined speech to decide the output. The output speech is selected from the recommended speech stored in the database, which is the best match to the recognized speech. The proposed context-oriented language model can improve the speech recognition rate by 21% for working entirely offline. A decision module based on determining the similarity between the two texts using Levenshtein distance decides the output sign language. The output sign language corresponding to the recognized speech is generated as a set of sequential images. The speech to sign language converter is deployed on a Raspberry Pi Zero board for low-cost deaf assistive devices.

키워드

과제정보

This work belongs to the project grant No: T2020-39TD, funded by Ho Chi Minh City University of Technology and Education, Vietnam.

참고문헌

  1. U. Bellugi and S. Fischer, "A comparison of sign language and spoken language" Cognition, vol. 1, no. 2-3, pp. 173-200, 1972. https://doi.org/10.1016/0010-0277(72)90018-2
  2. O. Aran and L. Akarun, "Sign Language Processing and Interactive Tools for Sign Language Education," 2007 IEEE 15th Signal Processing and Communications Applications, Eskisehir, 2007, pp. 1-4.
  3. L. Boppana, R. Ahamed, H. Rane and R. K. Kodali, "Assistive Sign Language Converter for Deaf and Dumb," 2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData), Atlanta, GA, USA, 2019, pp. 302-307.
  4. N. C. Camgoz, S. Hadfield, O. Koller, H. Ney and R. Bowden, "Neural Sign Language Translation," 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, 2018, pp. 7784-7793.
  5. L. Kau, W. Su, P. Yu and S. Wei, "A real-time portable sign language translation system," 2015 IEEE 58th International Midwest Symposium on Circuits and Systems (MWSCAS), Fort Collins, CO, 2015, pp. 1-4.
  6. P. Lakkhanawannakun and C. Noyunsan, "Speech Recognition using Deep Learning," 2019 34th International Technical Conference on Circuits/Systems, Computers and Communications (ITC-CSCC), JeJu, Korea (South), 2019, pp. 1-4
  7. I. Gavat and D. Militaru, "Deep learning in acoustic modeling for Automatic Speech Recognition and Understanding - an overview," 2015 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), Bucharest, Romania, 2015, pp. 1-8
  8. A. Kumar, S. Verma and H. Mangla, "A Survey of Deep Learning Techniques in Speech Recognition," 2018 International Conference on Advances in Computing, Communication Control and Networking (ICACCCN), Greater Noida, India, 2018, pp. 179-185
  9. N. K. Mudaliar, K. Hegde, A. Ramesh and V. Patil, "Visual Speech Recognition: A Deep Learning Approach," 2020 5th International Conference on Communication and Electronics Systems (ICCES), Coimbatore, India, 2020, pp. 1218-1221.
  10. K. Lee, H. Hon, M. Hwang, S. Mahajan and R. Reddy, "The SPHINX speech recognition system," International Conference on Acoustics, Speech, and Signal Processing,, Glasgow, UK, 1989, pp. 445-448 vol.1, doi: 10.1109/ICASSP.1989.266459.
  11. K. Lee, H. Hon and R. Reddy, "An overview of the SPHINX speech recognition system," in IEEE Transactions on Acoustics, Speech, and Signal Processing, vol. 38, no. 1, pp. 35-45, Jan. 1990. https://doi.org/10.1109/29.45616
  12. D. Huggins-Daines, M. Kumar, A. Chan, A. W. Black, M. Ravishankar and A. I. Rudnicky, "Pocketsphinx: A Free, Real-Time Continuous Speech Recognition System for Hand-Held Devices," 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, Toulouse, 2006, pp. I-I.
  13. B. Lakdawala, F. Khan, A. Khan, Y. Tomar, R. Gupta and A. Shaikh, "Voice to Text transcription using CMU Sphinx A mobile application for healthcare organization," 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT), Coimbatore, 2018, pp. 749-753.
  14. D. B. C. Lima, R. M. B. da Silva Lima, D. de Farias Medeiros, R. I. S. Pereira, C. P. de Souza and O. Baiocchi, "A Performance Evaluation of Raspberry Pi Zero W Based Gateway Running MQTT Broker for IoT," 2019 IEEE 10th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), Vancouver, BC, Canada, 2019, pp. 0076-0081
  15. N. S. Yamanoor and S. Yamanoor, "High quality, low cost education with the Raspberry Pi," 2017 IEEE Global Humanitarian Technology Conference (GHTC), San Jose, CA, USA, 2017, pp. 1-5
  16. A. P. Jadhav and V. B. Malode, "Raspberry PI Based OFFLINE MEDIA SERVER," 2019 3rd International Conference on Computing Methodologies and Communication (ICCMC), Erode, India, 2019, pp. 531-533