Browse > Article
http://dx.doi.org/10.22156/CS4SMB.2021.11.05.017

CNN-based Online Sign Language Translation Counseling System  

Park, Won-Cheol (Division of Computer Engineering, Kongju National University)
Park, Koo-Rack (Division of Computer Science & Engineering, Kongju National University)
Publication Information
Journal of Convergence for Information Technology / v.11, no.5, 2021 , pp. 17-22 More about this Journal
Abstract
It is difficult for the hearing impaired to use the counseling service without sign language interpretation. Due to the shortage of sign language interpreters, it takes a lot of time to connect to sign language interpreters, or there are many cases where the connection is not available. Therefore, in this paper, we propose a system that captures sign language as an image using OpenCV and CNN (Convolutional Neural Network), recognizes sign language motion, and converts the meaning of sign language into textual data and provides it to users. The counselor can conduct counseling by reading the stored sign language translation counseling contents. Consultation is possible without a professional sign language interpreter, reducing the burden of waiting for a sign language interpreter. If the proposed system is applied to counseling services for the hearing impaired, it is expected to improve the effectiveness of counseling and promote academic research on counseling for the hearing impaired in the future.
Keywords
OpenCV(Open Source Computer Vision); CNN(Convolutional Neural Networks); Sign Language; Hearing-Impaired Person; Image Processing;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 R. Sigit & D. R. Kartika. (2016). 3D Sign language translator using optical flow. International Electronics Symposium(IES), 262-266. DOI : 10.1109/elecsym.2016.7861014   DOI
2 T. Bluche, H. Ney & C. Kermorvant. (2013). Feature extraction with convolutional neural networks for handwritten word recognition. In 2013 12th International Conference on Document Analysis and Recognition. IEEE, 285-289. DOI : 10.1109/ICDAR.2013.64   DOI
3 J. H. Kim, S. Y. Shim, B. J. Oh, J. A. Lee, H. W. Choi & W. O. Cha. (2006). Implementation of an educational interactive game using OpenCV and color recognition. Proceedings of the Korea Multimedia Society Conference, 30-34.
4 LeCun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. nature, 521(7553), 436-444. DOI : 10.1038/nature14539   DOI
5 Y. T. Baek, S. H. Lee & J. S. Kim. (2017). Intelligent missing persons index system Implementation based on the OpenCV image processing and TensorFlow Deep-running Image Processing. Journal of the Korea Society of Computer and Information, 22(1), 15-21. DOI : 10.9708/jksci.2017.22.01.015   DOI
6 G. H. Kim, K. S. Chong & J. H. Youn. (2013). Automatic Recognition of Direction Information in Road Sign Image Using OpenCV. Journal of the Korean Society of Surveying, Geodesy, Photogrammetry and Cartography, 31(4), 293-300. DOI : 10.7848/ksgpc.2013.31.4.293   DOI
7 K. C. Kim & C. Yu. (2017). Flower Recognition System Using OpenCV on Android Platform. Journal of the Korea Institute of Information and Communication Engineering, 21(1), 123-129. DOI : 10.6109/jkiice.2017.21.1.123   DOI
8 N. K. Lee, J. Y. Kim & J. H. Shim. (2021). Empirical Study on Analyzing Training Data for CNN-based Product Classification Deep Learning Model. The Journal of Society for e-Business Studies, 26(1), 107-126. DOI : 10.7838/jsebs.2021.26.1.107   DOI
9 J. R. Park. (2010). A qualitative study on the Deaf for Meaning of the Sign Language Experience and Sign Language interpretation services Experience based on narrative inquiry. Social Science Research Review, 26(4), 93-122.
10 Y. I. Cho, B. C. Yoon & U. J. Min. (2012). A Qualitative Study for a Educational Sign Language Interpretation Service of Deaf Students at G High School. The Journal of Special Children Education, 14(3), 237-255. DOI : 10.21075/kacsn.2012.14.3.237   DOI
11 H. E. Kim, H. J. Koh, D. I. Kim & S. M. Choi. (2019). Research Trends and Support Measures in Counselling Service for Children and Adolescent with Disabilities in Korea. Asian Journal of Education, 20(3), 831-852. DOI : 10.15753/aje.2019.09.20.3.831   DOI
12 S. M. Koo, I. G. Jang & Y. S. Son. (2018). An Open Source Hardware based Sign Language Interpreter Glove & Situation Awareness Auxiliary IoT Device for the Hearing Impaired. KIISE Transactions on Computing Practices, 24(4), 204-209. DOI : 10.5626/KTCP.2018.24.4.204   DOI
13 D. Rempel, M. J. Camilleri & D. L. Lee. (2017). The design of hand gestures for human-computer interaction: Lessons from sign language interpreters. International Journal of Human-Computer Studies, 72(10-11), 728-735. DOI : 10.1016/j.ijhcs.2014.05.003   DOI
14 G. T. Lee, T. H. Kim, J. H. Cho & M. K. Moon. (2017). Development of Parking Management System Using OpenCV for Handicapped Person. Proceedings of the HCI Society of Korea Conference, 1187-1189.
15 S. H. Lee & H. M. Ahn. (2012). Piano practice using OpenCV and the Android application project. Proceedings of the Korean Society of Computer Information Conference, 20(2), 267-268.