Journal of the Korea Society of Computer and Information (한국컴퓨터정보학회논문지)

Korean Society of Computer Information (한국컴퓨터정보학회)
권호 표지
DOI QR코드

DOI QR Code

Sign Language Image Recognition System Using Artificial Neural Network

  • Kim, Hyung-Hoon (Dept. of Cosmetic Science, Kwangju Womens University) ;
  • Cho, Jeong-Ran (Dept. of Health Administration, Kwangju Womens University)
  • Received : 2019.01.11
  • Accepted : 2019.02.12
  • Published : 2019.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Hearing impaired people are living in a voice culture area, but due to the difficulty of communicating with normal people using sign language, many people experience discomfort in daily life and social life and various disadvantages unlike their desires. Therefore, in this paper, we study a sign language translation system for communication between a normal person and a hearing impaired person using sign language and implement a prototype system for this. Previous studies on sign language translation systems for communication between normal people and hearing impaired people using sign language are classified into two types using video image system and shape input device. However, existing sign language translation systems have some problems that they do not recognize various sign language expressions of sign language users and require special devices. In this paper, we use machine learning method of artificial neural network to recognize various sign language expressions of sign language users. By using generalized smart phone and various video equipment for sign language image recognition, we intend to improve the usability of sign language translation system.

Keywords

CPTSCQ_2019_v24n2_193_f0001.png 이미지

Fig. 1. Structure of artificial neural network

CPTSCQ_2019_v24n2_193_f0002.png 이미지

Fig. 2. Artificial neural network concept of sign language video recognition system

CPTSCQ_2019_v24n2_193_f0003.png 이미지

Fig. 3. Part implementation code of artificial neural network

CPTSCQ_2019_v24n2_193_f0004.png 이미지

Fig. 4. Example of numeric sign language image

CPTSCQ_2019_v24n2_193_f0005.png 이미지

Fig. 5. Part implementation code of feed forward and backward propagation of artificial neural network

CPTSCQ_2019_v24n2_193_f0006.png 이미지

Fig. 6. Learning and evaluation accuracy of artificial neural network 1600-10-10

CPTSCQ_2019_v24n2_193_f0007.png 이미지

Fig. 7. Correct recognition by artificial neural network 1600-10-10

CPTSCQ_2019_v24n2_193_f0008.png 이미지

Fig. 8. Incorrect recognition by artificial neural network 1600-10-10

CPTSCQ_2019_v24n2_193_f0009.png 이미지

Fig. 9. Learning and evaluation accuracy of artificial neural network 1600-30-10

CPTSCQ_2019_v24n2_193_f0010.png 이미지

Fig. 10. Correct recognition by artificial neural network 1600-30-10

CPTSCQ_2019_v24n2_193_f0011.png 이미지

Fig. 11. Incorrect recognition by artificial neural network 1600-30-10

Table 1. Going out status of persons with disabilities - by year, by type of disability

CPTSCQ_2019_v24n2_193_t0001.png 이미지

Table 2. Experiences that disabled people can not get to the clinic - Types of disability

CPTSCQ_2019_v24n2_193_t0002.png 이미지

References

  1. Ryu M. W., "Design and Implementation of Bi-directional Sign Language System for Emergency Medical Situation", Pusan National University M.S, Feb. 2015.
  2. Korea Employment Agency for the Disabled, "2017 Disability statistics", Survey statistics 2017-01, Nov. 2017.
  3. Vijaykumar Sutariya, Anastasia Groshev, Prabodh Sadana, Deepak Bhatia, Yashwant Pathak, "Artificial Neural Network in Drug Delivery and Pharmaceutical Research", The Open Bioinformatics Journal, Jul. 2013.
  4. S. Ferrari and R. F. Stengel, "Smooth function approximation using neural networks", IEEE Trans Neural Network, Vol. 16, pp. 24-38, Jan. 2005. https://doi.org/10.1109/TNN.2004.836233
  5. M. N. Jadid and D. R. Fairbairn, "Neural-network applications in predicting moment-curvature parameters from experimental data", Engineering applications of artificial intelligence, Vol. 9, pp. 309-319, Jun. 1996. https://doi.org/10.1016/0952-1976(96)00021-8
  6. H. M. Carpenter WC, "Understanding neural network approximations and polynomial approximations helps neural network performance," AI Expert. Vol. 2, pp. 31-33, Mar. 1995.
  7. Lee Seung Seok, Heo Jeong Hyun, No Seung Woo, Yoon Hyeon Jin, Park So Hyun, Kim Chan Kyu, "Sign language translation system based on deep learning for speech disorders", Proceedings of Symposium of the Korean Institute of communications and Information Sciences, Jun. 2018.
  8. Kyung-Hyuk Kwon, Yo-Seop Woo, Hong-Ki Min, "Design and Implementation of a Korean Text to Sign Language Translation System", The transactions of the Korea Information Processing Society, Mar. 2000.
  9. Saito G.. "Deep running starting from the bottom", O'REILLY, Hanbit Media Inc, Jan. 2018.
  10. Hope, Tom Resheff, YehezkelS. Lieder, Itay, "Learning TensorFlow", Hanbit Media, May. 2018.
  11. F. Liu, M. J. Er, "A novel efficient learning algorithm for self-generating fuzzy neural network with applications", International Journal of Neural Systems, Vol. 22, pp21-35, Feb. 2012. https://doi.org/10.1142/S0129065712003067
  12. D. K. Kim, "C++ API OpenCV Programming", The Publish Company of KAME, May 2015.
  13. AmmarAnuar, KhairulMuzzammilSaipullah, NurulAtiqah Ismail, and Soo Yew Guan, "OpenCV Based Real-Time Video Processing Using Android Smartphone", International Journal of Computer Technology and Electronics Engineering, Vol. 1, No. 3, pp.58-63, 2011.
  14. Lee Hyun-Suk, Kim Seung-Pil, Chung Wan-Young, "Development of Sign Language Translation System using Motion Recognition of Kinect", Journal of the institute of signal processing and systems, Vol. 14, No. 4, pp.235-242, Oct. 2013.