Journal of IKEEE (전기전자학회논문지)

Institute of Korean Electrical and Electronics Engineers (한국전기전자학회)
권호 표지
DOI QR코드

DOI QR Code

Motion Study of Treatment Robot for Autistic Children Using Speech Data Classification Based on Artificial Neural Network

음성 분류 인공신경망을 활용한 자폐아 치료용 로봇의 지능화 동작 연구

  • Lee, Jin-Gyu (Dept. of Electrical Engineering, Semyung University) ;
  • Lee, Bo-Hee (Dept. of Electrical Engineering, Semyung University)
  • Received : 2019.12.11
  • Accepted : 2019.12.30
  • Published : 2019.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Currently, the prevalence of autism spectrum disorders in children is reported to be higher and shows various types of disorders. In particular, they are having difficulty in communication due to communication impairment in the area of social communication and need to be improved through training. Thus, this study proposes a method of acquiring voice information through a microphone mounted on a robot designed through preliminary research and using this information to make intelligent motions. An ANN(Artificial Neural Network) was used to classify the speech data into robot motions, and we tried to improve the accuracy by combining the Recurrent Neural Network based on Convolutional Neural Network. The preprocessing of input speech data was analyzed using MFCC(Mel-Frequency Cepstral Coefficient), and the motion of the robot was estimated using various data normalization and neural network optimization techniques. In addition, the designed ANN showed a high accuracy by conducting an experiment comparing the accuracy with the existing architecture and the method of human intervention. In order to design robot motions with higher accuracy in the future and to apply them in the treatment and education environment of children with autism.

현재 아이들의 자폐스펙트럼장애 유병률이 한층 더 높게 보고되고 있으며 다양한 형태의 장애 징후를 보이고 있다. 특히 이들은 사회적 의사소통 영역에서 의사소통장애로 인한 대화에 어려움을 겪고 있으며 이를 훈련을 통해 개선 시킬 필요가 대두된다. 이를 위해 본 연구에서는 사전 연구를 통해 설계된 로봇에 장착된 마이크를 통해 음성 정보를 취득하고 이러한 정보를 이용하여 지능적인 동작을 만드는 방식을 제안한다. 음성 정보를 로봇 동작으로 분류하기 위해 인공신경망을 이용하였으며 여러 신경망 기법중 합성곱 방식을 기본으로 한 순환신경망을 결합하여 정확도를 향상시키려고 하였다. 입력 음성 데이터의 전처리는 MFCC를 이용하여 분석하였으며 여러 데이터 정규화 및 인공신경망 최적화 기법을 활용하여 로봇의 동작을 추정하였다. 아울러 설계된 인공신경망은 기존에 사용한 구조 및 사람이 개입하여 분석하는 방법과의 정확도 비교 실험을 진행하여 분석 결과가 높은 정확도를 나타냈다. 향후 보다 높은 정확도를 가질 수 있는 로봇 동작을 설계하여 실제의 자폐아 치료 및 교육 환경에서 적용할 수 있기 위하여 다양한 형태의 데이터를 수집하고 효율적으로 전처리하는 방식에 대한 연구가 요구된다.

Keywords

References

  1. Kyung-Sook Lee, Hyeon-Sook Yoon, Huiseung Jeong, Hui-jeong Yoo, "A Study on Early Screening of Young Children with Autism Spectrum Disorder in Korea and Its Support System," Journal of the Korean Association for Persons with Autism(JKAPA), vol.15, no.2, pp.93-120, 2015.
  2. Jang-Won Moon, Byeong-Jong Jeong, "Issues and Tasks for Autism Spectrum Disorders in the DSM-5," Journal of Emotional & Behavioral Disorders(JEBD), vol.34, no.3, pp.251-270, 2018.
  3. Nadia Jmour, Sehla Zayen, Afef Abdelkrim, "Convolutional neural networks for image classification," 2018 International Conference on Advanced Systems and Electric Technologies (IC_ASET), pp.397-402, 2018. DOI: 10.1109/ASET.2018.8379889
  4. Sharath Adavanne, Pasi Pertilä, Tuomas Virtanen, "Sound event detection using spatial features and convolutional recurrent neural network," 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 771-775, 2017. DOI: 10.1109/ICASSP.2017.7952260
  5. Zahra Shah, Minsu Kim, Gil-Jin Jang, "Image Pattern Classification Using MFCC and HMM," 2018 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia), pp.1-4, 2018. DOI: 10.1109/ICCE-ASIA.2018.8552112
  6. Chandrasekhar Paseddula, Suryakanth V. Gangashetty, "DNN based Acoustic Scene Classification using Score Fusion of MFCC and Inverse MFCC," 2018 IEEE 13th International Conference on Industrial and Information Systems (ICIIS), pp.18-21, 2018. DOI: 10.1109/ICIINFS.2018.8721379
  7. Elvira Sukma Wahyuni, "Arabic speech recognition using MFCC feature extraction and ANN classification," 2017 2nd International conferences on Information Technology, Information Systems and Electrical Engineering (ICITISEE), pp.22-25, 2018. DOI: 10.1109/ICITISEE.2017.8285499
  8. Chi Zhang, Thang Nguyen, Shagan Sah, Raymond Ptucha, Alexander Loui, Carl Salvaggio, "Batchnormalized recurrent highway networks," 2017 IEEE International Conference on Image Processing (ICIP), pp.640-644, 2018. DOI: 10.1109/ICIP.2017.8296359
  9. Jose Bermudez Castro, Raul Queiroz Feitosa, Patrick Nigri Happ, "An Hybrid Recurrent Convolutional Neural Network for Crop Type Recognition Based on Multitemporal Sar Image Sequences," IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium, pp.3824-3827, 2018. DOI: 10.1109/IGARSS.2018.8517280
  10. Lichao Mou, Xiao Xiang Zhu, "A Recurrent Convolutional Neural Network for Land Cover Change Detection in Multispectral Images," IGARSS 2018-2018 IEEE International Geoscience and Remote Sensing Symposium, pp.4363-4366, 2018. DOI: 10.1109/IGARSS.2018.8517375
  11. Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, Andrew Rabinovich, "Going deeper with convolutions," 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.1-9, 2015. DOI: 10.1109/CVPR.2015.7298594
  12. Toshi Sinha, Brijesh Verma, Ali Haidar, "Optimization of convolutional neural network parameters for image classification," 2017 IEEE Symposium Series on Computational Intelligence (SSCI), pp.1-7, 2018. DOI: 10.1109/SSCI.2017.8285338
  13. Cecilia F. Silva, Clauirton A. Siebra, "An investigation on the use of convolutional neural network for image classification in embedded systems," 2017 IEEE Latin American Conference on Computational Intelligence (LA-CCI), pp.1-6, 2018. DOI: 10.1109/LA-CCI.2017.8285727
  14. Ekachai Phaisangittisagul, "An Analysis of the Regularization Between L2 and Dropout in Single Hidden Layer Neural Network," 2016 7th International Conference on Intelligent Systems, Modelling and Simulation (ISMS), pp.174-179, 2017. DOI: 10.1109/ISMS.2016.14
  15. Mirco Ravanelli, Philemon Brakel, Maurizio Omologo, Yoshua Bengio, "Batch-normalized joint training for DNN-based distant speech recognition," 2016 IEEE Spoken Language Technology Workshop (SLT), pp.28-34, 2017. DOI: 10.1109/SLT.2016.7846241
  16. Cesar Laurent, Gabriel Pereyra, Philemon Brakel, Ying Zhang, Yoshua Bengio, "Batch normalized recurrent neural networks," 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.2657-2661, 2016. DOI: 10.1109/ICASSP.2016.7472159