International Journal of Internet, Broadcasting and Communication

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Interactive Data Acquisition System based on Hand Tracking to evaluate Children's Cognitive Abilities

  • Ekaterina, Ten (Dept. Computer Engineering, Dongseo University) ;
  • Lee, Suk-Ho (Dept. Computer Engineering, Dongseo University)
  • Received : 2022.06.10
  • Accepted : 2022.06.16
  • Published : 2022.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

Autism (ASD) is a mental disorder characterized by a pronounced deficit in personal, social, speech, and other aspects of development and communication skills. Since autism is a complex developmental disorder that requires a lot of effort to recognize, this research was conducted to develop an interactive data Acquisition System and detect the first signs of ASD in children. The proposed system presents several variants of the tasks in an entertaining form, using hand tracking. Hand tracking is used to attract children's attention and interest them more to achieve more accurate results. The creation of the system is based on such libraries as OpenCV, PyGame, TensorFlow, and Mediapipe. The ultimate goal of the paper is to obtain data on the disease of autism in children for use in further diagnosis by medical experts.

Keywords

Acknowledgement

This work was supported by Dongseo University, "Dongseo Cluster Project" Research Fund of 2022 (DSU-20220001).

References

  1. T. Falkmer, K. Anderson, M. Falkmer, and C. Horlin, "Diagnostic procedures in autism spectrum disorders: a systematic literature review", Eur Child Adolesc Psychiatry, Vol. 22, pp. 329-340, 2013. DOI:https://doi.org/10.1007/s00787-013-0375-0
  2. R. Aburukba, F. Aloul, A. Mahmoud, K. Kamili, and S. Ajmal, "AutiAid: A learning mobile application for autistic children", in Proc. IEEE 19th International Conference on e-health Networking, Applications, and Services, Oct. 12-15, 2017, DOI:https://doi.org/10.1109/HealthCom.2017.8210788
  3. P. Sharma, M. D. Upadhaya, A. Twanabasu, J. Barroso, S. R. Khanal, and H. Paredes, "Express Your Feelings: An Interactive Application for Autistic Patients", HCII: Universal Access in Human-Computer Interaction. Multimodality and Assistive Environments, Lecture Notes in Computer Science, Vol. 11573, pp.160-170, 2019. DOI:https://doi.org/10.1007/978-3-030-23563-5_14
  4. A. Zaffke, N. Jain, N. Johnson, M. A. U. Alam, M. Magiera; S. I. Ahamed: "iCanLearn: A Mobile Application for Creating Flashcards and Social StoriesTM for Children with Autism", ICOST: Smart Homes and Health Telematics, Lecture Notes in Computer Science, Vol. 8456, pp. 225-230, December 2014. DOI:https://doi.org/10.1007/978-3-319-14424-5_25
  5. M. L. King; K. Takeguchi, S. E.Barry; R. A. Rehfeldt; V. E.Boyer; T. L. Mathews, "Evaluation of the iPad in the acquisition of requesting skills for children with autism spectrum disorder", Research in Autism Spectrum Disorders, Vol. 8, No. 9, pp. 1107-1120, September 2014. DOI:https://doi.org/10.1016/j.rasd.2014.05.011
  6. Snake game: https://youtu.be/ZjWFi66BkiI
  7. Wechsler test/ Children's version/ Subtest 9, Kohs block design test: https://psylab.info/Тест_Векслера/Детский_вариант/Субтест_9._Кубики_Косса
  8. D. M. Buffington, P. J. Krantz, L. E. McClannahan, and C. L. Poulson, "Procedures for teaching appropriate gestural communication skills to children with autism", Journal of Autism and Developmental Disorders, Vol. 28, pp. 535-545. 1998. DOI:https://doi.org/10.1023/A:1026056229214