Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Non-contact Input Method based on Face Recognition and Pyautogui Mouse Control

얼굴 인식과 Pyautogui 마우스 제어 기반의 비접촉식 입력 기법

  • Park, Sung-jin (Applied Artificial Intelligence, Sungkyunkwan University) ;
  • Shin, Ye-eun (Applied Artificial Intelligence, Sungkyunkwan University) ;
  • Lee, Byung-joon (Department of Mathematics, Sungkyunkwan University) ;
  • Oh, Ha-young (College of Computing and Informatics, Sungkyunkwan University)
  • Received : 2022.07.31
  • Accepted : 2022.08.17
  • Published : 2022.09.30
Download PDF
⟨ Previous Next ⟩

Abstract

This study proposes a non-contact input method based on face recognition and Pyautogui mouse control as a system that can help users who have difficulty using input devices such as conventional mouse due to physical discomfort. This study includes features that help web surfing more conveniently, especially screen zoom, scroll function, and also solves the problem of eye fatigue, which has been suggested as a limitation in existing non-contact input systems. In addition, various set values can be adjusted in consideration of individual physical differences and Internet usage habits. Furthermore, no high-performance CPU or GPU environment is required, and no separate tracker devices or high-performance cameras are required. Through these studies, we intended to contribute to the realization of barrier-free access by increasing the web accessibility of the disabled and the elderly who find it difficult to use web content.

신체적 불편함으로 인해 기존의 마우스와 같은 입력 장치의 사용이 힘든 사용자에게 도움이 될 수 있는 시스템으로 얼굴 인식과 Pyautogui 마우스 제어 기반의 비접촉식 입력 기법을 제안한다. 본 연구는 특히 화면 확대/축소나 스크롤 기능과 같이 웹 서핑을 보다 편리하게 돕는 기능이 포함되어 있으며, 개인의 신체적 차이 및 웹 사용 습관을 고려해 여러 설정값을 조정할 수 있도록 하였다. 또한, 기존의 시스템에서 한계점으로 제시되었던 눈 피로도에 대한 문제도 해결하였다. 추가로 고성능 CPU나 GPU 환경이 요구되지 않고 별도의 트래커 장치나 고성능 카메라 또한 필요하지 않다. 이러한 연구를 통해 손을 쓰기 어려운 장애인 및 노인들의 웹 접근성을 높여 배리어프리 실현에 기여하고자 한다.

Keywords

Acknowledgement

This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) (No. NRF-2022 R1F1A1074696).

References

  1. H. Y. Kim, "Improvement of Web Accessibility through Auto-generated OCR Based Alternative Text," M. S. thesis, Hanbat University, Daejeon, 2022.
  2. KOSIS. Informatization statistical survey [Internet]. Available:https://kosis.kr/statHtml/statHtml.do?orgId=127&tblId=DT_120008_2019B001&conn_path=I2.
  3. KOSIS. Informatization statistical survey [Internet]. Available:https://kosis.kr/statHtml/statHtml.do?orgId=127&tblId=DT_120008_2019B005&conn_path=I2.
  4. J. H. Park, S. R. Jo, and S. B. Lim. "Object Magnification and Voice Command in Gaze Interface for the Upper Limb Disabled," Journal of Korea Multimedia Society, vol. 24, no. 7, pp. 903-912, Jul. 2021. https://doi.org/10.9717/KMMS.2021.24.7.903
  5. J. E. Seo, "An Empirical Study of the Quality of Assistive Technology for Improving Web Accessibility," M. S. thesis, Soongsil University, Seoul, 2012.
  6. Y. Kartynnik, A. Ablavatski, I. Grishchenko, and M. Grundmann. "Real-time Facial Surface Geometry from Monocular Video on Mobile GPUs," arXiv preprint arXiv:1907.06724, 2019.
  7. H. Hayashi, R. Inomata, R. Fujishiro, Y. Ouchi, K. Suzuki, and T. Nanami, "Development of Pre-Crash Safety System with Pedestrian Collision Avoidance Assist," in Proceedings of the 23rd International Conference on the Enhanced Safety of Vehicles, Seoul, Republic of Korea, pp. 13-0271. 2013.
  8. T. Soukupova and J. Cech, "Eye-Blink Detection Using Facial Landmarks," in Proceedings of 21st computer vision winter workshop, Rimske Toplice, Slovenia, 2016.
  9. Y. W. Suh, K. H. Kim, S. Y. Kang, S. W. Kim, J. R. Oh, H. M. Kim, and J. S. Song, "The Objective Methods to Evaluate Ocular Fatigue Associated With Computer Work," Journal of The Korean Ophthalmological Society, vol. 51, no. 10, pp. 1327-1332, Oct. 2010. https://doi.org/10.3341/jkos.2010.51.10.1327