Browse > Article
http://dx.doi.org/10.9708/jksci.2020.25.06.049

Efficient Mobile Writing System with Korean Input Interface Based on Face Recognition  

Kim, Jong-Hyun (Dept. of Software Application, Kangnam University)
Publication Information
Journal of the Korea Society of Computer and Information / v.25, no.6, 2020 , pp. 49-56 More about this Journal
Abstract
The virtual Korean keyboard system is a method of inputting characters by touching a fixed position. This system is very inconvenient for people who have difficulty moving their fingers. To alleviate this problem, this paper proposes an efficient framework that enables keyboard input and handwriting through video and user motion obtained through the RGB camera of the mobile device. To develop this system, we use face recognition to calculate control coordinates from the input video, and develop an interface that can input and combine Hangul using this coordinate value. The control position calculated based on face recognition acts as a pointer to select and transfer the letters on the keyboard, and finally combines the transmitted letters to integrate them to perform the Hangul keyboard function. The result of this paper is an efficient writing system that utilizes face recognition technology, and using this system is expected to improve the communication and special education environment for people with physical disabilities as well as the general public.
Keywords
RGB camera; Mobile device; Face recognition; Korean input interface; Korean keyboard system; Special education environment;
Citations & Related Records
연도 인용수 순위
  • Reference
1 S. E. Park, S. Y. Moon, H. S. Yoon, D. C. Shin, and H. S. Jeong, "An Input Method for Hangul Character and Implementation for Personal Digital Assistant," Proc. of Fall Conference of KIISE, vol. 21, no. 2, pp. 657-660, 1994.
2 W. H. Choi, S. C. Kim, and K. R. Song, "Apparatus and Method for Inputting the Korean Alphabet Based on the Character-Generative Principles of HUNMINJUNGEUM," Patent no. 1999-052648, 1999.
3 J. W. Kim, "Apparatus for Inputting Key and Method for Inputting Character by Direction Key," Patent no. 2001-0006208, 2001.
4 Share-typing, https://www.sharetyping.com/.
5 Kang, Seung-Shik, Hahn, Kwangsoo, "Hangul Vowel Input System for Hand-held Devices ", KIPS Transactions on Software and Data Engineering, pp. 507-512, 2005. DOI : 10.3745/KIPSTB.2005.12B.4.507.
6 T. J. Eom, J. B. Lee, and B. K. Kim, "Design of Smart Phone-Based Braille Keyboard System for Visually Impaired People," Journal of Information Security, vol. 12, no. 1, pp. 63-70, 2012.
7 Y. C. Byun, "2011 Survey on Impaired People," Research Report 2011-82, 2012.
8 H. Y. Kim, Gesture Recognition Interface by Keyboard Insertion IR Module, Master Thesis, KAIST, 2011.
9 D. H. Kim, Y. H. Kwon, and J. H. Kim, "A Hangul Input Interface Using Continuous Gestures and Language Models," Proc. of KCC2005, pp. 586-588, 2005.
10 Won Il Lee, "Natural Language Processing for Korean Speech Recognition," Proc. of KCC2005, pp. 586-588, 2005.
11 Sang Hun Jeong, "The Study on Phonetical Information for Speech Recognition", The Contents and Language Phenomena of Seokbosangjeol, vol. 49, pp.135-160, 2007.
12 Keyboard for Android : http://swypeinc.com/.
13 Lim, Chee Peng, Siew Chan Woo, Aun Sim Loh, and Rohaizan Osman. "Speech recognition using artificial neural networks." In Proceedings of the First International Conference on Web Information Systems Engineering, vol. 1, pp. 419-423, 2000. DOI : 10.1109/ICCSP.2018.8524333.
14 Jared Cechanowicz, Steven Dawson, Matt Victor, Sriram Subramanian, "Stylus based text input using expanding CIRRIN," AVI '06, May. 2006. DOI : 10.1145/1133265.1133299.
15 Gennaro Costagliola, Vittorio Fuccella, Michele DiCapua, Giovanni Guardi, "Performances of Multiple-Selection Enabled Menus in Soft Keyboards", DMS'09 - The 15th International Conference on Distributed Multimedia Systems., pp. 359-364, 2009.
16 Amberkar, Aditya, Parikshit Awasarmol, Gaurav Deshmukh, and Piyush Dave. "Speech Recognition using Recurrent Neural Networks." In International Conference on Current Trends towards Converging Technologies, pp. 1-4, 2018. DOI : 10.1109/ICCTCT.2018.8551185.
17 Schneider, Steffen and Baevski, Alexei and Collobert, Ronan and Auli, Michael, "wav2vec: Unsupervised pre-training for speech recognition", arXiv preprint arXiv:1904.05862, 2019.
18 Graves, Alex, Navdeep Jaitly, and Abdel-rahman Mohamed. "Hybrid speech recognition with deep bidirectional LSTM." In IEEE workshop on automatic speech recognition and understanding, pp. 273-278, 2013. DOI : 10.1109/ASRU.2013.6707742.
19 Sundermeyer, Martin, Ralf Schlüter, and Hermann Ney. "LSTM neural networks for language modeling." In Thirteenth annual conference of the international speech communication association. 2012.
20 Wang-Jun Kyung, Dae-Chul Kim, Yeong-Ho Ha, "Enhancement of Faded Images Using Integrated Compensation Coefficients Based on Multi-Scale Gray World Algorithm", vol. 39, pp. 459-466, The Journal of Korea Information and Communications Society, 2012. DOI : 10.7840/kics.2014.39A.8.459
21 Sriram, Anuroop and Jun, Heewoo and Gaur, Yashesh and Satheesh, Sanjeev, "Robust speech recognition using generative adversarial networks, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 5639-5643. 2017. DOI : 10.1109/ICASSP.2018.8462456
22 Schneider, Steffen and Baevski, Alexei and Collobert, Ronan and Auli, Michael, "wav2vec: Unsupervised pre-training for speech recognition", arXiv preprint arXiv:1904.05862, 2019.
23 Tian, Xu and Zhang, Jun and Ma, Zejun and He, Yi and Wei, Juan and Wu, Peihao and Situ, Wenchang and Li, Shuai and Zhang, Yang, "Deep LSTM for large vocabulary continuous speech recognition, arXiv preprint arXiv:1703.07090, 2017.