Journal of the Institute of Electronics Engineers of Korea SC (전자공학회논문지SC)

The Institute of Electronics and Information Engineers (대한전자공학회)
권호 표지

Betterment of Mobile Sign Language Recognition System

모바일 수화 인식 시스템의 개선에 관한 연구

  • 박광현 (한국과학기술원 전자전산학과)
  • Published : 2006.07.01
Download PDF
⟨ Previous Next ⟩

Abstract

This paper presents a development of a mobile sign language recognition system for daily communication of deaf people, who are sign dependent to access language, with hearing people. The system observes their sign by a cap-mounted camera and accelerometers equipped on wrists. To create a real application working in mobile environment, which is a harder recognition problem than lab environment due to illumination change and real-time requirement, a robust hand segmentation method is introduced and HMMs are adopted with a strong grammar. The result shows 99.07% word accuracy in continuous sign.

본 논문에서는 수화를 의사소통 수단으로 사용하는 청각 장애인이 일반인과 일상 대화를 할 수 있도록 도와주는 모바일 수화 인식 시스템을 다룬다. 개발된 시스템은 모자에 부착된 카메라와 손목에 착용한 가속도 센서를 통해 사용자의 수화 동작을 관찰하는데, 모바일 환경에서 실제 적용할 수 있도록 조명 변화에 둔감하고 실시간 처리가 가능하도록 개발하였다. 이를 위해 조명 변화에 강인한 손 영역 분할 방법을 제안하고 추출된 손 영역 정보를 히든 마르코프 모델의 입력으로 사용하여 연속적인 수화에 대해 99.07%의 단어 정확도를 얻었다.

Keywords

References

  1. C. Vogler and D. Metaxas, 'Adapting hidden Markov models for ASL recognition by using three-dimensional computer vision methods,' Proceedings of the IEEE International Conference on Systems, Man and Cybernetics, pp. 156-161, 1997 https://doi.org/10.1109/ICSMC.1997.625741
  2. C. Vogler and D. Metaxas, 'ASL recognition based on a coupling between HMMs and 3D motion analysis,' Proceedings of the IEEE International Conference on Computer Vision, pp. 363-369, 1998 https://doi.org/10.1109/ICCV.1998.710744
  3. T. Starner, J. Weaver and A. Pentland, 'Real-time American sign language recognition using desk and wearable computer based video,' IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no.12, pp. 1371-1375, 1998 https://doi.org/10.1109/34.735811
  4. J.-S. Kim, W. Jang and Z. Bien, 'A dynamic gesture recognition system for the Korean sign language KSL,' IEEE Transactions on System, Man and Cybernetics - Part B, vol. 26, no. 2, pp. 354-359, 1996 https://doi.org/10.1109/3477.485888
  5. J.-B. Kim and Z. Bien, 'Recognition of continuous Korean sign language using gesture tension model and soft computing technique,' IEICE Transactions on Information and Systems, vol. 87, no. 5, pp. 1265-1270, 2004
  6. L. Yoshino, T. Kawashima and Y. Aoki, 'Recognition of Japanese sign language from image sequence using color combination,' Proceedings of International Conference on Image Processing, pp. 511-514, 1996 https://doi.org/10.1109/ICIP.1996.560543
  7. R. Liang and M. Ouhyoung, 'A real-time continuous gesture recognition system for sign language,' Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, pp. 558-565, 1998 https://doi.org/10.1109/AFGR.1998.671007
  8. G. Fang, W. Gao and D. Zhao, 'Large vocabulary sign languge recognition based on fuzzy decision trees,' IEEE Transactions on Systems, Man and Cybernetics - Part A, vol. 34, no. 3, pp. 305-314, 2004 https://doi.org/10.1109/TSMCA.2004.824852
  9. W. Gao, G. Fang, D. Zhao and Y. Chen, 'Transition movement models for large vocabulary continuous sign language recognition(CSL),' Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 553-558, 2004 https://doi.org/10.1109/AFGR.2004.1301591
  10. T. Starner and A. Pentland, 'Visual recognition of American sign language using hidden Markov models,' Proceedigns of International Workshop on Automatic Face and Gesture Recognition, pp. 189-194, 1995
  11. H. Brashear, T. Starner, P. Lukowicz and H. Junker, 'Using mutiple sensors for mobile sign language recognition,' Proceedings of the Seventh IEEE International Symposium on Wearable Computers, pp. 45-52, 2003
  12. J. L. Hernandez-Rebollar, N. Kyriakopouls and R. W. Lindeman, 'A new instrumented approach for translating American sign language into sound and text,' Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 547-552, 2004 https://doi.org/10.1109/AFGR.2004.1301590
  13. R. M. McGuire, J. Hernandz-Rebollar, T. Starner, V. Henderson, H. Brashear and D. S. Ross, 'Towards a one-way American sign language translator,' Proceedings of the Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 620-625, 2004 https://doi.org/10.1109/AFGR.2004.1301602
  14. M.-C. Su, 'A fuzzy rule-based approach to spatio-temporal hand gesture recognition,' IEEE Transactions on Systems, Man and Cybernetics-Part C, vol. 30, no. 2, pp. 276-281, 2000 https://doi.org/10.1109/5326.868448
  15. L. R. Rabiner, 'A tutorial on hidden Markov models and selected applications in speech recognition,' Proceedings of the IEEE, vol. 77, no. 2, pp. 257-286, 1989 https://doi.org/10.1109/5.18626
  16. K. Lyons, 'Everyday wearable computer use: a case study of an expert user,' Proceedings of the 5th International Symposium of Mobile HCI, 2003
  17. N. Oliver, A. P. Pentland and F. Berard, 'Lafter: lips and face real time tracker,' Proceedings of the IEEE Conference on Computer Vision and Patten Recognition, pp. 123-129, 1997 https://doi.org/10.1109/CVPR.1997.609309
  18. J. Yang, L. Weier and A. Waibel, 'Skin-color modeling and adaptation,' Proceedings of Asian Conference on Computer Vision, pp. 687-694, 1998
  19. M. Storring, H. J. Andersen and E. Graunm, 'Skin colour detection under changing lighting conditions,' Proceedings of the Seventh Symposium on Intelligent Robotics Systems, pp. 187-195, 1999
  20. M. Storring, H. J. Andersen and E. Graunm, 'Estimation of the illuminant colour from human skin colour,' Proceedings of the Fouth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 64-69, 2000 https://doi.org/10.1109/AFGR.2000.840613
  21. L. Sigal, S. Sclaroff and V. Athitsos, 'Skin color-based video segmentation under time-varying illumination,' IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 7, pp. 862-877, 2004 https://doi.org/10.1109/TPAMI.2004.35
  22. M. J. Jones and J. M. Rehg, 'Statistical color models with application to skin detection,' Proceedings of the IEEE Conference on Computer Vision and Patten Recognition, pp. 274-280, 1999 https://doi.org/10.1109/CVPR.1999.786951
  23. http://htk.eng.cam.ac.uk