Browse > Article

A study on the lip shape recognition algorithm using 3-D Model  

남기환 (관동대학교)
배철수 (관동대학교)
Publication Information
Journal of the Korea Institute of Information and Communication Engineering / v.6, no.5, 2002 , pp. 783-788 More about this Journal
Abstract
Recently, research and developmental direction of communication system is concurrent adopting voice data and face image in speaking to provide more higher recognition rate then in the case of only voice data. Therefore, we present a method of lipreading in speech image sequence by using the 3-D facial shape model. The method use a feature information of the face image such as the opening-level of lip, the movement of jaw, and the projection height of lip. At first, we adjust the 3-D face model to speeching face Image sequence. Then, to get a feature information we compute variance quantity from adjusted 3-D shape model of image sequence and use the variance quality of the adjusted 3-D model as recognition parameters. We use the intensity inclination values which obtaining from the variance in 3-D feature points as the separation of recognition units from the sequential image. After then, we use discrete HMM algorithm at recognition process, depending on multiple observation sequence which considers the variance of 3-D feature point fully. As a result of recognition experiment with the 8 Korean vowels and 2 Korean consonants, we have about 80% of recognition rate for the plosives md vowels.
Keywords
HMM;
Citations & Related Records
연도 인용수 순위
  • Reference
1 E.Petajan, B.Bischoff, D.Bodoff, and N. M. Brooke, 'An Improved Automatic Lipreading System to enhance Speech Recognition.' In ACM SIGCHI, 1988
2 Mase and A.Pentland.'LIP Reading. Automatic Visual Recognition of Spoken Word.' Proc. Image Understanding and Machin Vision, Optical of America, June. 1989
3 L.R.Raider, 'Mathematical Foundations of Hidden Markov Models', Recent Advances in speech understanding and Digital systems
4 L.R.Raider and B.H.Juang, 'An Introduction to Hidden Markov Models,' IEEE ASSP Magazine Vol. 3, No.1, 99.4-16, Jan 1986   DOI   ScienceOn
5 K. Mase and A. Pentland. 'Lip Reading: Automatic Visual Recognition of poken Words.' Technical Report 117, M.I.T. Media Lab Vision Science, 1989
6 Danial Reisfeld and Yehezkel Yeshurun, 'Robust Detection of Facial Features by Generalized Symmetry,' Proc. ICPR, pp.117-120 ,1992
7 K. E. Finn and A. A. 'Montgomery. Automatic Optically-Based Recognition of Speech.' Pattern Recognition Letters, 8:159 -164, 1988   DOI
8 Young Dong Lee, Chong Seak Choi, Kap Seak Choi, 'Lip Shape Synthesis of Korean Syllable for Human Interface.' Korea Institut Comunication, vol 19, pp.614-623