• The KIPS Transactions:PartB
• /
• v.15B no.3
• /
• pp.229-236
• /
• 2008

This paper examines effectiveness of using color images instead of grayscale ones for automatic lipreading. First, we show the effect of color information for performance of humans' lipreading. Then, we compare the performance of automatic lipreading using features obtained by applying principal component analysis to grayscale and color images. From the experiments for various color representations, it is shown that color information is useful for improving performance of automatic lipreading; the best performance is obtained by using the RGB color components, where the average relative error reductions for clean and noisy conditions are 4.7% and 13.0%, respectively.

• Proceedings of the Korea Multimedia Society Conference
• /
• 2002.05c
• /
• pp.299-302
• /
• 2002

컴퓨터를 이용한 독순 연구는 Man Machine Interface, 지적부호화에 있어서의 송신측 기술, 청각 장애인의 독순 훈련 시스템 등 다방면에서 그 응용이 기대된다. 본 논문은, 움직임 정보는 입술의 에지영역에 집중하고 있음에 주목하여, 입술 에지영역의 Optical Flow 추정값을 독순정보로 이용하는 방법을 제안한다. 휘도값을 갖지 않는 에지에, 선형 가상 휘도값를 정해주어 Optical Flow를 추정하는 VGM을 도입해 특징 파라미터를 계산하고, 마할라노비스 평방거리(Mahalanobis's square distance)에 기초한 최대우도판별함수를 이용하여 단모음을 분석하는 알고리즘을 제안한다.

• The KIPS Transactions:PartB
• /
• v.14B no.7
• /
• pp.523-530
• /
• 2007

This paper proposes a new stochastic optimization algorithm for hidden Markov models (HMMs) used as a recognizer of automatic lipreading. The proposed method combines a global stochastic optimization method, the simulated annealing technique, and the local optimization method, which produces fast convergence and good solution quality. We mathematically show that the proposed algorithm converges to the global optimum. Experimental results show that training HMMs by the method yields better lipreading performance compared to the conventional training methods based on local optimization.

• The KIPS Transactions:PartB
• /
• v.15B no.2
• /
• pp.123-130
• /
• 2008

Automatic lipreading is to recognize speech by observing the movement of a speaker's lips. It has received attention recently as a method of complementing performance degradation of acoustic speech recognition in acoustically noisy environments. One of the important issues in automatic lipreading is to define and extract salient features from the recorded images. In this paper, we propose a feature extraction method by using a new filtering technique for obtaining improved recognition performance. The proposed method eliminates frequency components which are too slow or too fast compared to the relevant speech information by applying a band-pass filter to the temporal trajectory of each pixel in the images containing the lip region and, then, features are extracted by principal component analysis. We show that the proposed method produces improved performance in both clean and visually noisy conditions via speaker-independent recognition experiments.

• The KIPS Transactions:PartB
• /
• v.15B no.1
• /
• pp.53-60
• /
• 2008

This paper proposes a new multiobjective optimization method for discriminative training of hidden Markov models (HMMs) used as the recognizer for automatic lipreading. While the conventional Baum-Welch algorithm for training HMMs aims at maximizing the probability of the data of a class from the corresponding HMM, we define a new training criterion composed of two minimization objectives and develop a global optimization method of the criterion based on simulated annealing. The result of a speaker-dependent recognition experiment shows that the proposed method improves performance by the relative error reduction rate of about 8% in comparison to the Baum-Welch algorithm.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 1998.11a
• /
• pp.181-185
• /
• 1998

Recently, research and developmental direction of communication system is concurrent adopting voice data and face image in speaking to provide more higher recognition rate then in the case of only voice data. Therefore, we present a method of lipreading in speech image sequence by using the 3-D facial shape model. The method use a feature information of the face image such as the opening-level of lip, the movement of jaw, and the projection height of lip. At first, we adjust the 3-D face model to speeching face image sequence. Then, to get a feature information we compute variance quantity from adjusted 3-D shape model of image sequence and use the variance quality of the adjusted 3-D model as recognition parameters. We use the intensity inclination values which obtaining from the variance in 3-D feature points as the separation of recognition units from the sequential image. After then, we use discrete HMM algorithm at recognition process, depending on multiple observation sequence which considers the variance of 3-D feature point fully. As a result of recognition experiment with the 8 Korean vowels and 2 Korean consonants, we have about 80% of recognition rate for the plosives and vowels.

• Journal of Music and Human Behavior
• /
• v.5 no.2
• /
• pp.41-66
• /
• 2008

This study investigated the effects of traditional percussion improvisation to hearing-impaired college students who are under stress. For the research, between 21 to 22 years old four hearing-impaired college students, who could do lip reading, were chosen. In quantity program, improved version of college student stress measuring method which invented by Gyoung-gu Jun and Gyo-hyeon Kim(1991) were applied, and graphs has been used for analysis. In quality program, for reliability, the researcher and two music therapists observed and analysed it. The period of research was from Dec 26, 2007 to Feb 21, 2008. There were total twenty sessions and two sessions were assigned for each week. One was 40 minutes individual session, and the other one was 50 minutes group session. Even though auditory function is critical in music playing or listening, this study showed the positive results of the therapeutic use of music on stress management for college students with hearing impairment. Future studies are important to continue to investigate the effectiveness of music therapy for hearing impaired clients who are under stress with various age range.