• Proceedings of the Korean Information Science Society Conference
• /
• 2004.04b
• /
• pp.631-633
• /
• 2004

최근 잡음환경에서 신뢰도 높은 음성인식을 위해 음성정보와 영상정보를 융합하는 이중모드 음성인식 방법이 활발히 연구되고 일다. 본 논문에서는 보다 음성 인식률을 향상시키기 위해 사용자가 말한 단어들의 순차 패턴을 나타내는 문맥정보를 이용한 후처리 방법을 제안한다. 이러한 문맥정보를 인식하기 위해 다층퍼셉트론 구조를 갖는 문맥정보 인식기를 제안한다 이중모드 음성인식기와 문맥정보 인식기 결과를 효율적으로 결합하기 위한 후처리 방법으로 순차 결합방법을 제안한다. 문맥정보를 이용한 이중모드 음성인식이 잡음 환경에서 90%이상의 인식률을 보였다 본 논문은 잡음환경에서 강인한 음성인식을 위해 문맥정보와 같은 사용자 행동패턴이 새로운 정보로 이용될 수 있다는 가능성을 제시한다.

• The Journal of the Acoustical Society of Korea
• /
• v.22 no.1
• /
• pp.28-34
• /
• 2003

Recent researches have been focusing on jointly using lip motions (i.e. visual speech) and speech for reliable speech recognitions in noisy environments. This paper also deals with the method of combining the result of the visual speech recognizer and that of the conventional speech recognizer through putting weights on each result: the paper proposes the method of determining proper weights for each result and, in particular, the weights are autonomously determined, depending on the amounts of noise in the speech and the image quality. Simulation results show that combining the audio and visual recognition by the proposed method provides the recognition performance of 84% even in severely noisy environments. It is also shown that in the presence of blur in images, the newly proposed weighting method, which takes the blur into account as well, yields better performance than the other methods.

• The Journal of the Acoustical Society of Korea
• /
• v.20 no.6
• /
• pp.51-56
• /
• 2001

Recent researches have been focusing on jointly using lip motions and speech for reliable speech recognitions in noisy environments. To this end, this paper proposes the method of combining the visual speech recognizer and the conventional speech recognizer with each output properly weighted. In particular, we propose the method of autonomously determining the weights, depending on the amounts of noise in the speech. The correlations between adjacent speech samples and the residual errors of the LPC analysis are used for this determination. Simulation results show that the speech recognizer combined in this way provides the recognition performance of 83 ％ even in severely noisy environments.

• Journal of the Institute of Electronics Engineers of Korea CI
• /
• v.41 no.3
• /
• pp.67-77
• /
• 2004

The recent research has been focused on fusion of audio and visual features for reliable speech recognition in noisy environments. In this paper, we propose a neural network based model of robust speech recognition by integrating audio, visual, and contextual information. Bimodal Neural Network(BMNN) is a multi-layer perception of 4 layers, each of which performs a certain level of abstraction of input features. In BMNN the third layer combines audio md visual features of speech to compensate loss of audio information caused by noise. In order to improve the accuracy of speech recognition in noisy environments, we also propose a post-processing based on contextual information which are sequential patterns of words spoken by a user. Our experimental results show that our model outperforms any single mode models. Particularly, when we use the contextual information, we can obtain over 90% recognition accuracy even in noisy environments, which is a significant improvement compared with the state of art in speech recognition. Our research demonstrates that diverse sources of information need to be integrated to improve the accuracy of speech recognition particularly in noisy environments.