• Proceedings of the Korea Information Processing Society Conference
• /
• 2023.11a
• /
• pp.299-302
• /
• 2023

입 모양 인식은 음성인식의 중요한 부분 중 하나로 이를 개선하기위한 다양한 연구가 진행되어 왔다. 기존의 연구에서는 주로 입술주변 영역을 관찰하고 인식하는데 초점을 두었으나, 본 논문은 음성인식 시스템에서 기존의 입술영역과 함께 입술, 턱, 뺨 등 다른 관심 영역을 고려하여 음성인식 시스템의 입모양 인식 성능을 비교하였다. 입 모양 인식의 관심 영역을 자동으로 검출하기 위해 객체 탐지 인공신경망을 사용하며, 이를 통해 다양한 관심영역을 실험하였다. 실험 결과 입술영역만 포함하는 ROI 에 대한 결과가 기존의 93.92%의 평균 인식률보다 높은 97.36%로 가장 높은 성능을 나타내었다.

• Journal of the Korea Computer Industry Society
• /
• v.8 no.4
• /
• pp.281-288
• /
• 2007

In this paper, we set 6 interesting points around lips. Analyzed and characterized is the distance change of these 6 interesting points when people pronounces 5 vowels of Korean language. 450 data are gathered and analyzed. Based on this analysis, the system is constructed and the recognition experiments are performed. In this system, we used the camera connected to computer to measure the distance vector between 6 interesting points. In the experiment, 80 normal persons were sampled. The observational error between samples was corrected using normalization method. We analyzed with 30 persons and experimented with 50 persons. We constructed three recognition systems and of those the neural net system gave the best recognition result of 87.44 %.

• Journal of the Institute of Electronics Engineers of Korea CI
• /
• v.40 no.5
• /
• pp.289-297
• /
• 2003

The performance of a bimodal system is affected by the accuracy of the endpoint detection from the input signal as well as the performance of the speech recognition or lipreading system. In this paper, we propose the endpoint detection method which detects the endpoints from the audio and video signal respectively and utilizes the signal to-noise ratio (SNR) estimated from the input audio signal to select the reliable endpoints to the acoustic noise. In other words, the endpoints are detected from the audio signal under the high SNR and from the video signal under the low SNR. Experimental results show that the bimodal system using the proposed endpoint detector achieves satisfactory recognition rates, especially when the acoustic environment is quite noisy.

• Journal of the Korea Society of Computer and Information
• /
• v.14 no.4
• /
• pp.39-46
• /
• 2009

Many lip region detection methods have been developed in PC environment. But the existing methods are difficult to run on real-time in resource limited mobile devices. To solve the problem, this paper proposes a real-time lip region detection method for lipreading in Mobile device. It detects face region by using adaptive face color information. After that, it detects lip region by using geometrical relation between eyes and lips. The proposed method is implemented in a smart phone with Intel PXA 270 embedded processor and 386MB memory. Experimental results show that the proposed method runs at the speed 9.5 frame/see and the correct detection rate was 98.8% for 574 images.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.6 no.5
• /
• pp.783-788
• /
• 2002

Recently, research and developmental direction of communication system is concurrent adopting voice data and face image in speaking to provide more higher recognition rate then in the case of only voice data. Therefore, we present a method of lipreading in speech image sequence by using the 3-D facial shape model. The method use a feature information of the face image such as the opening-level of lip, the movement of jaw, and the projection height of lip. At first, we adjust the 3-D face model to speeching face Image sequence. Then, to get a feature information we compute variance quantity from adjusted 3-D shape model of image sequence and use the variance quality of the adjusted 3-D model as recognition parameters. We use the intensity inclination values which obtaining from the variance in 3-D feature points as the separation of recognition units from the sequential image. After then, we use discrete HMM algorithm at recognition process, depending on multiple observation sequence which considers the variance of 3-D feature point fully. As a result of recognition experiment with the 8 Korean vowels and 2 Korean consonants, we have about 80% of recognition rate for the plosives md vowels.

• Journal of the Korea Society of Computer and Information
• /
• v.14 no.3
• /
• pp.173-182
• /
• 2009

In this paper, we studied about the extraction of the parameter and implementation of speechreading system to recognize the Korean 8 vowel. Face features are detected by amplifying, reducing the image value and making a comparison between the image value which is represented for various value in various color space. The eyes position, the nose position, the inner boundary of lip, the outer boundary of upper lip and the outer line of the tooth is found to the feature and using the analysis the area of inner lip, the hight and width of inner lip, the outer line length of the tooth rate about a inner mouth area and the distance between the nose and outer boundary of upper lip are used for the parameter. 2400 data are gathered and analyzed. Based on this analysis, the neural net is constructed and the recognition experiments are performed. In the experiment, 5 normal persons were sampled. The observational error between samples was corrected using normalization method. The experiment show very encouraging result about the usefulness of the parameter.

• The Journal of the Acoustical Society of Korea
• /
• v.21 no.5
• /
• pp.471-477
• /
• 2002

Recently, lip-reading has been studied actively as an auxiliary method of automatic speech recognition(ASR) in noisy environments. However, almost of research results were obtained based on the database constructed in indoor condition. So, we dont know how developed lip-reading algorithms are robust to dynamic variation of image. Currently we have developed a lip-reading system based on image-transform based algorithm. This system recognize 22 words and this word recognizer achieves word recognition of up to 53.54%. In this paper we present how stable the lip-reading system is in environmental variance and what the main variant factors are about dropping off in word-recognition performance. For studying lip-reading robustness we consider spatial valiance (translation, rotation, scaling) and illumination variance. Two kinds of test data are used. One Is the simulated lip image database and the other is real dynamic database captured in car environment. As a result of our experiment, we show that the spatial variance is one of degradations factors of lip reading performance. But the most important factor of degradation is not the spatial variance. The illumination variances make severe reduction of recognition rates as much as 70%. In conclusion, robust lip reading algorithms against illumination variances should be developed for using lip reading as a complementary method of ASR.

• Journal of the Korea Institute of Information and Communication Engineering
• /
• v.3 no.1
• /
• pp.59-68
• /
• 1999

Recently, research and developmental direction of communication system is concurrent adopting voice data and face image in speaking to provide more higher recognition rate then in the case of only voice data. Therefore, we present a method of lipreading in speech image sequence by using the 3-D facial shape model. The method use a feature information of the face image such as the opening-level of lip, the movement of jaw, and the projection height of lip. At first, we adjust the 3-D face model to speeching face image sequence. Then, to get a feature information we compute variance quantity from adjusted 3-D shape model of image sequence and use the variance quality of the adjusted 3-D model as recognition parameters. We use the intensity inclination values which obtaining from the variance in 3-D feature points as the separation of recognition units from the sequential image. After then, we use discrete HMM algorithm at recognition process, depending on multiple observation sequence which considers the variance of 3-D feature point fully. As a result of recognition experiment with the 8 Korean vowels and 2 Korean consonants, we have about 80% of recognition rate for the plosives and vowels. We propose that usability with visual distinguishing factor that using feature vector because as a result of recognition experiment for recognition parameter with the 10 korean vowels, obtaining high recognition rate.