• Proceedings of the Korea Multimedia Society Conference
• /
• 2003.05b
• /
• pp.524-527
• /
• 2003

최근 들어 립리딩은 멀티모달 인터페이스 기술의 응용분야에서 많은 관심을 모으고 있다. 동적 영상을 이용한 립리딩 시스템에서 해결해야 할 주된 문제점은 상황 변화에 독립적인 얼굴 영역과 입술 영역을 추출하는 것이다. 본 논문에서는 움직임이 있는 영상에서 화자의 얼굴영역과 입술영역을 컬러, 조명등의 변화에 독립적으로 추출하기 위해 HSI 모델과 블록 매칭을 이용하였고 특징 점 추출에는 이미지 기반 방법인 PCA 기법을 이용하였다. 추출된 입술 파라미터와 음성 데이터에 각각 HMM 기반 패턴 인식 방법을 개별적으로 적용하여 단어를 인식하였고 각각의 인식 결과를 가중치를 주어 합병하였다. 실험 결과에 의하면 잡음으로 음성 인식률이 낮아지는 경우에 음성인식과 립리딩을 함께 사용함으로써 전체적인 인식 결과를 향상시킬 수 있었다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2023.11a
• /
• pp.299-302
• /
• 2023

입 모양 인식은 음성인식의 중요한 부분 중 하나로 이를 개선하기위한 다양한 연구가 진행되어 왔다. 기존의 연구에서는 주로 입술주변 영역을 관찰하고 인식하는데 초점을 두었으나, 본 논문은 음성인식 시스템에서 기존의 입술영역과 함께 입술, 턱, 뺨 등 다른 관심 영역을 고려하여 음성인식 시스템의 입모양 인식 성능을 비교하였다. 입 모양 인식의 관심 영역을 자동으로 검출하기 위해 객체 탐지 인공신경망을 사용하며, 이를 통해 다양한 관심영역을 실험하였다. 실험 결과 입술영역만 포함하는 ROI 에 대한 결과가 기존의 93.92%의 평균 인식률보다 높은 97.36%로 가장 높은 성능을 나타내었다.

• Journal of the Korea Convergence Society
• /
• v.9 no.8
• /
• pp.1-8
• /
• 2018

In the field of biometric security such as face and iris recognition, it is essential to extract facial features such as eyes and lips. In this paper, we have studied a method of detecting eye and lip region in face image using faster R-CNN. The faster R-CNN is an object detection method using deep running and is well known to have superior performance compared to the conventional feature-based method. In this paper, feature maps are extracted by applying convolution, linear rectification process, and max pooling process to facial images in order. The RPN(region proposal network) is learned using the feature map to detect the region proposal. Then, eye and lip detector are learned by using the region proposal and feature map. In order to examine the performance of the proposed method, we experimented with 800 face images of Korean men and women. We used 480 images for the learning phase and 320 images for the test one. Computer simulation showed that the average precision of eye and lip region detection for 50 epoch cases is 97.7% and 91.0%, respectively.

• The Transactions of the Korea Information Processing Society
• /
• v.13 no.4
• /
• pp.166-173
• /
• 2024

This study explores lip-sync detection, focusing on the synchronization between lip movements and voices in videos. Typically, lip-sync detection techniques involve cropping the facial area of a given video, utilizing the lower half of the cropped box as input for the visual encoder to extract visual features. To enhance the emphasis on the articulatory region of lips for more accurate lip-sync detection, we propose utilizing a pre-trained visual attention-based encoder. The Visual Transformer Pooling (VTP) module is employed as the visual encoder, originally designed for the lip-reading task, predicting the script based solely on visual information without audio. Our experimental results demonstrate that, despite having fewer learning parameters, our proposed method outperforms the latest model, VocaList, on the LRS2 dataset, achieving a lip-sync detection accuracy of 94.5% based on five context frames. Moreover, our approach exhibits an approximately 8% superiority over VocaList in lip-sync detection accuracy, even on an untrained dataset, Acappella.

• The korean journal of orthodontics
• /
• v.30 no.2 s.79
• /
• pp.185-196
• /
• 2000

Patients who want to reduce their lip protrusion usually estimate the severity of the lip protrusion on the frontal aspect. Most orthodontists have a perplexed experience of a reduced thin line of vermilion border on the frontal aspect as incisors we retracted, even though the lip protrusion is thought to be reduced favorably on the sagittal aspect. Some patients also look older after orthodontic treatment because of severe lip thinning. This unaesthetic reduction of vermilion border urges us to study the vertical lip change during orthodontic procedure. The purpose of this study was to evaluate the vertical lip and perioral soft tissue changes in respect to incisor retraction in an effort to analyze which factors might be responsible for their vertical changes, using the multiple regression analysis. The results were as follows. 1. Upper and lower lip philtrum length(SnLs, LiB') were increased after retraction of anterior teeth, where as upper and lower vermilion height(LsSuls, StmiLi), and vermilion length(LsLi) were decreased. 2. Upper and lower lip length(SnStms, StmiB'), and soft tissue lower anterior facial height(SnMe') did not show any significant difference after treatment. 3. The increase of the upper lip philtrum length was mainly influenced by the extrusion of upper anterior teeth(${\Delta}U1V$), and the increase of the lower lip philtrum length was mainly influenced by the initial overjet before treatment. 4. The decrease of the upper and lower lip vermilion height was mainly influenced by the decrease of upper lip thickness.

• The korean journal of orthodontics
• /
• v.41 no.6
• /
• pp.411-422
• /
• 2011

Objective: The aim of this study was to evaluate the lip and perioral soft tissue changes after bracket bonding. Methods: The soft tissue changes in 45 adult patients (age greater than 18 years and less than 29 years) without severe skeletal discrepancy were evaluated using three-dimensional images acquired with a laser scanner before and after bracket bonding was performed using 4 types of labial orthodontic brackets. Results: Among the statistically significant changes in distance observed for the landmarks, the biggest change was observed in forward movement. The landmarks on the lateral sides also showed significant changes. While the landmarks on the upper lip showed significant upward movement, those on the lower lip showed significant downward movement. However, the changes were smaller for the landmarks on the upper lip (average, 0.87 mm) than for the landmarks on the lower lip (average, 1.21 mm). The type of bracket used did not significantly affect the soft tissue changes. Conclusions: These findings will help predict soft tissue changes after bracket bonding for orthodontic treatment.

• Journal of the Institute of Convergence Signal Processing
• /
• v.11 no.4
• /
• pp.289-297
• /
• 2010

In this paper, A method to prevent the external acoustic noise from being misrecognized as the speech recognition object is presented in the speech activity detection process for the speech recognition. Also this paper confirmed besides the acoustic energy to the lip movement image signals. First of all, the successive images are obtained through the image camera for personal computer and the lip movement whether or not is discriminated. The next, the lip movement image signal data is stored in the shared memory and shares with the speech recognition process. In the mean time, the acoustic energy whether or not by the utterance of a speaker is verified by confirming data stored in the shared memory in the speech activity detection process which is the preprocess phase of the speech recognition. Finally, as a experimental result of linking the speech recognition processor and the image processor, it is confirmed to be normal progression to the output of the speech recognition result if face to the image camera and speak. On the other hand, it is confirmed not to the output the result of the speech recognition if does not face to the image camera and speak. Also, the initial feature values under off-line are replaced by them. Similarly, the initial template image captured while off-line is replaced with a template image captured under on-line, so the discrimination of the lip movement image tracking is raised. An image processing test bed was implemented to confirm the lip movement image tracking process visually and to analyze the related parameters on a real-time basis. As a result of linking the speech and image processing system, the interworking rate shows 99.3% in the various illumination environments.

• Journal of the Society of Cosmetic Scientists of Korea
• /
• v.45 no.1
• /
• pp.95-103
• /
• 2019

Lips have a defect in maintenance of moisture due to their thin layer. As aging progresses, lips lose volume and redness, and become wrinkled. Fat grafting and filler surgery have been used to achieve attractive lips, but little research has been reported to develop better materials to replace the present methods. Recently, a study suggests that the increase of adipocyte number can be enhancing the expansion endogenous fat. In previous study, we identified that the efficacy of Magnolia officinalis bark extract (MOBE) was effective on the induction of adipogenic differentiation. In this study, we confirmed that MOBE enhanced the differentiation of human adipose-derived stem cells on the fat mimic 3D structure built by 3D bioprinting method From further experiments in human, we established a method to quantify the severity of lip wrinkle by measurement of standard deviation of gray value using Image J software. Finally, we found that topical treatment with 1% MOBE formulated lip balm significantly improved the lip wrinkle after using for 12 weeks. In conclusion, these findings suggest that MOBE has great potential, as a cosmetic ingredient, to reduce the lip wrinkle through the effect of promoting adipogenic differentiation.

• KIPS Transactions on Software and Data Engineering
• /
• v.9 no.12
• /
• pp.431-438
• /
• 2020

In this paper, the method to extract lip contour by multiple threshold is proposed. Spyridonos et. el. proposed a method to extract lip contour. First step is get Q image from transform of RGB into YIQ. Second step is to find lip corner points by change point detection and split Q image into upper and lower part by corner points. The candidate lip contour can be obtained by apply threshold to Q image. From the candidate contour, feature variance is calculated and the contour with maximum variance is adopted as final contour. The feature variance 'D' is based on the absolute difference near the contour points. The conventional method has 3 problems. The first one is related to lip corner point. Calculation of variance depends on much skin pixels and therefore the accuracy decreases and have effect on the split for Q image. Second, there is no analysis for color systems except YIQ. YIQ is a good however, other color systems such as HVS, CIELUV, YCrCb would be considered. Final problem is related to selection of optimal contour. In selection process, they used maximum of average feature variance for the pixels near the contour points. The maximum of variance causes reduction of extracted contour compared to ground contours. To solve the first problem, the proposed method excludes some of skin pixels and got 30% performance increase. For the second problem, HSV, CIELUV, YCrCb coordinate systems are tested and found there is no relation between the conventional method and dependency to color systems. For the final problem, maximum of total sum for the feature variance is adopted rather than the maximum of average feature variance and got 46% performance increase. By combine all the solutions, the proposed method gives 2 times in accuracy and stability than conventional method.

• Journal of Convergence for Information Technology
• /
• v.11 no.2
• /
• pp.194-200
• /
• 2021

The aim of this study was proceed for 6weeks to determine the effect of middle-aged women's upper lips ms by SUKI programs. A total of 20 subjects were tested, and the SUKI arbitration programs of Pattern T1, Pattern T2, Pattern T3, Pattern T4 were adopted three times a week in six weeks. The results were as this. It showed a significant difference in the EG, although some of the intra-group comparisons of the upper lip ms. Therefore, SUKI programs have been shown to help improve the elasticity of the upper lip ms(p<.05). In this study, the SUKI program was used based on the fact that although limited, the elasticity of the upper lip ms, located in the facial part of a middle-aged woman, causes various wrinkles around her mouth, making them lose confidence in their appearance. This also suggests that normal facial ms can affect the appearance of the female middle-aged woman, and it was intended to suggest some ways to manage the appearance by proper management of facial muscle according to the life cycle of the female middle-aged woman. In conclusion, we hope that in the future, various experiments will be used as new research data on how to prevent women's facial skin beauty and wrinkles and help improve elasticity of facial muscles around the mouth.