• Speech Sciences
• /
• v.11 no.2
• /
• pp.193-210
• /
• 2004

In this paper, we develop a real time lip-synch system that activates 2-D avatar's lip motion in synch with an incoming speech utterance. To realize the 'real time' operation of the system, we contain the processing time by invoking merge and split procedures performing coarse-to-fine phoneme classification. At each stage of phoneme classification, we apply the support vector machine (SVM) to reduce the computational load while retraining the desired accuracy. The coarse-to-fine phoneme classification is accomplished via two stages of feature extraction: first, each speech frame is acoustically analyzed for 3 classes of lip opening using Mel Frequency Cepstral Coefficients (MFCC) as a feature; secondly, each frame is further refined in classification for detailed lip shape using formant information. We implemented the system with 2-D lip animation that shows the effectiveness of the proposed two-stage procedure in accomplishing a real-time lip-synch task. It was observed that the method of using phoneme merging and SVM achieved about twice faster speed in recognition than the method employing the Hidden Markov Model (HMM). A typical latency time per a single frame observed for our method was in the order of 18.22 milliseconds while an HMM method applied under identical conditions resulted about 30.67 milliseconds.

• 한국HCI학회:학술대회논문집
• /
• 2006.02a
• /
• pp.843-851
• /
• 2006

본 논문에서는 음성에 따른 얼굴 애니메이션을 사실적으로 표현하기 위한 조음기관(혀, 성대 등)의 움직임을 시각화하는 방법을 제시한다. 이를 위해서, 음성에 따른 얼굴 애니메이션을 위한 말뭉치(Corpus)를 생성하고, 생성된 말뭉치에 대해서 음소 단위의 분석(Phoneme alignment) 처리를 한 후, 각 음소에 따른 조음기관의 움직임을 생성한다. 본 논문에서는 조음기관의 움직임 생성을 위해서 얼굴 애니메이션 처리에서 널리 사용되고 있는 기저 모델 기반 형태 혼합 보간 기법(Blend shape Interpolation)을 사용하였다. 그리고 이를 통하여 프레임/키프레임 기반 움직임 생성 사용자 인터페이스를 구축하였다. 구축된 인터페이스를 통해 언어치료사가 직접 각 음소 별 조음기관의 정확한 모션 데이터를 생성토록 한다. 획득된 모션 데이터를 기반으로 각 음소 별 조음기관의 3차원 기본 기저를 모델링하고, 새롭게 입력된 음소 시퀀스(phoneme sequence)에 대해서 동기화된 3차원 조음기관의 움직임을 생성한다. 이를 통해 자연스러운 3차원 얼굴 애니메이션에 적용하여 얼굴과 동기화된 조음 기관의 움직임을 만들어 낼 수 있다.

• Cartoon and Animation Studies
• /
• s.14
• /
• pp.1-13
• /
• 2008

The infinite imagination and variety of material, and freedom of expression are fundamental elements of the comics. However, although the freedom of comics expression is linked with the liberty of speech and press, it's expression is constrained by a public authority and social pressure, self-regulation. This paper explains some problems and desirable freedom of expression thought researches on constraints of comics expression of Korea and Japan.

• Speech Sciences
• /
• v.15 no.3
• /
• pp.117-131
• /
• 2008

The purpose of this study was to determine the effects of whole language approach on the development of language abilities and story structure concepts for hearing impaired children. For this end, two research questions have been established. First, what is the effect of whole language program using story books on hearing impaired children’s language abilities? Second, what is the effect of whole language program using story books on hearing impaired children's story structure concept? Three subjects participated in the study. Each subject was scheduled for a 40-minute session two times a week. Subjects received 36 sessions of use animation activities for 3 months. The study used a multiple baseline across the subjects. The followings were the findings of this study. First, the whole language program using story books improved hearing impaired children's language abilities. Second, the whole language program using story books improved hearing impaired children's story structure concept.

• Speech Sciences
• /
• v.10 no.4
• /
• pp.63-71
• /
• 2003

This study aims to experiment on learning French intonation, based on the visualization of melody, which was employed in the early sixties to reeducate those with communication disorders. The visualization of melody in this paper, however, was used to the foreign language learning and produced successful results in many ways, especially in learning foreign intonation. In this paper, we used the PitchWorks to visualize some French intonation samples and experiment on learning intonation based on the bitmap picture projected on a screen. The students could see the melody curve while listening to the sentences. We could observe great achievement on the part of the students in learning intonations, as verified by the result of this experiment. The students were much more motivated in learning and showed greater improvement in recognizing intonation contour than just learning by hearing. But lack of animation in the bitmap file could make the experiment nothing but a boring pattern practices. It would be better if we can use a sound analyser, as like for instance a PitchWorks, which is designed to analyse the pitch, since the students can actually see their own fluctuating intonation visualized on the screen.

• The Journal of the Korea Contents Association
• /
• v.10 no.9
• /
• pp.9-17
• /
• 2010

Nowadays, many research have been conducted on robots or interactive characters that properly respond to the users affection. In this paper, we develop an intelligent messenger that provides appropriate responses to text inputs according to user's intention and affection. In order to properly respond, the intelligent messenger adapts methods to recognize user's speech act and affection. And it uses an AIML-based interactive script to which tags are additionally attached to express affection and speech act. If the intelligent messenger finds a proper reply in the interactive scripts, it displays the reply in a dialog window, and an animation character expresses emotion assimilated with a user's affection. If the animation character is synchronized with a content robot through a wireless link, the robot in the same space with the user can provide emotional response.

• The Journal of the Korea Contents Association
• /
• v.10 no.4
• /
• pp.1-9
• /
• 2010

Authoring tools for sketching the motion of characters to be animated have been studied. However the natural interface for sound editing has not been sufficiently studied. In this paper, I present a novel method that sound sample is selected by speaking sound-imitation words(onomatopoeia). Experiment with the method based on statistical models, which is generally used for pattern recognition, showed up to 97% in the accuracy of recognition. In addition, to address the difficulty of data collection for newly enrolled sound samples, the GLR Test based on only one sample of each sound-imitation word showed almost the same accuracy as the previous method.

• Journal of Korea Game Society
• /
• v.20 no.5
• /
• pp.77-88
• /
• 2020

In game characters, the direction of character design varies depending on the expression or production method. It is necessary to perform game character personality design by reflecting the importance. This study derives the importance of the game character personality model using AHP for the efficient decision and presents game character personality design check tool. In design elements, the importance of external appearance was the highest in all of style. In personality factors, action in 3D realism, external appearance in 3D animation, and speech in 2D animation were the most important.

• Journal of Korea Multimedia Society
• /
• v.17 no.10
• /
• pp.1239-1252
• /
• 2014

The OST (Original Sound Track) in the film plays a vital role in increasing consensus and concentration to the storyline. The selected 4 animations are classified into 17 Narrative Processes (NP) by using NP Classification Model [1]. For the NPs each having OSTs, the authors have investigated 6 kinds of objective musical elements of the OST such as sound (speech, music, effect), tonality, tempo, range, intensity, and instrumentation. It is found that there are 33.3% common musical elements among all of them for the NPs with OSTs commonly. Among them, it is also found that there are 71.9% of common properties of the musical element. This research is meaningful by firstly showing that there are common properties of objective musical elements in each NP and the corresponding OST.

• The Journal of Korea Institute of Information, Electronics, and Communication Technology
• /
• v.12 no.5
• /
• pp.512-520
• /
• 2019

It is quite useful to represent speeches visually for learners who study foreign languages as well as the hearing impaired who cannot directly hear speeches, and a number of researches have been presented in the literature. They remain, however, at the level of representing the characteristics of speeches using colors or showing the changing shape of lips and mouth using the animation-based representation. As a result of such approaches, those methods cannot tell the users how far their pronunciations are away from the standard ones, and moreover they make it technically difficult to develop such a system in which users can correct their pronunciation in an interactive manner. In order to address these kind of drawbacks, this paper proposes a speech visualization model based on the relative distance between the user's speech and the standard one, furthermore suggests actual implementation directions by applying the proposed model to the visualization of Korean vowels. The method extract three formants F1, F2, and F3 from speech signals and feed them into the Kohonen's SOM to map the results into 2-D screen and represent each speech as a pint on the screen. We have presented a real system implemented using the open source formant analysis software on the speech of a Korean instructor and several foreign students studying Korean language, in which the user interface was built using the Javascript for the screen display.