• 한국HCI학회:학술대회논문집
• /
• 한국HCI학회 2007년도 학술대회 1부
• /
• pp.667-674
• /
• 2007

Facial expressions provide significant clues about one's emotional state; however, it always has been a great challenge for machine to recognize facial expressions effectively and reliably. In this paper, we report a method of feature-based adaptive motion energy analysis for recognizing facial expression. Our method optimizes the information gain heuristics of ID3 tree and introduces new approaches on (1) facial feature representation, (2) facial feature extraction, and (3) facial feature classification. We use minimal reasonable facial features, suggested by the information gain heuristics of ID3 tree, to represent the geometric face model. For the feature extraction, our method proceeds as follows. Features are first detected and then carefully "selected." Feature "selection" is finding the features with high variability for differentiating features with high variability from the ones with low variability, to effectively estimate the feature's motion pattern. For each facial feature, motion analysis is performed adaptively. That is, each facial feature's motion pattern (from the neutral face to the expressed face) is estimated based on its variability. After the feature extraction is done, the facial expression is classified using the ID3 tree (which is built from the 1728 possible facial expressions) and the test images from the JAFFE database. The proposed method excels and overcomes the problems aroused by previous methods. First of all, it is simple but effective. Our method effectively and reliably estimates the expressive facial features by differentiating features with high variability from the ones with low variability. Second, it is fast by avoiding complicated or time-consuming computations. Rather, it exploits few selected expressive features' motion energy values (acquired from intensity-based threshold). Lastly, our method gives reliable recognition rates with overall recognition rate of 77%. The effectiveness of the proposed method will be demonstrated from the experimental results.

• ETRI Journal
• /
• 제35권6호
• /
• pp.1021-1028
• /
• 2013

Automatic facial expression recognition is a widely studied problem in computer vision and human-robot interaction. There has been a range of studies for representing facial descriptors for facial expression recognition. Some prominent descriptors were presented in the first facial expression recognition and analysis challenge (FERA2011). In that competition, the Local Gabor Binary Pattern Histogram Sequence descriptor showed the most powerful description capability. In this paper, we introduce hybrid facial representations for facial expression recognition, which have more powerful description capability with lower dimensionality. Our descriptors consist of a block-based descriptor and a pixel-based descriptor. The block-based descriptor represents the micro-orientation and micro-geometric structure information. The pixel-based descriptor represents texture information. We validate our descriptors on two public databases, and the results show that our descriptors perform well with a relatively low dimensionality.

• Asian Journal for Public Opinion Research
• /
• 제8권3호
• /
• pp.180-196
• /
• 2020

This paper considers Japanese political interviews to integrate conversation and facial expression analysis. The behaviors of political leaders will be disclosed by analyzing questions and responses by using the turn-taking system in conversation analysis. Additionally, audiences who cannot understand verbal expressions alone will understand the psychology of political leaders by analyzing their facial expressions. Integral analyses promote understanding of the types of facial and verbal expressions of politicians and their effect on public opinion. Politicians have unique techniques to convince people. If people do not know these techniques and ways of various expressions, they will become confused, and politics may fall into populism as a result. To avoid this, a complete understanding of verbal and non-verbal behaviors is needed. This paper presents two analyses. The first analysis is a qualitative analysis that deals with Prime Minister Shinzō Abe and shows that differences between words and happy facial expressions occur. That result indicates that Abe expresses disgusted facial expressions when faced with the same question from an interviewer. The second is a quantitative multiple regression analysis where the dependent variables are six facial expressions: happy, sad, angry, surprised, scared, and disgusted. The independent variable is when politicians have a threat to face. Political interviews that directly inform audiences are used as a tool by politicians. Those interviews play an important role in modelling public opinion. The audience watches political interviews, and these mold support to the party. Watching political interviews contributes to the decision to support the political party when they vote in a coming election.

• 한국멀티미디어학회논문지
• /
• 제25권7호
• /
• pp.964-972
• /
• 2022

The Uncanny Valley phenomenon has already been deemed as theoretical, and the characteristics of game character expression elements for the Uncanny Valley phenomenon were recognized through case analysis as well. By theoretical consideration and case studies, it was found out that the influential elements of the Uncanny Valley phenomenon can be classified as two primary factors: character surface treatment and facial expression animation. The prepared experimental materials and adjectives were measured to be Five-Point Likert Scale. The measured results were evaluated for both influence and comparative analysis through essential statistical analysis and Repeated Measuring ANOVA in SPSS. The conclusions which were drawn from this research are as follows: The surface treatment of characters did not substantially affect the Uncanny Valley phenomenon. Instead, character's expression animation had a significant impact on the Uncanny Valley phenomenon, which also led to another conclusion that the facial expression animation had an overall deeper impact on Uncanny Valley phenomenon compared with character's surface treatment. It was the unnatural facial expression animation that controlled all of the independent variables and also caused the Uncanny Valley phenomenon. In order for game characters to evade the Uncanny Valley phenomenon and enhance game immersion, the facial expression animation of the character must be done spontaneously.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• 제10권3호
• /
• pp.1390-1403
• /
• 2016

Facial expression recognition (FER) plays a very significant role in computer vision, pattern recognition, and image processing applications such as human computer interaction as it provides sufficient information about emotions of people. For video-based facial expression recognition, depth cameras can be better candidates over RGB cameras as a person's face cannot be easily recognized from distance-based depth videos hence depth cameras also resolve some privacy issues that can arise using RGB faces. A good FER system is very much reliant on the extraction of robust features as well as recognition engine. In this work, an efficient novel approach is proposed to recognize some facial expressions from time-sequential depth videos. First of all, efficient Local Binary Pattern (LBP) features are obtained from the time-sequential depth faces that are further classified by Generalized Discriminant Analysis (GDA) to make the features more robust and finally, the LBP-GDA features are fed into Hidden Markov Models (HMMs) to train and recognize different facial expressions successfully. The depth information-based proposed facial expression recognition approach is compared to the conventional approaches such as Principal Component Analysis (PCA), Independent Component Analysis (ICA), and Linear Discriminant Analysis (LDA) where the proposed one outperforms others by obtaining better recognition rates.

• ETRI Journal
• /
• 제32권5호
• /
• pp.784-794
• /
• 2010

Automatic facial expression recognition has many potential applications in different areas of human computer interaction. However, they are not yet fully realized due to the lack of an effective facial feature descriptor. In this paper, we present a new appearance-based feature descriptor, the local directional pattern (LDP), to represent facial geometry and analyze its performance in expression recognition. An LDP feature is obtained by computing the edge response values in 8 directions at each pixel and encoding them into an 8 bit binary number using the relative strength of these edge responses. The LDP descriptor, a distribution of LDP codes within an image or image patch, is used to describe each expression image. The effectiveness of dimensionality reduction techniques, such as principal component analysis and AdaBoost, is also analyzed in terms of computational cost saving and classification accuracy. Two well-known machine learning methods, template matching and support vector machine, are used for classification using the Cohn-Kanade and Japanese female facial expression databases. Better classification accuracy shows the superiority of LDP descriptor against other appearance-based feature descriptors.

• 한국HCI학회:학술대회논문집
• /
• 한국HCI학회 2008년도 학술대회 1부
• /
• pp.487-492
• /
• 2008

Facial expression recognition is an intensive research area for designing Human Computer Interfaces. In this work, we present a new facial expression recognition system utilizing Enhanced Independent Component Analysis (EICA) for feature extraction and discrete Hidden Markov Model (HMM) for recognition. Our proposed approach for the first time deals with sequential images of emotion-specific facial data analyzed with EICA and recognized with HMM. Performance of our proposed system has been compared to the conventional approaches where Principal and Independent Component Analysis are utilized for feature extraction. Our preliminary results show that our proposed algorithm produces improved recognition rates in comparison to previous works.

• 한국멀티미디어학회논문지
• /
• 제17권9호
• /
• pp.1076-1083
• /
• 2014

We propose an emotional facial avatar that portrays the user's facial expressions with an emotional emphasis, while achieving visual and behavioral realism. This is achieved by unifying automatic analysis of facial expressions and animation of realistic 3D faces with details such as facial hair and hairstyles. To augment facial appearance according to the user's emotions, we use emotional templates representing typical emotions in an artistic way, which can be easily combined with the skin texture of the 3D face at runtime. Hence, our interface gives the user vision-based control over facial animation of the emotional avatar, easily changing its moods.

• 인터넷정보학회논문지
• /
• 제6권1호
• /
• pp.85-93
• /
• 2005

본 논문은 다량의 고차원 얼굴 표정 모션 데이터를 2차원 공간에 분포시키고, 애니메이터가 이 공간을 항해하면서 원하는 표정들을 실시간 적으로 선택함으로써 얼굴 표정 애니메이션을 생성하는 방법을 기술한다. 본 논문에서는 약 2400여개의 얼굴 표정 프레임을 이용하여 표정공간을 구성하였다. 표정공간의 생성은 임의의 두 표정간의 최단거리의 결정으로 귀결된다. 표정공간은 다양체 공간으로서 이 공간내의 두 점간의 거리는 다음과 같이 근사적으로 표현한다. 임의의 마커간의 거리를 표시하는 거리행렬을 사용하여 각 표정의 상태를 표현하는 표정상태벡터를 정의한 후, 두 표정이 인접해 있으면, 이를 두 표정 간 최단거리(다양체 거리)에 대한 근사치로 간주한다. 그리하여 인접 표정들 간의 인접거리가 결정되면, 이들 인접거리들을 연결하여 임의의 두 표정 상태간의 최단거리를 구하는데, 이를 위해 Floyd 알고리즘을 이용한다. 다차원 공간인 표정공간을 가시화하기 위해서는 CCA 투영기법을 이용하여 2차원 평면에 투영시켰다 얼굴 애니메이션은 사용자 인터베이스를 사용하여 애니메이터들이 2차원 공간을 항해하면서 실시간으로 생성한다.

• 디자인융복합연구
• /
• 제16권6호
• /
• pp.137-152
• /
• 2017

본 연구는 실버세대를 위한 컴패니언 로봇의 인터랙션 경험 디자인을 위해 사용자 태스크- 로봇 기능 적합도 매핑에 기반한 로봇 유형 분석과 멀티모달 대화 인터랙션에서의 로봇 감정표현 연구를 수행하였다. 노인의 니즈 분석을 위해 노인과 자원 봉사자를 대상으로 FGI, 에스노그래피를 진행하였으며 로봇 지원 기능과 엑추에이터 매칭을 통해 로봇 기능 조합 유형에 대한 분석을 하였다. 도출된 4가지 유형의 로봇 중 표정 기반 대화형 로봇 유형으로 프로토타이핑을 하였으며 에크만의 얼굴 움직임 부호화 시스템(Facial Action Coding System: FACS)을 기반으로 6가지 기본 감정에 대한 표정을 시각화하였다. 사용자 실험에서는 로봇이 전달하는 정보의 정서코드에 맞게 로봇의 표정이 변화할 때와 로봇이 인터랙션 사이클을 자발적으로 시작할 때 사용자의 인지와 정서에 미치는 영향을 이야기 회상 검사(Story Recall Test: STR)와 표정 감정 분석 소프트웨어 Emotion API로 검증하였다. 실험 결과, 정보의 정서코드에 맞는 로봇의 표정 변화 그룹이 회상 검사에서 상대적으로 높은 기억 회상률을 보였다. 한편 피험자의 표정 분석에서는 로봇의 감정 표현과 자발적인 인터랙션 시작이 피험자들에게 정서적으로 긍정적 영향을 주고 선호되는 것을 확인하였다.