Browse > Article
http://dx.doi.org/10.5391/JKIIS.2008.18.1.020

Emotion Recognition and Expression System of User using Multi-Modal Sensor Fusion Algorithm  

Yeom, Hong-Gi (중앙대학교 전자전기공학부)
Joo, Jong-Tae (중앙대학교 전자전기공학부)
Sim, Kwee-Bo (중앙대학교 전자전기공학부)
Publication Information
Journal of the Korean Institute of Intelligent Systems / v.18, no.1, 2008 , pp. 20-26 More about this Journal
Abstract
As they have more and more intelligence robots or computers these days, so the interaction between intelligence robot(computer) - human is getting more and more important also the emotion recognition and expression are indispensable for interaction between intelligence robot(computer) - human. In this paper, firstly we extract emotional features at speech signal and facial image. Secondly we apply both BL(Bayesian Learning) and PCA(Principal Component Analysis), lastly we classify five emotions patterns(normal, happy, anger, surprise and sad) also, we experiment with decision fusion and feature fusion to enhance emotion recognition rate. The decision fusion method experiment on emotion recognition that result values of each recognition system apply Fuzzy membership function and the feature fusion method selects superior features through SFS(Sequential Forward Selection) method and superior features are applied to Neural Networks based on MLP(Multi Layer Perceptron) for classifying five emotions patterns. and recognized result apply to 2D facial shape for express emotion.
Keywords
Emotion Recognition and Expression; Speech Signal; Facial Image; Feature Fusion; Decision Fusion;
Citations & Related Records
Times Cited By KSCI : 3  (Citation Analysis)
연도 인용수 순위
1 Carlos Busso and Zhigang Deng et al, "Analysis of Emotion Recognition using Facial Expressions, Speech and Multimodal Information", ICMI 2004, pp. 205-211, 2004
2 Mase K., "Recognition of facial expression from optical flow", IEICE Trans., vol. 74, no. 10, pp 3474-3483, 1991
3 H. Guncs and M. Piccardi, "Bi-modal emotion recognition from expressive face and body gesture", Journal of Network and Computer Application, pp. 1-12, 2006
4 D. Silval and P. C. Nag, "Bimodal emotion recognition", Proc. of Fourth IEEE International Conference on Automatic Face and Gesture Recognition 2000, pp. 332-335, 2000
5 양현승, 서용호, 정일웅, 한태우, 노동현, "서비스 로 봇을 위한 감성 인터페이스 기술," 로봇공학회 논문지, 제1권, 제1호, pp. 58-65, 2006
6 C. H. Park and K. B. Sim, "Pattern Recognition Methods for Emotion Recognition with speech signal", International Journal of Fuzzy Logic and Intelligent Systems", vol. 6, no. 2, pp. 150-154, 2006   과학기술학회마을   DOI   ScienceOn
7 Ho-Duck Kim, Hyun-Chang Yang, Chang-Hyun Park, and Kwee-Bo Sim, "Emotion Recognition Method of Facial Image using PCA ", Journal of Korea Fuzzy Logic and Intelligent Systems Society(KFIS), vol.16, no.6, pp. 772-776, Dec. 2006   과학기술학회마을   DOI   ScienceOn
8 New T.L., Wei F.S. and De Silva L.C., "Speech based emotion classification", TENCON 2001, vol. 1, pp. 297-301, 2001
9 J. Nicholson and K. Takahashi, R. Nakatsu, "Emotion recognition in speech using neural networks" Proc. of ICONIP, Vol. 2, 1996
10 Mingli Song, Jiajun Bu, Chun Chen and Nan Li, "Audio-visual based emotion recognition", CVPR'04, vol. 2, pp. 1020-1025, 2004
11 심귀보, 변광섭, 박창현, "동적 감성 공간에 기반한 감정 표현 시스템," 한국퍼지 및 지능시스템학회 논문지, 제15권, 제1호, pp18-23, 2005   과학기술학회마을   DOI
12 Yoshitomi Y, S. I. Kim, Kawano T and Kilazoe T, "Effect of sensor fusion for recognition of emotional states using voice, face image and thermal image of face", Robot and Human Interactive Communication 2000, pp. 178-184, 2000
13 Lee C.M., Narayanan S.S. and Pieraccini. R., "Classifying emotions in human - machine spoken dialogs", ICME'02, vol. 1, pp. 737-740, 2002
14 Fumio Hara, "Artificial Emotion of Face Robot through Learning in Communicative Interactions with Human", Proceeding of the 2004 IEEE International Workshop on Robot and Human Interactive Communication, pp. 7-15, 2004