• Asia pacific journal of information systems
• /
• v.27 no.1
• /
• pp.38-53
• /
• 2017

As sensor technologies and image processing technologies make collecting information on users' behavior easy, many researchers have examined automatic emotion recognition based on facial expressions, body expressions, and tone of voice, among others. Specifically, many studies have used normal cameras in the multimodal case using facial and body expressions. Thus, previous studies used a limited number of information because normal cameras generally produce only two-dimensional images. In the present research, we propose an artificial neural network-based model using a high-definition webcam and Kinect to recognize users' emotions from facial and bodily expressions when watching a movie trailer. We validate the proposed model in a naturally occurring field environment rather than in an artificially controlled laboratory environment. The result of this research will be helpful in the wide use of emotion recognition models in advertisements, exhibitions, and interactive shows.

• Science of Emotion and Sensibility
• /
• v.14 no.2
• /
• pp.245-256
• /
• 2011

This study consists of two sections to figure out the relationship between autistic features and empathizing-systemizing traits. For the first section, the research involved 355 university students to measure their EQ, SQ-R and AQ. As a result, it is found that AQ was negatively correlated to EQ, and D score(relative difference between EQ and SQ-R of the individuals), but it was not significantly related to SQ-R. It means that the subject has high AQ if he has relatively lower EQ than SQ-R. For the second section, the subjects were divided into two groups based on their AQ score; the subjects who had a tendency of autism and the subjects who did not. The test measured how these two groups were different in terms of facial expressions' recognition according to the tendency of autism, facial expression presenting areas(whole face, eyes-alone, mouth-alone) and different types of emotions(basic and complex emotions). As a result, the subjects who had a tendency of autism were lower at judging facial expressions than the subjects who did not. Also, the results showed that the subjects judged better on the condition of basic emotions more than complex emotions, the whole face more than eyes-alone and mouth-alone. Especially, for the eyes-alone condition, the subjects who had a tendency of autism were lower at judging facial expressions than the subjects who did not. This study suggests that empathizing traits and facial expressions' recognition are related to the tendency of autism.

• Journal of Software Assessment and Valuation
• /
• v.17 no.2
• /
• pp.1-10
• /
• 2021

The ability to recognize human emotions by computer vision is a very important task, with many potential applications. Therefore the demand for emotion recognition using not only RGB images but also thermal images is increasing. Compared to RGB images, thermal images has the advantage of being less affected by lighting conditions but require a more sophisticated recognition method with low-resolution sources. In this paper, we propose a Divide and Conquer-based CNN training strategy to improve the performance of facial thermal image-based emotion recognition. The proposed method first trains to classify difficult-to-classify similar emotion classes into the same class group by confusion matrix analysis and then divides and solves the problem so that the emotion group classified into the same class group is recognized again as actual emotions. In experiments, the proposed method has improved accuracy in all the tests than when recognizing all the presented emotions with a single CNN model.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.16 no.3
• /
• pp.1076-1094
• /
• 2022

Technology for emotion recognition is an essential part of human personality analysis. To define human personality characteristics, the existing method used the survey method. However, there are many cases where communication cannot make without considering emotions. Hence, emotional recognition technology is an essential element for communication but has also been adopted in many other fields. A person's emotions are revealed in various ways, typically including facial, speech, and biometric responses. Therefore, various methods can recognize emotions, e.g., images, voice signals, and physiological signals. Physiological signals are measured with biological sensors and analyzed to identify emotions. This study employed two sensor types. First, the existing method, the binary arousal-valence method, was subdivided into four levels to classify emotions in more detail. Then, based on the current techniques classified as High/Low, the model was further subdivided into multi-levels. Finally, signal characteristics were extracted using a 1-D Convolution Neural Network (CNN) and classified sixteen feelings. Although CNN was used to learn images in 2D, sensor data in 1D was used as the input in this paper. Finally, the proposed emotional recognition system was evaluated by measuring actual sensors.

• The Journal of the Korea Contents Association
• /
• v.5 no.2
• /
• pp.147-155
• /
• 2005

In this paper, we present a FAES(a Facial Animation with Emotion and Speech) system for speech-driven face animation with emotions. We animate face cartoons not only from input speech, but also based on emotions derived from speech signal. And also our system can ensure smooth transitions and exact representation in animation. To do this, after collecting the training data, we have made the database using SVM(Support Vector Machine) to recognize four different categories of emotions: neutral, dislike, fear and surprise. So that, we can make the system for speech-driven animation with emotions. Also, we trained on Korean young person and focused on only Korean emotional face expressions. Experimental results of our system demonstrate that more emotional areas expanded and the accuracies of the emotional recognition and the continuous speech recognition are respectively increased 7% and 5% more compared with the previous method.

• The Journal of the Institute of Internet, Broadcasting and Communication
• /
• v.22 no.5
• /
• pp.23-28
• /
• 2022

Today, the online market has grown rapidly due to the development of digital platforms and the pandemic situation. Therefore, unlike the existing offline market, the distinctiveness of the online market has prompted users to check online reviews. It has been established that reviews play a significant part in influencing the user's purchase intention through precedents of several studies. However, the current review writing method makes it difficult for other users to understand the writer's emotions by expressing them through elements like tone and words. If the writer also wanted to emphasize something, it was very cumbersome to thicken the parts or change the colors to reflect their emotions. Therefore, in this paper, we propose a technique to check the user's emotions through facial expression recognition using a camera, to automatically set colors for each emotion using research on existing emotions and colors, and give colors based on the user's intention.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.10 no.3
• /
• pp.1390-1403
• /
• 2016

Facial expression recognition (FER) plays a very significant role in computer vision, pattern recognition, and image processing applications such as human computer interaction as it provides sufficient information about emotions of people. For video-based facial expression recognition, depth cameras can be better candidates over RGB cameras as a person's face cannot be easily recognized from distance-based depth videos hence depth cameras also resolve some privacy issues that can arise using RGB faces. A good FER system is very much reliant on the extraction of robust features as well as recognition engine. In this work, an efficient novel approach is proposed to recognize some facial expressions from time-sequential depth videos. First of all, efficient Local Binary Pattern (LBP) features are obtained from the time-sequential depth faces that are further classified by Generalized Discriminant Analysis (GDA) to make the features more robust and finally, the LBP-GDA features are fed into Hidden Markov Models (HMMs) to train and recognize different facial expressions successfully. The depth information-based proposed facial expression recognition approach is compared to the conventional approaches such as Principal Component Analysis (PCA), Independent Component Analysis (ICA), and Linear Discriminant Analysis (LDA) where the proposed one outperforms others by obtaining better recognition rates.

• Korean Journal of Cognitive Science
• /
• v.21 no.1
• /
• pp.127-143
• /
• 2010

This study explores children's sex differences of emotion discrimination from facial expressions based on two dimensional model of emotion. The study group consisted of 92 children, of 40, 52, and 64 months of age, and the rate of male and female children was male children (50%) and female children (50%). Children of 92 were required to choose facial expressions related the twelve emotion terms. Facial expressions applied for experiment are used the photographs rated the degree of expression in each of the two dimensions (pleasure-displeasure dimension and arousal-sleep dimension) on a nine-point scale from 54 university students. The experimental findings appeared that the sex differences were distinctly the arousal-sleep dimension than the pleasure-displeasure dimension. In the arousal-sleep dimensionoussleepness, anger, comfort, and loneliness' emotions showed large sex differences over 1 value. Especially, while male children showed high arousal more than female children in the emotions like 'sleepiness, anger and loneliness', female children showed high arousal more than male children in 'comfort' emotion.

• Korean Journal of Cognitive Science
• /
• v.14 no.4
• /
• pp.33-42
• /
• 2003

This paper presents a new system for facial expression recognition based in dimension model of internal states. The information of facial expression are extracted to the three steps. In the first step, Gabor wavelet representation extracts the edges of face components. In the second step, sparse features of facial expressions are extracted using fuzzy C-means(FCM) clustering algorithm on neutral faces, and in the third step, are extracted using the Dynamic Model(DM) on the expression images. Finally, we show the recognition of facial expression based on the dimension model of internal states using a multi-layer perceptron. The two dimensional structure of emotion shows that it is possible to recognize not only facial expressions related to basic emotions but also expressions of various emotion.

• ETRI Journal
• /
• v.37 no.6
• /
• pp.1199-1210
• /
• 2015

Automatic emotion recognition based on facial cues, such as facial action units (AUs), has received huge attention in the last decade due to its wide variety of applications. Current computer-based automated two-phase facial emotion recognition procedures first detect AUs from input images and then infer target emotions from the detected AUs. However, more robust AU detection and AU-to-emotion mapping methods are required to deal with the error accumulation problem inherent in the multiphase scheme. Motivated by our key observation that a single AU detector does not perform equally well for all AUs, we propose a novel two-phase facial emotion recognition framework, where the presence of AUs is detected by group decisions of multiple AU detectors and a target emotion is inferred from the combined AU detection decisions. Our emotion recognition framework consists of three major components - multiple AU detection, AU detection fusion, and AU-to-emotion mapping. The experimental results on two real-world face databases demonstrate an improved performance over the previous two-phase method using a single AU detector in terms of both AU detection accuracy and correct emotion recognition rate.