• Journal of Korea Multimedia Society
• /
• v.24 no.7
• /
• pp.880-889
• /
• 2021

In everyday life, recognizing people's emotions from their frames is essential and is a popular research domain in the area of computer vision. Visual emotion has a severe class imbalance in which most of the data are distributed in specific categories. The existing methods do not consider class imbalance and used accuracy as the performance metric, which is not suitable for evaluating the performance of the imbalanced dataset. Therefore, we proposed a method for recognizing visual emotion using balanced data augmentation to address the class imbalance. The proposed method generates a balanced dataset by adopting the random over-sampling and image transformation methods. Also, the proposed method uses the Focal loss as a loss function, which can mitigate the class imbalance by down weighting the well-classified samples. EfficientNet, which is the state-of-the-art method for image classification is used to recognize visual emotion. We compare the performance of the proposed method with that of conventional methods by using a public dataset. The experimental results show that the proposed method increases the F1 score by 40% compared with the method without data augmentation, mitigating class imbalance without loss of classification accuracy.

• Journal of KIISE:Software and Applications
• /
• v.34 no.9
• /
• pp.869-879
• /
• 2007

This paper proposes a neural network based approach for automatic human emotion recognition in textile images. To investigate the correlation between the emotion and the pattern, the survey is conducted on 20 peoples, which shows that a emotion is deeply affected by a pattern. Accordingly, a neural network based classifier is used for recognizing the pattern included in textiles. In our system, two schemes are used for describing the pattern; raw-pixel data extraction scheme using auto-regressive method (RDES) and wavelet transformed data extraction scheme (WTDES). To assess the validity of the proposed method, it was applied to recognize the human emotions in 100 textiles, and the results shows that using WTDES guarantees better performance than using RDES. The former produced the accuracy of 71%, while the latter produced the accuracy of 90%. Although there are some differences according to the data extraction scheme, the proposed method shows the accuracy of 80% on average. This result confirmed that our system has the potential to be applied for various application such as textile industry and e-business.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.14 no.3
• /
• pp.1188-1203
• /
• 2020

The human expression contains a lot of information that can be used to detect complex conditions such as pain and fatigue. After deep learning became the mainstream method, the traditional feature extraction method no longer has advantages. However, in order to achieve higher accuracy, researchers continue to stack the number of layers of the neural network, which makes the real-time performance of the model weak. Therefore, this paper proposed an expression recognition framework based on densely concatenated convolutional neural networks to balance accuracy and latency and apply it to humanoid robots. The techniques of feature reuse and parameter compression in the framework improved the learning ability of the model and greatly reduced the parameters. Experiments showed that the proposed model can reduce tens of times the parameters at the expense of little accuracy.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.15 no.4
• /
• pp.717-724
• /
• 2020

Due to the important role played by emotion in human interaction, affective computing is dedicated in trying to understand and regulate emotion through human-aware artificial intelligence. By understanding, emotion mental diseases such as depression, autism, attention deficit hyperactivity disorder, and game addiction will be better managed as they are all associated with emotion. Various studies for emotion recognition have been conducted to solve these problems. In applying machine learning for the emotion recognition, the efforts to reduce the complexity of the algorithm and improve the accuracy are required. In this paper, we investigate emotion Electroencephalogram (EEG) feature reduction and classification using Stack AutoEncoder (SAE) and Long-Short-Term-Memory/Recurrent Neural Networks (LSTM/RNN) classification respectively. The proposed method reduced the complexity of the model and significantly enhance the performance of the classifiers.

• The Transactions of the Korea Information Processing Society
• /
• v.13 no.6
• /
• pp.284-290
• /
• 2024

Speech emotion recognition (SER) is a technique that is used to analyze the speaker's voice patterns, including vibration, intensity, and tone, to determine their emotional state. There has been an increase in interest in artificial intelligence (AI) techniques, which are now widely used in medicine, education, industry, and the military. Nevertheless, existing researchers have attained impressive results by utilizing acted-out speech from skilled actors in a controlled environment for various scenarios. In particular, there is a mismatch between acted and spontaneous speech since acted speech includes more explicit emotional expressions than spontaneous speech. For this reason, spontaneous speech-emotion recognition remains a challenging task. This paper aims to conduct emotion recognition and improve performance using spontaneous speech data. To this end, we implement deep learning-based speech emotion recognition using the VGG (Visual Geometry Group) after converting 1-dimensional audio signals into a 2-dimensional spectrogram image. The experimental evaluations are performed on the Korean spontaneous emotional speech database from AI-Hub, consisting of 7 emotions, i.e., joy, love, anger, fear, sadness, surprise, and neutral. As a result, we achieved an average accuracy of 83.5% and 73.0% for adults and young people using a time-frequency 2-dimension spectrogram, respectively. In conclusion, our findings demonstrated that the suggested framework outperformed current state-of-the-art techniques for spontaneous speech and showed a promising performance despite the difficulty in quantifying spontaneous speech emotional expression.

• Journal of the Institute of Electronics and Information Engineers
• /
• v.54 no.7
• /
• pp.71-78
• /
• 2017

There are several algorithms to classify emotion, such as Support-vector-machine (SVM), Bayesian decision rule, etc. However, many researchers have insisted that these methods have minor problems. Therefore, in this paper, we propose a novel method for emotion recognition based on Electroencephalogram (EEG) signal using the Gradient method which was proposed by Han. We also utilize a database for emotion analysis using physiological signals (DEAP) to obtain objective data. And we acquire four channel brainwaves, including Fz (${\alpha}$), Fp2 (${\beta}$), F3 (${\alpha}$), F4 (${\alpha}$) which are selected in previous study. We use 4 features which are power spectral density (PSD) of the above channels. According to performance evaluation (4-fold cross validation), we could get 85% accuracy in valence axis and 87.5% in arousal. It is 5-7% higher than existing method's.

• The Journal of the Convergence on Culture Technology
• /
• v.10 no.1
• /
• pp.617-623
• /
• 2024

Identifying emotions through speech poses a significant challenge due to the complex relationship between language and emotions. Our paper aims to take on this challenge by employing feature engineering to identify emotions in speech through a multimodal classification task involving both speech and text data. We evaluated two classifiers-Convolutional Neural Networks (CNN) and Long Short-Term Memory (LSTM)-both integrated with a BERT-based pre-trained model. Our assessment covers various performance metrics (accuracy, F-score, precision, and recall) across different experimental setups). The findings highlight the impressive proficiency of two models in accurately discerning emotions from both text and speech data.

• Journal of the Korea Society of Computer and Information
• /
• v.29 no.5
• /
• pp.177-187
• /
• 2024

177-Existing music recommendation systems do not sufficiently consider the discrepancy between the intended emotions conveyed by song lyrics and the actual emotions felt by users. In this study, we generate topic vectors for lyrics and user comments using the LDA model, and construct a user preference model by combining user behavior trajectories reflecting time decay effects and playback frequency, along with statistical characteristics. Empirical analysis shows that our proposed model recommends music with higher accuracy compared to existing models that rely solely on lyrics. This research presents a novel methodology for improving personalized music recommendation systems by integrating emotion recognition and user behavior analysis.

• Science of Emotion and Sensibility
• /
• v.15 no.2
• /
• pp.307-316
• /
• 2012

Two experiments were conducted to test effects of ethnicity and gender of face stimuli and lineup procedure on face recognition. In experiment 1, all stimuli were frontal male pictures of Southeast Asian and Northeast Asian. It was tested that whether accuracy of face recognition improve when ethnicity of participant and stimulus was identical (own-ethnicity advantage). In experiment 2, the 'own-gender advantage' was tested using images of Northeast Asian's frontal male and female faces. All participants were Northeast Asian. The results showed that the own-ethnicity advantage or the effect of line-up procedure was not found, but accuracy of face recognition was high when target face was Southeast Asian. Interestingly, the own-gender advantages was observed in only case of female participants. It was discussed that face recognition could be affected by attention.

• Journal of the Korea Society of Computer and Information
• /
• v.28 no.3
• /
• pp.35-43
• /
• 2023

Emotion recognition while driving is an essential task to prevent accidents. Furthermore, in the era of autonomous driving, automobiles are the subject of mobility, requiring more emotional communication with drivers, and the emotion recognition market is gradually spreading. Accordingly, in this research plan, the driver's emotions are classified into seven categories using psychological and behavioral data, which are relatively easy to collect. The latent vectors extracted through the auto-encoder model were also used as features in this classification model, confirming that this affected performance improvement. Furthermore, it also confirmed that the performance was improved when using the framework presented in this paper compared to when the existing EEG data were included. Finally, 81% of the driver's emotion classification accuracy and 80% of F1-Score were achieved only through psychological, personal information, and behavioral data.