전기전자학회논문지 (Journal of IKEEE)

  • 제22권4호
  • /
  • Pages.1140-1146
  • /
  • 2018
  • /
  • 1226-7244(pISSN)
  • /
  • 2288-243X(eISSN)
한국전기전자학회 (Institute of Korean Electrical and Electronics Engineers)
권호 표지
DOI QR코드

DOI QR Code

멀티모달 감정인식률 향상을 위한 웨이블릿 기반의 통계적 잡음 검출 및 감정분류 방법 연구

Wavelet-based Statistical Noise Detection and Emotion Classification Method for Improving Multimodal Emotion Recognition

  • Yoon, Jun-Han (Dept. of Computer Engineering, Seokyeong University) ;
  • Kim, Jin-Heon (Dept. of Computer Engineering, Seokyeong University)
  • 투고 : 2018.12.08
  • 심사 : 2018.12.20
  • 발행 : 2018.12.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

최근 인간의 감정을 인식하는 연구 중 딥러닝 모델을 사용하여 복합적인 생체 신호를 분석하는 방법론이 대두되고 있다. 이때 학습하고자 하는 데이터의 종류에 따른 평가 방법 및 신뢰성은 감정 분류의 정확성에 있어 중요한 요소이다. 생체 신호의 경우 데이터의 신뢰성이 잡음 비율에 따라 결정되므로 잡음 검출 방법이 우수할수록 신뢰도가 올라가며, 감정을 정의하는 방법론에 따라 그에 맞는 적절한 감정 평가 방법이 수반될 때보다 정확하게 감정을 분류할 수 있다. 본 논문에서는 Valence와 Arousal로 라벨링 된 멀티모달 생체 신호 데이터에 대해 데이터의 신뢰성을 검증하기 위한 웨이블릿 기반의 잡음 임곗값 설정 알고리듬 및 감정 평가 시 데이터 신뢰도와 Valence-Arousal 값에 따른 가중치를 부여하여 감정 인식률을 향상하는 방법을 제안한다. 웨이블릿 변환을 이용해 신호의 웨이블릿 성분을 추출 후, 해당 성분의 왜도와 첨도를 구하여 햄펄 식별자를 통해 계산된 임곗값으로 잡음을 검출한 후, 원신호에 대한 잡음 비율을 고려하여 데이터의 신뢰성을 평가하고 가중치로 환산한다. 더불어 감정 데이터 분류 시 Valence-Arousal 평면의 중앙값과의 유클리디언 거리를 가중치로 환산하고, 감정 인식률에 대한 종합 평가 시 두 요소를 반영한다. ASCERTAIN 데이터셋을 활용하여 나타난 감정 인식률 개선 정도를 통해 제안된 알고리듬의 성능을 검증한다.

Recently, a methodology for analyzing complex bio-signals using a deep learning model has emerged among studies that recognize human emotions. At this time, the accuracy of emotion classification may be changed depending on the evaluation method and reliability depending on the kind of data to be learned. In the case of biological signals, the reliability of data is determined according to the noise ratio, so that the noise detection method is as important as that. Also, according to the methodology for defining emotions, appropriate emotional evaluation methods will be needed. In this paper, we propose a wavelet -based noise threshold setting algorithm for verifying the reliability of data for multimodal bio-signal data labeled Valence and Arousal and a method for improving the emotion recognition rate by weighting the evaluation data. After extracting the wavelet component of the signal using the wavelet transform, the distortion and kurtosis of the component are obtained, the noise is detected at the threshold calculated by the hampel identifier, and the training data is selected considering the noise ratio of the original signal. In addition, weighting is applied to the overall evaluation of the emotion recognition rate using the euclidean distance from the median value of the Valence-Arousal plane when classifying emotional data. To verify the proposed algorithm, we use ASCERTAIN data set to observe the degree of emotion recognition rate improvement.

키워드

JGGJB@_2018_v22n4_1140_f0001.png 이미지

Fig. 1. Valence-Arousal plane. 그림 1. Valence-Arousal 면

JGGJB@_2018_v22n4_1140_f0002.png 이미지

Fig. 2. Deep learning Model Block diagram. 그림 2. 딥러닝 모델 블록 다이어그램

Table 1. Comparison of emotion recognition rate according to data reliability evaluation technique. 표 1. 데이터 신뢰성 평가 기법에 따른 감정인식률 비교

JGGJB@_2018_v22n4_1140_t0001.png 이미지

Table 2. Valence-Arousal Comparison of Emotion Recognition Rate by Weights Applying. 표 2. Valence-Arousal 가중치 적용 여부에 따른 감정식률 비교

JGGJB@_2018_v22n4_1140_t0002.png 이미지

Table 3. Comparison of Emotion Recognition Rate with and without All of the Proposed Methods. 표 3. 제안한 방법을 모두 적용한 경우와 그렇지 않은 경우의 감정인식률 비교

JGGJB@_2018_v22n4_1140_t0003.png 이미지

참고문헌

  1. M. Soleymani, J. Lichtenauer, T. Pun, M. Pantic, "A multi-modal database for affect recognition and implicit tagging," IEEE Trans. Affect. Comput. Vol.3 pp.42-55, 2012. DOI:10.1109/T-AFFC.2011.25
  2. G. Fanelli, J. Gall, H. Romsdorfer, T. Weise, L. Van Gool, "A 3-D audio-visual cor-pus of affective communication," IEEE Trans. on Multimedia Vol. 12 pp.591-598. 2010. DOI:10.1109/TMM.2010.2052239
  3. Z. Yin, J. Zhang, "Operator functional state classification using least-square sup-port vector machine based recursive feature elimination technique," Comput. Methods Prog. Biomed. Vol. 113, pp.101-115. 2014. DOI:10.1016/j.cmpb.2013.09.007
  4. J. Kim, E. Andre, "Emotion-specific dichotomous classification and feature-level fusion of multichannel biosignals for automatic emotion recognition," in: Pro-ceedings of IEEE Inter-national Conference on Multisensor Fusion and Integration for Intelligent Systems 2008. DOI:10.1109/MFI.2008.4648119
  5. H. Lee, A. Shackman, D. Jackson, R. Davidson, "Test-retest reliability of voluntary emotion regulation," Psychophysiology Vol.46 pp.874-879. 2009. DOI:10.1111/j.1469-8986.2009.00830.x
  6. O. AlZoubi, S. K. D'Mello, R. A. Calvo, "Detecting naturalistic expressions of non-basic affect using physiological signals," IEEE Trans. Affect. Comput. Vol.3 pp.298-310. 2012. DOI:10.1109/T-AFFC.2012.4
  7. L. Brown, B. Grundlehner, J. Penders, "Towards wireless emotional valence de-tection from EEG," in: Proceedings of Engineering in Medicine and Biology Society, EMBC, 2011. DOI:10.1109/IEMBS.2011.6090412
  8. R. J. Davidson, "Affective neuroscience and psychophysiology: toward a synthesis," Psychophysiology Vol.40, pp.655-665, 2003. DOI:10.1111/1469-8986.00067
  9. Z. Yin, J. Zhang, "Identification of temporal variations in mental workload using locally-linear-embedding-based EEG feature reduction and support-vec tor-machine-based clustering and classification technique," Comput. Methods Prog. Biomed. Vol.115, pp.119-134. 2014. DOI:10.1016/j.cmpb.2014.04.011
  10. G. K. Verma, U. S. Tiwary, "Multimodal fusion framework: a multiresolution ap-proach for emotion classification and recognition from physiological signal," NeuroImage Vol.102 pp.162-172. 2014. DOI:10.1016/j.neuroimage.2013.11.007
  11. M. Khezri, M. Firoozabadi, A. R. Sharafat, "Reliable emotion recognition system based on dynamic adaptive fusion of forehead biopotentials and physiological signals," Comput. Methods Prog. Biomed. Vol.122 pp.149-164. 2015. DOI:10.1016/j.cmpb.2015.07.006
  12. C. Li, C. Xu, Z. Feng, "Analysis of physiological for emotion recognition with the IRS model," Neurocomputing Vol.178 pp.103-111, 2016. DOI:10.1016/j.neucom.2015.07.112
  13. Senthil, R. Arumuganathan, K. Sivakumar, and C. Vimal, "Removal of ocular artifacts in the EEG through wavelet transform without using an EOG reference channel," Int.J. Open Problems Compt. Math., Vol.1, No.3, 2008. DOI:10.1.1.502.6932
  14. Mrs V Baby Deepa, Dr P Thangaraj, Dr S Chitra, "Investigating the performance improvement by sampling techniques in EEG data," International Journal on Computer Science and Engineering, Vol.2, pp.2025-2028, 2010.
  15. J. Hernandez et al. "Call center stress recognition with person-specific models," Proceedings of the Affective Computing and Intelligent Interaction, Memphis, 2011. DOI:10.1007/978-3-642-24600-5_16
  16. M. Poh et al. "A Wearable Sensor for Unobtrusive, Long-Term Assessment of Electrodermal Activity," IEEE Trans. Biomed. Eng, 2010. DOI:10.1109/TBME.2009.2038487
  17. M. Poh et al. "Continuous monitoring of electrodermal activity during epileptic seizures using a wearable sensor," Annual Int. Conf. of the IEEE Engineering in Medicine and Biology Society (EMBC), 2010. DOI:10.1109/IEMBS.2010.5625988
  18. R. R. Coifman and D. L. "Donoho. in Wavelets and statistics," Vol.103, pp.281-299, 1995.
  19. D. P. Subha, P. K. Joseph, R. Acharya, C. M. Lim, "EEG Signal Analysis: A Survey," Journal of Medical Systems, Vol.34, No.2, pp 195-212, 2010. DOI:10.1007/s10916-008-9231-z
  20. Chmelka and J. Kozumplik, "Wavelet-based Wiener Filter for electrocardiogram signal denoising," Comput. Cardiol, vol.32, pp.771-774, 2005. DOI:10.1109/CIC.2005.1588218
  21. Maarten Jansen, "wavelet Thresholding and Noise Reductionm," Katholieke Universiteit Leuven-Faculteit Toegepaste Wetenschappen Arenbergkasteel, 2000.
  22. R. Subramanian et al., "ASCERTAIN: Emotion and Personality Recognition using Commercial Sensors," IEEE Transactions on Affective Computing, pp.1-14, 2016.
  23. P. Baldi, "Boolean AutoEncoder," pp.37-50, 2012.
  24. R. Memisevic and C. Zach, "Gated softmax classification," Advances in Neural, pp.1-9, 2010.