Journal of the Korea Computer Graphics Society (한국컴퓨터그래픽스학회논문지)

Korea Computer Graphics Society (한국컴퓨터그래픽스학회)
권호 표지
DOI QR코드

DOI QR Code

Emotion Training: Image Color Transfer with Facial Expression and Emotion Recognition

감정 트레이닝: 얼굴 표정과 감정 인식 분석을 이용한 이미지 색상 변환

  • Received : 2018.03.04
  • Accepted : 2018.08.15
  • Published : 2018.09.01
Download PDF
⟨ Previous Next ⟩

Abstract

We propose an emotional training framework that can determine the initial symptom of schizophrenia by using emotional analysis method through facial expression change. We use Emotion API in Microsoft to obtain facial expressions and emotion values at the present time. We analyzed these values and recognized subtle facial expressions that change with time. The emotion states were classified according to the peak analysis-based variance method in order to measure the emotions appearing in facial expressions according to time. The proposed method analyzes the lack of emotional recognition and expressive ability by using characteristics that are different from the emotional state changes classified according to the six basic emotions proposed by Ekman. As a result, the analyzed values are integrated into the image color transfer framework so that users can easily recognize and train their own emotional changes.

본 논문은 얼굴의 표정 변화를 통해 감정을 분석하는 방법으로 조현병의 초기 증상을 스스로 인지할 수 있는 감정 트레이닝 프레임워크를 제안한다. 먼저, Microsoft의 Emotion API를 이용하여 캡처된 얼굴 표정의 사진으로부터 감정값을 얻고, 피크 분석 기반 표준편차로 시간에 따라 변화하는 얼굴 표정의 미묘한 차이를 인식해 감정 상태를 각각 분류한다. 그리하여 Ekman이 제안한 여섯 가지 기본 감정 상태에 반하는 감정들의 정서 및 표현능력이 결핍된 부분에 대해 분석하고, 그 값을 이미지 색상 변환 프레임워크에 통합시켜 사용자 스스로 감정의 변화를 쉽게 인지하고 트레이닝 할 수 있도록 하는 것이 최종목적이다.

Keywords

References

  1. E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, "Color transfer between images," IEEE Computer graphics and applications, vol. 21, no. 5, pp. 34-41, 2001. https://doi.org/10.1109/38.946629
  2. G. R. Greenfield and D. H. House, "Image recoloring induced by palette color associations," 2003.
  3. L. Neumann and A. Neumann, "Color style transfer techniques using hue, lightness and saturation histogram matching," in Computational Aesthetics. Citeseer, 2005, pp. 111-122.
  4. Y.-W. Tai, J. Jia, and C.-K. Tang, "Local color transfer via probabilistic segmentation by expectation-maximization," in Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on, vol. 1. IEEE, 2005, pp. 747-754.
  5. N. Bonneel, K. Sunkavalli, S. Paris, and H. Pfister, "Examplebased video color grading," ACM Transactions on Graphics, vol. 32, no. 4, pp. 39-1, 2013.
  6. H. Hristova, O. Le Meur, R. Cozot, and K. Bouatouch, "Styleaware robust color transfer," in Proceedings of the workshop on Computational Aesthetics. Eurographics Association, 2015, pp. 67-77.
  7. R. Plutchik, "Emotions: A general psychoevolutionary theory," Approaches to emotion, vol. 1984, pp. 197-219, 1984.
  8. S. S. Tomkins, "The role of facial response in the experience of emotion: A reply to tourangeau and ellsworth." 1981.
  9. P. Ekman, "Facial action coding system (facs)," A human face, 2002.
  10. P. Ekman, "An argument for basic emotions," Cognition & emotion, vol. 6, no. 3-4, pp. 169-200, 1992. https://doi.org/10.1080/02699939208411068
  11. C. C. Bennett and S. Sabanovic, "Deriving minimal features for human-like facial expressions in robotic faces," International Journal of Social Robotics, vol. 6, no. 3, pp. 367-381, 2014. https://doi.org/10.1007/s12369-014-0237-z
  12. R. C. Gur, R. J. Erwin, R. E. Gur, A. S. Zwil, C. Heimberg, and H. C. Kraemer, "Facial emotion discrimination: Ii. behavioral findings in depression," Psychiatry research, vol. 42, no. 3, pp. 241-251, 1992. https://doi.org/10.1016/0165-1781(92)90116-K
  13. R. L. Morrison, A. S. Bellack, and K. T. Mueser, "Deficits in facial-affect recognition and schizophrenia." Schizophrenia Bulletin, vol. 14, no. 1, p. 67, 1988. https://doi.org/10.1093/schbul/14.1.67
  14. J. A. Russell, "Is there universal recognition of emotion from facial expression? a review of the cross-cultural studies." Psychological bulletin, vol. 115, no. 1, p. 102, 1994. https://doi.org/10.1037/0033-2909.115.1.102
  15. K. Kulkarni, C. A. Corneanu, I. Ofodile, S. Escalera, X. Baro, S. Hyniewska, J. Allik, and G. Anbarjafari, "Automatic recognition of facial displays of unfelt emotions," Journal of IEEE Transactions on Affective Computing, 2017.
  16. M. A. Aghevli, J. J. Blanchard, and W. P. Horan, "The expression and experience of emotion in schizophrenia: a study of social interactions," Psychiatry research, vol. 119, no. 3, pp. 261-270, 2003. https://doi.org/10.1016/S0165-1781(03)00133-1
  17. R. D. Yee, R. W. Baloh, S. Marder, D. Levy, S. Sakala, and V. Honrubia, "Eye movements in schizophrenia." Investigative ophthalmology & visual science, vol. 28, no. 2, pp. 366-374, 1987.
  18. J. R. Carter, "Facial expression analysis in schizophrenia," Unpublished doctoral dissertation, University of Western Ontario, 2000.
  19. L. Scutt, E. Chow, R.Weksberg,W. Honer, and A. S. Bassett, "Patterns of dysmorphic features in schizophrenia," American journal of medical genetics, vol. 105, no. 8, pp. 713-723, 2001. https://doi.org/10.1002/ajmg.1612
  20. C.-K. Yang and L.-K. Peng, "Automatic mood-transferring between color images," IEEE computer graphics and applications, vol. 28, no. 2, 2008.
  21. R. M. Haralick, K. Shanmugam et al., "Textural features for image classification," IEEE Transactions on Systems, Man, and Cybernetics, no. 6, pp. 610-621, 1973.
  22. B. Arbelot, R. Vergne, T. Hurtut, and J. Thollot, "Automatic texture guided color transfer and colorization," in Proceedings of the Joint Symposium on Computational Aesthetics and Sketch Based Interfaces and Modeling and Non- Photorealistic Animation and Rendering. Eurographics Association, 2016, pp. 21-32.
  23. H. S. Faridul, T. Pouli, C. Chamaret, J. Stauder, E. Reinhard, D. Kuzovkin, and A. Tremeau, "Colour mapping: A review of recent methods, extensions and applications," in Computer Graphics Forum, vol. 35, no. 1. Wiley Online Library, 2016, pp. 59-88. https://doi.org/10.1111/cgf.12671
  24. O. Frigo, N. Sabater, V. Demoulin, and P. Hellier, "Optimal transportation for example-guided color transfer," in Asian Conference on Computer Vision. Springer, 2014, pp. 655-670.