Korean Journal of Cognitive Science (인지과학)

The Korean Society for Cognitive Science (한국인지과학회)
권호 표지
DOI QR코드

DOI QR Code

A comparison of emotion detection for faces that are wearing or not wearing a face mask

마스크 착용·미착용 얼굴에 대한 정서탐지 비교

  • YuSeon Son (Department of Psycholty, Pusan National University) ;
  • DongHoon Lee (Department of Psycholty, Pusan National University) ;
  • YoonKyoung Lee (Department of Psycholty, Pusan National University) ;
  • Bia Kim (Department of Psycholty, Pusan National University)
  • Received : 2024.10.10
  • Accepted : 2024.10.28
  • Published : 2024.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

This study investigated in facial emotion detection according to emotional facial expressions(anger, fearful, happy), especially face that are wearing or not wearing a face mask. after presenting the emotional facial expression(anger, fearful, happy) to be detected as a directive, the subjects were asked to detect whether the target stimulus or the non-target stimulus was observed by exposure to emotion facial expressions for 40 ms. The result showed that there were significant differences in sensitivity, response criterion, response times. Specifically, when wearing a face mask, sensitivity of fearful expression and anger expression was more higher compared to happy expression and in the case of response criterion, happy expression was not accurately detected when happy expression wearing face mask was presented because happy expression was different response criterion depending on whether the face mask was worn or not. The subjects responded that when wearing mask, they were longer reaction time for happy expression, compared to anger expression and fearful expression. This seems to have occurred as happy expression mainly recognized facial emotions through the mouth area, but the mouth area is covered by wearing face mask. The results of this study suggest that difficulty of detecting emotional facial emotion according to whether the face mask was worn or not and the possibility of bias in happy expression.

본 연구는 마스크 착용 유무에 따라 정서얼굴표정(분노, 두려움, 행복)의 탐지 양상이 다르게 나타나는지 확인하였다. 이를 위해 탐지해야 할 정서표정(분노, 두려움, 행복)을 지시문으로 제시한 후 40ms 동안 나타나는 얼굴표정을 보고 표적자극인지 비표적자극인지 탐지하게 하였다. 그 결과, 마스크 착용 유무와 정서에 따라 민감도, 반응기준, 반응시간에서 유의한 차이가 나타났다. 구체적으로 마스크 착용 시 행복표정보다 두려움표정과 분노표정의 민감도가 상대적으로 높았으며, 반응기준의 경우 행복 표정이 마스크 착용 유무에 따라 달라져 마스크 착용 시 행복표정이 제시되었을 때 행복표정을 정확히 탐지하지 못하였다. 반응시간의 경우 참가자들은 마스크 착용시 분노나 두려움표정을 제시하였을 때 보다 행복표정이 제시되었을 때 더 느린 반응을 보였다. 주로 행복정서를 인식할 때 입 영역이 주요 단서가 되는데 마스크 착용으로 인해 해당 영역이 가려지게 되면서 나타난 결과로 보인다. 본 연구의 결과는 마스크 착용 유무에 따른 정서탐지의 어려움을 보여주며 특히 행복 표정의 편향 가능성을 시사한다.

Keywords

References

  1. 김민우, 설선혜, 문양호 (2013). 분노표정과 두려움표정의 지각이 위기상황에서의 판단과 대응에 미치는 영향. 한국심리학회지: 인지 및 생물, 25(4), 445-461. https://doi.org/10.22172/COGBIO.2013.25.4.004
  2. 박경옥, 박기환 (2006). 정서강도와 정서인식의 명확성, 정서조절양식이 대인관계에 미치는 영향. 가톨릭대학교대학원 석사학위 논문.
  3. 보건복지부 (2020). 생활 속 거리 두기 기본 지침 개인방역. http://ncov.mohw.go.kr/guidelineView.do?brdId=6&brdGubun=61&dataGubun=&ncvContSeq=2152&contSeq=2152&board_id=&gubun=
  4. 송유미 (2016) 사회불안 경향성 집단의 인지적 해석 편향 : 얼굴표정 정서인식을 중심으로. 대구가톨릭대학교 대학원 석사학위 논문.
  5. 안길성, 양재원 (2019). 자기애 성향과 얼굴표정 정서인식의 민감성과 반응 기준. 인지행동치료, 19(3), 319-339.
  6. 이정현, 조경자, 박수진, 한광희, 김혜리 (2007). 얼굴표정의 제시유형과 제시영역에 따른 정서인식 효과. 감성과학, 10(1), 113-126.
  7. 최해연 (2013). 정서처리유형의 구분에 따른 정서경험과 대처방식 이해. 한국심리학회지: 사회 및 성격, 27(1), 1-15. https://doi.org/10.21193/kjspp.2013.27.1.001
  8. Barrick, E. M., Thornton, M. A., & Tamir, D. I. (2021). Mask exposure during COVID-19 changes emotional face processing. PloS One, 16(10), 1-22. https://doi.org/10.1371/journal.pone.0258470
  9. Baumeister, R. F., & Leary, M. R. (1995). The need to belong: desire for interpersonal attachments as a fundamental human motivation. Psychological Bulletin, 117(3), 497-529. https://doi.org/10.1037/0033-2909.117.3.497
  10. Becker, D. V., Anderson, U. S., Mortensen, C. R., Neufeld, S. L., & Neel, R. (2011). The face in the crowd effect unconfounded: happy faces, not angry faces, are more efficiently detected in single-and multiple-target visual search tasks. Journal of Experimental Psychology: General, 140(4), 637-659. https://doi.org/10.1037/a0024060
  11. Borghi, A. M., & Cimatti, F. (2010). Embodied cognition and beyond: Acting and sensing the body. Neuropsychologia, 48(3), 763-773. https://doi.org/10.1016/j.neuropsychologia.2009.10.029
  12. Cacioppo, J. T., Martzke, J. S., Petty, R. E., & Tassinary, L. G. (1988). Specific forms of facial EMG response index emotions during an interview: From darwin to the continuous flow hypothesis of affect-laden information processing. Journal of Personality and Social Psychology, 54(4), 592-604. https://doi.org/10.1037/0022-3514.54.4.592
  13. Calvo, M. G., & Beltrán, D. (2013). Recognition advantage of happy faces: tracing the neurocognitive processes. Neuropsychologia, 51(11), 2051-2061. https://doi.org/10.1016/j.neuropsychologia.2013.07.010
  14. Calvo, M. G., & Lundqvist, D. (2008). Facial expressions of emotion (KDEF): Identification under different display-duration conditions. Behavior Research Methods, 40(1), 109-115. https://doi.org/10.3758/BRM.40.1.109
  15. Carbon, C. C. (2020). Wearing face masks strongly confuses counterparts in reading emotions. Frontiers in Psychology, 11, 2526-2534. https://doi.org/10.3389/fpsyg.2020.566886
  16. Cartaud, A., Quesque, F., & Coello, Y. (2020). Wearing a face mask against Covid-19 results in a reduction of social distancing. PLoS ONE, 15(12), 1-9. https://doi.org/10.1371/journal.pone.0243023
  17. Chiroro, P., & Valentine, T. (1995). An investigation of the contact hypothesis of the own-race bias in face recognition. The Quarterly Journal of Experimental Psychology Section A, 48(4), 879-894. https://doi.org/10.1080/14640749508401421
  18. Davis, F. C., Somerville, L. H., Ruberry, E. J., Berry, A. B., Shin, L. M., & Whalen, P. J. (2011). A tale of two negatives: differential memory modulation by threat-related facial expressions. Emotion, 11(3), 647-655. https://doi.org/10.1037/a0021625
  19. Doty, T. J., Japee, S., Ingvar, M., & Ungerleider, L. G. (2013). Fearful face detection sensitivity in healthy adults correlates with anxiety-related traits. Emotion, 13(2), 183-188. https://doi.org/10.1037/a0031373
  20. Ekman, P. (1982). Methods for measuring facial action. In K. R. Schere & P. Ekman(Eds.), Handbook of Methods in Nonverbal Behavior Research, Cambridge: Cambridge University Press, 45-90.
  21. Elsherif, M. M., Saban, M. I., & Rotshtein, P. (2017). The perceptual saliency of fearful eyes and smiles: A signal detection study. PLoS ONE, 12(3), 1-30 https://doi.org/10.1371/journal.pone.0173199
  22. Eriksen, C. W., & Yeh, Y. (1985). Allocation of attention in the visual field. Journal of Experimental Psychology: Human Perception and Performance, 11(5), 583-597. https://doi.org/10.1037/0096-1523.11.5.583
  23. Fenske, M. J., & Eastwood, J. D. (2003). Modulation of focused attention by faces expressing emotion: evidence from flanker tasks. Emotion, 3(4), 327-343. https://doi.org/10.1037/1528-3542.3.4.327
  24. Freud, E., Di Giammarino, D., Stajduhar, A., Rosenbaum, R. S., Avidan, G., & Ganel, T. (2022). Recognition of masked faces in the era of the pandemic: No improvement despite extensive natural exposure. Psychological Science, 33(10), 1635-1650. https://doi.org/10.1177/09567976221105459
  25. Gao, X., & Maurer, D. (2009). Influence of intensity on children's sensitivity to happy, sad, and fearful facial expressions. Journal of Experimental Child Psychology, 102(4), 503-521. https://doi.org/10.1016/j.jecp.2008.11.002
  26. Gelder, B., Snyder, J., Greve, D., Gerard, G., & Hadjikhani, N. (2004). Fear fosters flight: a mechanism for fear contagion when perceiving emotion expressed by a whole body. Proceedings of the National Academy of Sciences of the United States of America, 101, 16701-16706. https://doi.org/10.1073/pnas.0407042101
  27. Grahlow, M., Rupp, C. I., & Derntl, B. (2022). The impact of face masks on emotion recognition performance and perception of threat. PLoS Oone, 17(2), e0262840. https://doi.org/10.1371/journal.pone.0262840
  28. Halberstadt, A. G., Denham, S. A., & Dunsmore, J. C. (2001). Affective social competence. Social Development, 10(1), 79-119. https://doi.org/10.1111/1467-9507.00150
  29. Hall, J. K., Hutton, S. B., & Morgan, M. J. (2010). Sex differences in scanning faces: Does attention to the eyes explain female superiority in facial expression recognition? Cognition & Emotion, 24(4), 629-637. https://doi.org/10.1080/02699930902906882
  30. Hung, S., Nieh, C., & Hsieh, P. (2016). Unconscious processing of facial attractiveness: invisible attractive faces orient visual attention. Scientific Reports, 6(1), 1-8. https://doi.org/10.1038/s41598-016-0001-8
  31. Kaitz, M., Bar-Haim, Y., Lehrer, M., & Grossman, E. (2004). Adult attachment style and interpersonal distance. Attachment & Human Development, 6(3), 285-304. https://doi.org/10.1080/14616730412331281520
  32. Kret, M., & De Gelder, B. (2012). Islamic headdress influences how emotion is recognized from the eyes. Frontiers in Psychology, 3, 110-123. https://doi.org/10.3389/fpsyg.2012.00110
  33. Leible, T. L., & Snell Jr, W. E. (2004). Borderline personality disorder and multiple aspects of emotional intelligence. Personality and Individual Differences, 37(2), 393-404. https://doi.org/10.1016/j.paid.2003.09.011
  34. Lerner, J. S., & Keltner, D. (2000). Beyond valence: Toward a model of emotion-specific influences on judgement and choice. Cognition & Emotion, 14(4), 473-493. https://doi.org/10.1080/026999300402763
  35. Lerner, J. S., & Keltner, D. (2001). Fear, anger, and risk. Journal of Personality and Social Psychology, 81(1), 146-159. https://doi.org/10.1037/0022-3514.81.1.146
  36. Liu, L., & Ioannides, A. A. (2010). Emotion Separation Is Completed Early and It Depends on Visual Field Presentation. PLoS ONE, 5(3), 1-14 https://doi.org/10.1371/journal.pone.0009790
  37. MacMillan, N. A., & Creelman. C. D. (2005). Detection theory: A user's guide (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
  38. Marini, M., Ansani, A., Paglieri, F., Caruana, F., & Viola, M. (2021). The impact of facemasks on emotion recognition, trust attribution and re-identification. Scientific Reports, 11(1), 1-14. https://doi.org/10.1038/s41598-020-79139-8
  39. Molnar-Szakacs, I., Uddin, L. Q., & Heffernan, M. B. (2021). The face behind the mask: The future of interpersonal interaction. Neuron, 109(12), 1918-1920. https://doi.org/10.1016/j.neuron.2021.05.030
  40. Narumoto, J., Okada, T., Sadato, N., Fukui, K., & Yonekura, Y. (2001). Attention to emotion modulatesf-MRI activity in human right superior temporal sulcus. Cognitive Brain Research, 12(2), 225-231. https://doi.org/10.1016/S0926-6410(01)00053-2
  41. Neath, K. N., & Itier, R. J. (2014). Facial expression discrimination varies with presentation time but not with fixation on features: A backward masking study using eye-tracking. Cognition & Emotion, 28(1), 115-131. https://doi.org/10.1080/02699931.2013.812557
  42. Perry, A., Rubinsten, O., Peled, L., & Shamay-Tsoory, S. G. (2013). Don't stand so close to me: A behavioral and ERP study of preferred interpersonal distance. NeuroImage, 83, 761-769. https://doi.org/10.1016/j.neuroimage.2013.07.042
  43. Pichon, S., de Gelder, B., & Grèzes, J. (2009). Two different faces of threat. Comparing the neural systems for recognizing fear and anger in dynamic body expressions. Neuroimage, 47(4), 1873-1883. https://doi.org/10.1016/j.neuroimage.2009.03.084
  44. Pinkham, A. E., Griffin, M., Baron, R., Sasson, N. J., & Gur, R. C. (2010). The face in the crowd effect: anger superiority when using real faces and multiple identities. Emotion, 10(1), 141-146. https://doi.org/10.1037/a0017387
  45. Rhodes, G., Hickford, C., & Jeffery, L. (2000). Sex typicalityand attractiveness: Are supermale and superfemale facessuper attractive? British Journal of Psychology, 91(1), 125-140. https://doi.org/10.1348/000712600161718
  46. Ruffman, T., Henry, J. D., Livingstone, V., & Phillips, L. H. (2008). A meta-analytic review of emotion recognition and aging: Implications for neuropsychological models of aging. Neuroscience & Biobehavioral Reviews, 32(4), 863-881. https://doi.org/10.1016/j.neubiorev.2008.01.001
  47. Sato, W., Fujimura, T., Kochiyama, T., & Suzuki, N. (2013). Relationships among facial mimicry, emotional experience, and emotion recognition. PLoS ONE, 8(3), 1-8. https://doi.org/10.1371/journal.pone.0057889
  48. Schurgin, M. W., Nelson, J., Iida, S., Ohira, H., Chiao, J. Y., & Franconeri, S. L. (2014). Eye movements during emotion recognition in faces. Journal of Vision, 14(13), 1-16. https://doi.org/10.1167/14.13.14
  49. Smith, M. L., Cottrell, G. W., Gosselin, F., & Schyns, P. G. (2005). Transmitting and decoding facial expressions. Psychological Science, 16(3), 184-189. https://doi.org/10.1111/j.0956-7976.2005.00801.x
  50. Stanislaw, H. , & Todorov, N. (1999). Calculation of signaldetection theory measures. Behavior Research Methods, Instruments, & Computers, 31(1), 137-149. https://doi.org/10.3758/BF03207704
  51. Vakhrusheva, J., Zemon, V., Bar, M., Weiskopf, N. G., Tremeau, F., Petkova, E., Su Z., Abeles I., Butler, P. D. (2014). Forming first impressions of others in schizophrenia: Impairments in fast processing and in use of spatial frequency information. Schizophrenia Research, 160(1-3), 142-149. https://doi.org/10.1016/j.schres.2014.10.012
  52. Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: the PANAS scales. Journal of Personality and Social Psychology, 54(6), 1063-1070. https://doi.org/10.1037/0022-3514.54.6.1063
  53. Wegrzyn, M., Vogt, M., Kireclioglu, B., Schneider, J., & Kissler, J. (2017). Mapping the emotional face. How individual face parts contribute to successful emotion recognition. PloS One, 12(5), e0177239.
  54. Whalen, P. J., Shin, L. M., McInerney, S. C., Fischer, H., Wright, C. I., & Rauch, S. L. (2001). A functional MRI study of human amygdala responses to facial expressions of fear versus anger. Emotion, 1, 70-83. https://doi.org/10.1037/1528-3542.1.1.70