Science of Emotion and Sensibility (감성과학)

Korean Society for Emotion and Sensibility (한국감성과학회)
권호 표지
DOI QR코드

DOI QR Code

Testing Modality-Generality and Valence Models using Representational Similarity Analysis

표상 유사성 분석을 이용한 감각양상에 따른 정서표상 모델과 정서가 모델의 검증

  • Received : 2022.08.29
  • Accepted : 2022.12.01
  • Published : 2023.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Among the discussions on affective representation, the first is to explain the affective representation in the dimensions, and the second is to explain the affective representation according to the modality. In previous studies, to explain affective representation, valence models (signed valence, unsigned valence) and Modality-generality models (modality-general, modality-specific) were presented. In this study, we compared models presented in the previous study using the recently published ASMR to confirm which models explain affective representation well. The data used in this study were behavioral rating values collected by Kim & Kim (2022), and these were obtained for ASMR stimuli that were divided into three affective types (negative, neutral, and positive) and two modalities (auditory and audiovisual). Then, a multidimensional scaling, a representational similarity analysis with a two-way repeated measures ANOVA, and a multiple regression analysis with a two-way repeated measures ANOVA were performed. The results revealed that signed valence and modality-general distinguished between affective types of stimuli better than unsigned valence and modality-specific. Similar to the results of multidimensional scaling, the results of a representational similarity analysis and a multiple regression also showed that the signed valence and modality-general significantly explained affective representation better than the unsigned valence and the modality-specific. These results suggest that the model in which positive and negative are located at the opposite ends of the one dimension explains the affective representation of ASMR well, and that the affective representation was consistent regardless of modality.

정서표상에 대한 논의 중 첫 번째는 정서가 차원에서의 정서표상, 두 번째는 감각양상에 따른 정서표상을 설명하는 것이다. 선행연구에서는 정서표상을 설명하기 위해 정서가 모델(부호 정서가, 비부호 정서가), 감각양상에 따른 정서표상 모델(감각보편성, 감각특징성)들이 제시되었다. 본 연구에서는 최근에 등장한 ASMR을 이용하여 기존 연구에서 제시된 모델들을 비교하여 어떠한 모델이 정서표상을 잘 설명하는지 확인하고자 하였다. 본 연구에서 사용한 자료는 Kim & Kim(2022)에서 수집한 3개의 정서유형(부정, 중립, 긍정) 및 2개의 감각양상(청각, 시청각)으로 구분된 ASMR 자극에 대한 정서평정자료를 사용하였다. 이후, 해당 자료에 대한 다차원척도법, 표상 유사성 분석 및 이원 변량분석, 다중회귀분석 및 이원 변량분석을 실시하였다. 다차원척도법 결과, 비부호 정서가에 비해 부호 정서가, 감각특징성에 비해 감각보편성에서 자극의 정서유형 간 구분이 잘 이루어졌다. 다차원척도법 결과와 유사하게, 표상 유사성 분석 및 다중회귀분석 결과 또한 비부호 정서가에 비해 부호 정서가, 감각특징성에 비해 감각보편성이 유의하게 정서표상을 잘 설명하였다. 이러한 결과는 정서가 모델 중 1차원의 양극단에 긍정과 부정이 위치하는 모델이 ASMR에 대한 정서표상을 잘 설명하며, 감각양상과 상관없이 정서표상이 일관적임을 시사한다.

Keywords

Acknowledgement

이 논문은 한국연구재단 4단계 BK21사업(전북대학교 심리학과)의 지원을 받아 연구되었음(No. 4199990714213).

References

  1. Barrett, L. F., & Bliss-Moreau, E. (2009). Affect as a psychological primitive. Advances in Experimental Social Psychology, 41, 167-218. DOI: 10.1016/S0065-2601(08)00404-8
  2. Baucom, L. B., Wedell, D. H., Wang, J., Blitzer, D. N., & Shinkareva, S. V. (2012). Decoding the neural representation of affective states. NeuroImage, 59(1), 718-727. DOI: 10.1016/j.neuroimage.2011.07.037
  3. Bradley, M. M., & Lang, P. J. (2000). Affective reactions to acoustic stimuli. Psychophysiology, 37(2), 204-215. DOI: 10.1111/1469-8986.3720204
  4. Chikazoe, J., Lee, D., Kriegeskorte, N., & Anderson, A. K. (2014). Population coding of affect across stimuli, modalities and individuals. Nature Neuroscience, 17(8), 1114-1122. DOI: 10.1038/ nn.3749
  5. Codispoti, M., Bradley, M. M., & Lang, P. J. (2001). Affective reactions to briefly presented pictures. Psychophysiology, 38(3), 474-478. DOI: 10.1111/1469-8986.3830474
  6. Dalenberg, J. R., Weitkamp, L., Renken, R. J., & ter Horst, G. J. (2018). Valence processing differs across stimulus modalities. NeuroImage, 183, 734-744. DOI: 10.1016/j.neuroimage.2018.08.059
  7. Dellacherie, D., Bigand, E., Molin, P., Baulac, M., & Samson, S. (2011). Multidimensional scaling of emotional responses to music in patients with temporal lobe resection. Cortex, 47(9), 1107-1115. DOI: 10.1016/j.cortex.2011.05.007
  8. Gao, C., & Shinkareva, S. V. (2021). Modality-general and modality-specific audiovisual valence processing. Cortex, 138, 127-137. DOI: 10.1016/j.cortex.2021.01.022
  9. Gomez, P., Stahel, W. A., & Danuser, B. (2004). Respiratory responses during affective picture viewing. Biological Psychology, 67(3), 359-373. DOI: 10.1016/j.biopsycho.2004.03.013
  10. Kim, H., & Kim. J (2022). Affective responses to ASMR using multidimensional scaling and classification. Science of Emotion & Sensibility, 25(3), 47-62. DOI: 0.14695/KJSOS.2022.25.3.47 https://doi.org/10.14695/KJSOS.2022.25.3.47
  11. Kim, J. (2021). Representation of facial expressions of different ages: A multidimensional scaling study. Science of Emotion and Sensibility, 24(3), 71-80. DOI: 10.14695/KJSOS.2021.24.3.71
  12. Kim, J., & Wedell, D. H. (2016). Comparison of physiological responses to affect eliciting pictures and music. International Journal of Psychophysiology, 101, 9-17. DOI: 10.1016/j.ijpsycho.2015.12.011
  13. Kim, J., Shinkareva, S. V., & Wedell, D. H. (2017). Representations of modality-general valence for videos and music derived from fMRI data. NeuroImage, 148, 42-54. DOI: 10.1016/J.NEUROIMAGE.2017.01.002
  14. Kim, J., Weber, C. E., Gao, C., Schulteis, S., Wedell, D. H., & Shinkareva, S. V. (2020). A study in affect: Predicting valence from fMRI data. Neuropsychologia, 143, 107473. DOI: 10.1016/j.neuropsychologia.2020.107473
  15. Klasen, M., Kenworthy, C. A., Mathiak, K. A., Kircher, T. T. J., & Mathiak, K. (2011). Supramodal representation of emotions. Journal of Neuroscience, 31(38), 13635-13643. DOI: 10.1523/JNEUROSCI.2833-11.2011
  16. Kriegeskorte, N., Mur, M., & Bandettini, P. (2008). Representational similarity analysis - connecting the branches of systems neuroscience. Frontiers in Systems Neuroscience, 2, 1-28. DOI: 10.3389/neuro.06.004.2008
  17. Larsen, J. T., Norris, C. J., & Cacioppo, J. T. (2003). Effects of positive and negative affect on electromyographic activity over zygomaticus major and corrugator supercilii. Psychophysiology, 40(5), 776-785. DOI: 10.1111/1469-8986.00078
  18. Lindquist, K. A., Satpute, A. B., Wager, T. D., Weber, J., & Barrett, L. F. (2016). The brain basis of positive and negative affect: Evidence from a meta-analysis of the human neuroimaging literature. Cerebral Cortex, 26(5), 1910-1922. DOI: 10.1093/cercor/bhv001
  19. Nastase, S. A., Connolly, A. C., Oosterhof, N. N., Halchenko, Y. O., Guntupalli, J. S., Visconti di Oleggio Castello, M., Gors, J., Gobbini, M. I., & Haxby, J. V. (2017). Attention selectively reshapes the geometry of distributed semantic representation. Cerebral Cortex, 27(8), 4277-4291. DOI: 10.1093/cercor/bhx138
  20. Peelen, M. V., Atkinson, A. P., & Vuilleumier, P. (2010). Supramodal representations of perceived emotions in the human brain. Journal of Neuroscience, 30(30), 10127-10134. DOI: 10.1523/JNEUROSCI.2161-10.2010
  21. Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161. DOI: 10.1037/h0077714
  22. Russell, J. A., & Bullock, M. (1985). Multidimensional scaling of emotional facial expressions: Similarity from preschoolers to adults. Journal of Personality and Social Psychology, 48(5), 1290-1298. DOI: 10.1037/0022-3514.48.5.1290
  23. Russell, J. A., & Carroll, J. M. (1999). On the bipolarity of positive and negative affect. Psychological Bulletin, 125(1), 3-30. DOI: 10.1037//0033-2909.125.1.3
  24. Shinkareva, S. V., Wang, J., & Wedell, D. H. (2013). Examining similarity structure: Multidimensional scaling and related approaches in neuroimaging. Computational and Mathematical Methods in Medicine, 2013. DOI: 10.1155/2013/796183
  25. Shinkareva, S. V., Wang, J., Kim, J., Facciani, M. J., Baucom, L. B., & Wedell, D. H. (2014). Representations of modality-specific affective processing for visual and auditory stimuli derived from functional magnetic resonance imaging data. Human Brain Mapping, 35(7), 3558-3568. DOI: 10.1002/HBM.22421
  26. Vieillard, S., & Guidetti, M. (2009). Children's perception and understanding of (dis)similarities among dynamic bodily/facial expressions of happiness, pleasure, anger, and irritation. Journal of Experimental Child Psychology, 102(1), 78-95. DOI: 10.1016/j.jecp.2008.04.005
  27. Viinikainen, M., Katsyri, J., & Sams, M. (2012). Representation of perceived sound valence in the human brain. Human Brain Mapping, 33(10), 2295-2305. DOI: 10.1002/hbm.21362
  28. Weaverdyck, M. E., Lieberman, M. D., & Parkinson, C. (2020). Tools of the trade multivoxel pattern analysis in fMRI: A practical introduction for social and affective neuroscientists. Social Cognitive and Affective Neuroscience, 15(4), 487-509. DOI: 10.1093/scan/nsaa057