International journal of advanced smart convergence

The Institute of Internet, Broadcasting and Communication (한국인터넷방송통신학회)
권호 표지
DOI QR코드

DOI QR Code

Understanding the Importance of Presenting Facial Expressions of an Avatar in Virtual Reality

  • Kim, Kyulee (Department of Computer Science and Engineering, Ewha Womans University) ;
  • Joh, Hwayeon (Department of Computer Science and Engineering, Ewha Womans University) ;
  • Kim, Yeojin (Department of Computer Science and Engineering, Ewha Womans University) ;
  • Park, Sohyeon (Department of Computer Science and Engineering, Ewha Womans University) ;
  • Oh, Uran (Department of Computer Science and Engineering, Ewha Womans University)
  • Received : 2022.10.18
  • Accepted : 2022.10.25
  • Published : 2022.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

While online social interactions have been more prevalent with the increased popularity of Metaverse platforms, little has been studied the effects of facial expressions in virtual reality (VR), which is known to play a key role in social contexts. To understand the importance of presenting facial expressions of a virtual avatar under different contexts, we conducted a user study with 24 participants where they were asked to have a conversation and play a charades game with an avatar with and without facial expressions. The results show that participants tend to gaze at the face region for the majority of the time when having a conversation or trying to guess emotion-related keywords when playing charades regardless of the presence of facial expressions. Yet, we confirmed that participants prefer to see facial expressions in virtual reality as well as in real-world scenarios as it helps them to better understand the contexts and to have more immersive and focused experiences.

Keywords

Acknowledgement

This research was supported by the ITRC (Information Technology Research Center) program (IITP-2022-2020-0-01460) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation).

References

  1. Stylianos Mystakidis. 2022. Metaverse. Encyclopedia 2, 1 (2022), 486-497. https://doi.org/10.3390/encyclopedia2010031
  2. Christoph Anthes, Ruben Jesus Garcia-Hernandez, Markus Wiedemann, and Dieter Kranzlmuller. 2016. State of the art of virtual reality technology. In 2016 IEEE aerospace conference. IEEE, 1-19. DOI: 10.1109/AERO.2016.7500674
  3. CNW Geraets, S Klein Tuente, BP Lestestuiver, M Van Beilen, SA Nijman, JBC Marsman, and W Veling. 2021. Virtual reality facial emotion recognition in social environments: An eye-tracking study. Internet Interventions 25 (2021), 100432. DOI : https://doi.org/10.1016/j.invent.2021.100432
  4. Steven Hickson, Nick Dufour, Avneesh Sud, Vivek Kwatra, and Irfan Essa. 2019. Eyemotion: Classifying facial expressions in VR using eye-tracking cameras. In 2019 IEEE winter conference on applications of computer vision (WACV). IEEE, 1626-1635. DOI : 10.1109/WACV.2019.00178
  5. Chris Frith. 2009. Role of facial expressions in social interactions. Philosophical Transactions of the Royal Society B: Biological Sciences 364, 1535 (2009), 3453-3458. DOI : https://doi.org/10.1098/rstb.2009.0142
  6. Diane E Marian and Arthur P Shimamura. 2013. Contextual influences on dynamic facial expressions. The American Journal of Psychology 126, 1 (2013), 53-66. DOI : https://doi.org/10.5406/amerjpsyc.126.1.0053
  7. Lisa E Ishii, Jason C Nellis, Kofi Derek Boahene, Patrick Byrne, and Masaru Ishii. 2018. The importance and psychology of facial expression. Otolaryngologic Clinics of North America 51, 6 (2018), 1011-1017. https://doi.org/10.1016/j.otc.2018.07.001
  8. Gary Bente, Felix Eschenburg, and Lisa Aelker. 2007. Effects of simulated gaze on social presence, person perception and personality attribution in avatar-mediated communication. In Presence 2007: Proceedings of the 10th Annual International Workshop on Presence, October 25-27, 2007, Barcelona, Spain. Citeseer, 207-14.
  9. Rosanna E Guadagno, Jim Blascovich, Jeremy N Bailenson, and Cade McCall. 2007. Virtual humans and persuasion: The effects of agency and behavioral realism. Media Psychology 10, 1 (2007), 1-22. DOI : : https://doi.org/10.1080/15213260701300865
  10. Jonathon Derek Hart, Thammathip Piumsomboon, Gun A Lee, Ross T Smith, and Mark Billinghurst. 2021. Manipulating Avatars for Enhanced Communication in Extended Reality. In 2021 IEEE International Conference on Intelligent Reality (ICIR). IEEE, 9-16. DOI : 10.1109/ICIR51845.2021.00011
  11. Soo Youn Oh, Jeremy Bailenson, Nicole Kramer, and Benjamin Li. 2016. Let the avatar brighten your smile: Effects of enhancing facial expressions in virtual environments. PloS one 11, 9 (2016), e0161794. DOI : https://doi.org/10.1371/journal.pone.0161794
  12. Caroline J Falconer, Janek S Lobmaier, Marina Christoforou, Sunjeev K Kamboj, John A King, Paul Gilbert, and Chris R Brewin. 2019. Compassionate faces: Evidence for distinctive facial expressions associated with specific prosocial motivations. PloS one 14, 1 (2019), e0210283. https://doi.org/10.1371/journal.pone.0210283
  13. Giacomo Rizzolatti, LJARN Craighero, et al. 2004. The mirror-neuron system. (2004).
  14. Peter G Enticott, Patrick J Johnston, Sally E Herring, Kate E Hoy, and Paul B Fitzgerald. 2008. Mirror neuron activation is associated with facial emotion processing. Neuropsychologia 46, 11 (2008), 2851-2854. https://doi.org/10.1016/j.neuropsychologia.2008.04.022
  15. Gennaro Ruggiero, Francesca Frassinetti, Yann Coello, Mariachiara Rapuano, Armando Schiano Di Cola, and Tina Iachini. 2017. The effect of facial expressions on peripersonal and interpersonal spaces. Psychological research 81, 6 (2017), 1232-1240. DOI : https://doi.org/10.1007/s00426-016-0806-x
  16. Aulia Rahmah and Yuli Astutik. 2020. Charades game: Does it affect students' learning on English vocabulary. EnJourMe (English Journal of Merdeka) 5, 1 (2020), 75-89. DOI : 10.26905/enjourme.v4i2.4258
  17. Kil-Soo Suh, Hongki Kim, and Eung Kyo Suh. 2011. What if your avatar looks like you? Dual-congruity perspectives for avatar use. MIs Quarterly (2011), 711-729. DOI : https://doi.org/10.2307/23042805
  18. Plutchik Robert. 1980. Emotion: Theory, Research, and Experience. Vol. 1: Theories of Emotion. DOI : https://doi.org/10.1016/B978-0-12-558701-3.50007-7
  19. Masahiro Mori. 1970. The uncanny valley: the original essay by Masahiro Mori. IEEE Spectrum (1970).
  20. Nidhi N Khatri, Zankhana H Shah, and Samip A Patel. 2014. Facial expression recognition: A survey. International Journal of Computer Science and Information Technologies (IJCSIT) 5, 1 (2014), 149-152.
  21. Fernando De la Torre and Jeffrey F Cohn. 2011. Facial expression analysis. Visual analysis of humans (2011), 377-409. DOI : https://doi.org/10.1007/978-0-85729-997-0_19
  22. Hannah Scott, Jonathan P Batten, and Gustav Kuhn. 2019. Why are you looking at me? It's because I'm talking, but mostly because I'm staring or not doing much. Attention, Perception, & Psychophysics 81, 1 (2019), 109-118. DOI : https://doi.org/10.3758/s13414-018-1588-6
  23. Charlotte N Gunawardena and Frank J Zittle. 1997. Social presence as a predictor of satisfaction within a computer-mediated conferencing environment. American journal of distance education 11, 3 (1997), 8-26. DOI : https://doi.org/10.1080/08923649709526970
  24. Juliann Cortese and Mihye Seo. 2012. The role of social presence in opinion expression during FtF and CMC discussions. Communication Research Reports 29, 1 (2012), 44-53. https://doi.org/10.1080/08824096.2011.639913
  25. Elias Giannopoulos, Victor Eslava, Maria Oyarzabal, Teresa Hierro, Laura Gonzalez, Manuel Ferre, and Mel Slater. 2008. The effect of haptic feedback on basic social interaction within shared virtual environments. In International Conference on Human Haptic Sensing and Touch Enabled Computer Applications. Springer, 301-307. DOI : https://doi.org/10.1007/978-3-540-69057-3_36
  26. Ki Joon Kim, Eunil Park, and S Shyam Sundar. 2013. Caregiving role in human-robot interaction: A study of the mediating effects of perceived benefit and social presence. Computers in Human Behavior 29, 4 (2013), 1799-1806. DOI : https://doi.org/10.1016/j.chb.2013.02.009
  27. Bouhlal Meriem, Habib Benlahmar, Mohamed Amine Naji, Elfilali Sanaa, and Kaiss Wijdane. 2022. Determine the Level of Concentration of Students in Real Time from their Facial Expressions. International Journal of Advanced Computer Science and Applications 13, 1 (2022).
  28. Prabin Sharma, Meltem Esengonul, Salik Ram Khanal, Tulasi Tiwari Khanal, Vitor Filipe, and Manuel JCS Reis. 2018. Student concentration evaluation index in an e-learning context using facial emotion analysis. In International Conference on Technology and Innovation in Learning, Teaching and Education. Springer, 529-538. DOI : https://doi.org/10.1007/978-3-030-20954-4_40