Browse > Article
http://dx.doi.org/10.15701/kcgs.2016.22.5.37

Affective interaction to emotion expressive VR agents  

Choi, Ahyoung (Department of Software, Gachon University)
Publication Information
Journal of the Korea Computer Graphics Society / v.22, no.5, 2016 , pp. 37-47 More about this Journal
Abstract
This study evaluate user feedback such as physiological response and facial expression when subjects play a social decision making game with interactive virtual agent partners. In the social decision making game, subjects will invest some of money or credit in one of projects. Their partners (virtual agents) will also invest in one of the projects. They will interact with different kinds of virtual agents which behave reciprocated or unreciprocated behavior while expressing socially affective facial expression. The total money or credit which the subject earns is contingent on partner's choice. From this study, I observed that subject's appraisal of interaction with cooperative/uncooperative (or friendly/unfriendly) virtual agents in an investment game result in increased autonomic and somatic response, and that these responses were observed by physiological signal and facial expression in real time. For assessing user feedback, Photoplethysmography (PPG) sensor, Galvanic skin response (GSR) sensor while capturing front facial image of the subject from web camera were used. After all trials, subjects asked to answer to questions associated with evaluation how much these interaction with virtual agents affect to their appraisals.
Keywords
Emotion expressive agent; physiological signals; nonverbal user feedback; social game; decision making;
Citations & Related Records
연도 인용수 순위
  • Reference
1 J. C. Lester, S. A. Converse, S. E. Kahler, S. T. Barlow, B. A. Stone, and R. S. Bhogal, "The persona effect: Affective impact of animated pedagogical agents," In Proceedings of CHI, pp. 359-366, 1997.
2 S. W. McQuiggan, J. L. Robison, R. Phillips, J. C. Lester, "Modeling parallel and reactive empathy in virtual agents: an inductive approach," In Proceedings of AAMAS, pp. 167-174, 2008.
3 H. C. van Vugt, J. F. Hoorn, E. A. Konijn, "Interactive engagement with embodied agents: an empirically validated framework," In Proceedings of CASA, pp.195-204, 2009.
4 A. Camurri, A. Coglio, "An architecture for emotional agents," IEEE Multimedia, pp. 24-33, 1998.
5 J. K. Rilling, D. R. Goldsmith, A. L. Glenn AL, M. R. Jairam, H. A. Elfenbein, J. E. Dagenais, C. D. Murdock, G. Pagnoni, "The neural correlates of the affective response to unreciprocated cooperation," Neuropsychologia, Vol. 46, No. 5, pp.1256-66, 2008.   DOI
6 S. Kiesler, L. Sproull, K. Waters, "A Prisoner's dilemma experiment on cooperation with people and human like computers," Journal of Personality and Social Psychology, Vol. 70, No. 1, pp. 47-65, 1996.   DOI
7 J. Scharlemann, C. Eckel, A. Kacelnik, R. Wilson, "The value of smile: Game theory with a human face," Journal of Economic Psychology, Vol. 22, No. 5, pp. 617-640, 2001.   DOI
8 S. Kopelman, A. Rosette, L. Thompson, "The three faces of Eve: Strategic displays of positive, negative and neutral emotions in negotiations," Organizational Behavior and Human Decision Processes, Vol.99, No.1, pp.81-101, 2006.   DOI
9 W. Yoshida, B. Seymour, K. Friston, and R. Dolan, "Neural Mechanisms of Belief Inference during Cooperative Games," Journal of Neuroscience, Vol. 30, No.32, pp.10744-10751, 2010.   DOI
10 D. Watson, L. A. Clark, A. Tellegen, A, "Development and validation of brief measures of positive and negative affect: The PANAS scale," Journal of Personality and Social Psychology, Vol. 54, pp.1063-1070, 1988.   DOI
11 T. W. Buchanan, "Retrieval of Emotional Memories," Psychological Bulletin, Vol. 133, No.5, pp.764-779, 2008.
12 J. Cacioppo, G. Berntson, J. Larsen, K. Poehlmann, T. Ito, "The psychophysiology of emotion," M. Lewis, J. Haviland- Jones, (Eds.), Handbook of emotions, 2nd ed. The Guilford Press: New York, NY, pp. 173-191, 2000.
13 The impact of positive, negative and neutral stimuli in a virtual reality cognitive-motor rehabilitation task: a pilot study with stroke patients
14 J. N. Bailenson, E. Aharoni, A. C. Guadagno, R. E. Guadagno, A. Dimov, J. Blascovich, "Comparing behavioral and selfreport measures of embodied agents' social presence in immersive virtual environments," In Proceedings of the 7th Annual International Workshop on presence, 2004.
15 J. N. Bailenson, E.D. Pontikakis, I.B. Mauss, J.J Gross, M.E. Jabon, C.A. Hutcherson, C. Nass, O. John, "Real-time classification of evoked emotions using facial feature tracking and physiological responses," International Journal of Human Machine Studies, Vol. 66, pp. 303-317, 2008.   DOI
16 H. Prendinger, S. Mayer, J. Mori, and M. Ishizuka. "Persona effect revisited: Using bio-signals to measure and reflect the impact of character-based interfaces," In Proceedings of the 4th International Working Conference on Intelligent Virtual Agents, pp. 283-291, 2003.
17 C. Melo, L. Zheng, J. Gratch, "Expression of Moral Emotions in Cooperating Agents," In Proceedings of International Conference on Intelligent Virtual Agents, 2009.
18 J. N. Bailenson, K. Swinth, C. Hoyt, S. Persky, A. Dimov, J. Blascovich, "The Independent and Interactive Effects of Embodied-Agent Appearance and Behavior on Self Report, Cognitive, and Behavioral Markers of Copresence in Immersive Virtual Environments," Presence, Vol. 14, No.4, pp.379-393, 2005.   DOI
19 M. Rodrigo, R. Baker, J. Agapito, J. Nabos, M. Repalam, S. Reyes, M. Pedro, "The Effects of an Interactive Software Agent on Student Affective Dynamics while Using an Intelligent Tutoring System," IEEE Transactions on Affective Computing, Vol.3, No.2, pp.224-236, 2012.   DOI
20 M.M. Bradley, M. Codispoti, B.N. Cuthbert, P.J. Lang, "Emotion and motivation I: Defensive and appetitive reactions in picture processing," Emotion, Vol. 1, pp.276-298, 2001.   DOI
21 P.J. Lang, "A bio-informational theory of emotional imagery," Psychophysiology, Vol. 16, No. 6, pp. 495-512, 1979.   DOI
22 M. S. Cameirao, A. L. Faria, T. Paulino, J. Alves, and S. B. Bermudez, "The impact of positive, negative and neutral stimuli in a virtual reality cognitive-motor rehabilitation task: a pilot study with stroke patients," Journal of NeuroEngineering and Rehabilitation, Vol. 13, No. 1, pp.1-15, 2016.   DOI
23 H. Prendinger, J. Mori, M. Ishizuka, "Using human physiology to evaluate subtle expressivity of a virtual quizmaster in a mathematical game," International Journal of Human-Computer Studies, Vol. 62, No. 2, pp.231-245, 2005   DOI