Browse > Article
http://dx.doi.org/10.5392/JKCA.2021.21.06.019

The Effects of Chatbot's Error Types and Structures of Error Message on User Experience  

Lee, Mi-Jin (연세대학교 심리학과)
Han, Kwang-Hee (연세대학교 심리학과)
Publication Information
The Journal of the Korea Contents Association / v.21, no.6, 2021 , pp. 19-34 More about this Journal
Abstract
The aim of this study is verifying the effects of chatbot's error types and structures of error message on attitude, behavior intention towards the chatbot and perceived usability of the chatbot. The error types of chatbot are divided into 'experience' error and 'agency' error, which set different expectancy level, according to mind perception theory. The structures of error message were either unstructured condition composed of error specification only or structured condition composed of apology, explanation and willingness of improvement. It was found that score of perceived usability was higher in experience error condition than agency error condition. Also, all three scores of dependent variables were higher in structured error message condition than unstructured error message condition. Furthermore, expectation gap of experience didn't predict the dependent variables but expectation gap of agency predicted all three dependent variables. Finally, the tendency of interaction effect between the error type and the structure of the error message on expectation gap of agency was observed. This study confirmed the mitigating effect of structured error messages and the possibility that these effects may vary by the type of error. The result is expected to be applicable to design of error coping strategies that enhance user experience.
Keywords
Chatbot; Error; User Experience; Mind Perception; Expectancy Violation;
Citations & Related Records
연도 인용수 순위
  • Reference
1 H. M. Gray, K. Gray, and D. M. Wegner, "Dimensions of mind perception," Science, Vol.315 No.5812, pp.619-619, 2007.   DOI
2 M. Fishbein and I. Ajzen, "Belief, attitude, intention, and behavior: An introduction to theory and research," The Journal of Philosophy and Rhetoric, Vol.10, No.2, pp.130-132, 1977.
3 P. R. Spence, D. Westerman, C. Edwards, and A. Edwards, "Welcoming our robot overlords: Initial expectations about interaction with a robot," Communication Research Reports, Vol.31, No.3, pp.272-280, 2014.   DOI
4 T. Ringberg, G. Odekerken-Schroder, and G. L. Christensen, "A cultural models approach to service recovery," Journal of Marketing, Vol.71, No.3, pp.194-214, 2007.   DOI
5 C. Nass, J. Steuer, and E. R. Tauber, "Computers are social actors," In Proceedings of the SIGCHI conference on Human factors in computing systems, pp.72-78, 1994.
6 R. L. Oliver, "A cognitive model of the antecedents and consequences of satisfaction decisions," Journal of marketing research, Vol.17, No.4, pp.460-469, 1980.   DOI
7 R. A. Spreng, G. D. Harrell, and R. D. Mackoy, "Service recovery: impact on satisfaction and intentions," Journal of services marketing, 1995.
8 C. Edwards, A. Edwards, P. R. Spence, and D. Westerman, "Initial interaction expectations with robots: Testing the human-to-human interaction script," Communication Studies, Vol.67, No.2, pp.227-238, 2016.   DOI
9 M. Akgun, K. Cagiltay, and D. Zeyrek, "The effect of apologetic error messages and mood states on computer users' self-appraisal of performance," Journal of Pragmatics, Vol.42, No.9, pp.2430-2448, 2010.   DOI
10 J. Y. Tzeng, "Toward a more civilized design: studying the effects of computers that apologize," International Journal of Human-Computer Studies, Vol.61, No.3, pp.319-345, 2004.   DOI
11 E. Go and S. S. Sundar, "Humanizing chatbots: The effects of visual, identity and conversational cues on humanness perceptions," Computers in Human Behavior, Vol.97, pp.304-316, 2019.   DOI
12 J. Weizenbaum, "ELIZA-a computer program for the study of natural language communication between man and machine," Communications of the ACM, Vol.9, No.1, pp.36-45, 1966.   DOI
13 T. Komatsu and S. Yamada, "Adaptation gap hypothesis: How differences between users' expected and perceived agent functions affect their subjective impression," Journal of Systemics, Cybernetics and Informatics, Vol.9, No.1, pp.67-74, 2011.
14 J. K. Burgoon, "Interpersonal expectations, expectancy violations, and emotional communication," Journal of Language and Social Psychology, Vol.12, No.1-2, pp.30-48, 1993.   DOI
15 F. D. Davis, R. P. Bagozzi, and P. R. Warshaw, "User acceptance of computer technology: A comparison of two theoretical models," Management science, Vol.35, No.8, pp.982-1003, 1989.   DOI
16 S. Yuan, B. Bruggemeier, S. Hillmann, and T. Michael, "User Preference and Categories for Error Responses in Conversational User Interfaces," In Proceedings of the 2nd Conference on Conversational User Interfaces, pp.1-8, 2020.
17 E. Olshtain, "Apologies across languages," Cross-cultural pragmatics: Requests and apologies, Vol.155173, 1989.
18 A. Cohen and E. Olshtain, "Apology: A speech act set," Sociolinguistics and language acquisition, Vol.18, p.35, 1983
19 F. D. Davis, "A technology acceptance model for empirically testing new end-user information systems: Theory and results," Massachusetts Institute of Technology, 1985.
20 A. Folstad and P. B. Brandtzaeg, "Chatbots and the new world of HCI," Interactions, Vol.24, No.4, pp.38-42, 2017.   DOI
21 문용식, 인공지능 기반의 챗봇 서비스의 국내외 동향 분석 및 발전 전망, 한국정보문화진흥원, 2018.
22 Z. Ashktorab, M. Jain, Q. V. Liao, and J. D. Weisz, "Resilient chatbots: repair strategy preferences for conversational breakdowns," In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp.1-12, 2019.
23 J. K. Burgoon, "A communication model of personal space violations: Explication and an initial test," Human communication research, Vol.4, No.2, pp.129-142, 1978.   DOI
24 D. J. Brooks, M. Begum, and H. A. Yanco, "Analysis of reactions towards failures and recovery strategies for autonomous robots," In 2016 25th IEEE International Symposium on Robot and Human Interactive Communication, pp.487-492, 2016.
25 S. Honig and T. Oron-Gilad, "Understanding and resolving failures in human-robot interaction: Literature review and model development," Frontiers in psychology, Vol.9, p.861, 2018.   DOI
26 C. R. Berger and R. J. Calabrese, "Some explorations in initial interaction and beyond: Toward a developmental theory of interpersonal communication," Human communication research, Vol.1, No.2, pp.99-112, 1974.   DOI
27 T. Komatsu, R. Kurosawa, and S. Yamada, "How does the difference between users' expectations and perceptions about a robotic agent affect their behavior?," International Journal of Social Robotics, Vol.4, No.2, pp.109-116, 2012.   DOI
28 A. C. Horstmann and N. C. Kramer, "When a Robot Violates Expectations: The Influence of Reward Valence and Expectancy Violation on People's Evaluation of a Social Robot," In Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, pp.254-256, 2020.
29 T. Komatsu and S. Yamada, "How do robotic agents' appearances affect people's interpretations of the agents' attitudes?," In CHI'07 Extended Abstracts on Human Factors in Computing Systems, pp.2519-2524, 2007.
30 M. Appel, D. Izydorczyk, S. Weber, M. Mara, and T. Lischetzke, "The uncanny of mind in a machine: Humanoid robots as tools, agents, and experiencers," Computers in Human Behavior, Vol.102, pp.274-286, 2020.   DOI
31 Z. Ashktorab, M. Jain, Q. V. Liao, and J. D. Weisz, "Resilient chatbots: Repair strategy preferences for conversational breakdowns," In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, pp.1-12, 2019.
32 M. K. Lee, S. Kiesler, J. Forlizzi, S. Srinivasa, and P. Rybski, "Gracefully mitigating breakdowns in robotic services," In 2010 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp.203-210, 2010.
33 K. C. Yam, Y. E. Bigman, P. M. Tang, R. Ilies, D. De Cremer, H. Soh, and K. Gray, "Robots at work: People prefer-And forgive-Service robots with perceived feelings," The Journal of Applied Psychology, 2020. retrived from https://doi.org/10.1037/apl0000834   DOI
34 M. Salem, F. Eyssel, K. Rohlfing, S. Kopp, and F. Joublin, "To err is human (-like): Effects of robot gesture on perceived anthropomorphism and likability," International Journal of Social Robotics, Vol.5, No.3, pp.313-323, 2013.   DOI
35 S. J. Park, C. M. MacDonald, and M. Khoo, "Do you care if a computer says sorry? User experience design through affective messages," In Proceedings of the designing interactive systems conference, pp.731-740, 2012.
36 R. J. Lewicki, B. Polin, and R. B. Lount Jr, "An exploration of the structure of effective apologies," Negotiation and Conflict Management Research, Vol.9, No.2, pp.177-196, 2016.   DOI