Knowledge Management Research (지식경영연구)

The Knowledge Management Society of Korea (한국지식경영학회)
권호 표지
DOI QR코드

DOI QR Code

Can Generative AI Replace Human Managers? The Effects of Auto-generated Manager Responses on Customers

생성형 AI는 인간 관리자를 대체할 수 있는가? 자동 생성된 관리자 응답이 고객에 미치는 영향

  • Received : 2023.12.02
  • Accepted : 2023.12.19
  • Published : 2023.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Generative AI, especially conversational AI like ChatGPT, has recently gained traction as a technological alternative for automating customer service. However, there is still a lack of research on whether current generative AI technologies can effectively replace traditional human managers in customer service automation, and whether they are advantageous in some situations and disadvantageous in others, depending on the conditions and environment. To answer the question, "Can generative AI replace human managers in customer service activities?", this study conducted experiments and surveys on customer online reviews of a food delivery platform. We applied the perspective of the elaboration likelihood model to generate hypotheses about whether there is a difference between positive and negative online reviews, and analyzed whether the hypotheses were supported. The analysis results indicate that for positive reviews, generative AI can effectively replace human managers. However, for negative reviews, complete replacement is challenging, and human managerial intervention is considered more desirable. The results of this study can provide valuable practical insights for organizations looking to automate customer service using generative AI.

최근 생성형 AI, 특히 ChatGPT와 같은 대화형 인공지능이 고객 서비스를 자동화하는 기술적 대안으로 주목받고 있다. 그러나 고객 서비스 자동화에 있어 현재의 생성형 AI 기술이 기존 인간 관리자를 효과적으로 대체할 수 있는지, 조건이나 환경에 따라 어떤 상황에서는 유리하고 다른 상황에서는 불리한지에 대한 연구는 아직 충분히 이루어지지 않은 상태이다. 이러한 배경에서 본 연구는 "고객 서비스 활동과 관련하여 생성형 AI가 인간 관리자를 대체할 수 있는가?"라는 질문에 답하기 위해, 음식 배달 플랫폼의 고객 온라인 리뷰에 대한 실험과 설문조사를 수행하였다. 또한 고객의 온라인 리뷰가 긍정적일 때와 부정적일 때에 따라 차이가 있는지 정교화 가능성 모델의 관점을 적용하여 가설을 도출하고 해당 가설이 지지되는지를 분석을 통해 확인하였다. 분석 결과, 긍정적인 리뷰에 대해서는 생성형 AI가 인간 관리자를 효과적으로 대체할 수 있지만, 부정적인 리뷰에 대해서는 완벽한 대체가 어려워 인간 관리자의 개입이 더 바람직한 것으로 확인되었다. 이러한 본 연구의 결과는 생성형 AI를 이용하여 고객 서비스 자동화하고자 하는 기업들에게 유의미한 실무적인 통찰을 제공해 줄 수 있을 것이다.

Keywords

References

  1. 김유진 (2023). 부정적 온라인 리뷰에 대한 사업자 피드백 효과 연구. 석사학위논문, 연세대학교 정보대학원, 서울. 
  2. 김태하, 차훈상, 박찬희, 위정현 (2020). 온라인 쇼핑몰 챗봇 사용자의 활용의도에 영향을 미치는 요인에 대한 실증 연구. 지식경영연구, 21, 211-225.  https://doi.org/10.15813/KMR.2020.21.4.011
  3. 양박 (2021). 부정적인 온라인 리뷰와 판매자의 대응전략이 소비자의 신뢰도와 구매의도에 미치는 영향. 석사학위논문, 동국대학교 대학원, 서울. 
  4. 윤성준 (2000). 웹쇼핑몰 사이트 신뢰도의 결정요인과 구매의향에 미치는 영향에 관한 시뮬레이션 접근방법. 경영학연구, 29(3), 353-376. 
  5. 이규하, 곽기영 (2014). 인터넷 쇼핑몰에서 구전과 보증이 신뢰에 미치는 영향: 제품평가 용이성의 조절효과를 중심으로. 지식경영연구, 15(3), 141-168.  https://doi.org/10.15813/KMR.2014.15.3.007
  6. 이동욱 (2016). 소상공인 창업교육이 교육성과 및 창업의도에 미치는 영향. 석사학위논문, 중앙대학교 대학원. 
  7. 이미경 (2002). 전자상거래의 불확실성과 소비자 선택이론. 박사학위논문, 고려대학교 대학원, 서울. 
  8. 정병규, 정상희 (2018). 디지털 고객경험이 브랜드 충성도 및 구매에 미치는 영향: 4 개 상품군을 중심으로. 유통경영학회지, 21(6), 109-118.  https://doi.org/10.17961/JDMR.21.6.201812.109
  9. 정희정, 구철모, 정남호 (2021). 호텔 고객의 비대면 서비스 이용의도의 영향요인에 대한 연구. 지식경영연구, 22(3), 235-252.  https://doi.org/10.15813/KMR.2021.22.3.013
  10. 최인혁, 김갑식 (2008). 소비자관점에서의 기업신뢰도 측정 척도 개발. 한국심리학회지: 소비자.광고, 9(1), 69-85. 
  11. 홍종필 (2008). 인터넷 소비자의 전자 구전행동(electronic word-of-mouth)에 대한 개념적 모형 및 연구명제의 고찰. 광고학연구, 19(4), 165-184.
  12. Anziano, T. (2023, October). Responding to online reviews can help your business. Business News Daily, https://www.businessnewsdaily.com/9187-respond-to-online-reviews.html 
  13. Arango, L., Singaraju, S. P., & Niininen, O. (2023). Consumer responses to AI-Generated charitable giving ads. Journal of Advertising, 1-18. 
  14. Arsenijevic, U., & Jovic, M. (2019, September). Artificial intelligence marketing: Chatbots. In 2019 International Conference on Artificial Intelligence: Applications and Innovations(IC-AIAI) (pp. 19-193). IEEE. 
  15. Bitner, M. J., Booms, B. H., & Tetreault, M. S. (1990). The service encounter: Diagnosing favorable and unfavorable incidents. Journal of Marketing, 54(1), 71-84.  https://doi.org/10.1177/002224299005400105
  16. Day, E., & Crask, M. R. (2000). Value assessment: The antecedent of customer satisfaction. Journal of Consumer Satisfaction, Dissatisfaction and Complaining Behavior, 13, 52-60. 
  17. Du, D., Zhang, Y., & Ge, J. (2023, July). Effect of AI generated content advertising on consumer engagement. In International Conference on Human-Computer Interaction (pp. 121-129). Cham: Springer Nature Switzerland. 
  18. Garvin, D. (1983). Quality on the line. Harv. Bus. Rev., 65-75. 
  19. Gefen, D., Straub, D., & Boudreau, M. C. (2000). Structural equation modeling and regression: Guidelines for research practice. Communications of the Association for Information Systems, 4(1), 7. 
  20. Gronroos, C. (1988). Service quality: The six criteria of good perceived service. Review of Business, 9(3), 10. 
  21. Hess Jr, R. L., Ganesan, S., & Klein, N. M. (2003). Service failure and recovery: The impact of relationship factors on customer satisfaction. Journal of the Academy of Marketing Science, 31(2), 127-145.  https://doi.org/10.1177/0092070302250898
  22. Hinton, G. E., Osindero, S., & Teh, Y. W. (2006). A fast learning algorithm for deep belief nets. Neural Computation, 18(7), 1527-1554.  https://doi.org/10.1162/neco.2006.18.7.1527
  23. Holbrook, M. B., & Hirschman, E. C. (1982). The experiential aspects of consumption: Consumer fantasies, feelings, and fun. Journal of Consumer Research, 9(2), 132-140.  https://doi.org/10.1086/208906
  24. Karinshak, E., Liu, S. X., Park, J. S., & Hancock, J. T. (2023). Working with AI to persuade: Examining a large language model's ability to generate pro-vaccination messages. Proceedings of the ACM on Human-Computer Interaction, 7(CSCW1), 1-29. 
  25. Katsiuba, D., Kew, T., Dolata, M., & Schwabe, G. (2022). Supporting online customer feedback management with automatic review response generation. Proceedings of the 55th Hawaii International Conference on Systems Sciences(HICSS-55), 226-236. 
  26. Lee, J., Park, D. H., & Han, I. (2008). The effect of negative online consumer reviews on product attitude: An information processing view. Electronic Commerce Research and Applications, 7(3), 341-352.  https://doi.org/10.1016/j.elerap.2007.05.004
  27. Maheswaran, D., & Meyers-Levy, J. (1990). The influence of message framing and issue involvement. Journal of Marketing Research, 27(3), 361-367.  https://doi.org/10.1177/002224379002700310
  28. Meyer, C., & Schwager, A. (2007). Understanding customer experience. Harvard Business Review, 85(2), 116. 
  29. Miller, J. L., Craighead, C. W., & Karwan, K. R. (2000). Service recovery: A framework and empirical investigation. Journal of Operations Management, 18(4), 387-400.  https://doi.org/10.1016/S0272-6963(00)00032-2
  30. Parasuraman, A. B. L. L., Zeithaml, V. A., & Berry, L. (1988). SERVQUAL: A multiple-item scale for measuring consumer perceptions of service quality (Vol. 64, No. 1). Routledge, pp. 12-40. 
  31. Parasuraman, A., Zeithaml, V. A., & Berry, L. L. (1985). A conceptual model of service quality and its implications for future research. Journal of Marketing, 49(4), 41-50.  https://doi.org/10.1177/002224298504900403
  32. Pine, B. J., & Gilmore, J. H. (1998). Welcome to the experience economy (Vol. 76, No. 4). Cambridge, MA, USA: Harvard Business Review Press, pp. 97-105. 
  33. Pine, B. J., & Gilmore, J. H. (2011). The experience economy. Harvard Business Press. 
  34. Rathjens, B., Van Der Heide, B., Pham, D., Earle, K., Ulusoy, E., Mason, A. J., ... & Bredland, A. (2023). Negative online reviews and manager response: Applying expectancy disconfirmation theory in a CMC context. Communication Reports, 36(2), 110-122.  https://doi.org/10.1080/08934215.2022.2154815
  35. ReviewTrackers. (2022, February). Powerful examples of how to respond to negative reviews. ReviewTrackers, https://www.reviewtrackers.com/guides/examples-responding-reviews/ 
  36. Rosenblatt, F. (1957). The perceptron, a perceiving and recognizing automaton Project Para. Cornell Aeronautical Laboratory. 
  37. Samuel, A. L. (1959). Some studies in machine learning using the game of checkers. IBM Journal of Research and Development, 3(3), 210-229.  https://doi.org/10.1147/rd.33.0210
  38. Schmitt, B. H. (1999). Experiential marketing: How to get customers to sense, feel, think, act, and relate to your company and brands. Free Press. 
  39. Sheehan, B., Jin, H. S., & Gottlieb, U. (2020). Customer service chatbots: Anthropomorphism and adoption. Journal of Business Research, 115, 14-24.  https://doi.org/10.1016/j.jbusres.2020.04.030
  40. Sparks, B. A., So, K. K. F., & Bradley, G. L. (2016). Responding to negative online reviews: The effects of hotel responses on customer inferences of trust and concern. Tourism Management, 53, 74-85.  https://doi.org/10.1016/j.tourman.2015.09.011
  41. Taylor, S. A., & Baker, T. L. (1994). An assessment of the relationship between service quality and customer satisfaction in the formation of consumers' purchase intentions. Journal of Retailing, 70(2), 163-178.  https://doi.org/10.1016/0022-4359(94)90013-2
  42. Van Noort, G., & Willemsen, L. M. (2012). Online damage control: The effects of proactive versus reactive webcare interventions in consumer-generated and brand-generated platforms. Journal of Interactive Marketing, 26(3), 131-140.  https://doi.org/10.1016/j.intmar.2011.07.001
  43. Wang, Y., & Chaudhry, A. (2018). When and how managers' responses to online reviews affect subsequent reviews. Journal of Marketing Research, 55(2), 163-177.  https://doi.org/10.1509/jmr.15.0511
  44. Weizenbaum, J. (1966). ELIZA-a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36-45.  https://doi.org/10.1145/365153.365168
  45. Yang, S., Zhou, C., & Chen, Y. (2021). Do topic consistency and linguistic style similarity affect online review helpfulness? An elaboration likelihood model perspective. Information Processing & Management, 58(3), 102521. 
  46. Zeithaml, V. A., Berry, L. L., & Parasuraman, A. (1993). The nature and determinants of customer expectations of service. Journal of the academy of Marketing Science, 21, 1-12. https://doi.org/10.1177/0092070393211001
  47. Chui, M. (2023, August). The state of AI in 2023: Generative AI's breakout year. McKinsey Digital, https://www.mckinsey.com/capabilities/quantumblack/our-insights/the-state-of-ai-in-2023-generative-ais-breakout-year#/ 
  48. IBM. (2023a). What are AI hallucinations? https://www.ibm.com/topics/ai-hallucinations 
  49. IBM. (2023b). What is generative AI? https://research.ibm.com/blog/what-is-generative-AI 
  50. ITWorld. (2023, 8월). "생성형 AI라는 거부할 수 없는 물결" 2023 국내 AI 도입 및 활용 현황 조사-Market Pulse. https://www.itworld.co.kr/techlibrary/302617 
  51. Leonhardt, M. (2023, March 2023). Some workers are worried that ChatGPT will replace their jobs. They might be right. Fortune, https://fortune.com/2023/03/25/workers-worried-about-chat-gpt-ai-taking-jobs/ 
  52. Markets and Markets. (2023, June). https://www.marketsandmarkets.com/Market-Reports/chatbot-market-72302363.html 
  53. McAfee, D. R., & Brynjolfsson, E. (2023). How to capitalize on generative AI. Harvard Business Review. https://hbr.org/2023/11/how-to-capitalize-on-generative-ai 
  54. Qualtrics. (2023). Digital customer experience: Everything to know in 2023. https://www.qualtrics.com/kr/experience-management/customer/what-is-digital-cx/