Journal of the Korea Academia-Industrial cooperation Society (한국산학기술학회논문지)

The Korea Academia-Industrial cooperation Society (한국산학기술학회)
권호 표지
DOI QR코드

DOI QR Code

Development of AI-based Real Time Agent Advisor System on Call Center - Focused on N Bank Call Center

AI기반 콜센터 실시간 상담 도우미 시스템 개발 - N은행 콜센터 사례를 중심으로

  • Ryu, Ki-Dong (Graduate School of Public Policy and Information Technology, Seoul National University of Science & Technology) ;
  • Park, Jong-Pil (NH Bank) ;
  • Kim, Young-min (NH Bank) ;
  • Lee, Dong-Hoon (Saltlux) ;
  • Kim, Woo-Je (Department of Industrial & Information Systems Engineering, Seoul National University of Science and Technology)
  • Received : 2018.10.25
  • Accepted : 2019.02.01
  • Published : 2019.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

The importance of the call center as a contact point for the enterprise is growing. However, call centers have difficulty with their operating agents due to the agents' lack of knowledge and owing to frequent agent turnover due to downturns in the business, which causes deterioration in the quality of customer service. Therefore, through an N-bank call center case study, we developed a system to reduce the burden of keeping up business knowledge and to improve customer service quality. It is a "real-time agent advisor" system that provides agents with answers to customer questions in real time by combining AI technology for speech recognition, natural language processing, and questions & answers for existing call center information systems, such as a private branch exchange (PBX) and computer telephony integration (CTI). As a result of the case study, we confirmed that the speech recognition system for real-time call analysis and the corpus construction method improves the natural speech processing performance of the query response system. Especially with name entity recognition (NER), the accuracy of the corpus learning improved by 31%. Also, after applying the agent advisor system, the positive feedback rate of agents about the answers from the agent advisor was 93.1%, which proved the system is helpful to the agents.

기업의 대고객 접점으로써 콜센터의 중요성은 커지고 있다. 하지만, 콜센터는 상담사의 지식 부족과 업무 부적응에 따른 잦은 이직으로 인해 상담사 운영이 어렵고, 이로 인한 고객 서비스 품질 저하의 문제를 안고 있다. 이에 본 연구에서는 상담사에게 업무 지식에 대한 부하를 줄이고 서비스 품질을 향상 시키기 위해 음성 인식 기술과 자연어 처리 및 질의응답을 지원하는 AI 기술과 PBX, CTI 등의 콜센터 정보시스템을 결합하여 실시간으로 상담사에게 고객의 질의 내용에 대한 답변을 제공해주는 "실시간 상담 도우미" 시스템 개발 방안에 대해 N은행 콜센터 사례를 통해 연구하였다. 사례연구 결과, 실시간 통화 분석을 위한 음성인식 시스템의 구성방안과, 질의응답 시스템의 자연어처리 성능 향상을 위한 말뭉치 구축 방안을 확인 할 수 있었으며, 특히 개체명 인식기의 경우 도메인에 맞는 말뭉치 학습 후 정확도가 31% 향상됨을 확인하였다. 또한, 상담 도우미 시스템을 적용한 후 상담 도우미의 답변에 대한 상담사들의 긍정적 피드백 비율이 93.1%로써 충분히 상담사 업무에 도움을 주고 있음을 확인하였다.

Keywords

SHGSCZ_2019_v20n2_750_f0001.png 이미지

Fig. 1. Block Diagram of the classic automatic speech recognition and understanding system[29]

SHGSCZ_2019_v20n2_750_f0002.png 이미지

Fig. 2. N-Bank Call Center AI System Concept

SHGSCZ_2019_v20n2_750_f0003.png 이미지

Fig. 3. Agent Advisor screen example

SHGSCZ_2019_v20n2_750_f0004.png 이미지

Fig. 4. Block Diagram of the Real Time Agent Advisor System

SHGSCZ_2019_v20n2_750_f0005.png 이미지

Fig. 5. Architecture of AI Engine

SHGSCZ_2019_v20n2_750_f0006.png 이미지

Fig. 6. Block Diagram of the STT Engine[33]

SHGSCZ_2019_v20n2_750_f0007.png 이미지

Fig. 7. Q & A Development Methodology

SHGSCZ_2019_v20n2_750_f0008.png 이미지

Fig. 8. Analysis of major consultation types

SHGSCZ_2019_v20n2_750_f0009.png 이미지

Fig. 9. A Part of the Ontology Class

SHGSCZ_2019_v20n2_750_f0010.png 이미지

Fig. 10. Result of learning model verification with query-response evaluation set

SHGSCZ_2019_v20n2_750_f0011.png 이미지

Fig. 11. Graph of the User Feedback

Table 1. Elements of Intelligent Virtual Secretary Technology

SHGSCZ_2019_v20n2_750_t0001.png 이미지

Table 4. Accuracy of STT via Acoustic Learning

SHGSCZ_2019_v20n2_750_t0002.png 이미지

Table 2. Morphological analyzer improvement result

SHGSCZ_2019_v20n2_750_t0003.png 이미지

Table 3. NER improvement result

SHGSCZ_2019_v20n2_750_t0004.png 이미지

Table 4. Example of Competency Question

SHGSCZ_2019_v20n2_750_t0005.png 이미지

Table 5. Knowledge base (ontology) construction status

SHGSCZ_2019_v20n2_750_t0006.png 이미지

Table 7. Actual user feedback during operation

SHGSCZ_2019_v20n2_750_t0007.png 이미지

References

  1. Y. S. Kim and S. M. Rue, "4th Industrial Revolution and IoT-A.I Platform", Korea Institute of Information Technology Magazine, Vol. 15, No. 1, pp.1-7, 2017
  2. C. B. Frey and M. A. Osborne, "The future of employment: how susceptible are jobs to computerisation?", Technological forecasting and social change, Vol. 114, No. , pp.254-280, 2013 https://doi.org/10.1016/j.techfore.2016.08.019
  3. K. D. Ryu and W. J. Kim, "A Study on Contact Center Evaluation Model Using AHP and Content Analysis", Journal of the Korea Academia-Industrial cooperation Society, Vol. 19, No. 5, pp.106-116, 2018
  4. Z. Askin, M. Armony, and V. Mehrotra, "The Modern Call Center: A Multi-Disciplinary Perspective on Operations Management Research", PRODUCTION AND OPERATIONS MANAGEMENT, Vol. 16, No. 6, pp.665-688, 2007 https://doi.org/10.1111/j.1937-5956.2007.tb00288.x
  5. R. Rijo, J. Varajao, and R. Goncalves, "Contact center: information systems design. Journal of Intelligent Manufacturing", Journal of Intelligent Manufacturing, Vol. 23, No. 3, pp.497-515, 2012 https://doi.org/10.1007/s10845-010-0389-0
  6. D. Fensel, F. Van Harmelen, I. Horrocks, D. L. McGuinness, and P. F. Patel-Schneider, "OIL: An ontology infrastructure for the semantic web", IEEE intelligent systems, Vol. 16, No. 2, pp.38-45, 2001 https://doi.org/10.1109/MIS.2001.956080
  7. S. L. Han, S. H. Nam, Y. S. Kim, D. J. Kim, J. Y. Kim, and N. Han, "A Study on Survey and policy of the call center industry", National IT Industry Promotion Agency, pp.12, 2010
  8. H. J. We, J. L. Sim, B. S. Kim, and H. J. Kim, "A Study on Improvement of Call Center Personnel Supply and Demand", National IT Industry Promotion Agency, 2011
  9. KOSIS, "2011 Annual employment statistics", http://kosis.kr/statHtml/statHtml.do?orgId=118&tblId=DT_118N_MONA31&conn_path=I3(accessed:Sep., 29, 2018)
  10. M. Gilbert, J. G. Wilpon, B. Stern, and G. Di Fabbrizio, "Intelligent Virtual Agents for Contact Center Automation", IEEE Signal Processing Magazine, Vol. 22, No. 5, pp.32-41, 2005 https://doi.org/10.1109/MSP.2005.1511822
  11. M. Saberi, O. K. Hussain, and E. Chang, "Past, present and future of contact centers: a literature review", Business Process Management Journal, Vol. 23, No. 3, pp.574-597, 2017 https://doi.org/10.1108/BPMJ-02-2015-0018
  12. O. W. Kwon, T. G. Hong, J. X. Huang, and Y. K. Kim, "An Analysis for Dialogue Processing Technologies and Service Trends of Virtual Personal Assistants", Communications of the Korean Institute of Information Scientists and Engineers, Vol. 35, No. 8, pp.19-27, 2017
  13. H. M. Lee, and I. Seung, "Comparative Study on the usability of mobile intelligent personal assistance service based on voice recognition technology-Focused on 'Samsung S Voice' and 'Apple Siri'", Journal of Digital Design, Vol. 14, No. 1, pp.231-240, 2014 https://doi.org/10.17280/jdd.2014.14.1.023
  14. D. A. Park, "A Study on Conversational Public Administration Service of the Chatbot Based on Artificial Intelligence", Journal of Korea Multimedia Society, Vol. 20, No. 8, pp.1347-1356, 2017 https://doi.org/10.9717/KMMS.2017.20.8.1347
  15. S. S. Kang, "AI Assistant Apple's Siri and Wolfram Alpha", Communications of the Korean Institute of Information Scientists and Engineers, Vol. 35, No. 8, pp.42-50, 2017
  16. K. O. Park, "A Review on AI Virtual Assistants' Secretarial Job Competencies", JOURNAL OF SECRETARIAL STUDIES, Vol. 26, No. 2, pp.253-272, 2017 https://doi.org/10.35605/jss.2017.09.26.2.253
  17. D. S. Chang, J. W. Sung, and J. H. Im, "Intelligent Personal Assistant: GiGA Genie Assistant", Communications of the Korean Institute of Information Scientists and Engineers, Vol. 35, No. 8, pp.28-35, 2017
  18. S. H. Hwang and J. Y. Yun, "An User Experience Analysis of Virtual Assistant Using Grounded Theory", Journal of the HCI Society of Korea , Vol. 12, No. 2, pp.31-40, 2017 https://doi.org/10.17210/jhsk.2017.05.12.2.31
  19. H. S. Kim, "Natural Language Processing Techniques of Spoken Assistant Systems based on Artificial Intelligence", Communications of the Korean Institute of Information Scientists and Engineers, Vol. 35, No. 8, pp.9-18, 2017
  20. J. H. Kim, D. S. Chang, and H. S. Kim, "Statistical Generation of Korean Chatting Sentences Using Multiple Feature Information", KOREAN JOURNAL OF COGNITIVE SCIENCE, Vol. 20, No. 4, pp.421-437, 2009 https://doi.org/10.19066/cogsci.2009.20.4.002
  21. S. Wilske, "Form and Meaning in Dialogue- based Computer-Assisted Language Learning", Ph.D. thesis, Saarland University, Saarbrucken, 2015
  22. M. R. Kangavari, S. Ghandchi, and M. Golpour, "Information Retrieval: Improving Question Answering Systems by Query Reformulation and Answer Validation.", World Academy of Science, Engineering and Technology, Vol. 24, No. 12, pp.1275-1282, 2008
  23. S. Y. Park, S. C. Kwon, B. S. Kim, S. D. Han, H. S. Shim, and G. G. Lee, "Question Answering System using Multiple Information Source and Open Type Answer Merge", Proceedings of NAACL-HLT 2015, pp.111-115, 2015
  24. W. Cui, Y. Xiao, and W. Wang, "KBQA: An Online Template Based Question Answering System over Freebase", Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI-16), pp.4240-4241, 2016
  25. K. Jeong, Y. Ham, and K. Lee, "Dual-Spiral methodology for knowledgebase constructions", In 2016 International Conference on Big Data and Smart Computing (BigComp). IEEE, pp.477-480, 2016
  26. A. Santoro, D. Raposo, D. G. Barrett, M. Malinowski, R. Pascanu, P. Battaglia, and T. Lillicrap, "A simple neural network module for relational reasoning", Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA., pp.4967-4976, 2017
  27. S. Furui, "Speech Recognition-Past, Present, and Future -", NTT Review, Vol. 7, No. 2, pp.13-18, 1995
  28. J. T. Kim and H. Jung, "Trends of Speech Recognition Technology and Its Application to the Navy's Information and Communication Sector-Focusing on Speech To Text", Defense & Technology, pp.120-127, 2017
  29. I. Gavat and D. Militaru, "New trends in machine learning for speech recognition", Annual Symposium of the Institute of Solid Mechanics and Session of the Commission of Acoustics SISOM, At Bucharest, Romania, Vol. 2015, No. , pp.271-276, 2015
  30. S. J. Choi and J. B. Kim, "Comparison Analysis of Speech Recognition Open APIs' Accuracy", Asia-pacific Journal of Multimedia Services Convergent with Art, Humanities, and Sociology, Vol. 7, No. 8, pp.411-418, 2017
  31. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A. R. Mohamed, N. Jaitly, and B. Kingsbury, "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups", IEEE Signal processing magazine, Vol. 29, No. 6, pp.82-97, 2012 https://doi.org/10.1109/MSP.2012.2205597
  32. S. W. Lee, H. M. Jeong, P. Kim, and D. M. Seo, "Inference Technology Research Trend", NIPA, pp.1-12, 2010
  33. J. G. Park, "Speech Recognition Technology for Call Center Service Version 2.0", ETRI, 2015
  34. N. F. Noy and D. L. McGuinness, "Ontology Development 101: A Guide to Creating Your First Ontology", Stanford University, 2001