Browse > Article
http://dx.doi.org/10.7472/jksii.2021.22.2.69

A Method of Automated Quality Evaluation for Voice-Based Consultation  

Lee, Keonsoo (Research Lab, CNAI Ltd.)
Kim, Jung-Yeon (ICT Convergence Research Center, Soonchunhyang University)
Publication Information
Journal of Internet Computing and Services / v.22, no.2, 2021 , pp. 69-75 More about this Journal
Abstract
In a contact-free society, online services are becoming more important than classic offline services. At the same time, the role of a contact center, which executes customer relation management (CRM), is increasingly essential. For supporting the CRM tasks and their effectiveness, techniques of process automation need to be applied. Quality assurance (QA) is one of the time and resource consuming, and typical processes that are suitable for automation. In this paper, a method of automatic quality evaluation for voice based consultations is proposed. Firstly, the speech in consultations is transformed into a text by speech recognition. Then quantitative evaluation based on the QA metrics, including checking the elements in opening and closing mention, the existence of asking the mandatory information, the attitude of listening and speaking, is executed. 92.7% of the automated evaluations are the same to the result done by human experts. It was found that the non matching cases of the automated evaluations were mainly caused from the mistranslated Speech-to-Text (STT) result. With the confidence of STT result, this proposed method can be employed for enhancing the efficiency of QA process in contact centers.
Keywords
Call Center; Consultation; Quality Assurance; Speech Recognition; Process Automation;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Graf, S., Herbig, T., Buck, M. et al. "Features for voice activity detection: a comparative analysis". EURASIP J. Adv. Signal Process. vol. 91, 2019. https://doi.org/10.1186/s13634-015-0277-z   DOI
2 M. Rim, K. Thomas, j. Chandramouli, S. Barrus, and N. Nickman. "Implementation and quality assessment of a pharmacy services call center for outpatient pharmacies and specialty pharmacy services in an academic health system". The Bulletin of the American Society of Hospital Pharmacists, vol. 75, no. 10, pp. 633-641, 2018. https://doi.org/10.2146/ajhp170319   DOI
3 S. Ahmed, "Integrating DMAIC approach of Lean Six Sigma and theory of constraints toward quality improvement in healthcare." Reviews on environmental health, vol. 34, no. 4, pp. 427-434, 2019. https://doi.org/10.1515/reveh-2019-0003   DOI
4 X. Yang, S. Gao, Z. He, and M. Zhang, "Application of Design for Six Sigma tools in telecom service improvement". Production Planning & Control, vol. 29, no. 12, pp. 959-971, 2018. https://doi.org/10.1080/09537287.2018.1486469   DOI
5 E. Suarez, A. Calvo-Mora, J. Roldan, and R. Perianez-Cristobal, "Quantitative research on the EFQM excellence model: A systematic literature review (1991-2015)". European Research on Management and Business Economics, vol. 23, no. 3, pp. 147-156, 2017. https://doi.org/10.1016/j.iedeen.2017.05.002   DOI
6 J. Li, A. Sun, J. Han, and C. Li, "A survey on deep learning for named entity recognition". IEEE Transactions on Knowledge and Data Engineering. 2020. https://doi.org/10.1109/TKDE.2020.2981314   DOI
7 J. Devlin, M. Chang, K. Lee, and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding". 2018. arXiv preprint, https://arxiv.org/abs/1810.04805
8 K. Clark, M. Luong, Q. Le, and C. Manning, "Electra: Pre-training text encoders as discriminators rather than generators". 2020. arXiv preprint, https://arxiv.org/abs/2003.10555
9 M. Lee, and D. Lee. ""Untact": a new customer service strategy in the digital age." Service Business vol. 14. no. 1, pp. 1-22. 2020. http://dx.doi.org/10.1007/s11628-019-00408-2   DOI
10 A. Prashar, "Adopting PDCA (Plan-Do-Check-Act) cycle for energy optimization in energy-intensive SMEs". Journal of cleaner production, vol. 145, pp. 277-293, 2017. https://doi.org/10.1016/j.jclepro.2017.01.068   DOI