• Journal of the Semiconductor & Display Technology
• /
• v.21 no.4
• /
• pp.14-17
• /
• 2022

Due to the development of IT technology and the on-going Coronavirus disease, non-face-to-face services have been activated. To overcome the inconvenience of non-face-to-face service, service providers have adopted chatbots as a way to feel like a human being. As the increasing chatbot services, chatbot builders have emerged, which can help non-developers to build them. Although its popularity has increased, its performance evaluation has not been conducted on such chatbot builders. In this paper, we implement a prototype chatbot that classifies hospital departments in the medical field using Dialogflow and Rasa, which are popular chatbot builders. By measuring the accuracy of the chatbot's classification of medical subjects, we evaluated the level of accuracy that the most used chatbot builder can have when they are used to build a chatbot service. The simulation results showed that Dialogflow had 87%, 65%, and 60%, and Rasa did 64%, 70%, and 63% in surgery dermatology, and otolaryngology, respectively.

• Journal of the Korean Society for information Management
• /
• v.37 no.4
• /
• pp.151-179
• /
• 2020

This study explores the potential of using a library chatbot to improve the non-face-to-face digital reference services for academic library users by designing and implementing a library chatbot. Through data analysis, user needs and library services were analyzed, and a scenario was designed by selecting an appropriate development method. For user-friendly interaction, the personality of the chatbot and user interface was designed to evaluate its usability. In addition, the accuracy was verified through the response accuracy evaluation and performance evaluation of the chatbot, and the effectiveness of the chatbot was evaluated through a user satisfaction survey. In order to manage the operation and maintain service quality, the chatbot is improved by monitoring user-chatbot conversations and reflecting user feedback. Based on these findings, recommendations for designing and implementing a library chatbot were made to help improve library reference services.

• Journal of Information Technology Services
• /
• v.17 no.3
• /
• pp.17-36
• /
• 2018

In this study, we developed a chatbot (Dialogue agent) using small Q & A data and evaluated its performance. The chatbot developed in this study was developed in the form of an FAQ chatbot that responds promptly to customer inquiries. The development of chatbot was conducted in three stages : 1. Analysis and planning, 2. Content creation, 3. API and messenger interworking. During the analysis and planning phase, we gathered and analyzed the question data of the customers and extracted the topics and details of the customers' questions. In the content creation stage, we created scenarios for each topic and sub-items, and then filled out specific answers in consultation with business owners. API and messenger interworking is KakaoTalk. The performance of the chatbot was measured by the quantitative indicators such as the accuracy that the chatbot grasped the inquiry of the customer and correctly answered, and then the questionnaire survey was conducted on the chatbot users. As a result of the survey, it was found that the chatbot not only provided useful information to the users but positively influenced the image of the pension. This study shows that it is possible to develop chatbots by using easily obtainable data and commercial API regardless of the size of business. It also implies that we have verified the validity of the development process by verifying the performance of developed chatbots as well as an explicit process of developing FAQ chatbots.

• Korean Journal of Acupuncture
• /
• v.41 no.1
• /
• pp.27-32
• /
• 2024

Objectives : This study developed an AI-based patient chatbot and examined the usability and educational effectiveness of the chatbot in the context of Korean medicine education. Methods : The patient chatbot was developed using the AI chatbot builder 'Danbee', and a total of five experts were surveyed and interviewed to determine the usability, effectiveness, advantages, disadvantages, and improvement points of the chatbot. Results : The patient chatbot was found to have high usability and educational effectiveness. The advantages of the patient chatbot were 1) it provided students with practical experience in performing clinical skills, 2) it provided instructors with assessment materials while reducing their teaching burden, and 3) it could be effectively used for horizontal and vertical integration education. The disadvantages and improvements of the patient chatbot were 1) improving the accuracy of intention inference, 2) providing students with specific instructions for problem-solving activities, and 3) providing assessment results and feedback about students' activities. Conclusions : This study is significant in that it proposes a new training method to overcome the limitations of the existing doctor-patient simulation. It is hoped that this study will stimulate further research on the improvement of students' clinical skills using artificial intelligence.

• Journal of Information Technology Services
• /
• v.19 no.4
• /
• pp.31-47
• /
• 2020

Recently, as the COVID-19 has spread and prolonged worldwide, the 'Untact' society is becoming routinized, and various smart technologies are leading to the spread of the 'Ontact' culture. This is because the desire of consumers to purchase a product and use the service has increased while minimizing the direct contact. In order to quickly respond to this circumstance, the percentage of the companies which are adopting Chatbot in various fields such as orders, delivery, and inquiries is increasing and they are getting a positive result. However as the demand for building Chatbot increases dramatically, there are many confusions among the companies which want to introduce Chatbot to their system, due to the lack of professional technicians and difficulties in understanding AI technologies and how to build them effectively. I believe that in the post COVID-19 era, much more companies will adopt Chatbot, and this will intensify the problem. The purpose of this study was to derive the needs for a guide on the method of buiilding a Chatbot through considering the prior research on Chatbot and analysis of the recent surge in the use of Chatbot services related to COVID-19. There are implications to presenting 5 phases of universal Chatbot implementation methodology using the platform to the stakeholders who want to introduce Chatbot to their customer so that they can understand and build Chatbot more easily and use AI Chatbot actively in response to the POST COVID-19 era.

• Journal of Information Technology Services
• /
• v.20 no.3
• /
• pp.75-86
• /
• 2021

As artificial intelligence is actively studied, chatbot systems are being applied to various fields. In particular, many chatbot systems for psychological counseling have been studied that can comfort modern people. However, while most psychological counseling chatbots are studied as rule-base and deep learning-based chatbots, there are large limitations for each chatbot. To overcome the limitations of psychological counseling using such chatbots, we proposes a novel psychological counseling AI chatbot system. The proposed system consists of a GPT-2 model that generates output sentence for Korean input sentences and an Electra model that serves as sentiment analysis and anxiety cause classification, which can be provided with psychological tests and collective intelligence functions. At the same time as deep learning-based chatbots and conversations take place, sentiment analysis of input sentences simultaneously recognizes user's emotions and presents psychological tests and collective intelligence solutions to solve the limitations of psychological counseling that can only be done with chatbots. Since the role of sentiment analysis and anxiety cause classification, which are the links of each function, is important for the progression of the proposed system, we experiment the performance of those parts. We verify the novelty and accuracy of the proposed system. It also shows that the AI chatbot system can perform counseling excellently.

• Journal of Information Technology Services
• /
• v.22 no.6
• /
• pp.37-53
• /
• 2023

With the development of the 4th industrial revolution, companies are trying to introduce new AI technologies and improve their performance. In particular, chatbot technology has developed and can not only communicate smoothly with humans, but also perform many complex tasks, so it has high market potential. However, there is still little research on chatbots in the field of e-commerce. Accordingly, this study aims to suggest ways to improve corporate performance through chatbot user satisfaction analysis. With the rapid development of China's e-commerce platform, In this study, through previous studies, the characteristics of chatbots were classified into accessibility, accuracy, empathy, reliability, and intimacy as factors influencing perceived usefulness, perceived ease, and perceived enjoyment of the Technology Acceptance Model (TAM). Five were selected and used as independent variables, and a model that affects customer satisfaction was set up. This paper sets user satisfaction as an important indicator of chatbot service and analyzes the path that affects user satisfaction, thereby improving chatbot service technology. It is important in that it provides a way to improve the smart chatbot service by understanding the degree of user acceptance in depth.

• Journal of Oriental Neuropsychiatry
• /
• v.35 no.2
• /
• pp.153-162
• /
• 2024

Objectives: Among OECD countries, South Korea has been having the highest suicide rate since 2018, with 24.1 deaths per 100,000 people reported in 2020. The objectie of this study was to examine the use of generative artificial intellicence (AI) chatbots to train third-year Korean medicine (KM) students in conducting suicide risk assessments for patients with depressive disorders to train students for their clinical practice skills. Methods: The Claude 3 Sonnet model was utilized for chatbot simulations. Students performed mock consultations using standardized suicide risk assessment tools including Ask Suicide-Screening Questions (ASQ) tool and ASQ Brief Suicide Safety Assessment. Experiences and attitudes were collected through an anonymous online survey. Responses were rated on a 1~5 Likert scale. Results: Thirty-six students aged 22~30 years participated in this study. Their scores for interest and appropriateness (4.66±0.57), usefulness (4.60±0.61), and overall experience (4.63±0.60) were high. Their evaluation of the usability of artificial intelligence chatbot was also high at 4.58±0.70 points. However, their trust in chatbot responses (Q12) was lower (3.86±0.99). Common issues related to dissatisfaction included conversation disruptions due to token limits and inadequate chatbot responses. Conclusions: This is the first study investigating generative AI chatbots for suicide risk assessment training in KM education. Students reported high satisfaction, although their trust in chatbot accuracy was moderate. Technical limitations affected their experience. These preliminary findings suggest that generative AI chatbots hold promise for clinical training, particularly for education in psychiatry. However, improvements in response accuracy and conversation continuity are needed.

• Journal of the Korea Convergence Society
• /
• v.8 no.9
• /
• pp.271-276
• /
• 2017

This project has been conducted to improve the usability of Chatbot Services such as Google(Allo) and Facebook M(Messenger. Based on the evaluation, this study aims to suggest the solutions to improve the usability of domestic Chatbot services and future directions for their development. It provides the overall understanding of the AI Chatbot service and the feature of Chatbot service through literature search. Furthermore, we summarized the current standing and the prospect of domestic messenger-based assistant Chatbot services. For conducting user evaluation, Peter Morville's honeycomb model is applied to in-depth user interviews. The followings are elements that could be amended to improve the service. The service should be incorporated by intuitive elements for users' understanding its functions and eliminate any elements that interfere with usability. The accuracy should be increased to improve the user satisfaction. This research will provide the future guidelines to improve the usability of Chabot services through continuous evaluation by users.

• Education of Primary School Mathematics
• /
• v.27 no.3
• /
• pp.303-320
• /
• 2024

This study presents the development and performance evaluation of a custom GPT-based chatbot tailored to provide solutions following Polya's problem-solving stages. A beta version of the chatbot was initially deployed to assess its mathematical capabilities, followed by iterative error identification and correction, leading to the final version. The completed chatbot demonstrated an accuracy rate of approximately 89.0%, correctly solving an average of 57.8 out of 65 image-based problems from a 6th-grade elementary mathematics textbook, reflecting a 4 percentage point improvement over the beta version. For a subset of 50 problems, where images were not critical for problem resolution, the chatbot achieved an accuracy rate of approximately 91.0%, solving an average of 45.5 problems correctly. Predominant errors included problem recognition issues, particularly with complex or poorly recognizable images, along with concept confusion and comprehension errors. The custom chatbot exhibited superior mathematical performance compared to the general-purpose ChatGPT. Additionally, its solution process can be adapted to various grade levels, facilitating personalized student instruction. The ease of chatbot creation and customization underscores its potential for diverse applications in mathematics education, such as individualized teacher support and personalized student guidance.