• Annual Conference on Human and Language Technology
• /
• 2023.10a
• /
• pp.228-233
• /
• 2023

인간의 수동 평가 시 시간과 비용의 소모, 주석자 간의 의견 불일치, 평가 결과의 품질 등 불가피한 한계가 발생한다. 본 논문에서는 맥락을 고려하고 긴 문장 입출력이 가능한 ChatGPT를 활용한 한국어 요약문 평가가 인간 평가를 대체하거나 보조하는 것이 가능한가에 대해 살펴보았다. 이를 위해 ChatGPT가 생성한 요약문에 정량적 평가와 정성적 평가를 진행하였으며 정량적 지표로 BERTScore, 정성적 지표로는 일관성, 관련성, 문법성, 유창성을 사용하였다. 평가 결과 ChatGPT4의 경우 인간 수동 평가를 보조할 수 있는 가능성이 있음을 확인하였다. ChatGPT가 영어 기반으로 학습된 모델임을 고려하여 오류 발견 성능을 검증하고자 한국어 오류 요약문으로 추가 평가를 진행하였다. 그 결과 ChatGPT3.5와 ChatGPT4의 오류 요약 평가 성능은 불안정하여 인간을 보조하기에는 아직 어려움이 있음을 확인하였다.

• Journal of Korea Society of Digital Industry and Information Management
• /
• v.19 no.4
• /
• pp.87-96
• /
• 2023

ChatGPT, a leader in generative AI, can use natural expressions like humans based on large-scale language models (LLM). The ability to grasp the context of the language and provide more specific answers by algorithms is excellent. It also has high-quality conversation capabilities that have significantly developed from past Chatbot services to the level of human conversation. In addition, it is expected to change the operation method of the tourism industry and improve the service by utilizing ChatGPT, a generative AI in the tourism sector. This study was conducted to explore ChatGPT trends and paradigms in tourism. The results of the study are as follows. First, keywords such as tourism, utilization, creation, technology, service, travel, holding, education, development, news, digital, future, and chatbot were widespread. Second, unlike other keywords, service, education, and Mokpo City data confirmed the results of a high degree of centrality. Third, due to CONCOR analysis, eight keyword clusters highly relevant to ChatGPT in the tourism sector emerged.

• Knowledge Management Research
• /
• v.22 no.1
• /
• pp.287-308
• /
• 2021

This study aims to explore how the effects of the perceived social presence on trust and live chat adoption intention vary with the types of live chats in e-commerce context. As technology develops, live chat with the seller in e-commerce is rapidly replaced by AI-assisted live chat called chat-bot. However, it is not well known how the buyers perceive the difference between the chat with seller and the chat-bot. This study therefore proposes first, the perceived social presence toward the seller will influence trust and the live chat adoption. Second, the effects of social presence will be stronger when using live chat with seller than using chat-bot. To validate, we collect data from 232 e-commerce users and confirm the first proposition. However, the higher level of the social presence effect of live chat with seller is not clearly revealed. This study is expected to provide researchers and managers who are interested in AI-based chatbots with useful theoretical and practical implications.

• Journal of Digital Policy
• /
• v.2 no.4
• /
• pp.1-10
• /
• 2023

This study conducted preliminary study to identify effective ways to use ChatGPT in traffic policing by analyzing ChatGPT's responses to the driver's license test and the road traffic accident appraiser test. I collected ChatGPT responses for the driver's license test item pool and the road traffic accident appraiser test using the OpenAI API with Python code for 30 iterative experiments, and analyzed the percentage of correct answers by test, year, section, and consistency. First, the average correct answer rate for the driver's license test and the for road traffic accident appraisers test was 44.60% and 35.45%, respectively, which was lower than the pass criteria, and the correct answer rate after 2022 was lower than the average correct answer rate. Second, the percentage of correct answers by section ranged from 29.69% to 56.80%, showing a significant difference. Third, it consistently produced the same response more than 95% of the time when the answer was correct. To effectively utilize ChatGPT, it is necessary to have user expertise, evaluation data and analysis methods, design a quality traffic law corpus and periodic learning.

• Journal of Intelligence and Information Systems
• /
• v.29 no.4
• /
• pp.111-127
• /
• 2023

This study is a study on user perception of ChatGPT use. The goal of this study is to analyze the relationship between user policy expectations and user innovativeness on ChatGPT technology acceptance and intention to use using variables of TRA (Theory of Reasoned Action). The impact of policy expectations and user innovativeness on the intention to use by mediating usefulness and hedonic motivation, and the impact of subjective norms on the usefulness and intention to use were analyzed by dividing them into the MZ generation and the non-MZ generation. It was verified whether there was a moderating effect on the effect of age differences on usefulness by interacting with policy expectations. An online survey was conducted on 300 ChatGPT users using PLS (Partial Least Square) structural equations and SPSS Package, and statistical analysis was performed using PLS and SPSS. According to the analysis results, it was confirmed that the higher the initial user's innovativeness, the higher the intention to use ChatGPT. In addition, the moderating effect analysis comparing the differences between the MZ generation and the non-MZ generation showed that policy expectations had a negative effect on the usefulness of ChatGPT use.

• Journal of Internet Computing and Services
• /
• v.25 no.1
• /
• pp.79-90
• /
• 2024

This study focused on user trust in ChatGPT, a generative AI technology, and explored the factors that affect usage status and intention to continue using, and whether the influence of trust varies depending on the purpose. For this purpose, the survey was conducted targeting people in their 20s and 30s who use ChatGPT the most. The statistical analysis deploying IBM SPSS 27 and SmartPLS 4.0. A structural equation model was formulated on the foundation of Bhattacherjee's Expectation-Confirmation Model (ECM), employing path analysis and Multi-Group Analysis (MGA) for hypothesis validation. The main findings are as follows: Firstly, ChatGPT is mainly used for specific needs or objectives rather than as a daily tool. The majority of users are cognizant of its hallucination effects; however, this did not hinder its use. Secondly, the hypothesis testing indicated that independent variables such as expectation- confirmation, perceived usefulness, and user satisfaction all exert a positive influence on the dependent variable, the intention for continuance intention. Thirdly, the influence of trust varied depending on the user's purpose in utilizing ChatGPT. trust was significant when ChatGPT is used for information retrieval but not for creative purposes. This study will be used to solve reliability problems in the process of introducing generative AI in society and companies in the future and to establish policies and derive improvement measures for successful employment.

• Journal of dental hygiene science
• /
• v.24 no.1
• /
• pp.62-70
• /
• 2024

Background: This study aimed to evaluate ChatGPT's performance accuracy in responding to questions from the national dental hygienist examination. Moreover, through an analysis of ChatGPT's incorrect responses, this research intended to pinpoint the predominant types of errors. Methods: To evaluate ChatGPT-3.5's performance according to the type of national examination questions, the researchers classified 200 questions of the 49th National Dental Hygienist Examination into recall, interpretation, and solving type questions. The researchers strategically modified the questions to counteract potential misunderstandings from implied meanings or technical terminology in Korea. To assess ChatGPT-3.5's problem-solving capabilities in applying previously acquired knowledge, the questions were first converted to subjective type. If ChatGPT-3.5 generated an incorrect response, an original multiple-choice framework was provided again. Two hundred questions were input into ChatGPT-3.5 and the generated responses were analyzed. After using ChatGPT, the accuracy of each response was evaluated by researchers according to the types of questions, and the types of incorrect responses were categorized (logical, information, and statistical errors). Finally, hallucination was evaluated when ChatGPT provided misleading information by answering something that was not true as if it were true. Results: ChatGPT's responses to the national examination were 45.5% accurate. Accuracy by question type was 60.3% for recall and 13.0% for problem-solving type questions. The accuracy rate for the subjective solving questions was 13.0%, while the accuracy for the objective questions increased to 43.5%. The most common types of incorrect responses were logical errors 65.1% of all. Of the total 102 incorrectly answered questions, 100 were categorized as hallucinations. Conclusion: ChatGPT-3.5 was found to be limited in its ability to provide evidence-based correct responses to the Korean national dental hygiene examination. Therefore, dental hygienists in the education or clinical fields should be careful to use artificial intelligence-generated materials with a critical view.

• The Journal of the Convergence on Culture Technology
• /
• v.10 no.2
• /
• pp.565-574
• /
• 2024

ChatGPT is causing significant reverberations across all sectors of our society, and this holds true for the field of education as well. However, scholarly and societal discussions regarding ChatGPT in academic settings have primarily focused on issues such as plagiarism, with relatively limited research on the positive effects of utilizing generative AI. Additionally, amidst the educational crisis of the post-COVID era, there is a growing recognition of the need to enhance academic engagement. In light of these concerns, we investigated how academic engagement varies based on students' levels of ChatGPT literacy and examined whether students' academic confidence and perceived academic competence serve as mediators between ChatGPT literacy and academic engagement. An analysis using SPSS was conducted on the data collected from 406 college students. The results showed that ChatGPT literacy had a positive effect on academic engagement, and academic confidence mediated the relationship between ChatGPT literacy and academic engagement. Also, when the mediating effect of perceived academic competence was significant only when it was serially mediated. Based on these findings, we discussed the theoretical contributions of identifying the theoretical mechanism between ChatGPT literacy and academic engagement. In addition, practical implications regarding the importance of ChatGPT literacy education were described.

• Korean Journal of Biological Psychiatry
• /
• v.17 no.4
• /
• pp.218-225
• /
• 2010

Objectives : The potential association between choline acetyltransferase(CHAT) polymorphism and the risk of mild cognitive impairment(MCI) has not been investigated in Korea. We examined the main effect of CHAT polymorphism and its interaction with apolipoprotein E(APOE) polymorphism in the development of MCI in elderly Korean sample. Methods : We analyzed CHAT 2384G > A polymorphism and APOE polymorphism among 149 MCI subjects with MCI and 298 normal controls. We tested the association between MCI and CHAT A allele status using a logistic regression model. In addition, we employed generalized multifactor dimensionality reduction(GMDR) to investigate the interaction between CHAT and APOE with regard to the risk of MCI. Results : The CHAT A allele was associated with AD risk(OR = 1.59, 95% CI = 1.02-2.48, p = 0.042). No significant gene-gene interaction between CHAT and APOE was found in GMDR method(testing balanced accuracy = 0.540, p = 0.055). Conclusion : The CHAT A allele was associated with MCI risk in the Korean elderly. Its interaction with the APOE ${\varepsilon}4$ allele was not significant with regard to the development of MCI.

• Journal of the Korea Society of Computer and Information
• /
• v.26 no.8
• /
• pp.221-227
• /
• 2021

Since there are almost no scales for measuring the size of We-Chat applets, and most of the existing We-Chat applets are grafted through the original APP application, At present, the application scope of We-chat applets which is mainly in /shopping/life/food application. Thus, the purpose of this research is to focus on the iPhone app store, collect data on the top five of APP-STORE through users' comments and The high-frequency words will be obtained for statistics, and the variables of this study will be set up. Last, develop relevant Empirical research on the size and measurement scale of the We-Chat applet. Therefore, how to use We-Chat applets to improve user experience, we can create their own user private domain traffic for We-Chat applets and achieve long-term market competitiveness.