• The Mathematical Education
• /
• v.63 no.3
• /
• pp.467-484
• /
• 2024

The rapid advancement of generative AI has ushered in an era where anyone can create and freely utilize personalized chatbots without the need for programming expertise. This study aimed to develop a customized chatbot based on OpenAI's GPTs for the purpose of pre-service teacher education and to analyze its educational performance in mathematics as assessed by educators guiding pre-service teachers. Responses to identical questions from a general-purpose chatbot (ChatGPT), a customized GPTs-based chatbot, and an elementary mathematics education expert were compared. The expert's responses received an average score of 4.52, while the customized GPTs-based chatbot received an average score of 3.73, indicating that the latter's performance did not reach the expert level. However, the customized GPTs-based chatbot's score, which was close to "adequate" on a 5-point scale, suggests its potential educational utility. On the other hand, the general-purpose chatbot, ChatGPT, received a lower average score of 2.86, with feedback indicating that its responses were not systematic and remained at a general level, making it less suitable for use in mathematics education. Despite the proven educational effectiveness of conventional customized chatbots, the time and cost associated with their development have been significant barriers. However, with the advent of GPTs services, anyone can now easily create chatbots tailored to both educators and learners, with responses that achieve a certain level of mathematics educational validity, thereby offering effective utilization across various aspects of mathematics education.

• Education of Primary School Mathematics
• /
• v.27 no.3
• /
• pp.303-320
• /
• 2024

This study presents the development and performance evaluation of a custom GPT-based chatbot tailored to provide solutions following Polya's problem-solving stages. A beta version of the chatbot was initially deployed to assess its mathematical capabilities, followed by iterative error identification and correction, leading to the final version. The completed chatbot demonstrated an accuracy rate of approximately 89.0%, correctly solving an average of 57.8 out of 65 image-based problems from a 6th-grade elementary mathematics textbook, reflecting a 4 percentage point improvement over the beta version. For a subset of 50 problems, where images were not critical for problem resolution, the chatbot achieved an accuracy rate of approximately 91.0%, solving an average of 45.5 problems correctly. Predominant errors included problem recognition issues, particularly with complex or poorly recognizable images, along with concept confusion and comprehension errors. The custom chatbot exhibited superior mathematical performance compared to the general-purpose ChatGPT. Additionally, its solution process can be adapted to various grade levels, facilitating personalized student instruction. The ease of chatbot creation and customization underscores its potential for diverse applications in mathematics education, such as individualized teacher support and personalized student guidance.

• Annual Conference of KIPS
• /
• 2024.10a
• /
• pp.517-518
• /
• 2024

본 논문에서는 오픈소스(Open Source) LLM(Large Language Model)인 Llama3을 이용하여 음성으로 동작하는 맞춤형 로컬 챗봇을 구현하고자 한다 이를 위해 PEFT(Parameter Efficient Fine-Tuning) 방식과 RAG(Retrival Augmented Generation) 방식을 혼합하는 하이브리드 방식을 사용해 Llama3을 파인 튜닝하고, Ollama을 이용하여 로컬 컴퓨터에서 동작하는 챗봇을 구현하였다. 챗봇의 배포를 위해 서버 부분은 LangServe와 Ngrok을 사용하였고 클라이언트 부분은 모바일 환경에서 원활히 동작하는지 확인하기 위해 교육용 임베디드 시스템인 Raspberry Pi 5를 사용하여 구현하였다. 또한 사용자의 편의성을 위해 음성인식 기능을 추가하였다. 또한 구현한 챗봇에 성능평가를 진행하였다. 성능 측정 방식은 정확도를 사용하였고, 데이터 셋은 총 18개로 각 쿼리마다 5번씩 총 90개의쿼리로 성능 평가를 진행하였다. 성능 평가 결과 하이브리드 방식이 가장 우수한 성능을 보여주었다.

• Journal of The Korean Association For Science Education
• /
• v.44 no.3
• /
• pp.231-248
• /
• 2024

This study developed a chatbot for first-year high school students, employing open-source software and the Korean Sentence-BERT model for AI-powered document classification. The chatbot utilizes the Sentence-BERT model to find the six most similar Q&A pairs to a student's query and presents them in a carousel format. The initial dataset, built from online resources, was refined and expanded based on student feedback and usability throughout over the operational period. By the end of the 2023 academic year, the chatbot integrated a total of 30,819 datasets and recorded 3,457 student interactions. Analysis revealed students' inclination to use the chatbot when prompted by teachers during classes and primarily during self-study sessions after school, with an average of 2.1 to 2.2 inquiries per session, mostly via mobile phones. Text mining identified student input terms encompassing not only science-related queries but also aspects of school life such as assessment scope. Topic modeling using BERTopic, based on Sentence-BERT, categorized 88% of student questions into 35 topics, shedding light on common student interests. A year-end survey confirmed the efficacy of the carousel format and the chatbot's role in addressing curiosities beyond integrated science learning objectives. This study underscores the importance of developing chatbots tailored for student use in public education and highlights their educational potential through long-term usage analysis.

• Proceedings of the Korean Institute of Information and Commucation Sciences Conference
• /
• 2022.10a
• /
• pp.84-86
• /
• 2022

Task-oriented chatbots prevail in various filed with the artificial intelligent dialogue system. The need for chatbots in customer services is growing, especially in education businesses given that there are many user inquiries and consultation requests. However, current dialogue systems only function as simple reactions or predetermined and frequently used actions. Meanwhile, the research about customized recommendation systems through artificial intelligence is very active with a wide variety of educational content. Although a dialogue system and a recommendation system is a core element in this domain, it has a limitation in that it is being conducted separately. Therefore, we present a study on a recommendation system that can recommend user-customized lectures combined with a dialogue system. With this combination, our system can respond to additional functions beyond these limitations. Through our research, we expect that work efficiency and user satisfaction will be improved by applying chatbots in education domains that are becoming more diversified and personalized.

• Annual Conference of KIPS
• /
• 2024.10a
• /
• pp.519-520
• /
• 2024

본 논문은 오픈소스 LLM(Large Language Model)인 Llama3를 기반으로 음성 인터페이스를 갖춘 맞춤형 로컬 챗봇 시스템을 개발하였다. 이 시스템은 PEFT(Parameter Efficient Fine-Tuning)와 AutoRAG(Auto Retrieval-Augmented Generation)로 최적화된 RAG(Retrieval-Augmented Generation) 방식을 결합한 하이브리드 접근법을 통해 Llama3를 전이학습 하였다. Ollama를 사용하여 로컬 환경에서 챗봇을 구현하였으며, LangServe와 Ngrok을 활용해 배포하였다. Raspberry Pi 5에 구현하여 모바일 환경으로 동작 가능하게 하였고 음성인식 기능을 추가하여 사용자 편의성을 높였다. 연구한 모델의 성능 평가는 총 18 종류의 데이터셋에 대해 각 질문당 5회씩, 총 90회의 질문으로 정확도를 측정하였다. 실험결과, PEFT 학습 모델과 Advanced RAG를 결합한 시스템이 가장 우수한 성능을 나타냈다.

• The Journal of the Korea institute of electronic communication sciences
• /
• v.19 no.6
• /
• pp.1281-1292
• /
• 2024

The purpose of this study is to investigate the use of large-scale language models (LLMs), which have been increasing in recent years, and the market size of RAG (Retrieval Augmented Generation) chatbots specialized for each company is constantly growing. However, the process of learning the necessary materials and creating a service using them is a trial-and-error and time-consuming task, and it requires a lot of burden on the company, such as building a high-performance server. In this paper, we developed a customized chatbot for a company using Google Gemma2 9b, an open source model, and LangChain, a RAG development framework. Jetson Xavier NX was used as a client for safe operation, and GPU server was used as a chatbot server. This paper presents the process of learning using RAG and the process of building the service operation of the learned chatbot as a management system.

• The Journal of the Korea Contents Association
• /
• v.22 no.2
• /
• pp.255-269
• /
• 2022

In this study, it was necessary to examine consumer acceptance attitudes from an in-depth and multifaceted perspective at a time when the need for chatbot services in various industrial fields is increasing and being activated in earnest. Accordingly, this study conducted a structural equation model to examine not only the structural relationship between ease, usefulness, and playfulness among the main functions of chatbot services and their acceptance attitudes, but also whether there is a mediating effect of service value in the relationship. As a result of the main study of this study, it was identified that the relationship between the ease, usefulness, and playfulness factors, which are the main functional sub-factors of the chatbot service, and their acceptance attitude and service value had a statistically static influence relationship. Based on these research results, the main research conclusions suggest that when companies in various fields provide chatbot services in the future, it is necessary to clearly determine the influencing factors that can affect the chatbot service acceptance attitude and provide these services. Through this, it is expected that the AI chatbot service will strengthen communication with consumers and establish itself as a customized and personalized counseling service.

• Journal of the Korea Academia-Industrial cooperation Society
• /
• v.21 no.5
• /
• pp.543-550
• /
• 2020

Interest in food recipe retrieval systems has been increasing recently. Most computer-based recipe retrieval systems are searched by cooking name or ingredient name. Since each recipe provides information in different weighing units, recalculations to the desired amount are necessary and inconvenient. This paper introduces a computer system that addresses these inconveniences. The system is a chatbot system, based on web-based recipe recommendations, for users familiar with the use of messenger conversation systems. After selecting the most popular recipes by their names, and pre-processing to extract only information required for the recipes, the system recommends recipes based on the 100,000 data. Recipes are then searched by the names of food ingredients (included and excluded). Recalculations are performed based on the number of servings entered by the user. A satisfaction rate for the systems' recommendations was 90.5%.

• The Mathematical Education
• /
• v.63 no.4
• /
• pp.767-789
• /
• 2024

This study conducted a scoping review of 19 studies on math chatbots conducted in South Korea. The study analyzed key research themes and types, mathematical content and its selection criteria, research methods, roles and design principles of chatbots, technologies employed in chatbots, and the roles of teachers and students in math chatbot research. Through this analysis, the current landscape and growth potential of the math chatbot research ecosystem were identified, leading to several directions for future research. First, 'studies on AI and convergence', 'exploratory research', 'research on geometry content', and 'literature reviews' should be expanded. Second, further studies should establish a consensus on the safety of personalized instructional chatbots and then develop technical solutions for their implementation. Third, additional studies are needed on teachers' roles as ethical supervisors and students' roles as ethical adherents regarding AI and chatbots. Fourth, future research should allocate adequate research resources to studies on teachers' noticing and specialized content knowledge (SCK) in chatbot-based teaching and learning environments, as well as the impact of chatbots on students' thinking processes. Finally, three research topics for a systematic literature review, which should be conducted once sufficient chatbot research has been accumulated, were suggested. The study provided specific directions to ensure the systematic and continuous growth of the math chatbot research ecosystem in South Korea through this analysis.