• Proceedings of the Korean Society for Cognitive Science Conference
• /
• 2006.06a
• /
• pp.9-14
• /
• 2006

본고는 한국어 교육방식의 하나라고 할 수 있는 한국어 어휘를 대상으로 한문제 출제 방식에서 문제 은행식 출제 방식이 갖고 있는 여러 가지 문제점을 해소할 수 있는 하나의 방법으로서 한국어 어휘 학습 시스템을 위한 자동문제 생성 기술을 제시한다. 먼저 기존 한국어 어휘 문제의 문항 분석 결과를 바탕으로 8가지 어휘력 평가 유형 및 각 유형별 자동 문제 생성 패턴을 구축하고, 한국어 어휘에 대한 풍부한 정보를 담고 있는 국어사전을 기반으로 한 자동 한국어 어휘 문제 생성 기술을 제시한다.

• Journal of The Korean Association For Science Education
• /
• v.36 no.3
• /
• pp.389-397
• /
• 2016

This study aims to test the efficacy of English-based automated computer scoring models and machine translation to score Korean college students' written responses on natural selection concept items. To this end, I collected 128 pre-service biology teachers' written responses on four-item instrument (total 512 written responses). The machine translation software (i.e., Google Translate) translated both original responses and spell-corrected responses. The presence/absence of five scientific ideas and three $na{\ddot{i}}ve$ ideas in both translated responses were judged by the automated computer scoring models (i.e., EvoGrader). The computer-scored results (4096 predictions) were compared with expert-scored results. The results illustrated that no significant differences in both average scores and statistical results using average scores was found between the computer-scored result and experts-scored result. The Pearson correlation coefficients of composite scores for each student between computer scoring and experts scoring were 0.848 for scientific ideas and 0.776 for $na{\ddot{i}}ve$ ideas. The inter-rater reliability indices (Cohen kappa) between computer scoring and experts scoring for linguistically simple concepts (e.g., variation, competition, and limited resources) were over 0.8. These findings reveal that the English-based automated computer scoring models and machine translation can be a promising method in scoring Korean college students' written responses on natural selection concept items.

• The Mathematical Education
• /
• v.63 no.2
• /
• pp.187-207
• /
• 2024

In this study, we explored the feasibility of automatically scoring descriptive assessment items using GPT-4 based ChatGPT by comparing and analyzing the scoring results between teachers and GPT-4 based ChatGPT. For this purpose, three descriptive items from the permutation and combination unit for first-year high school students were selected from the KICE (Korea Institute for Curriculum and Evaluation) website. Items 1 and 2 had only one problem-solving strategy, while Item 3 had more than two strategies. Two teachers, each with over eight years of educational experience, graded answers from 204 students and compared these with the results from GPT-4 based ChatGPT. Various techniques such as Few-Shot-CoT, SC, structured, and Iteratively prompts were utilized to construct prompts for scoring, which were then inputted into GPT-4 based ChatGPT for scoring. The scoring results for Items 1 and 2 showed a strong correlation between the teachers' and GPT-4's scoring. For Item 3, which involved multiple problem-solving strategies, the student answers were first classified according to their strategies using prompts inputted into GPT-4 based ChatGPT. Following this classification, scoring prompts tailored to each type were applied and inputted into GPT-4 based ChatGPT for scoring, and these results also showed a strong correlation with the teachers' scoring. Through this, the potential for GPT-4 models utilizing prompt engineering to assist in teachers' scoring was confirmed, and the limitations of this study and directions for future research were presented.

• Annual Conference on Human and Language Technology
• /
• 2006.10e
• /
• pp.15-21
• /
• 2006

본 논문에서는 올바른 한국어 생활과 한국어 실력 향상을 위하여, 한국어 어휘에 대한 풍부한 정보를 담고 있는 한국어사전, 사용자 어휘지능망(User-Word Intelligent Network : U-WIN)등의 언어자원을 이용한 자동문제생성기술을 소개하고, 이를 이용한 한국어 어휘학습시스템을 제시한다. 대부분의 학습시스템에서 사용하는 문제 은행식 출제 방식의 문제점을 해소할 수 있는 하나의 방법으로서, 기존의 한국어 어휘문제의 문항을 분석하여 8가지 문제 유형으로 재편성하고, 각 유형별 자동 문제 생성패턴에 따라 언어자원이 가지고 있는 한국어 어휘의 형태적 정보, 의미적 정보를 이용하여 한국어 어휘 문제를 자동 출제하는 한국어 어휘학습시스템을 구현하였다.

• KIPS Transactions on Software and Data Engineering
• /
• v.1 no.2
• /
• pp.91-102
• /
• 2012

Computer aided language learning has become popular. But the level of automation of constructing a Korean learning assistant system is not so high because a practical language learning system needs large scale knowledge resources, which is very hard to acquire. In this paper, we propose a Korean learning assistant system that utilizes easily obtainable knowledge resources like a corpus, web documents and a lexicon. Our system has three modules - problem solving, pronunciation marker and writing assistant. Automatic problem generator uses a corpus and a lexicon to make problems with one correct answer and three distracters, then verifies their suitability by utilizing frequency information from web documents. We analyze pronunciation rules for a pronunciation marker and recommend appropriate words and sentences in real-time by using data extracted from a corpus. In experiment, we evaluate 400 automatically generated problems, which show 89.9% problem suitability and 64.9% example suitability.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2009.01a
• /
• pp.153-158
• /
• 2009

SCORM(Sharable Content Object Reference Model)은 세계 e-Learning 표준화 분야에서 가장 주목을 받고 있는 ADL(Advanced Distributed Learning)의 표준화 모델이다. SCORM2004 RTE(Run-Time Environment) 에서 상호작용 데이터 모델(Interaction Data Model)의 기능을 활용하면 LMS(Learning Management System)가 문항을 자동 생성하여 문제은행을 보다 쉽게 구현할 수 있다. 내용학습 후에 형성평가를 실시하기 위한 문항을 학습자가 원하는 만큼 공급할 수 있다. 본 연구는 일반계 고등학교 수학교과의 삼각함수 성질을 학습하는 데 있어 RTE의 상호작용 데이터 모델로 구현한 문제은행을 갖춘 반복학습 콘텐츠를 개발하여 학습효과를 높이고자 한다.

• KIISE Transactions on Computing Practices
• /
• v.22 no.5
• /
• pp.218-224
• /
• 2016

This paper proposes a computer-based practical automatic scoring system for Korean short answers through student answer analysis and natural language processing techniques. The proposed system reduces the overall scoring time and budget, while improving the ease-of-use to write answer templates from student answers as well as the accuracy and reliability of automatic scoring system. To evaluate the application of the automatic scoring system and compare to the human scoring process, we performed an experiment using the student answers of social science subject in 2014 National Assessment of Educational Achievement.

• The Mathematical Education
• /
• v.63 no.2
• /
• pp.165-186
• /
• 2024

Despite the growing attention on artificial intelligence-based automated scoring technology as a support method for the introduction of descriptive items in school environments and large-scale assessments, there is a noticeable lack of foundational research in mathematics compared to other subjects. This study developed an automated scoring model for two descriptive items in first-year middle school mathematics using the Random Forest algorithm, evaluated its performance, and explored ways to enhance this performance. The accuracy of the final models for the two items was found to be between 0.95 to 1.00 and 0.73 to 0.89, respectively, which is relatively high compared to automated scoring models in other subjects. We discovered that the strategic selection of the number of evaluation categories, taking into account the amount of data, is crucial for the effective development and performance of automated scoring models. Additionally, text preprocessing by mathematics education experts proved effective in improving both the performance and interpretability of the automated scoring model. Selecting a vectorization method that matches the characteristics of the items and data was identified as one way to enhance model performance. Furthermore, we confirmed that oversampling is a useful method to supplement performance in situations where practical limitations hinder balanced data collection. To enhance educational utility, further research is needed on how to utilize feature importance derived from the Random Forest-based automated scoring model to generate useful information for teaching and learning, such as feedback. This study is significant as foundational research in the field of mathematics descriptive automatic scoring, and there is a need for various subsequent studies through close collaboration between AI experts and math education experts.

• Journal of Convergence for Information Technology
• /
• v.9 no.10
• /
• pp.154-162
• /
• 2019

Despite the development of information and communication technology, which has a great impact on society and culture, there is no platform that provides a systematic classification and state-of-the-art information retrieval system on the traditional culture. Therefore, this paper developed a traditional culture retrieval system capable of convergence services with systematic classification and retrieval based on automatically generate and visualize the 3D timeline. This system provides the function of collecting traditional culture contents, classifying and storing collected traditional culture contents, and automatically generating 3D digital timeline based on stored traditional culture contents. In addition, a system satisfaction questionnaire was developed to evaluate the usability of the system, and 19 students participated in verifying the system. As a result of the experiment, the satisfaction of the system showed that all items were 'satisfied' on average.