• Journal of the Korean School Mathematics Society
• /
• v.17 no.3
• /
• pp.409-435
• /
• 2014

This study examines relations between the 5th graders' fraction operation skills and error types on constructed-response items. As results, first, the participants have lower fraction operation skills on 'multiplication of fraction' than 'addition and subtraction of fraction'. Second, the participants have different error types depend on their constructed-response items. Most of error types which group with high ability made was 'leap of solving process', both groups error type with medium ability as well as low ability is 'misunderstanding of questions'. Third, the operation skills on 'addition and subtraction of fraction' have an influence on their operation skills on 'multiplication of fraction', and error types of 'understanding of questions' and 'understanding of solving process' have the most effects on the influence.

• Journal of Elementary Mathematics Education in Korea
• /
• v.14 no.3
• /
• pp.885-905
• /
• 2010

This study questions that mathematical evaluations strive to memorize fragmentary knowledge and have an objective test. To solve these problems on mathematical education We did descriptive test. Through the descriptive test, students think and express their ideas freely using mathematical terms. We want to know if that procedure is correct or not, and, if they understand what was being presented. We studied this because We want to analyze where and what kinds of faults they committed, and be able to correct an error so as to establish a correct mathematical concept. The result from this study can be summarized as the following; First, the mistakes students make when solving the descriptive tests can be divided into six things: error of question understanding, error of concept principle, error of data using, error of solving procedure, error of recording procedure, and solving procedure omissions. Second, students had difficulty with the part of the descriptive test that used logical thinking defined by mathematical terms. Third, errors pattern varied as did students' ability level. For high level students, there were a lot of cases of the solving procedure being correct, but simple calculations were not correct. There were also some mistakes due to some students' lack of concept understanding. For middle level students, they couldn't understand questions well, and they analyzed questions arbitrarily. They also have a tendency to solve questions using a wrong strategy with data that only they can understand. Low level students generally had difficulty understanding questions. Even when they understood questions, they couldn't derive the answers because they have a shortage of related knowledge as well as low enthusiasm on the subject.

• The Mathematical Education
• /
• v.63 no.2
• /
• pp.187-207
• /
• 2024

In this study, we explored the feasibility of automatically scoring descriptive assessment items using GPT-4 based ChatGPT by comparing and analyzing the scoring results between teachers and GPT-4 based ChatGPT. For this purpose, three descriptive items from the permutation and combination unit for first-year high school students were selected from the KICE (Korea Institute for Curriculum and Evaluation) website. Items 1 and 2 had only one problem-solving strategy, while Item 3 had more than two strategies. Two teachers, each with over eight years of educational experience, graded answers from 204 students and compared these with the results from GPT-4 based ChatGPT. Various techniques such as Few-Shot-CoT, SC, structured, and Iteratively prompts were utilized to construct prompts for scoring, which were then inputted into GPT-4 based ChatGPT for scoring. The scoring results for Items 1 and 2 showed a strong correlation between the teachers' and GPT-4's scoring. For Item 3, which involved multiple problem-solving strategies, the student answers were first classified according to their strategies using prompts inputted into GPT-4 based ChatGPT. Following this classification, scoring prompts tailored to each type were applied and inputted into GPT-4 based ChatGPT for scoring, and these results also showed a strong correlation with the teachers' scoring. Through this, the potential for GPT-4 models utilizing prompt engineering to assist in teachers' scoring was confirmed, and the limitations of this study and directions for future research were presented.

• Journal of the Korean School Mathematics Society
• /
• v.11 no.4
• /
• pp.543-567
• /
• 2008

The current 2007 Mathematics Curriculum in Korea emphasizes mathematical problem solving, advanced mathematical thinking and effective mathematical communicating. Therefore, in order to assess and evaluate these important thinking attributes, performance evaluation using descriptive and essay type of assessments are emphasized. In this paper, analysis of the elementary mathematics curriculum is used to develop descriptive assessment problems and grading rubrics that could be used objectively and consistently by teachers of the grade 6 school mathematics. The assessment problems were developed, pilot tested, revised, implemented and analyzed in detail to understand the overall effectiveness of the descriptive evaluation method in school mathematics.

• Journal of The Korean Association For Science Education
• /
• v.24 no.3
• /
• pp.519-531
• /
• 2004

In recent years, there has been an increased emphasis on performance assessment to evaluate students' abilities. Our nation has introduced a change in testing and assessment. Additional work on the efficacy, reliability, and comparability in order to develop the performance assessment item has been needed in the enforcement of the 7th National Science Curriculum. Also, criteria for professional and technical standards has been needed to be developed. The purpose of this study was to draw out various key concepts and to develop achievement standards, assessment standards and performance assessment items based on the 7th National Science Curriculum on the subject matter of reproduction(chapter 13) and biological accumulation(chapter 17). And also, this study examined the validity of completed performance assessment items based on classical test theory and polytomous item response theory. Twelve key concepts in chapter 13(reproduction) and four from chapter 17(biological accumulation) were abstracted. Twenty-six achievement standards in chapter 13(reproduction), and nine in chapter 17(biological accumulation) were developed. The achievement standards were determined in terms of knowledge(K), process skill(P) and attitude(A). Twenty-five assessment standards in chapter 13(reproduction) and nine in chapter 17(biological accumulation) were developed. Based on the developed achievement standards and assessment standards, twenty-two performance assessment items(seventeen open-ended questions, three essays, and two portfolios) with concrete grading criteria were developed. Eight open-ended items were applied to 240 10th graders to evaluate reliabilities of the test which consisted of four items per each chapter. The results would be suggested that the applied items were valid for performance assessment because item difficulties and item discriminations were proper. There was not much differences in item discrimination between interpretation from classical test theory and that from polytomous item response theory. However, there were some differences in item difficulties between the interpretations of two theories because the characteristics of examinees were reflected in classical test theory.

• Journal of the Korean School Mathematics Society
• /
• v.15 no.2
• /
• pp.239-258
• /
• 2012

The present study examined the differences of system and problems of descriptive assessment at the national, district level, and school's levels between Korea and U.S.A. (focused on NCTM perspective). As results, both Korea and U.S.A. showed that their goals for descriptive assessment at the national level pointed out the importance of assessment of process rather than result. In addition, States of U.S.A. demonstrated concrete rubrics and examples to help many school teachers to use easily, while many Districts of Education in Korea presented implementation rate in school in order to give a official direction to teachers.

• Journal of the Korean School Mathematics Society
• /
• v.11 no.3
• /
• pp.377-397
• /
• 2008

In today's secondary school mathematics curriculum, assessment is to be focused on evaluating the student's mathematical thinking rather than finding the correct solution to the problem. A descriptive evaluation method is therefore introduced to the school districts and suggested as an alternative assessment method in K-12 school mathematics. Descriptive evaluation method is widely available for use in schools, but there are barriers to using it since the teachers are forced to use the method by the school districts. In this research, we surveyed 120 secondary school mathematics teachers in Seoul and the surrounding metropolitan area to understand the status and the perception about using descriptive evaluation. The goal of the study was to find and understand the direct implications of using the new assessment method in secondary mathematics classes. The study showed that most of the mathematics teachers used the descriptive assessment method during their regular exam periods which is given twice per semester. Most of the open-ended problems used for descriptive evaluation were medium or high level math questions which were graded by the teachers at least 3 times to ensure objective evaluation. The teachers stated that objectivity in grading, administrative situation, and development of descriptive problems were the difficult barriers in descriptive evaluation. The teachers in the survey also commented that teachers' administrative responsibility should be reduced and that school environment in general should be improved for the new assessment method to become successful. Finally, the study showed that development of more descriptive problems with specific grading guidelines need to be developed for each grade level.

• Journal of The Korean Association of Information Education
• /
• v.20 no.3
• /
• pp.255-262
• /
• 2016

In this paper, we proposes a method to effectively evaluate learner competencies of computational thinking. Developed descriptive performance assessment tool is the use of assessment items presented in a creative computing MIT. This tool is selected and modified items through expert delphi technique. The performance assessment tool is composed of four areas that are an experimenting and iterating, testing and debugging, reusing and remixing and abstracting and modularizing. We apply to software classes for the pre-service teachers and conducted this descriptive performance assessment. Applying result of the assessment, the developed assessment tool shows the reliability $Cronbach-{\alpha}$ values higher than 0.6. In addition, the validity of the test results is higher in experts delphi test. Descriptive assessment tool suggested in this study are identified as a useful assessment tool to effectively measure student CT competencies.

• Annual Conference on Human and Language Technology
• /
• 2012.10a
• /
• pp.121-122
• /
• 2012

컴퓨터를 사용한 서술형 답안의 자동채점은 채점의 편의성과 객관성을 제고하기 위하여 많은 연구자들이 연구해 왔으며 자동채점의 성능을 향상시키기 위해 여러 가지 방법들이 제안되었다. 본 논문은 서술어 정규화를 통하여 서술형 답안의 자동채점 정확도를 높이고자 하였다. 기존의 다른 채점 방법들과 비교했을때 서술어 정규화 기법을 적용한 채점 방식은 기존의 방법들보다 유사도 계산 정확도가 향상되어 정답 판별 정확도가 향상되는 것을 확인할 수 있었다. 서술어 정규화는 기존의 모든 서술형 답안 채점 방법에 추가적으로 적용할 수 있는 범용성을 가지고 있다. 따라서 서술어 정규화는 기존 방법들의 자동채점 정확도를 향상시켜 보다 정확하게 서술형 답안을 채점할 수 있다.

• Journal of Educational Research in Mathematics
• /
• v.23 no.4
• /
• pp.533-551
• /
• 2013

The purpose of this study is to investigate how middle school mathematics teachers understand the definition and the purpose of constructed-response assessments as well as what they expect from the assessments. By examining the responses of 189 middle school mathematics teachers to the survey, we found that there is discrepancy between their perceptions and reality of the constructed-response assessments in practice. In other words, teachers' practice is not well aligned with their perceptions and expectations due to the restraint of school system and environment. The findings give us some meaningful implication for the improvement of effective constructed-response assessments and professional development.