• KIISE Transactions on Computing Practices
• /
• v.22 no.5
• /
• pp.218-224
• /
• 2016

This paper proposes a computer-based practical automatic scoring system for Korean short answers through student answer analysis and natural language processing techniques. The proposed system reduces the overall scoring time and budget, while improving the ease-of-use to write answer templates from student answers as well as the accuracy and reliability of automatic scoring system. To evaluate the application of the automatic scoring system and compare to the human scoring process, we performed an experiment using the student answers of social science subject in 2014 National Assessment of Educational Achievement.

• Journal of Science Education
• /
• v.38 no.1
• /
• pp.168-181
• /
• 2014

The purpose of this study was to monitor evaluation in science education at primary and secondary school in Korea. In order to do this, the survey was administered. The subject was 518 teachers; 292 elementary school teachers, 110 middle school science teachers and 116 general high school science teachers. It was found that the ratio of paper examination increased while that of performance evaluation decreased according to the grade of school. In the science paper examination, the ratio of selection type item was higher than that of supply type item, and the ratio of item belonging to knowledge area was higher than that of item belonging to the inquiry area regardless of the grade of school, teaching career, and teacher's gender. The teachers recognized the necessity of supply type item, but they suffered from the reliability of the result of that item. And it was hard for them to do performance evaluation because it took much time and effort. This study suggested that the high-quality in-service teacher training be provided and laboratory assistants who help the science experiment and grading test papers be employed in order to fulfil the performance evaluation and increase the ratio of supply type item and items belonging to the inquiry area.

• Journal of The Korean Association For Science Education
• /
• v.43 no.1
• /
• pp.17-28
• /
• 2023

In line with the entry into the digital-based intelligent information society, the science curriculum emphasizes the cultivation of scientific competencies, and computer-based test (CBT) is drawing attention for assessment of competencies. CBT has advantages to develop items that have high fidelity, and to establish a feedback system by accumulating results into the database. However, it is necessary to solve the problems of improving validity of assessment results, lowering measurement efficiency, and increasing management factors. To examine students' responses to the introduction of the new assessment tools in the process of transitioning from paper-based test (PBT) to CBT, in this study, we analyzed the results of the PBT and the CBT conducted in 2021 National Assessment of Educational Achievement (NAEA). In particular, we sought to find the effects on student achievement when only the mode of assessment was changed without change of items, and the effect on student achievement when the items were composed including technology enhanced features that take advantage of CBT. This study is derived through the analysis of the results of 7,137 third-grade middle school students taking one among the three kinds of assessments, which were the PBT or two kinds of CBT. After the assessment, the percentage of correct answers and the item discriminations were collected for each group, and expert opinions on characteristics of response were collected through the expert council involving 8 science teachers with experience in NAEA. According to the results, there was no significant difference between students' achievement results in the PBT and the CBT-M, which means simple mode conversion type of CBT, so it could be explained that the mode effect did not appear. However, it was confirmed that the percentage of correct answers for the construct response items was somewhat high in the CBT, and this result was analyzed to be related to the convenience of the response. On the other hand, there were the items with a difference of more than 10%p from the correct answer rate of similar items, among the items to which technology enhanced functions were applied following the introduction of CBT. According to the analysis of response rate of options, these results could be explained that the students' level of understanding could be more closely grasped through the innovative items developed through the technology enhanced function. Based on the results, we discussed some guidance to be considered when introducing CBT and developing items through CBT, and presented implications.

• Journal of The Korean Association For Science Education
• /
• v.31 no.8
• /
• pp.1158-1174
• /
• 2011

In this study, we examined the compositions, basic principles, and the area of the National Higher Education Entrance Examination (GaoKao) in 2009, we also analyzed the categories and characteristics of items. Also, the GaoKao was analyzed in terms of test specifications, the number of items, item patterns, difficulty levels, and implications of the College Scholastic Ability Test(CSAT) were explored. Results show Natural Science section of the National Test 1, 2 are 300 points per 150 min, and Natural Science, and Chemistry of Shanghai is 150 points each per 120 min. Also, the GaoKao contained multiple choice and fill in the blanks questions, and the description items are composed of experiments of various types. The GaoKao Natural Science section is composed of physics, chemistry, biology but not earth science, which is different from the CSAT. GaoKao requires basic understanding or the observation ability to reasoning, the complex thinking ability, especially emphasized on the experiment ability. The range of possible questions is in the examination outline, not the curriculum, and the ratio of questions from the University level is high. In the analysis of the behavioral domain, the ratios of the understanding and application items is higher than the CSAT, and inquiry items is lower, but the inquiry items are deeper. In case of the ratio of the expected correct answer, National Test 1 and National Test 2 is similar, but the difficult items or about 20~39% of the test is 4~5 times to that of the CSAT, making the GaoKao very difficult. The peculiar characteristics of GaoKao is the emphasis on the experiment, and even though the practical items is of lower ratio, they are very useful in life.

• Journal of The Korean Association For Science Education
• /
• v.34 no.1
• /
• pp.1-9
• /
• 2014

This study aimed to explore the over-efficacy in problem solving and overconfidence of knowledge of students performing assessments in two different test formats: multiple-choice and supply-type. Two hundred and four female middle school students participated in this study. Multiple-choice and supply-type formats of tests on photosynthesis were used, and each item contained scales indicating one's self-efficacy on problem-solving and confidence of knowledge. The results showed that the correlation coefficients of performance between the two different assessment formats were less than 0.5 and the correlation coefficients between efficacy/confidence and actual performance were less than 0.45. Moreover, students tended to exhibit more over-efficacy and overconfidence in multiple-choice formats. The percentage of over-efficacy and overconfidence was higher in the group that completed the multiple-choice test first followed by the supply-type assessment than in the group that started with the supply-type followed by the multiple-choice assessment. From this study, it can be suggested that more use of supply-type assessment is required in science education. If test administrators require the combination of both multiple-choice and supply-type in an assessment, the supply-type assessment format should come first so that students can maintain the appropriate level of efficacy and confidence. In addition, science educators need to develop new learning programs to enhance students' self-monitoring skills of their problem-solving ability and knowledge.

• Journal of the Korean School Mathematics Society
• /
• v.18 no.4
• /
• pp.431-448
• /
• 2015

Elementary school teacher selection test has to be implemented for knowledge(MKT) to estimate and to be a step in the expansionary direction. For this, It has to be analyzed closely in terms of teacher's MKT first that questions set from selection tests until now. In this research, some standards were established by type of question, grade, contents field and subject from 84 mathematics questions set at examinations. And it was analyzed concretely by general tendency and produced the results of analysis related to elementary teacher's MKT in elementary school teacher selection test since year 2000. The results showed that descriptive and short answer type was mostly set(71.4%) in type of question and integrated grade type was mostly set(41.7%) in type of grade in elementary school teacher selection test. And in type of contents field, rated from highest to lowest, number and operations(31.0%), measurement(23.8%), figure(15.5%) were set. Textbook/manual(34.5%), solution of problem(26.2%), learning psychology(13.1%) in status of subject were followed in order, but there's no relationship with estimation. This research was based on a standard of precedent study and made an analysis of elementary mathematics questions of selection test since year 2000. Therefore, it has a meaning because of produced results from analysis related to elementary teacher's MKT.