• The Mathematical Education
• /
• v.43 no.2
• /
• pp.151-162
• /
• 2004

For the past several years, performance assessment has been widely used by mathematics teachers. The superiority of performance assessment items compare to multiple choice items has been discussed by many researchers, however these discussions tend to be lack of empirical data. Thus, this study aims to examine the effectiveness of tree response items in comparison with multiple choice items. Using the information function in Item Response Theory(IRT), item information of free response items and multiple choice items from the Third International Mathematics and Science Study-Repeat(TIMSS-R) were obtained and compared. Test informations of the whole mathematics area as well as each content area of mathematics were computed. On average, tree response items yielded more information than multiple choice items, especially in measurement and data interpretation. This study also revealed that free response items estimated students' mathematics ability more accurately than multiple choice items with smaller number of items.

• Journal of Korean Elementary Science Education
• /
• v.23 no.4
• /
• pp.332-343
• /
• 2004

Multiple choice items have been widely used. However the difficulties in understanding and solving the items have not been known well. The purpose of this study is to analyze the difficulties and errors in the process of solving multiple choice items. Twelve multiple choice items were developed based on the Unit 5 Separation of Mixtures in the 4th grade. Four items which students had hardly given the correct answer were selected and six students were chosen for interview. Interview results were analyzed with regard to the errors in the process of solving the multiple choice items. The findings of this study are as follows: I) The students who misread and misunderstand the questions choose the incorrect answers. 2) Most of the students activate daily knowledge in the process of problem solving. 3) The students who have misconception with the daily knowledge or have no experiences choose incorrect answers, while students who activate both daily knowledge and school knowledge choose correct answer. 4) The students of high level commit errors mainly in the latter part of problem solving process, but the students of low level do from early.

• Journal of The Korean Association For Science Education
• /
• v.18 no.4
• /
• pp.643-650
• /
• 1998

In order to assess students' higher mental abilities, such as scientific inquiry thinking skills, the essay type items would be more adequate than the multiple choice itmes. However, due to the present condition in which a huge number of students take the examination at the same time, it is inevitable to use the multiple choice type. For this reason, it is necessary to develop a new type of multiple choice items which can reduce the disadvantages of the traditional multiple choice type and can achieve a similar level of validity as subjective type assessment. The three-tier multiple choice test items which can be used for a large sample of students and especially for scientific inquiry thinking abilities, are proposed and examined. The three-tier multiple choice test items asked firstly conclusion or the results of calculation or experimental apparatus, secondly the processes of calculation or of developing conclusion, thirdly asking relevant scientific concepts. For the item analysis, 1 point was given to the correct answer, while 0 point was given to the wrong one. The data were processed through the computer program developed in Turbo C 2.0 language with an IBM compatable personal computer. The average score in the sub-items asking for scientific concepts was lower than that in the sub-items asking for results or processes. The score of guessing by chance in the three-tier multiple choice items was only 0.13%, so that the probability of making correct answers by just guessing would be extremely low. The three-tier multiple choice items, even if they are objective items, are thought to assess thinking skills of the scientific inquiry meaningfully excluding the possibility of guessing by chance.

• Journal of The Korean Association For Science Education
• /
• v.31 no.6
• /
• pp.887-900
• /
• 2011

Teachers use explanations to communicate important scientific ideas to students. Consequently, all biology teachers should be evaluated to determine how effective they are at constructing and communicating biological explanations. Open response questions are required to detect pre-service biology teachers' abilities to communicate robust and accurate scientific explanations. Nevertheless, multiple-choice questions are typically preferred by educators because of the common drawbacks of using open-response instruments, such as scoring time, inter-rater scoring disagreements, and delayed feedback to test takers. This study aims to measure pre-service biology teachers' competence in building scientific explanations and to investigate how accurately multiple-choice questions predict the results of open-response questions. One hundred twenty four pre-service biology teachers participated in the study and were administered 20 multiple-choice items and three open-response items designed to measure the accuracy and quality of their explanations of evolutionary change. The results demonstrated that pre-service teachers displayed higher competence when tested with multiple choice items than when tested with open response items. Moreover, scores derived from multiple-choice items poorly predicted the scores derived from open-response items. Multiple-choice items were also found to be poor measures of the consistency, purity and abundance of conceptual elements in teachers' evolutionary explanations. Additionally, many teachers held mixed-models composed of both scientific and naive ideas, which were difficult to detect using multiple-choice formats. Overall, the study indicates that multiple-choice formats are poorly suited to measuring several aspects of biology teachers' knowledge of evolution, including their ability to generate scientific explanations. This study suggests that open-response items should be used in teacher education programs to assess pre-service teachers' explanatory competency prior to being permitted to teach science to children.

• Journal of The Korean Association For Science Education
• /
• v.22 no.1
• /
• pp.122-131
• /
• 2002

Recently, performance assessment that makes growing use of free response items in a large scale assessment has been emphasized. This study is an empirical examination of the effectiveness of free response items in comparison with multiple choice items. Using the information function in Item Response Theory (IRT) framework, item information of free response items and multiple-choice items from the Third International Mathematics and Science Study-Repeat (TlMSS-R) were obtained. Test information of the whole science area as well as each area of science contents was computed. On average, free response items yielded more information than multiple choice items, especially in earth science, physics, chemistry, and life science. This study also showed that free response items were appropriate for students in high science ability. Also, free response items estimated students' science ability more accurately than multiple choice items with smaller number of free response items.

• Journal of Korean Elementary Science Education
• /
• v.32 no.2
• /
• pp.139-158
• /
• 2013

This study investigated elementary students' (grade 4~6) learning progressions for water cycling drawn from iterative assessments using ordered multiple-choice (OMC) items. An assessment system, which consisted of construct map, item design, outcome space, and measurement model, was employed in this study to examine children's learning progressions. At the first stage of the assessment system, a construct map was designed on which children's conceptual understandings from naive to most sophisticated were represented. At the item design stage, 8 OMC items were drawn from the construct map. Each item option of the OMC items was scored from 0 to 3 according to its level of understanding at the stage of outcome space. As a measurement model, Rasch model, a branch of item response theory, was applied to interpreting the outcomes of the OMC items. This cycle of assessment system was furtherly implemented iteratively in order to elaborate on the first version of water cycling learning progression. In conclusion, children's understanding of water cycling could be described in two aspects: water distribution and water movement. We identified children's conjectural developmental pathways about water cycling existed from superficial and naive accounts to more complex and abstract accounts.

• Journal of The Korean Association For Science Education
• /
• v.24 no.6
• /
• pp.1143-1157
• /
• 2004

The purpose of this study was to develop HS-VOSTS(High School Students' Views On Science-Technology-Society), which is an instrument that monitors high school students' views on STS. For this study, 23 multiple-choice items were developed on four categories. First, a categorial scheme was constructed. The categorial scheme was based on many instruments that evaluate students' understanding on STS, literature review, and STS learning goal that many researchers had asserted. Second, the multiple-choice items were developed. The development of the items was carried out through four steps. In the first step, some pairs of statement on each subordinate category were formed. Next, the student statement questionaries which were based on the pairs of statement were responded by 772 students. In the second step, the response written by the students were analyzed to common viewpoints and the first multiple-choice items were constructed. In the third step, the semistructured interview with 28 high school students was implemented and the second multiple-choice items were constructed. In the fourth step, the final version of the instrument was developed through the analysis of the students' response on the second multiple-choice items. HS-VOSTS is an useful instrument for monitoring students' beliefs and viewpoints on STS topics. The results of the students' responses to HS-VOSTS items reflect the students' own thoughts directly. These results will give some implications for the curriculum developers, the authors of text books and the educational policy makers as well as the teachers.

• The Journal of Korean Association of Computer Education
• /
• v.23 no.1
• /
• pp.29-36
• /
• 2020

Various items were used as evaluation tools that identify the student's abilities accurately to confirm the completion of learning. Among them, the multiple choice item has the advantages of high objectivity and reliability in scoring, but it cannot remove the factor of guessing. In this paper, the multiple choice items are classified into two types according to the relationship between the questionnaire and the choices. One is the type used in the classical test theory with the probability of guessing 1/k for k choices and the other is the novel proposed type which introduces the concept of partial knowledge. In the proposed type, the probability of guessing when the number of knowledge i is (i+1)/k for k choices. Based on the assumptions of the previous theories about multiple choice items, we derive the guessing parameter about the proposed type. And we analyzed the effect of the guess on the correct answer in the existing type and the proposed type. This shows that the proposed type has more question guessing than the existing type.

• Journal of The Korean Association For Science Education
• /
• v.24 no.1
• /
• pp.17-28
• /
• 2004

The purpose of this study is to determine the patterns of differences between genders in science achievement. Eleventh grade, 140 female and male students were sampled from a school in Seoul. According to the analysis results of pilot study, 20 items were finally selected for the main study. To sharpen our interpretations of the factors and provide some confirmation, we supplemented the statistical analysis with a more detailed cognitive study of the items using think-aloud protocols and interviews with student test takers. The analysis of this study took into account the different item formats, contexts, and presentation styles. The findings are as follows: First, there was no significant gender difference between multiple-choice and open-ended items. Second, male students achieved significantly better in the context of everyday life in multiple-choice items. Third, male students favored items presented as written texts. Fourth, in problem-solving process, female students tend to apply their science concepts, whereas male students tend to apply their everyday experiences. The results of this investigation indicate that gender difference in science achievement depends heavily on item characteristics.

• Journal of the Korea Society of Computer and Information
• /
• v.23 no.12
• /
• pp.1-9
• /
• 2018

The multiple-choice multidimensional knapsack problem (MMKP) is a variant of the well known 0-1 knapsack problem, which is known as an NP-hard problem. This paper proposes a method for solving the MMKP using the integer programming-based local search (IPbLS). IPbLS is a kind of a local search and uses integer programming to generate a neighbor solution. The most important thing in IPbLS is the way to select items participating in the next integer programming step. In this paper, three ways to select items are introduced and compared on 37 well-known benchmark data instances. Experimental results shows that the method using linear programming is the best for the MMKP. It also shows that the proposed method can find the equal or better solutions than the best known solutions in 23 data instances, and the new better solutions in 13 instances.