• Journal of The Korean Association For Science Education
• /
• v.31 no.6
• /
• pp.887-900
• /
• 2011

Teachers use explanations to communicate important scientific ideas to students. Consequently, all biology teachers should be evaluated to determine how effective they are at constructing and communicating biological explanations. Open response questions are required to detect pre-service biology teachers' abilities to communicate robust and accurate scientific explanations. Nevertheless, multiple-choice questions are typically preferred by educators because of the common drawbacks of using open-response instruments, such as scoring time, inter-rater scoring disagreements, and delayed feedback to test takers. This study aims to measure pre-service biology teachers' competence in building scientific explanations and to investigate how accurately multiple-choice questions predict the results of open-response questions. One hundred twenty four pre-service biology teachers participated in the study and were administered 20 multiple-choice items and three open-response items designed to measure the accuracy and quality of their explanations of evolutionary change. The results demonstrated that pre-service teachers displayed higher competence when tested with multiple choice items than when tested with open response items. Moreover, scores derived from multiple-choice items poorly predicted the scores derived from open-response items. Multiple-choice items were also found to be poor measures of the consistency, purity and abundance of conceptual elements in teachers' evolutionary explanations. Additionally, many teachers held mixed-models composed of both scientific and naive ideas, which were difficult to detect using multiple-choice formats. Overall, the study indicates that multiple-choice formats are poorly suited to measuring several aspects of biology teachers' knowledge of evolution, including their ability to generate scientific explanations. This study suggests that open-response items should be used in teacher education programs to assess pre-service teachers' explanatory competency prior to being permitted to teach science to children.

• Journal of The Korean Association of Information Education
• /
• v.20 no.2
• /
• pp.197-206
• /
• 2016

A fill-in-the-blank with choices are widely used in classrooms in order to check whether students' understand what is being taught. Although there have been proposed many algorithms for generating this type of questions, most of them focus on preparing sentences with blanks rather than generating multiple choices. In this paper, we propose a novel algorithm for generating multiple choices, given a sentence with a blank. Because the algorithm is based on a statistical language model, we can generate relatively unbiased result and adjust the level of difficulty with ease. The experimental results show that our approach automatically produces similar multiple-choices to those of the exam writers.

• Journal of Science Education
• /
• v.37 no.3
• /
• pp.458-475
• /
• 2013

According to instruction of standard-based assessment system, teachers with higher level of competence in student assessment are required. The purpose of this study is to offer the basic data to enhance pre-service physics teachers' competency in student assessment. 17 pre-service physics teachers, who are sophomore in department of physics education, participated in the study for a whole semester. To do this, we gave the assignment of multiple choice question construction in physics conceptest for formative assessment to pre-service teachers and observed review questions constructed by them. We also analyzed multiple choice questions of 17 pre-service teachers. They had 5 classes for the review process of the question and improved their questions through it. 16 of 17 pre-service teachers completed test design. Our results show that pre-service teachers had a little low competence in test design for classroom assessment. In addition to, constructing and revising questions, almost of them didn't have various perspectives and didn't consider to get information about the pre-conceptions of students. These findings indicate that they have difficulties with constructing multiple choice question and more specific plan needs to enhance the overall student-assessment competency levels of pre-service teachers for successful settlement of the standard-based assessment system.

• Journal of Korean Elementary Science Education
• /
• v.23 no.4
• /
• pp.332-343
• /
• 2004

Multiple choice items have been widely used. However the difficulties in understanding and solving the items have not been known well. The purpose of this study is to analyze the difficulties and errors in the process of solving multiple choice items. Twelve multiple choice items were developed based on the Unit 5 Separation of Mixtures in the 4th grade. Four items which students had hardly given the correct answer were selected and six students were chosen for interview. Interview results were analyzed with regard to the errors in the process of solving the multiple choice items. The findings of this study are as follows: I) The students who misread and misunderstand the questions choose the incorrect answers. 2) Most of the students activate daily knowledge in the process of problem solving. 3) The students who have misconception with the daily knowledge or have no experiences choose incorrect answers, while students who activate both daily knowledge and school knowledge choose correct answer. 4) The students of high level commit errors mainly in the latter part of problem solving process, but the students of low level do from early.

• Journal of Korean Elementary Science Education
• /
• v.22 no.1
• /
• pp.15-28
• /
• 2003

The purpose of this study was to explore the relationships between the concept of heat and that of temperature for elementary students. Eight multiple choice type questions with explanation of reasons for selection were developed based on previous researches and the analysis of science curriculum for elementary students. The students of 9, 10, 11 years(n=292) were selected from two elementary schools in Taegu City. The responses of students' multiple choice and their explanations were analysed in each items χ² test used for the relationships between types of heat and temperature conceptions Half of elementary student could discriminate the two terms of heat and temperature, majority of them thought that heat is likely to be hot and temperature is the quantity of heat, which is not based on scientific conception. Elementary students thinkings about heat could be classified with material type of heat and molecular kinematics type. Material type of heat were more popular than with molecular kinematics type, although the latter is increased. Majority of students answered correctly in qualitative questions of mixing of hot and cold water, but about only one third of them answers in quantitative questions. Subtraction of cold temperature from hot temperature was the most popular explanation, even though one-quarter of students summed up the two temperature in quantitative situation of mixing hot and cold waters. Those who thought heat as the molecular kinetic responded more correctly in most difficult questions than those who as the material. Therefore, we concluded that the types of heat conceptions affected the concept formation of temperature.

• Journal of Engineering Education Research
• /
• v.24 no.6
• /
• pp.30-39
• /
• 2021

The present study aims to demonstrate that it is possible to objectively evaluate the competency (referred to as graduate attributes or program outcomes) of graduates of engineering education programs. To strengthen the link between engineering education accreditation and the qualification/certification system of engineering professionals, referred to as K-Engineering Professional (K-EP), individually assuring the quality of accredited graduates using multiple-choice test as main assessment tool is proposed. Test questions related to the basic vocational skills of NCS are developed for seven of the 10 program outcomes of the ABEEK KEC2015. The three program outcomes, PO1, PO3, and PO5, which need to fully accommodate the characteristics of each disciplinary field, are excluded. A pilot test involving graduates of eight accredited programs is conducted. Applying on Rasch model based on Item Response Theory (IRT), the item difficulty, fit and discrimination of multiple choice test are demonstrated. The pilot study strongly suggests that individual competency evaluation is possible at a certain level for seven program outcomes tested. For PO1, PO3, and PO5, however, questions that address the characteristics of each disciplinary field need to be devised. If a suitable pool of questions is built, it can be used as a program outcomes assessment tool by the accredited programs.

• Asia-pacific Journal of Multimedia Services Convergent with Art, Humanities, and Sociology
• /
• v.7 no.1
• /
• pp.253-263
• /
• 2017

As needs for knowledge on Korean history and the attention of the people are rapidly increasing, various smartphone applications for learning the history have appeared during recent years. These applications provide multiple choice questions to users through their own problem banks. But, since these questions are selected from the fixed set of problems that are stored previously, the learning efficiency of users is inevitably decreased when they use the applications repeatedly. In this paper, we present a question generation system named K-History which generates multiple choice questions in an automatic way using the database on the history of Korea. In addition, we also describe the development of the application Korean History Infinite Challenge as a learning application for Korean history. To develop K-History, we classify typical types of learning problems through various problems based on Korean history learning materials, proposing algorithms to generate problems according to the types found. Through the developed techniques, various learning systems can reduce the cost for creating questions, while increasing the learning efficiency of users.

• The KIPS Transactions:PartA
• /
• v.15A no.2
• /
• pp.125-134
• /
• 2008

We implemented a system being able to suggest example sentences for multiple choice tests, considering the level of students. To build the system, we designed an automatic method for sentence generation, which made it possible to control the difficulty degree of questions. For the proper evaluation in the multiple choice tests, proper size of question pools is required. To satisfy this requirement, a system which can generate various and numerous questions and their example sentences in a fast way should be used. In this paper, we designed an automatic generation method using a linguistic resource called WordNet. For the automatic generation, firstly, we extracted keywords from the existing sentences with the morphological analysis and candidate terms with similar meaning to the keywords in Korean WordNet space are suggested. When suggesting candidate terms, we transformed the existing Korean WordNet scheme into a new scheme to construct the concept similarity matrix. The similarity degree between concepts can be ranged from 0, representing synonyms relationships, to 9, representing non-connected relationships. By using the degree, we can control the difficulty degree of newly generated questions. We used two methods for evaluating semantic similarity between two concepts. The first one is considering only the distance between two concepts and the second one additionally considers positions of two concepts in the Korean Wordnet space. With these methods, we can build a system which can help the instructors generate new questions and their example sentences with various contents and difficulty degree from existing sentences more easily.

• The Journal of Korean Association of Computer Education
• /
• v.22 no.3
• /
• pp.55-65
• /
• 2019

In programming language learning system of flip learning methods, the evaluation of cyber lectures generally proceeds from online to multiple choice questions. In this case, the questions are randomly extracted from the question bank and given to individual learners. In order for these evaluation results to be reflected in the grades, the equity of the examination question is more important than anything else. Especially in the programming language subject, the degree of difficulty that learners think can be different depending on the type of problem. In this paper, we classify the types of multiple-choice problems into two categories, and manage the difficulty level by each type. And we propose a question selection algorithm that considers both difficulty level and type of question. Considering the characteristics of the programming language, experimental results show that the proposed algorithm is more efficient and fair than the conventional method.

• The Journal of Korean Medicine
• /
• v.34 no.1
• /
• pp.69-79
• /
• 2013

Objectives: The aim of this survey was to investigate utilization, intended use, problems with and demand for medical devices by surveying members of the traditional Korean medical society. Methods: We distributed questionnaires to 13,957 traditional Korean medical doctors via e-mail, and received replies from 1,225. The questionnaire consisted of 4 multiple-choice questions for survey respondent information, 8 multiple-choice questions about the status of medical devices utilizing, and a short answer question about the demand for medical devices. Results: Use of medical devices in traditional Korean medical clinics is common. Diagnostic medical devices are mainly used to assess the patient's condition and to establish a close rapport with clients. In case of therapeutic medical devices, they are usually used for secondary treatment. Issues with traditional Korean medical devices currently in use were ineligibility for national health insurance, low reliability, uncertain validity, and high price. In development of traditional Korean medical equipment, the need for diagnostic medical devices was greater than for therapeutic, and the need for the recording and analysis of medical image data and visualization of medical information was great. Conclusions: There is growing demand for facilitating the development and commercialization of traditional Korean medical devices. To satisfy this demand, research on evaluation indicators that reflect functional and structural clinical information and how to clinically assess the indicators should proceed.