• Journal of The Korean Association For Science Education
• /
• v.28 no.1
• /
• pp.32-46
• /
• 2008

The purpose of this study was to analyse the qualitative and quantitative characteristics of paper-pencil tests for exploring its appropriateness as a selection tool of the gifted in science. For this purpose, we developed two (internal and external) item analysis frameworks, and applied these frameworks to analyse qualitative characteristics. Also, we analysed the relationship between two characteristics. The results of analysing qualitative characteristics revealed that the portion of items with acceleration context exceeding middle school curriculum level was relatively large, which caused low content validity. Furthermore, there was considerable deviation in content and context by subject matter and year, which caused test unstability. Items measuring knowledge domain was the most prevalent, and too much weight on data interpretation & analysis domain in inquiry process skills. In case of creativity test, the portion of items measuring convergent thinking was much larger than that of divergent or associative thinking. Most of these items were represented by using pictures and tables rather than using graphs. Item types of multiple-choice and short answers were superior to essay types. Discrimination index, on the whole, was appropriate (above 0.3), but item difficulty showed a vast deviation ($0.01{\sim}0.90$). Correlation coefficients among subject matters and test tools were very low, and test reliabilities were also low. Low item difficulty & high discrimination index item types were distinguishable. Items with acceleration context were more discriminating than enrichment context. Implications of developing quality paper-pencil test items in the selection of gifted students are discussed.

• (The) Korean Journal of Educational Psychology
• /
• v.23 no.3
• /
• pp.645-663
• /
• 2009

The purpose of this study is to develop and verify a Korean Diagnostic Scale of Multiple Intelligence(MI), which will be an alternative test to avoid problems with former Shearer's MI test and to adopt H. Gardner's suggestions to develop MI assessment. The test is developed 5 types; kindergartner, elementary lower grader, elementary upper grader, middle schooler, high schooler test. A form of test is diversified with 3 types; multiple-choice items for accomplishment, true or false items for ability, and self-reported items with likert scale for interest and ability. According to H. Gardner's suggestions, we have tried to reanalyze key component of MI, analyze an overlapping or hierarchical relationship between intelligences, develop intelligences-fair items, diversify form of item. We have developed a final standardized test through a primary, secondary preliminary-test analysis, and sampled 5,585 students by age, gender, and regional groups. As a result of this sampling test, we can get a norm score and compare individuals with other's score relatively. To verify this test, we analyzed behavior observation, mean, standard deviation, a percentage of correct answers, reliability of each test type, correlation between intelligence scales, Kruskal-Wallis test of mean rank of career choice by intelligences. As a result of correlation analysis between sub-intelligence scales, we can conclude that this MI test is satisfied with intelligence independent assumption. Besides, as non-parametric statistics test(Kruskal-Wallis) of career choice by intelligences, we can identify that MI is related with domain of career choice. This test is not a linguistic and logical-mathematical biased test but a intelligences-fair test. It makes us compare individual's potential with a norm score. Besides, it could be useful as a means of educational prescription or counsel in comparison with ability, interest, and accomplishment of individual. But this test is limited to do factor or correlation analysis between types of sub-test, because items are minimized for a time-constraint and a heavy burden of test receiver. But if it could be tested with increased items by two sessions, further research could be expected to get over this constraints and do a further validation analysis.

• Journal of The Korean Association For Science Education
• /
• v.36 no.6
• /
• pp.815-833
• /
• 2016

This study investigated children's (grade 3 to 9) responses to assessment items on energy as a cross-cutting concept in order to get basic information for a learning progression. The assessment consisted of 8 ordered multiple-choice items at the contexts of electric circuit, mechanical energy of falling objects, phase change of matter, dissolution, biological phenomena of a lizard, food chain, radiative equilibrium between Sun and Earth, and the system of water cycling. Children's responses to each item were analyzed with using cross-tabulations in terms of grades and item option levels and Wright map and Differential item functioning based on Rasch modeled item response analysis. The results offered empirical evidence of children's development of understanding energy from relation between energy and its phenomena, types of energy, transfer and conversion of energy, towards conservation and equilibrium of energy for all of eight contexts. Children of each grade did not fully understand energy conservation. As grade goes up, their understandings of energy transfer and conversion were differentiated across the contexts and topics of energy. According to Rasch analysis, children had easier understanding of energy on dissolution and poorer understanding of energy on water cycling than that on other contexts. It was discussed and suggested that the results of this study help us organize science topics with regard to energy when developing new national science curriculum.

• School Mathematics
• /
• v.13 no.1
• /
• pp.89-105
• /
• 2011

To provide some suggestions on the setting of mathematics test in the College Scholastic Ability Test(CSAT), this paper analyses the result of mathematics test in the CSAT from 2005 to 2011, on which the 7th national mathematics curriculum has been applied. From the result, four suggestions are drawn out. First, the mathematics test needs to be easier to reduce the burden of test-taker. Accordingly, the number of items and their scores need to be adjusted. Second, the proportion of multiple-choice items has to be reduced whereas that of short-answer items has to be increased to enhance the function of the CSAT as a selection test. Third, the sub-item system needs to be adopted. Fourth, new item-types have to be developed.

• Journal of Educational Research in Mathematics
• /
• v.25 no.1
• /
• pp.1-19
• /
• 2015

This paper aims to explore what we can improve in the curriculum, teaching-learning, and evaluation on the bases of the analyses of multiple-choice items set in National Assessment of Educational Achievement. For this goal, by using the distribution curves of the responses percentages, we will grasp the features of educational achievement which appear to students through an in-depth analysis about not only item itself but also the contents included in particular distracters. These analyses provide more information than the descriptive statistical values such as the mean of correct answer percentage and the discrimination of whole group and the mean of responses percentages of replies of subgroups. Because the distribution curves of the responses percentages reveal the transition from the lowest to the highest educational achievement very well. From these analyses we acquire the implications about the concept of prime factor or prime factorization, ratio(proportion) such as velocity, linear function, volume of cone, properties of solid figure, and probabilities of empty event and total event.

• Proceedings of the Korean Association for Survey Research Conference
• /
• 2002.06a
• /
• pp.8-34
• /
• 2002

This paper is concerned with evaluating the fidelity of a non-cognitive test adaptation for use in multiple languages and cultures using two differential item functioning(DIF) techniques: (a) PSIBTEST, and (b) Logistic Discriminant Function Analysis(LDFA). In particular, this study focused on how DIF research can best be extended to the problem of evaluating the equivalence of tests across cultures and languages. The Sixteen Personality Factor (16PF) questionnaire was administered in English to 844 American college students and in Korean to 538 Korean college students. This study attempted to identify the best matching criterion for the translated tests by using both a multivariate matching technique and iterative purification process. The results generally showed a small number of DIF items on each scale, except for scales A and N where about half of the items showed DIF. The choice of matching variables based on a combination of internal measures appeared to have little effect and the iterative purification method was unsuccessful. Finally, the results were discussed and methodological implications were also presented.

• Archives of design research
• /
• v.18 no.3 s.61
• /
• pp.191-198
• /
• 2005

Menu is the most common interaction tool to select and execute a specific menu item from multiple menu options. With the very rapid increasing amount of information, various new menu designs have been developed. In this research, the pull-down menu, fisheye menu and grid menu were tested to compare the performance time, error rate, simplicity, usefulness, user friendliness, and overall user preference of each menu type. The grid menu was more efficient in selection speed than the pull-down and fisheye menus when the number of menu-items was 50 and 100. The time needed to choose a menu-item with a grid menu was less affected by the size of menu. The pull-down and the grid menus were considered to be more satisfactory, simple, user friendly, and useful than the fisheye menu. 42.3 percent of subjects indicated that the grid menu was their preferred selection tool among the menus. The grid menu is an efficient and robust alternative menu choice for small and middle size menu list. Further study is required to examine the possibility of grid menu on mobile devices.

• Journal of Korea Society of Industrial Information Systems
• /
• v.27 no.2
• /
• pp.11-24
• /
• 2022

For the questions asked for LMS-based online evaluation the professor directly set exam questions, or use the automatic question-taking method according to the level of difficulty using the question bank divided by category. Among them, it is important to manage the difficulty of questions in an objective and efficient way, above all, in the automatic question-taking method according to difficulty. Because the questions presented to the evaluators may be different. In this paper, we propose an difficulty re-adjustment algorithm that considers not only the correct rate of a problem but also the time taken to solve the problem. For this, a logistic regression classification algorithm was used of machine learning, and a reference threshold was set based on the predicted probability value of the learning model and used to readjust the difficulty of each item. As a result, it was confirmed that there were many changes in the difficulty of each item that depended only on the existing correct rate. Also, as a result of performing group evaluation using the adjustment difficulty problem, it was confirmed that the average score improved in most groups compared to the difficulty problem based on the percentage of correct answers.

• Journal of The Korean Association For Science Education
• /
• v.27 no.1
• /
• pp.9-17
• /
• 2007

This study sets out to assess the scientific literacy of secondary school students and to describe their differences according to gender, grade, course. This study involved 112 middle school students and 213 high school students. Their scientific literacy was measured by the Scientific Literacy Test designed by Manhart (1997). A 70-item multiple-choice test was used to assess their scientific literacy. The constructs of science factor included 36 items making up physical science, life science, and earth science subtests. The social aspects of science factor consisted of 34 items in nature of scientific inquiry/knowledge, science as a human endeavor, science and technology, and societal perspectives. A two-way analysis of variance (ANOVA) and t-test were conducted using the SPSS program. The scientific literacy score of the middle school students was 45.17. There was no significant difference according to gender but boys tended to perform better than girls on both the constructs of science factor and the social aspects of science factor. The scientific literacy score of the high school students was 51.79. There was no significant difference according to gender. But, boys tended to perform better than girls on the constructs of science factor. Girls tended to perform better than boys on the social aspects of science factor. The students taking a course on natural science got statistically higher scores than the students taking a course on humanities. The high school students got statistically higher scores than the middle school students.

• Journal of The Korean Association For Science Education
• /
• v.21 no.4
• /
• pp.697-709
• /
• 2001

Researchers on conceptual change have been proposed that confronting a cognitive conflict situation would be important for a student to change his/her preexisting conception. There have been reported that there are three different methods of producing a cognitive conflict situation; the first is logical argument(LC), the second is demonstration of an actual phenomenon(DC), and the third is kinesthetic conflict which is a kind of physical experience(EC). In this study, the researcher tried to find out the differences in the conceptual changes by the three different conflict situations. Seventy two high school students were chosen in a high school in Kyungkido, Korea. The students were tested four times; pretest, posttest, one week delayed posttest, and one month delayed posttest. Six different test situations on mechanics were developed for this study. Test item for each situation was developed. Each item consisted of a multiple choice question and explanation of the choice. The result showed a clear differences among the three conflict groups. In general, kinesthetic conflict which is a kind of physical experience(EC) was proved to be the most efficient strategy for the conceptual change; however, logical argument(LC) seemed to be the least efficient. However, the effectiveness was not uniform from situation to situation. Results of some items showed that even the LC was quite good for the conceptual change. Therefore, it seems to be important to develope appropriate method for the target concept.