Browse > Article

Analysis of Korea Earth Science Olympiad Items for the Enhancement of Item Quality  

Lee Ki-Young (Hansung Science High School)
Kim Chan-Jong (Department of Earth Science Education, Seoul National University)
Publication Information
Journal of the Korean earth science society / v.26, no.6, 2005 , pp. 511-523 More about this Journal
Abstract
The purpose of this study is to analyze the 1st and 2nd Korea Earth Science Olympiad (KESO) items, in order to find informations to enhance item quality. To do this, internal and external item classification frameworks are developed. Item difficulty (P), discrimination index (DI), correlation, and reliability are estimated by using classical test theory. Generalizability is also estimated by applying the generalizability theory. The results of item classification are as follows: (1) ‘Geology’, ‘astronomy’ and ‘data analysis and interpretation’ are dominant in content and inquiry process domain, respectively. Nearly every item has textbook context. (2) There is no difference between the preliminary and final tests in terms of their thinking skills sections. (3) As a whole, the ratio of items with pictures is high in item representation. However, multiple-choice and short answer items are more common in preliminary competition, and essay type items are found more often in final competition. The ratio of simple items is high in middle school section and preliminary competition, but composite items are dominant in high school section and final competition. The findings of item analysis are as follows: (1) In the middle school section, P is low and DI is moderate. But in the high school section, there is a considerable differences between science high schools and other high schools in general. (2) The highest correlation is reported between the scores of meteorology domain and total score in middle school, whereas in high school astronomy domain and total score show the highest correlation. (3) General high school section show the highest Cronbach $\alpha$ and generalizability. (4) General high school section show acceptable generalizability coefficient (> 0.80), but middle and science high school section should increase the number of items to reach acceptable generalizability level.
Keywords
item classification framework; item analysis; olympiad; reliability; generalizability;
Citations & Related Records
연도 인용수 순위
  • Reference
1 김성숙, 김양분, 2001, 일반화가능도 이론. 교육과학사
2 한국지구과학회, 2003, 함께하는 지구과학교육. 2 (2), 10-19
3 한국지구과학회, 2004, 함께하는 지구과학교육. 3 (1), 9-13
4 Crick, J. E., Brennan, R. L., 1983, Manual of GENOVA: A GENeralized Analysis Of VAriance System. Iowa city, IA: American College Testing Program
5 Cangelosi, J. S, 1990, Designing tests for evaluating student achievement. New York: Longman
6 변창진, 최진승, 문수백, 김진규, 권대훈, 2001, 교육 평가. 학지사
7 홍미영, 전경문, 이범홍, 이양락, 2002, 대학수학능력시험 화학II 문항에 대한 학생들의 응답 분석. 한국과학교육학회지, 22 (1), 204-213
8 Ebel, R. L, 1965, Measuring Educational Achievement. Englewood Cliffs, NJ: Prentice-Hall
9 이기영, 2004, 평가유형과 채점 방식에 따른 중.고등학교 과학 수행평가의 일반화가능도에 관한 연구. 서울대학교 박사학위논문
10 성태제, 2002, 타당도와 신뢰도. 학지사
11 Burns, K. J., 1998, Beyond classical reliability: Using generalizability theory to assess dependability. Research in Nursing & Health, 21, 83-90   DOI   PUBMED   ScienceOn
12 Brennan, R. L., 2000, Performance assessments from the perspective of generalizability theory. Applied Psychological Measurement, 24 (4), 339-353   DOI   ScienceOn
13 Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N., 1972, The dependability of behavioral measurements: Theory of generalizability of scores and profiles. New York: John Wiley
14 우종옥, 이항로, 구창현, 1996, 과학 탐구 능력 평가 문항 유형 변화에 관한 종단적 연구. 한국과학교육학회지, 16 (3), 314-328