Browse > Article
http://dx.doi.org/10.5467/JKESS.2010.31.1.095

An Analysis of the Reliability of Group Assessment of Logical Thinking (GALT) using Generalizability Theory  

Ryu, Chun-Ryol (Department of Earth Science Education, Chungbuk National University)
Lee, Yong-Geun (Department of Earth Science Education, Chungbuk National University)
Publication Information
Journal of the Korean earth science society / v.31, no.1, 2010 , pp. 95-105 More about this Journal
Abstract
The purpose of this study lies in applying generalizability theory depending on the aim of the usage of GALT to analyze the sources of error of single-facet considering item and person only and to analyze the sources of error of multi-facet considering item, person and domain. The study was conducted with 1016 students of local elementary, middle, and high schools. The 21 items of a full version were answered for 40 minute and then the 12 items of short version were sampled to analyze reliability using generalizability theory. Both the full version and the short version of the items were analyzed using Cronbach's alpha for data analysis, and we applied generalizability theory and separate $p{\times}i$ design and $p{\times}(i:h)$ design, G study and D study were performed. Results of analysis are as follows: First, the result of D study after $p{\times}I$ design both on the full version and the short version showed that in the case of the full version, the generalizability coefficient was 0.87 exceeding a normal level of 0.80, and the normal level of generalizability coefficient was achieved in 13 items as well. In case of short version, when 12 items were evaluated, generalizability coefficient was 0.77 not reaching the normal level, and the normal level was achieved in case of more than 15 items. Second, the result of D study after $p{\times}(I:H)$ design on the short version showed that once one domain consists of 2 items in 6 domains, generalizability coefficient was 0.71 which is lower than the normal level of 0.80, the normal level was achieved in more than 5 item cases.
Keywords
GALT; generalizability theory; reliability;
Citations & Related Records
Times Cited By KSCI : 3  (Citation Analysis)
연도 인용수 순위
1 Dunbar, S.B., Koretz, D.M., and Hoover, H.D., 1992, Quality control in the development and use of performance assessment. Applied Mesurement in Education, 4, 289-303.
2 Lawson, A.E., 1995, Science teaching and the development of thinking. Wadsworth Publication, CA, USA, 593 p.
3 Piaget, J. and Inhelder, B., 1958, The growth of logical thinking from childhood to adolescence: An essay on the construction of formal operational structures. Basic Book, NY, USA, 356 p.
4 Popper, K.R., 1965, The logic of scientific discovery. Harper Torchbooks and The Science Library, NY, USA, 479 p.
5 Roadrangka, V., Yeany, R.H., and Padilla, M.J., 1983, The construction and validation of group assesment of logical thinking (GALT). Paper Presented at the Annual Meeting of the National Association for Research in Science Teaching, 5 p.
6 Shavelson, R.J., Webb, N., and Burstein, L., 1991, Generalizability theory: A primer. Sage Productions, CA, USA, 152 p.
7 Shavelson, R.J., Baxter, G.P., and Gao, X., 1993, Sampling variability of performance assessments. Journal of Educational Measurement, 30, 215-235.   DOI   ScienceOn
8 Sireci, S.G., Thissen, D., and Wainer, H., 1991, On the reliability of testlet-based tests. Journal of Educational Measerement, 28, 237-247.   DOI
9 Thondike, R.L., 1951, Reliability. In Lindquist, E.F. (ed.), Educational measurement. American Council on Education, Washington DC, USA, 560-620.
10 이규민, 2003, 단위검사 개념의 적용: 일반화가능도 이론을 중심으로. 교육평가연구, 16, 53-70.
11 이용원, 2003, Score reliability of a test composed of passage-based testlets. 영어교과교육, 2, 1-29.
12 임청환, 정진우, 1991, 고교생의 논리적사고력과 과학탐구 기능 사이의 상관관계에 관한 연구. 한국과학교육학회지, 11, 23-30.   과학기술학회마을
13 정연태, 김범기, 1991, 사고 훈련에 의한 우수 집단의 논리적 사고력 및 프로세스 스킬의 발달에 관한 연구. 과학교육논문집, 1, 241-247.
14 최영준, 최병순, 이원식, 1985, 중.고등학생들의 논리적 사고력 형성에 관한 연구(I). 한국과학교육학회지, 5, 1-9.   과학기술학회마을
15 최재환, 1993, 뇌기능 분화가 논리적 사고력과 자연과 단원별 성취도에 미치는 영향. 과학.수학 교육연구, 17, 53-89.
16 Anastasi, A., 1961, Psychological testing. Macmillan, NY, USA, 657 p.
17 Brennan, R., 1992, Elements of generalizability theory. The American College Testing Program, IA, USA, 161 p.
18 Brennan, R.L. and Lockwood, R.E., 1980, A comparison of nedelsky and angoff cutting score procedures using generalizability theoy. Applied Psychological Measurement, 4, 219-240.   DOI
19 Crick, J.E. and Brennan, R.L., 1983, Manual of GENOVA: A generalize analysis of variance system. American College Testing Program, IA, USA, 109 p.
20 Bunce, D.M. and Hutchinson, K.D., 1993, The use of GALT(Group Assessment of Logical Thinking) as a predictor of academic success in college chemistry. Journal of Chemical Education, 70, 183-187.   DOI
21 Cronbach, L.J., Glesser, G.C., Nanda, H., and Rajaratnam, N., 1972, The dependability of behavioral measurements: Theory of generalizability for scores and profiles. John Wiley, NY, USA, 410 p.
22 소원주, 우종옥, 1994, 중학생의 논리적 사고력과 통합적 과학탐구 능력에 관한 한, 일 비교 연구. 한국과학교육학회지, 14, 312-320.
23 김성숙, 김양분, 2001, 일반화가능도 이론. 교육과학사, 서울, 121 p.
24 김태선, 배덕진, 김범기, 2002, 중학생의 그래프 능력과 논리적사고력 및 과학탐구능력의 관계. 한국과학교육학회지, 22, 725-739.
25 성태제, 2002, 타당도와 신뢰도. 학지사, 서울, 191 p.
26 양지승, 이규민, 2007, 일반화가능도 이론을 적용한 단위검사 구성 검사점수의 신뢰도 추정. 교육평가연구, 20, 119-139.
27 유갑렬, 1988, 한국과 일본의 중학생의 과학적 사고력 및 탐구과정 능력에 관한 비교 연구. 서울대학교 석사학위논문, 102 p.