The Mathematical Education (한국수학교육학회지시리즈A:수학교육)

Korean Society of Mathematical Education (한국수학교육학회)
권호 표지
DOI QR코드

DOI QR Code

An Analysis of Measurement Equivalence in a Teaching Aptitude and Personality Test for Pre-service Mathematics Teachers between a Graduate School of Education and a College of Education

교육대학원과 사범대학 예비수학교사의 교직 적성·인성 검사에 대한 측정의 동등성 분석

  • Received : 2018.04.27
  • Accepted : 2018.05.23
  • Published : 2018.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

The purpose of this study was to investigate the measurement equivalence and to suggest application ways in teaching aptitude and personality test results for pre-service mathematics teachers between a graduate school of education and a college of education. This study analyzed the scores of the teaching aptitude and personality test of 36 pre-service mathematics teachers enrolled in a graduate school of education and 111 pre-service mathematics teachers in a college of education by performing a multivariate generalizability analysis. The main results were as follows. First, graduate's pre-service mathematics teachers had a higher level of teaching aptitude and personality than that of college's pre-service mathematics teachers based on the total scores. In addition, graduate's pre-service mathematics teachers had higher levels of teaching aptitude and personality than those of college's pre-service mathematics teachers except for a creativity application domain based on the sub-domain scores. Second, cognitive domains were measured more precisely but affective domains were measured less precisely for graduate's pre-service mathematics teachers than for college's pre-service mathematics teachers. Third, regardless of school levels, Cronbach's ${\alpha}$ values, which might be overestimated by applying the classical test theory, were higher than dependability coefficients. Fourth, this study showed a somewhat negative result in ensuring the measurement equivalence for a problem solving exploration domain. However, regardless of school levels, this study indicated that the overall measurement was generally reliable on composite scores. Based on these results, it was confirmed that multivariate generalizability methodologies' approach can be useful for exploring the measurement equivalence issues. Finally, this study suggests how to utilize the results of the test, how to apply a multivariate generalizability analysis for detecting the measurement equivalence, and how to develop future research based on limitations.

Keywords

References

  1. 교육부 (2015). 2015 문.이과 통합형 교육과정 총론 주요 사항, 세종: 교육부. (Ministry of Education. (2015). The main particular of 2015 National curriculum draft for an integration of arts and sciences, Sejong: Ministry of Education.)
  2. 김래영, 김은현 (2017). 교원양성교육과정과 교원임용후 보자 선정경쟁시험에 나타난 중등수학교사에게 요구 되는 지식 분석: 교수를 위한 내용 지식을 중심으로. 교과교육학연구 21(5), 610-623. (Kim, R. Y. & Kim, E. H. (2017). An analysis of the expected content knowledge for teaching in teacher education programs and teacher employment tests for secondary mathematics. Journal of Research in Curriculum Instruction 21(5), 610-623.)
  3. 김명화 (2005). 채점 자동화 시스템 구축을 위한 수학 구성형 문항 채점의 일반화 가능도 연구, 교육문제연구 22, 205-222. (Kim, M. H. (2005). An application of the generalizability to constructed-response items in mathematics ofr computer automated scoring. Journal of Research in Education 22, 205-222.)
  4. 김성숙, 김양분 (2001). 일반화가능도 이론, 서울: 교육과학사. (Kim, S. S. & Kim, Y. B. (2001). Generalizability theory, Seoul: Kyoyookgwahaksa.)
  5. 김성숙, 전경희 (2018). 정의적 성취특성 측정의 오차요인 및 성취수준별 척도의 동등성 분석. 교육과정평가연구 21(1), 153-172. (Kim, S. S. & Chon, K. H. (2018). An Analysis of measurement errors and invariance properties by proficiency level in the non-cognitive measures, The Journal of Curriculum and Evaluation 21(1), 153-172.)
  6. 김성연 (2016). 수학적 창의성 검사의 채점 영역별 가중치 분석. 수학교육 55(2), 147-169. (Kim, S. Y. (2016). Analysis of weights depending on scoring domains of the mathematical creativity test, The Mathematical Education 55(2), 147-169.)
  7. 김성연 (2017a). 교직 인성 검사에서의 문항 프로파일 분석, 중등교육연구 65(4), 705-729. (Kim, S. Y. (2017a). Item profile analysis of the personality test for teaching profession, Second Education Research 65(4), 705-729.)
  8. 김성연 (2017b). 수학적 창의성 태도 검사에서 수학영재 와 일반학생의 다집단 일반화가능도 분석, 수학교육논문집 31(1), 49-70. (Kim, S. Y. (2017b). Multigroup geveralizability analysis of creativity attitude scale-Korea for mathematically gifted and general students in middle schools, Communications of Mathematical Education 31(1), 49-70.)
  9. 김성연(2018a). 예비수학교사의 교직 적성.인성 검사에 서 분할점수 변화에 따른 다양한 신뢰도 탐색. 수학교육 60(1), 55-74. (Kim, S. Y. (2018a). Investigation of various reliability indices of pre-service mathematics teachers' teaching aptitude and personality test based on setting cut scores, The Mathematical Education 60(1), 55-74.)
  10. 김성연 (2018b). 교육대학원과 사범대학에서 교직 적성. 인성 검사의 일반화가능도 분석. 교원교육 34(2), 1-17. (Kim, S. Y. (2018b). Generalizability analysis of teaching aptitude and personality test for graduate school of education and college of education students, Korean Journal of Teacher Education 34(2), 1-17.)
  11. 김정환 (2004). 초등학교 기간제 교사의 교육능력 관련 요소의 인과관계 분석. 교육평가연구 17(1), 121-139. (Kim, J. H. (2004). An investigation on casual relationships among educational competence-related factors of period - limit teachers in primary schools. Journal of Educational Evaluation 17(1), 121-139.)
  12. 김정환, 남현우, 염시창, 임진영 (2012). 교직 적성.인성 검사 도구 개발 연구, 서울: 교육과학기술부. (Kim, J., Nam, H., Yeom, S., & Im, J. (2012). The development of teaching aptitude and personality test, Seoul: Ministry of Education, Science, and Technology.)
  13. 김진영, 이기종 (2013). 예비유아교사의 셀프리더십과 교직인.적성과의 관계, 한국교육 40(3), 5-25. (Kim, J. Y. & Lhee, K. J. (2013). The relationship between pre-service kindergarten teachers' self-leadership strategies and teaching personalities and aptitudes, The Journal of Korean Education 40(3), 5-25.)
  14. 김현진 (2013). 교사.학교장 신념과 중학생의 자율성 및 자기효능감, 학업성취도의 관계 분석. 교육학연구 51(2), 117-143. (Kim, H. J. (2013). An analysis of relations among teacher and principal's beliefs and 8th grade students' autonomy, self-efficacy and achievement, The Journal of Educational Research 51(2), 117-143.)
  15. 박정 (2007). 우리나라 중학생의 수학에 대한 정의적 특성 변화와 수학 성취에 미치는 영향력 분석. 수학교육 46(1), 19-31. (Park, J. (2007). The trend in the Korean middle school students' affective variables toward mathematics and its effect on their mathematics achievements, The Mathematical Education 46(1), 19-31.)
  16. 백인순 (2000). 교육대학원에 대한 교사의 인식 조사 연구. 석사학위논문, 숙명여자대학교. (Paek, I. S. (2000). A study on teacher's understanding of the graduate school of education. Master's thesis, Seoul: Sookmyung Woman's University.)
  17. 신준국, 부덕훈, 서보억 (2015). 수학수업에서 인성 함양을 위한 중학교 교수.학습 자료 개발 연구, 수학교육논문집 29(2), 241-265. (Shin, J. K., Boo, D. H., & Suh, B. E. (2015). A study on the development of teaching and learning materials for character education in middle school, Communications of Mathematical Education 29(2), 241-265.)
  18. 안도희, 김유리 (2015). 교직이수 학생들의 교직핵심역량, 교수능력 및 교직전문성 열망 간의 관계. 교원교육 31(1), 203-226. (Ahn, D. H. & Kim, Y. R. (2015). The relationships between core competencies in teaching, teaching ability and professional aspirations for teaching among student teachers, Korean Journal of Teacher Education 31(1), 203-226.)
  19. 안재희, 이숙정 (2012). 교직이수 학생들의 진로결정에 대한 인식 분석, 열린교육연구 20(2), 27-49. (Ahn, J. & Lee, S. (2012). Analysis of career choices made by student teachers, The Journal of Yeolin Education 20(2), 27-49.)
  20. 오유진 (2016). 영어과 임용준비생들의 서답형 영어쓰기 답안에 대한 인식과 이에 대비한 각 교육기관 교육과정의 효율성에 관한 인식, 석사학위논문, 숙명여자대학교, (Oh, Y. J. (2016). A study on the awareness of examination for secondary school English teachers' writing answer shiit by preliminary teacher and efficacy of curriculum of each English educational institution, Master's thesis, Seoul: Sookmyung Woman's University.)
  21. 원혜경, 이소정 (2017). 예비유아교사를 위한 간편형 교직 적.인성 척도 타당화 연구, 유아교육학논집 21(1), 133-165. (Won, H. K. & Lee, S. J. (2017). Validation of a short form of the aptitude and personality as a teacher scale for preservice early childhood teachers, Early Childhood Education Research & Review 21(1), 133-165.)
  22. 이규민, 황경현 (2007). 초등학교 과학과 수행평가의 총체적 채점과 분석적 채점 방식에 대한 일반화가능도 분석, 아동교육 16(4), 169-184. (Lee, G, M. & Hwang, K. H. (2007). A generalizability theory approach toward investigating the generalizability of scores from holistic and analytic scoring methods in performance assessments of an elementary school science class, The Korean Journal of Child Education 16(4), 169-184.)
  23. 이문수, 차동춘 (2016). 수학 창의력 문제 해결 검사에 서의 일반화가능도 이론과 다국면 라쉬모형의 비교 연구. 교육과정평가연구 19(2), 251-279. (Lee, M. & Cha, D. (2016). A comparison of generalizability theory and many facet Rasch measurement in an analysis of mathematics creative problem solving test, The Journal of Curriculum and Evaluation 19(2), 251-279.)
  24. 이선정, 신혜원 (2008). 서울소재 교육대학원 가정교육 전공 교육과정에 대한 운영실태와 교육대학원생의 인식. 교원교육 20(4), 173-186. (Lee, S. J. & Shin, H. W. (2008). The operation of home ecnomomics education course in graduate school of education and the graduate students' perception, Korean Journal of Teacher Education 20(4), 173-186.)
  25. 이현숙, 송미영 (2015). PISA 2012 수학 성취도를 설명하는 학생의 정의적 특성 및 교사 특성 분석을 위한 다층 구조방정식모형의 적용. 교과교육학연구 19(1), 137-158. (Yi, H. S. & Song, M Y. (2015). A multi-level SEM approach for the analysis of relationships between math-related educational context variables and math literacy of PISA 2012. Journal of Research in Curriculum Instruction 19(1), 137-158.)
  26. 전경희, 조안나 (2017). 예비교사의 입학전형 유형별 교직 적.인성 특성의 구조적 관계 분석. 학습자중심교과교육연구 17(17), 315-334. (Chon, K. H. & Cho, A. (2017). An analysis of the structural relationship among aptitude, character traits of preservice teachers by college admission types. Journal of Learner-Centered Curriculum and Instruction 17(17), 315-334.)
  27. 조운주 (2014). 예비유아교사를 위한 교직적성.인성 검사도구의 타당성 및 개선방안, 육아지원연구 9(2), 101-123. (Cho, W. (2014). Validation and modification of teaching aptitude test for pre-service early childhood teachers, Early Childhood Education and Care 9(2), 101-123.)
  28. 한혜숙, 최계현 (2011). 중등 수학 교사들의 정의적 특성에 대한 인식과 수업 실태 분석. 한국학교수학회논문집 14(4), 491-518. (Han, H. S. & Choi, K. H. (2011). Secondary mathematics teachers' recognition of the affective domain and analysis of condition in mathematics teaching. Journal of the Korean School Mathematics Society 14(4), 491-518.)
  29. 황혜정 (2011). 수학 수업의 교사 지식에 관한 평가 요 소 탐색-교수.학습 방법 및 평가를 중심으로, 한국학 교수학회논문집 14(3), 241-263. (Hwang, H. J. (2011). The study on the investigation of the mathematics teaching evaluation standards focused on teaching and learning methods and assessment. Journal of the Korean School Mathematics Society 14(3), 241-263.)
  30. Abedi, J. & Lord, C. (2001). The language factor in mathematics tests, Applied Measurement in Education 14(3), 219-234. https://doi.org/10.1207/S15324818AME1403_2
  31. Baxter, G. P. Shavelson, R. J., Herman, S. J., Brown, K. A., & Valadez, J. R. (1993). Mathematics performance assessment: Technical quality and diverse student impact, Journal for Research in Mathematics Education 24, 190-216. https://doi.org/10.2307/749344
  32. Brennan, R. L. (2001a). Generalizability Theory, New York, NY: Springer-Verlag.
  33. Brennan, R. L. (2001b). Manual for mGENOVA, Iowa City: Iowa Testing Programs, University of Iowa.
  34. Christ, T. J., Johnson-Gros, K. N., & Hintze, J. M. (2005). An examination of alternate assessment durations when assessing multiple-skill computational fluency: The generalizability and dependability of curriculum-based outcomes within the context of educational decisions, Psychology in the Schools 42(6), 615-622. https://doi.org/10.1002/pits.20107
  35. Cohen, J. (1988). Statistical power analysis for the behavioral sciences, Hilsdale. NJ: Lawrence Earlbaum Associates.
  36. Cronbach, L. J., Linn, R. L., Brennan, R. L., & Haertel, E. H. (1997). Generalizability analysis for performance assessments of student achievement or school effectiveness. Educational and Psychological Measurement 57(3), 373-399. https://doi.org/10.1177/0013164497057003001
  37. Darling-Hammond, L. (2000). Teacher quality and student achievement: A review of state policy evidence, Education Policy Analysis Archives 8(1), 1-44. https://doi.org/10.14507/epaa.v8n1.2000
  38. Dunbar, S. B., Koretz, D. M., & Hoover, H. D. (1991). Quality control in the development and use of performance assessments, Applied measurement in education 4(4), 289-303. https://doi.org/10.1207/s15324818ame0404_3
  39. Fyans, L. J. (1983). Generalizability theory: Inferences and practical applications, Jossey-Bass Inc Pub.
  40. Hill, H. C., Charalambous, C. Y., & Kraft, M. A. (2012). When rater reliability is not enough: Teacher observation systems and a case for the generalizability study. Educational Researcher 41(2), 56-64. https://doi.org/10.3102/0013189X12437203
  41. Hill, H. C., Rowan, B., & Ball, D. L. (2005). Effects of teachers' mathematical knowledge for teaching on student achievement, American educational research journal 42(2), 371-406. https://doi.org/10.3102/00028312042002371
  42. Ho, A. D. & Kane, T. J. (2013). The reliability of classroom observations by school personnel. Research Paper, MET Project, Bill & Melinda Gates Foundation.
  43. Houchard, M. A. (2005). Principal leadership, teacher morale, and student achievement in seven schools in Mitchell county, North Carolina, Electronic Theses and Dissertations.
  44. Klein, S. P., McCaffrey, D., Stecher, B., & Koretz, D. (1995). The reliability of mathematics portfolio scores: lessons from the Vermont experience, Applied Measurement in Education 8(3), 243-260. https://doi.org/10.1207/s15324818ame0803_4
  45. Klieme, E., Pauli, C., & Reusser, K. (2009). The Pythagoras study: Investigating effects of teaching and learning in Swiss and German mathematics classrooms, The power of video studies in investigating teaching and learning in the classroom, 137-160.
  46. Kukla-Acevedo, S. (2009). Do teacher characteristics matter? New results on the effects of teacher preparation on student achievement, Economics of Education Review 28(1), 49-57. https://doi.org/10.1016/j.econedurev.2007.10.007
  47. Kunter, M., Tsai, Y. M., Klusmann, U., Brunner, M., Krauss, S., & Baumert, J. (2008). Students' and mathematics teachers' perceptions of teacher enthusiasm and instruction, Learning and Instruction 18(5), 468-482. https://doi.org/10.1016/j.learninstruc.2008.06.008
  48. Lakin, J. M. & Lai, E. R. (2012). Multigroup generalizability analysis of verbal, quantitative, and nonverbal ability tests for culturally and linguistically diverse students. Educational and Psychological Measurement 72(1), 139-158. https://doi.org/10.1177/0013164411408074
  49. Lane, S., Liu, M., Ankenmann, R. D., & Stone, C. A. (1996). Generalizability and validity of a mathematics performance assessment. Journal of Educational Measurement 33(1), 71-92. https://doi.org/10.1111/j.1745-3984.1996.tb00480.x
  50. Lee, G. & Frisbie, D. A. (1999). Estimating reliability under a generalizability theory model for test scores composed of testlets, Applied Measurement in Education 12(3), 237-255. https://doi.org/10.1207/S15324818AME1203_2
  51. Li, D. & Brennan, R. (2007). A multi-group generalizability analysis of a large-scale reading comprehension test, In annual meeting of the National Council on Measurement in Education. Chicago, IL.
  52. Marty, M. C., Henning, J. M., & Willse, J. T. (2010). Accuracy and reliability of peer assessment of athletic training psychomotor laboratory skills. Journal of athletic training 45(6), 609-614. https://doi.org/10.4085/1062-6050-45.6.609
  53. Mastergeorge, A. M. & Martinez, J. F. (2010). Rating performance assessments of students with disabilities: A study of reliability and bias. Journal of Psychoeducational Assessment 28(6), 536-550. https://doi.org/10.1177/0734282909351022
  54. Matsumura, L. C., Slater, S. C., Junker, B., Peterson, M., Boston, M., Steele, M., & Resnick, L. (2006). Measuring reading comprehension and mathematics instruction in urban middle schools: A pilot study of the Instructional Quality Assessment, CSE Technical Report 681. National Center for Research on Evaluation, Standards, and Student Testing (CRESST).
  55. McBee, M. M. & Barnes, L. L. (1998). The generalizability of a performance assessment measuring achievement in eight-grade mathematics, Applied Measurement in Education 11(2), 179-194. https://doi.org/10.1207/s15324818ame1102_4
  56. Mottus, R., Allik, J. & Realo, A. (2012). The effect of response style on self-reported conscientiousness across 20 countries, Personality and Social Psychology Bulletin 38(11), 1423-1436. https://doi.org/10.1177/0146167212451275
  57. Narvaez, D. & Nucci, L. P. (2008). Handbook of moral and character education, New York, NY: Routledge.
  58. Powers, S. & Brennan, R. L. (2009). Multivariate generalizability analyses of mixed-format advanced placement exams, CASMA Research Report.
  59. Schnitzler, C., Button, C., Croft, J. L., & Seifert, L. (2015). A new qualitative typology to classify treading water movement patterns. Journal of sports science & medicine 14(3), 530.
  60. Shavelson, R. J., Baxter, G. P., & Gao, X. (1993). Sampling variability of performance assessments. Journal of Educational Measurement 30(3), 215-232. https://doi.org/10.1111/j.1745-3984.1993.tb00424.x
  61. Shavelson, R. J. & Webb, N. M. (1991). Generalizability theory: A primer, London: Sage Publications.
  62. Sluijsmans, D. (2001). Peer assessment in problem based learning, Studies in educational evaluation 27(2), 153-73. https://doi.org/10.1016/S0191-491X(01)00019-0
  63. Thompson, A. G. (1992). Teacher' belief and conceptions: A synthesis of the research, New York: Macmillan Publishing Company.
  64. Wayne, A. J. & Youngs, P. (2003). Teacher characteristics and student achievement gains: A review, Review of Educational Research 73(1), 89-122. https://doi.org/10.3102/00346543073001089
  65. Wilhelm, A. G. & Kim, S. (2015). Generalizing from observations of mathematics teachers' instructional practice using the instructional quality assessment, Journal for Research in Mathematics Education 46(3), 270-279. https://doi.org/10.5951/jresematheduc.46.3.0270
  66. Yin, P. (2005). A multivariate generalizability analysis of the multistate bar examination. Educational and psychological measurement 65(4), 668-686. https://doi.org/10.1177/0013164404273940