The Mathematical Education (한국수학교육학회지시리즈A:수학교육)

Korean Society of Mathematical Education (한국수학교육학회)
권호 표지
DOI QR코드

DOI QR Code

Feature analysis for competency and representation type of mathematics assessment

수학과 평가 문항의 역량 및 표현 형식 특성 분석

  • Received : 2021.04.19
  • Accepted : 2021.05.24
  • Published : 2021.05.31
Download PDF
⟨ Previous Next ⟩

Abstract

The purpose of this study is developed the Item Feature Analysis (IFA) frameworks for curriculum-based assessments, focusing on Math competency and representation in secondary schools and implemented the IFA in National Assessment of Educational Achievement. To conduct the study, previous studies were analyzed, and feasibility studies were conducted twice. As a result of the study, we structured the IFA framework based on the 2015 revised mathematics curriculum in Korea and developed a method to analyze the characteristics of the math items. The results of structuring the framework for math included two categories: math competency in the content aspects, and representation type in the formal aspects. Specifically, 12 features of math competency and 8 features of representation type were identified, and an item feature analysis framework composed of these features was developed. The math competency was developed based on the subject competency of 2015 national curriculum. Math assessments in high schools, which have been changed to the competency-based assessments, had more frequency of the feature of math competency compared to middle schools. In this study, implemented the IFA in National Assessment of Educational Achievement and explored the way of ensuring the validity. These have been proved as critical applications for ensuring the validity of curriculum-based student assessment as well as building a tool for assessment.

본 연구의 목적은 수학과 교육과정에 기반한 평가 문항을 출제할 때 핵심적으로 고려되는 수학 역량 및 표현 형식 특성을 규명하고, 수학과 평가 문항에서 수학 역량 및 표현 형식 특성이 나타나는 양상을 밝히는 것이다. 선행 연구 분석 및 2차에 걸친 타당성 조사를 통해, 수학과 교육과정에 기반한 평가 문항의 특성으로 수학 역량 범주는 12가지, 표현 형식 범주는 8가지를 추출하고, 이 특성들로 구성된 문항 특성 분석틀을 개발하였다. 또한 문항 특성 분석틀을 활용하여 2019년 국가수준 학업성취도 평가 문항을 분석한 결과, 중 고등학교의 수학 역량 및 표현 형식 특성에 대한 출제 경향에 차이가 있음을 밝혔다.

Keywords

References

  1. AERA, APA, & NCME (2014). Standards for educational and psychological testing. Wachington, D.C.: AERA.
  2. Baker, E. L., Madni, A., Michiuye, J. K., Choi, K., & Cai, L. (2015). Mathematical reasoning project quantitative analyses results: Grades 4, 8, and 11. LA: UCLA, CRESST.
  3. Baker, E. L., & Choi, K. (2019). Feature analysis approach: uses for national and international assessments. Presented to the Chilean Delegates (2019.10.1.).
  4. Bruner, J. S. (1966). Toward a theory of instruction. Cambridge, Mass.: Belkapp Press.
  5. Choi, K. (2019). Feature Analysis: An Introduction. Unpublished document.
  6. CRESST (2020). ELA and Math features_tentative list. Unpublished document.
  7. Dong, H. G., Kim, K. J., Jang, E. S., Sung, K. H., Yang, S. H., Kim, S. K., Lee, J. B., Ku, J. O., Park, S. B., Kim, S. Y., Choi, W. H., Kim, Y. J., & Lee, G. Y. (2018). A Study on the Development of National Assessment of Educational Achievement according to the 2015 Revised National Curriculum. Korea Institute for Curriculum and Evaluation RRE 2018-4.
  8. Jang, H. W. (1997). A study on the epresentations in mathematics learning. Doctoral dissertation, Seoul University, Seoul.
  9. Jonassen, D. H. (2011). Learning to Solve Problems : A Handbook for Designing Problem - Solving Learning Environments. NY: Routledge.
  10. Ko, H. K., & Yi, H. S. (2007). Factors of Predicting Difficulty of Mathematics Test Items in College Scholastic Ability Test. Journal of the Korean School Mathematics society, 10(1), 113-127.
  11. Kim, E. H, & Kim, R. Y. (2020). Interpretation and application of information processing competency as mathematical competency: A case of middle school mathematics textbooks under the 2015 revised curriculum. Journal of the Korean Society of Mathematics Education Series A: The Mathematical Education, 59(4), 389-403.
  12. Kwon, N. Y., & Oh, S. Y. (2020). Secondary mathematics teachers' perceptions on assessment. Journal of the Korean Society of Mathematics Education Series A: The Mathematical Education, 59(4), 295-312.
  13. Lawshe, C. H. (1975). A Quantitative Approach to Content Validity. Personnel Psychology, 28, 563-575. https://doi.org/10.1111/j.1744-6570.1975.tb01393.x
  14. Lee, J. S., Kim. S. H., Kim, J. C., Song, H. J., Park, M. H., Jang, K. S., & Seo, J. Y. (2003). A Study of Item Difficulty Prediction in College Scholastic Ability Test - the areas of Korean, Mathe, and English -. Korea Institute for Curriculum and Evaluation RRE-2002-14.
  15. Lee, J. S. (2006). Delphi method. Seoul: kyoyookbook.
  16. Lee, K. S., Lee, J. B., & Son, Y. L. (2020). Analysis of the National Assessment of Educational Achievement (NAEA) 2019 results -High school mathematics-. Korea Institute for Curriculum and Evaluation ORM 2020-23-7.
  17. Lee, M. E., Dong, H. G., Park, I. Y., Kim, W. S.., Seo, M. H., Jung, H. K., Kim, K. J., Kang, M. K., Jang, E. S., Sung, K. H., Lim, H. M., Kim, S. K., Bea, J. K., Kim, S. Y., Lee, J. B., Park, J. H., Yang, K. S., Kang, T. H., Shin, Y. J., Park, Y. J. (2017). A Study on the Development plan of National Assessment of Educational Achievement according to the 2015 Revised National Curriculum. Korea Institute for Curriculum and Evaluation CRE 2017-8.
  18. Lee, W. S. (2008). Unified View on Validity, Journal of Educational Evaluation, 22(4). 1079-1099.
  19. Lee, Y. M., & Jeon, P. K. (2005). An Analysis of Third Graders' Representations and Elaborating Processes of Representations in Mathematical Problem Solving. The Mathematical Education, 44(4), 627-651.
  20. Lesh, R., Landau, M. & Hamilton, E. (1983). Conceptual Models and Applied Mathemaitcal PRoblem-Solving Research, In Lesh, R, & Landau, M. (Ed.), Acquisition of Mathematics Concepts and Processes, Academic press.
  21. Madni, A., Michiuye, J. K., Choi, K., Baker, E. L., & Cai, L. (2015). Feature analysis qualitative and quantitative results for tests of mathematics and english language arts, Grades 4, 8, and 11. LA: UCLA, CRESST.
  22. Meyer, M. R. (2001). Representation in realistic mathematics education. In Albert A. Cuoco, Frances R. Curcio, National Council of Teachers of Mathematics (Eds.) The Role of Representation in School Mathematics (pp. 238-250). National Council of Teachers of Mathematics, 2001.
  23. Ministry of Education (2015). Mathematics curriculum. Proclamation of the Ministry of Education #2015-75 [Annex 8].
  24. Park, J. H. (2020a). Analysis of Middle School Students' Proficiency of Mathematics Curriculum Achievement Standards. The journal of Educational Research in Mathematics, 30(3), 553-573. https://doi.org/10.29275/jerm.2020.08.30.3.553
  25. Park, J. H., Lee, J. B., & Son, Y. L. (2020b). Analysis of the National Assessment of Educational Achievement (NAEA) 2019 results -Middle school mathematics-. Korea Institute for Curriculum and Evaluation ORM 2020-23-3.
  26. Park, J. H., Song, M. Y., Nam, M. W., & Choi, K. (2020c). Feature analysis approach for improving validity of student assessments. Korea Institute for Curriculum and Evaluation RRE 2020-1.
  27. Park, K. M., Lee. H. C., Park, S. H., Kang, E. J., Kim, S. H., Lim, H. M., Kim, S. Y., Jang, H. W., Kang, T. S., Kwon, J. R., Kim, M. J., Bang, J. S., Lee, H. Y., Lim, M. I., Lee, M. K., Kim, H. K., Yun, S. H., Lee, K. S., Lee, K. E., ... & Yeo, M. J. (2015). 2015 revised mathematics curriculum design development studyII. KOFAC BD15120005.
  28. Park, M. H. (2004). A Study of Variables Related to Item Difficulty in College Scholastic Ability Test. The journal of Educational Research in Mathematics, 14(1), 71-88.
  29. Seong, T. (2016). Modern Fundamental Statistics: Understanding and Application. Seoul: Hakjisa.