Browse > Article
http://dx.doi.org/10.7468/mathedu.2021.60.1.209

Feature analysis for competency and representation type of mathematics assessment  

Park, Ji Hyun (Korea Institute for Curriculum and Evaluation)
Publication Information
The Mathematical Education / v.60, no.2, 2021 , pp. 209-228 More about this Journal
Abstract
The purpose of this study is developed the Item Feature Analysis (IFA) frameworks for curriculum-based assessments, focusing on Math competency and representation in secondary schools and implemented the IFA in National Assessment of Educational Achievement. To conduct the study, previous studies were analyzed, and feasibility studies were conducted twice. As a result of the study, we structured the IFA framework based on the 2015 revised mathematics curriculum in Korea and developed a method to analyze the characteristics of the math items. The results of structuring the framework for math included two categories: math competency in the content aspects, and representation type in the formal aspects. Specifically, 12 features of math competency and 8 features of representation type were identified, and an item feature analysis framework composed of these features was developed. The math competency was developed based on the subject competency of 2015 national curriculum. Math assessments in high schools, which have been changed to the competency-based assessments, had more frequency of the feature of math competency compared to middle schools. In this study, implemented the IFA in National Assessment of Educational Achievement and explored the way of ensuring the validity. These have been proved as critical applications for ensuring the validity of curriculum-based student assessment as well as building a tool for assessment.
Keywords
Mathematics item features; Item feature analysis; Mathematics assessment; validity;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Lee, J. S., Kim. S. H., Kim, J. C., Song, H. J., Park, M. H., Jang, K. S., & Seo, J. Y. (2003). A Study of Item Difficulty Prediction in College Scholastic Ability Test - the areas of Korean, Mathe, and English -. Korea Institute for Curriculum and Evaluation RRE-2002-14.
2 AERA, APA, & NCME (2014). Standards for educational and psychological testing. Wachington, D.C.: AERA.
3 Baker, E. L., Madni, A., Michiuye, J. K., Choi, K., & Cai, L. (2015). Mathematical reasoning project quantitative analyses results: Grades 4, 8, and 11. LA: UCLA, CRESST.
4 Bruner, J. S. (1966). Toward a theory of instruction. Cambridge, Mass.: Belkapp Press.
5 Choi, K. (2019). Feature Analysis: An Introduction. Unpublished document.
6 Baker, E. L., & Choi, K. (2019). Feature analysis approach: uses for national and international assessments. Presented to the Chilean Delegates (2019.10.1.).
7 Lee, W. S. (2008). Unified View on Validity, Journal of Educational Evaluation, 22(4). 1079-1099.
8 Lee, M. E., Dong, H. G., Park, I. Y., Kim, W. S.., Seo, M. H., Jung, H. K., Kim, K. J., Kang, M. K., Jang, E. S., Sung, K. H., Lim, H. M., Kim, S. K., Bea, J. K., Kim, S. Y., Lee, J. B., Park, J. H., Yang, K. S., Kang, T. H., Shin, Y. J., Park, Y. J. (2017). A Study on the Development plan of National Assessment of Educational Achievement according to the 2015 Revised National Curriculum. Korea Institute for Curriculum and Evaluation CRE 2017-8.
9 Lee, Y. M., & Jeon, P. K. (2005). An Analysis of Third Graders' Representations and Elaborating Processes of Representations in Mathematical Problem Solving. The Mathematical Education, 44(4), 627-651.
10 Lesh, R., Landau, M. & Hamilton, E. (1983). Conceptual Models and Applied Mathemaitcal PRoblem-Solving Research, In Lesh, R, & Landau, M. (Ed.), Acquisition of Mathematics Concepts and Processes, Academic press.
11 Madni, A., Michiuye, J. K., Choi, K., Baker, E. L., & Cai, L. (2015). Feature analysis qualitative and quantitative results for tests of mathematics and english language arts, Grades 4, 8, and 11. LA: UCLA, CRESST.
12 Meyer, M. R. (2001). Representation in realistic mathematics education. In Albert A. Cuoco, Frances R. Curcio, National Council of Teachers of Mathematics (Eds.) The Role of Representation in School Mathematics (pp. 238-250). National Council of Teachers of Mathematics, 2001.
13 Ministry of Education (2015). Mathematics curriculum. Proclamation of the Ministry of Education #2015-75 [Annex 8].
14 Park, J. H. (2020a). Analysis of Middle School Students' Proficiency of Mathematics Curriculum Achievement Standards. The journal of Educational Research in Mathematics, 30(3), 553-573.   DOI
15 Park, J. H., Lee, J. B., & Son, Y. L. (2020b). Analysis of the National Assessment of Educational Achievement (NAEA) 2019 results -Middle school mathematics-. Korea Institute for Curriculum and Evaluation ORM 2020-23-3.
16 CRESST (2020). ELA and Math features_tentative list. Unpublished document.
17 Dong, H. G., Kim, K. J., Jang, E. S., Sung, K. H., Yang, S. H., Kim, S. K., Lee, J. B., Ku, J. O., Park, S. B., Kim, S. Y., Choi, W. H., Kim, Y. J., & Lee, G. Y. (2018). A Study on the Development of National Assessment of Educational Achievement according to the 2015 Revised National Curriculum. Korea Institute for Curriculum and Evaluation RRE 2018-4.
18 Jang, H. W. (1997). A study on the epresentations in mathematics learning. Doctoral dissertation, Seoul University, Seoul.
19 Jonassen, D. H. (2011). Learning to Solve Problems : A Handbook for Designing Problem - Solving Learning Environments. NY: Routledge.
20 Kim, E. H, & Kim, R. Y. (2020). Interpretation and application of information processing competency as mathematical competency: A case of middle school mathematics textbooks under the 2015 revised curriculum. Journal of the Korean Society of Mathematics Education Series A: The Mathematical Education, 59(4), 389-403.
21 Ko, H. K., & Yi, H. S. (2007). Factors of Predicting Difficulty of Mathematics Test Items in College Scholastic Ability Test. Journal of the Korean School Mathematics society, 10(1), 113-127.
22 Park, K. M., Lee. H. C., Park, S. H., Kang, E. J., Kim, S. H., Lim, H. M., Kim, S. Y., Jang, H. W., Kang, T. S., Kwon, J. R., Kim, M. J., Bang, J. S., Lee, H. Y., Lim, M. I., Lee, M. K., Kim, H. K., Yun, S. H., Lee, K. S., Lee, K. E., ... & Yeo, M. J. (2015). 2015 revised mathematics curriculum design development studyII. KOFAC BD15120005.
23 Park, M. H. (2004). A Study of Variables Related to Item Difficulty in College Scholastic Ability Test. The journal of Educational Research in Mathematics, 14(1), 71-88.
24 Seong, T. (2016). Modern Fundamental Statistics: Understanding and Application. Seoul: Hakjisa.
25 Lawshe, C. H. (1975). A Quantitative Approach to Content Validity. Personnel Psychology, 28, 563-575.   DOI
26 Lee, J. S. (2006). Delphi method. Seoul: kyoyookbook.
27 Lee, K. S., Lee, J. B., & Son, Y. L. (2020). Analysis of the National Assessment of Educational Achievement (NAEA) 2019 results -High school mathematics-. Korea Institute for Curriculum and Evaluation ORM 2020-23-7.
28 Kwon, N. Y., & Oh, S. Y. (2020). Secondary mathematics teachers' perceptions on assessment. Journal of the Korean Society of Mathematics Education Series A: The Mathematical Education, 59(4), 295-312.
29 Park, J. H., Song, M. Y., Nam, M. W., & Choi, K. (2020c). Feature analysis approach for improving validity of student assessments. Korea Institute for Curriculum and Evaluation RRE 2020-1.