Research in Mathematical Education (한국수학교육학회지시리즈D:수학교육연구)

Korean Society of Mathematical Education (한국수학교육학회)
권호 표지
DOI QR코드

DOI QR Code

Designing an Assessment to Measure Students' Inferential Reasoning in Statistics: The First Study, Development of a Test Blueprint

  • Received : 2013.05.21
  • Accepted : 2013.12.27
  • Published : 2013.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Accompanied with ongoing calls for reform in statistics curriculum, mathematics and statistics teachers purposefully have been reconsidering the curriculum and the content taught in statistics classes. Changes made are centered around statistical inference since teachers recognize that students struggle with understanding the ideas and concepts used in statistical reasoning. Despite the efforts to change the curriculum, studies are sparse on the topic of characterizing student learning and understanding of statistical inference. Moreover, there are no tools to evaluate students' statistical reasoning in a coherent way. In response to the need for a research instrument, in a series of research study, the researcher developed a reliable and valid measure to assess students' inferential reasoning in statistics (IRS). This paper describes processes of test blueprint development that has been conducted from review of the literature and expert reviews.

Keywords

References

  1. AERA; APA & NCME (1999). Standards for educational psychological testing. Washington, DC: AERA.
  2. American Statistical Association (2005). GAISE College Report. Retrieved from ASA GAISE College Report Web site: http://www.amstat.org/education/gaise/GAISECollege.htm
  3. Batanero, C. (2000). Controversies around the role of statistical tests in experimental research. Mathematical Thinking and Learning 2(1-2), 75-97. ME 2001f.04627 https://doi.org/10.1207/S15327833MTL0202_4
  4. Carver, R. P. (1978). The case against statistical significance testing. Harvard Educational Review 48(3), 378-399. Available from: http://scholasticadministrator.typepad.com/thisweekineducation/files/the_case_against_statistic al_significance_testing.pdf https://doi.org/10.17763/haer.48.3.t490261645281841
  5. Chance, B.; delMas, R. & Garfield, J. (2004). Reasoning about sampling distributions. In: D. Ben- Zvi & J. Garfield (Eds.), The Challenge of Developing Statistical Literacy, Reasoning and Thinking (pp. 295-323). Dordrecht, Netherlands: Kluwer Academic.
  6. Cohen, J. (1994). The earth is round (p<.05). American Psychologist, 49(12), 997-1003. https://doi.org/10.1037/0003-066X.49.12.997
  7. delMas, R. C.; Garfield, J. B. & Chance, B. L. (1999). A model of classroom research in action: developing simulation activities to improve students' statistical reasoning. J. Stat. Educ. 7(3), 80K. ME 2000b.01311 Available from: http://www.amstat.org/publications/jse/secure/v7n3/delmas.cfm
  8. Falk, R. (1986). Misconceptions of statistical significance. J. Struct. Learn. 9(1), 83-96. ME1986h.02861
  9. Falk, R. & Greenbaum, C. W. (1995). Significance tests die hard: The amazing persistence of a probabilistic misconception. Theory and Psychology 5(1), 75-98. https://doi.org/10.1177/0959354395051004
  10. Garfield, J. (1998). The Statistical Reasoning Assessment: Development and Validation of a Research Tool. In L. Pereira Mendoza (Ed.), Proceedings of the Fifth International Conference on Teaching Statistics (pp. 781-786). Voorburg, The Netherlands: International Statistical Institute.
  11. Garfield, J. & Ben-Zvi, D. (2008). Developing Students Statistical Reasoning: Connecting Research and Teaching Practice. Dordrecht, Netherlands: Springer. ME 2009b.00447
  12. Garfield, J., delMas, R., & Chance, B. (2002). "The Web-based ARTIST: Assessment Resource Tools for Improving Statistical Thinking" Project J. Stat. Educ. 7(3). [Online]. www.amstat.org/publications/jse/v5n3/giraud.html
  13. Haller, H. & Krauss, S. (2002). Misinterpretations of significance: A problem students share with their teachers? Methods of Psychological Research 7(1), 1-20.
  14. Lipson, A. (2003). The role of the sampling distribution in understanding statistical inference. Mathematical Educational Research Journal 15(3), 270-287. Retrieved from: http://files.eric.ed.gov/fulltext/EJ776331.pdf https://doi.org/10.1007/BF03217383
  15. Liu, Y. & Thompson, P. (2009). Mathematics teachers' understandings of proto-hypothesis testing. Pedagogies 4(2), 129-138.
  16. Makar, K. & Rubin, A. (2009). A framework for thinking about informal statistical inference. SERJ - Stat. Educ. Res. J. 8(1), 82-105. ME 2009e.00573 Retrieved from: http://www.stat.auckland.ac.nz/-iase/serj/SERJ8(1)_Makar_Rubin.pdf
  17. Metz, K. E. (1999). Why sampling works or why it can't: Ideas of young children engaged in research of their own design. In: R. Hitt and M. Santos (Eds.), Proceedings of the Twenty-First Annual Meeting of the North American Chapter of the International Group for the Psychology of Education (pp. 492-498). Columbus, OH: 1999 ERIC Clearinghouse of Science, Mathematics, and Environmental Education.
  18. Mittag, K. C. & Thompson, B. (2000). A national survey of AERA members' perceptions of statistical significance tests and other statistical issues. Educational Researcher 29(4), 14-20.
  19. Mokros, J. & Russell, S. J. (1995). Children's concepts of average and representativeness. J. Res. Mathe. Educ. 26(1), 20-39. ME 1995f.03881 https://doi.org/10.2307/749226
  20. Nickerson, R. S. (2000). Null hypothesis significance testing: a review of an old and continuing controversy. Psychological Methods 5(2), 241-301. https://doi.org/10.1037/1082-989X.5.2.241
  21. Pfannkuch, M. (2005). Probability and statistical inference: How can teachers enable learners to make the connection? In: G. Jones (Ed.), Exploring probability in school: Challenges for teaching and learning (pp. 267-294). Dordrecht, Netherlands: Kluwer Academic.
  22. Reed-Rhoads, T.; Murphy, T. J. & Terry, R. (2006). The Statistics Concept Inventory: An Instrument for Assessing Student Understanding of Statistics Concepts, SIGMAA on Statistics Education session First Steps for Implementing the Recommendations of the Guidelines for Assessment and Instruction in Statistics Education (GAISE) College Report, Joint Mathematics Meetings, San Antonio, January 2006.
  23. Rosenthal, R. (1993). Cumulating evidence. In: G. Keren (Ed.), A handbook of data analysis in the behavioral sciences: Methodological issues (pp. 519-559). Hillsdale, NJ: Erlbaum.
  24. Rubin, A.; Bruce, B. & Tenney, Y. (1991). Learning about sampling: Trouble at the core of statistics. In D. Vere-Jones (Ed.), Proceedings of the Third International Conference on Teaching Statistics (pp. 314-319). Dunedin, New Zealand: International Statistical Institute.
  25. Rubin, A.; Hammerman, J., & Konold, C. (2006). Exploring informal inference with interactive visualization software. In: A. Rossman & B. Chance (Eds.), Proceedings of the Seventh International Conference on Teaching Statistics (ICOTS) held at Salvador, Bahai, Brazil, July 2-7, 2006 (CD-ROM). Voorburg, Netherlands: International Statistical Institute.
  26. Saldanha, L. (2004). "Is this sample unusual?": An investigation of students exploring connections between sampling distributions and statistical inference. Unpublished Ph.D. Thesis. Nashville, TN: Vanderbilt University.
  27. Saldanha, L. & Thompson, P. (2002). Conceptions of sample and their relationship to statistical inference. Educ. Stud. Math. 51(3), 257-270. ME 2003d.03486 https://doi.org/10.1023/A:1023692604014
  28. Schwartz, D. L.; Goldman, S. R.; Vye, N. J.; Barron, B. J. & Cognition Technology Group at Vanderbilt (1998). Aligning everyday and mathematical reasoning: The case of sampling assumptions. In: S. Lajoie (Ed.), Reflections on Statistics: Learning, Teaching, and Assessment in Grades K-12 (pp. 233-273). Hillsdale, NJ: Erlbaum.
  29. Sedlemeier, P. & Gigerenzer, G. (1997). Intuitions about sample size: The empirical law of large numbers. Journal of Behavior Decision Making 10, 33-51. https://doi.org/10.1002/(SICI)1099-0771(199703)10:1<33::AID-BDM244>3.0.CO;2-6
  30. Sotos, A. E. C.; Vanhoof, S.; Van den Noortgate, W. & Onghena, P. (2007). Students' misconceptions of statistical inference: A review of the empirical evidence from research on statistics education. Educational Research Review 2, 98-113. https://doi.org/10.1016/j.edurev.2007.04.001
  31. Tversky, A. & Kahneman, D. (1971). Belief in the law of small numbers. Psychological Bulletin 76, 105-110. [Reprinted in: D. Kahneman, P. Slovic & A. Tversky (1982), Judgment under uncertainty: Heuristics and biases. Cambridge University Press.]
  32. Tversky, A. & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science 185(4157), 1124-1131. Retrieved from: http://files.eric.ed.gov/fulltext/EJ776331.pdf https://doi.org/10.1126/science.185.4157.1124
  33. Vallecillos, A. (1999). Some empirical evidences on learning difficulties about testing hypotheses. In: Proceedings of the 52 session of the International Statistical Institute (pp. 201-204). Helsinki: International Statistical Institute. Retrieved from: https://www.stat.auckland.ac.nz/-iase/publications/5/vall0682.pdf
  34. Vallecillos, A. (2002). Empirical evidence about understanding of the level of significance concept in hypotheses testing by university students. Themes in Education 3(2), 183-198.
  35. Vallecillos, A. & Batanero, C. (1997). Activated concepts in the statistical hypothesis contrast and their understanding by unversity students. Reserchers en Didactique des Mathematiques 17, 29-48.
  36. Vanhoof, S.; Sotos, A.; Onghena, P. & Verschaffel, L. (2007). Students' reasoning about sampling distribution before and after the sampling distribution activity. In: Proceedings of the 56 session of the International Statistical Institute, Lisbon, Spain, International Statistical Institute. [Online]: www.stat.auckland.ac.nz/-iase/publications/isi56/CPM80_Vanhoof.pdf.
  37. Wagner, D. A. & Gal, I. (1991). Project STARC: Acquisition of statistical reasoning in children. (Annual Report: Year 1, NSF Grant No. MDR90-50006). Philadelphia, PA: Literacy Research Center, University of Pennsylvania.
  38. Watson, J. M. & Moritz, J. B. (2000a). Developing concepts of sampling. J. Res. Math. Educ. 31(1), 44-70. ME 2000e.03613 https://doi.org/10.2307/749819
  39. Watson, J. M. & Moritz, J. B. (2000b). The longitudinal development of understanding of average. Math. Think. Learn. 2(1-2), 11-50. ME 2000d.02870 ME 2001f.05345 https://doi.org/10.1207/S15327833MTL0202_2
  40. Well, A.; Pollastek, A. & Boyce, S. (1990). Understanding of the effects of sample size on the variability of the mean. Organizational Behavior and Human Decision Processes 47, 289-312. https://doi.org/10.1016/0749-5978(90)90040-G
  41. Wild, C. K.; Pfannkuch, M.; Regan, M. & Horton, N. J. (2011). Towards more accessible conceptions of statistical inference. J. Royal Statistical Society A. 174(2), 1-23. Retrieved November 7, 2010, from: http://www.rss.org.uk/pdf/Wild_Oct._2010.pdf https://doi.org/10.1111/j.1467-985X.2010.00647.x
  42. Wilkerson, M. & Olson, J. R. (1997). Misconceptions about sample size, statistical significance, and treatment effect. Journal of Psychology 131(6), 627-631. https://doi.org/10.1080/00223989709603844
  43. Williams, A. M. (1999). Novice students' conceptual knowledge of statistical hypothesis testing. In: J. M. Truran and K.M. Truran (Eds.), Making the difference: Proceedings of the Twenty-second Annual Conference of the Mathematics Education Research Group of Australasia (pp. 554- 560). Adelaide, Australia: MERGA. ME 2000e.03645
  44. Zieffler, A.; Garfield, J.; delMas, R. & Reading, C. (2008). A framework to support research on informal inferential reasoning. SERJ - Stat. Edu. Res. J. 7(2), 40-58. ME 2009e.00120