Browse > Article
http://dx.doi.org/10.7468/jksmed.2013.17.4.243

Designing an Assessment to Measure Students' Inferential Reasoning in Statistics: The First Study, Development of a Test Blueprint  

Park, Jiyoon (Federation of State Boards)
Publication Information
Research in Mathematical Education / v.17, no.4, 2013 , pp. 243-266 More about this Journal
Abstract
Accompanied with ongoing calls for reform in statistics curriculum, mathematics and statistics teachers purposefully have been reconsidering the curriculum and the content taught in statistics classes. Changes made are centered around statistical inference since teachers recognize that students struggle with understanding the ideas and concepts used in statistical reasoning. Despite the efforts to change the curriculum, studies are sparse on the topic of characterizing student learning and understanding of statistical inference. Moreover, there are no tools to evaluate students' statistical reasoning in a coherent way. In response to the need for a research instrument, in a series of research study, the researcher developed a reliable and valid measure to assess students' inferential reasoning in statistics (IRS). This paper describes processes of test blueprint development that has been conducted from review of the literature and expert reviews.
Keywords
statistical inference; inferential reasoning in statistics; informal inference; formal inference;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Watson, J. M. & Moritz, J. B. (2000a). Developing concepts of sampling. J. Res. Math. Educ. 31(1), 44-70. ME 2000e.03613   DOI   ScienceOn
2 Watson, J. M. & Moritz, J. B. (2000b). The longitudinal development of understanding of average. Math. Think. Learn. 2(1-2), 11-50. ME 2000d.02870 ME 2001f.05345   DOI
3 Well, A.; Pollastek, A. & Boyce, S. (1990). Understanding of the effects of sample size on the variability of the mean. Organizational Behavior and Human Decision Processes 47, 289-312.   DOI
4 Wild, C. K.; Pfannkuch, M.; Regan, M. & Horton, N. J. (2011). Towards more accessible conceptions of statistical inference. J. Royal Statistical Society A. 174(2), 1-23. Retrieved November 7, 2010, from: http://www.rss.org.uk/pdf/Wild_Oct._2010.pdf   DOI   ScienceOn
5 Wilkerson, M. & Olson, J. R. (1997). Misconceptions about sample size, statistical significance, and treatment effect. Journal of Psychology 131(6), 627-631.   DOI
6 Williams, A. M. (1999). Novice students' conceptual knowledge of statistical hypothesis testing. In: J. M. Truran and K.M. Truran (Eds.), Making the difference: Proceedings of the Twenty-second Annual Conference of the Mathematics Education Research Group of Australasia (pp. 554- 560). Adelaide, Australia: MERGA. ME 2000e.03645
7 Zieffler, A.; Garfield, J.; delMas, R. & Reading, C. (2008). A framework to support research on informal inferential reasoning. SERJ - Stat. Edu. Res. J. 7(2), 40-58. ME 2009e.00120
8 AERA; APA & NCME (1999). Standards for educational psychological testing. Washington, DC: AERA.
9 American Statistical Association (2005). GAISE College Report. Retrieved from ASA GAISE College Report Web site: http://www.amstat.org/education/gaise/GAISECollege.htm
10 Batanero, C. (2000). Controversies around the role of statistical tests in experimental research. Mathematical Thinking and Learning 2(1-2), 75-97. ME 2001f.04627   DOI
11 Carver, R. P. (1978). The case against statistical significance testing. Harvard Educational Review 48(3), 378-399. Available from: http://scholasticadministrator.typepad.com/thisweekineducation/files/the_case_against_statistic al_significance_testing.pdf   DOI
12 Chance, B.; delMas, R. & Garfield, J. (2004). Reasoning about sampling distributions. In: D. Ben- Zvi & J. Garfield (Eds.), The Challenge of Developing Statistical Literacy, Reasoning and Thinking (pp. 295-323). Dordrecht, Netherlands: Kluwer Academic.
13 Cohen, J. (1994). The earth is round (p<.05). American Psychologist, 49(12), 997-1003.   DOI   ScienceOn
14 delMas, R. C.; Garfield, J. B. & Chance, B. L. (1999). A model of classroom research in action: developing simulation activities to improve students' statistical reasoning. J. Stat. Educ. 7(3), 80K. ME 2000b.01311 Available from: http://www.amstat.org/publications/jse/secure/v7n3/delmas.cfm
15 Falk, R. (1986). Misconceptions of statistical significance. J. Struct. Learn. 9(1), 83-96. ME1986h.02861
16 Falk, R. & Greenbaum, C. W. (1995). Significance tests die hard: The amazing persistence of a probabilistic misconception. Theory and Psychology 5(1), 75-98.   DOI   ScienceOn
17 Garfield, J. (1998). The Statistical Reasoning Assessment: Development and Validation of a Research Tool. In L. Pereira Mendoza (Ed.), Proceedings of the Fifth International Conference on Teaching Statistics (pp. 781-786). Voorburg, The Netherlands: International Statistical Institute.
18 Haller, H. & Krauss, S. (2002). Misinterpretations of significance: A problem students share with their teachers? Methods of Psychological Research 7(1), 1-20.
19 Garfield, J. & Ben-Zvi, D. (2008). Developing Students Statistical Reasoning: Connecting Research and Teaching Practice. Dordrecht, Netherlands: Springer. ME 2009b.00447
20 Garfield, J., delMas, R., & Chance, B. (2002). "The Web-based ARTIST: Assessment Resource Tools for Improving Statistical Thinking" Project J. Stat. Educ. 7(3). [Online]. www.amstat.org/publications/jse/v5n3/giraud.html
21 Lipson, A. (2003). The role of the sampling distribution in understanding statistical inference. Mathematical Educational Research Journal 15(3), 270-287. Retrieved from: http://files.eric.ed.gov/fulltext/EJ776331.pdf   DOI
22 Liu, Y. & Thompson, P. (2009). Mathematics teachers' understandings of proto-hypothesis testing. Pedagogies 4(2), 129-138.
23 Makar, K. & Rubin, A. (2009). A framework for thinking about informal statistical inference. SERJ - Stat. Educ. Res. J. 8(1), 82-105. ME 2009e.00573 Retrieved from: http://www.stat.auckland.ac.nz/-iase/serj/SERJ8(1)_Makar_Rubin.pdf
24 Metz, K. E. (1999). Why sampling works or why it can't: Ideas of young children engaged in research of their own design. In: R. Hitt and M. Santos (Eds.), Proceedings of the Twenty-First Annual Meeting of the North American Chapter of the International Group for the Psychology of Education (pp. 492-498). Columbus, OH: 1999 ERIC Clearinghouse of Science, Mathematics, and Environmental Education.
25 Mittag, K. C. & Thompson, B. (2000). A national survey of AERA members' perceptions of statistical significance tests and other statistical issues. Educational Researcher 29(4), 14-20.
26 Reed-Rhoads, T.; Murphy, T. J. & Terry, R. (2006). The Statistics Concept Inventory: An Instrument for Assessing Student Understanding of Statistics Concepts, SIGMAA on Statistics Education session First Steps for Implementing the Recommendations of the Guidelines for Assessment and Instruction in Statistics Education (GAISE) College Report, Joint Mathematics Meetings, San Antonio, January 2006.
27 Mokros, J. & Russell, S. J. (1995). Children's concepts of average and representativeness. J. Res. Mathe. Educ. 26(1), 20-39. ME 1995f.03881   DOI   ScienceOn
28 Nickerson, R. S. (2000). Null hypothesis significance testing: a review of an old and continuing controversy. Psychological Methods 5(2), 241-301.   DOI   ScienceOn
29 Pfannkuch, M. (2005). Probability and statistical inference: How can teachers enable learners to make the connection? In: G. Jones (Ed.), Exploring probability in school: Challenges for teaching and learning (pp. 267-294). Dordrecht, Netherlands: Kluwer Academic.
30 Rosenthal, R. (1993). Cumulating evidence. In: G. Keren (Ed.), A handbook of data analysis in the behavioral sciences: Methodological issues (pp. 519-559). Hillsdale, NJ: Erlbaum.
31 Rubin, A.; Bruce, B. & Tenney, Y. (1991). Learning about sampling: Trouble at the core of statistics. In D. Vere-Jones (Ed.), Proceedings of the Third International Conference on Teaching Statistics (pp. 314-319). Dunedin, New Zealand: International Statistical Institute.
32 Rubin, A.; Hammerman, J., & Konold, C. (2006). Exploring informal inference with interactive visualization software. In: A. Rossman & B. Chance (Eds.), Proceedings of the Seventh International Conference on Teaching Statistics (ICOTS) held at Salvador, Bahai, Brazil, July 2-7, 2006 (CD-ROM). Voorburg, Netherlands: International Statistical Institute.
33 Sedlemeier, P. & Gigerenzer, G. (1997). Intuitions about sample size: The empirical law of large numbers. Journal of Behavior Decision Making 10, 33-51.   DOI
34 Saldanha, L. (2004). "Is this sample unusual?": An investigation of students exploring connections between sampling distributions and statistical inference. Unpublished Ph.D. Thesis. Nashville, TN: Vanderbilt University.
35 Saldanha, L. & Thompson, P. (2002). Conceptions of sample and their relationship to statistical inference. Educ. Stud. Math. 51(3), 257-270. ME 2003d.03486   DOI   ScienceOn
36 Schwartz, D. L.; Goldman, S. R.; Vye, N. J.; Barron, B. J. & Cognition Technology Group at Vanderbilt (1998). Aligning everyday and mathematical reasoning: The case of sampling assumptions. In: S. Lajoie (Ed.), Reflections on Statistics: Learning, Teaching, and Assessment in Grades K-12 (pp. 233-273). Hillsdale, NJ: Erlbaum.
37 Sotos, A. E. C.; Vanhoof, S.; Van den Noortgate, W. & Onghena, P. (2007). Students' misconceptions of statistical inference: A review of the empirical evidence from research on statistics education. Educational Research Review 2, 98-113.   DOI   ScienceOn
38 Tversky, A. & Kahneman, D. (1971). Belief in the law of small numbers. Psychological Bulletin 76, 105-110. [Reprinted in: D. Kahneman, P. Slovic & A. Tversky (1982), Judgment under uncertainty: Heuristics and biases. Cambridge University Press.]
39 Tversky, A. & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science 185(4157), 1124-1131. Retrieved from: http://files.eric.ed.gov/fulltext/EJ776331.pdf   DOI   ScienceOn
40 Vallecillos, A. (1999). Some empirical evidences on learning difficulties about testing hypotheses. In: Proceedings of the 52 session of the International Statistical Institute (pp. 201-204). Helsinki: International Statistical Institute. Retrieved from: https://www.stat.auckland.ac.nz/-iase/publications/5/vall0682.pdf
41 Vallecillos, A. (2002). Empirical evidence about understanding of the level of significance concept in hypotheses testing by university students. Themes in Education 3(2), 183-198.
42 Vallecillos, A. & Batanero, C. (1997). Activated concepts in the statistical hypothesis contrast and their understanding by unversity students. Reserchers en Didactique des Mathematiques 17, 29-48.
43 Vanhoof, S.; Sotos, A.; Onghena, P. & Verschaffel, L. (2007). Students' reasoning about sampling distribution before and after the sampling distribution activity. In: Proceedings of the 56 session of the International Statistical Institute, Lisbon, Spain, International Statistical Institute. [Online]: www.stat.auckland.ac.nz/-iase/publications/isi56/CPM80_Vanhoof.pdf.
44 Wagner, D. A. & Gal, I. (1991). Project STARC: Acquisition of statistical reasoning in children. (Annual Report: Year 1, NSF Grant No. MDR90-50006). Philadelphia, PA: Literacy Research Center, University of Pennsylvania.