Browse > Article
http://dx.doi.org/10.7468/mathedu.2018.57.3.215

The Reliability and Validity of Online Peer Assessment on Proofs in a Number Theory Course  

Oh, Yaerin (Graduate School of Seoul National University)
Kwon, Oh Nam (Department of Mathematics Education, Seoul National University)
Park, Jooyong (Department of Psychology, Seoul National University)
Publication Information
The Mathematical Education / v.57, no.3, 2018 , pp. 215-229 More about this Journal
Abstract
Despite the importance of learning to do mathematical proofs, researchers have reported that not only secondary school students but also undergraduate students have difficulties in learning proofs. In this study, we introduced a new toll for learning proofs and explored the reliability and the validity of peer assessment on proofs. In the course of a university in Seoul, students were given weekly proof assignments prior to class. After solving the proofs, each student had to assess other students' proofs. The inter-rater reliabilities of weekly peer assessment was higher than .9 over 90 percent of the observed cases. To examine the validity of peer assessment, we check whether students' assessments were similar to expert assessment. Analysis showed that the equivalence has been quite high throughout the semester and the validity was low in the middle of the semester but rose by the end of the semester. Based on these results, we believe instructors can consider the application of peer assessment on proving tasks as a tool to help students learn.
Keywords
proof; peer assessment; online assessment; reliability; validity;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Kang, A., & Lee, G. (2006). A Generalizability Theory Approach to Investigating the Generalizability of Performance Assessment Using Student Peer Reviews. Journal of Education Evaluation 19(3), 107-121.
2 Kim, J. H., & Jo, Y. M. (2006). Effects of Evaluation Types according to Learning Styles on Students' Mathematical Disposition and Problem-solving Ability, Journal of Education Evaluation 19(2), 21-39.
3 Bae, S. J., & Park, J. Y. (2016). The validity of using cumulative peer assessed scores for final grades in college courses, Korean Journal of Cognitive Science 27(2), 221-245.   DOI
4 Shin, J. H. (2014). Study on Formative Peer Review of Teaching Program in Higher Education, Journal of Korean Teacher Education 31(3), 371-398.   DOI
5 Korean Society for Educational Evaluation (2004). Educational evaluation thesaurus, Seoul: Hakjisa.
6 Brown, D. E. & Michel, S. (2010). Assessing proofs with rubrics: the RVF method, In Proceedings of the 13th Annual Conference on Research in Undergraduate Mathematics Education, Raleigh, NC. Retrieved from http://sigmaa.maa.org/rume/crume2010/Archive/Brown_D.pdf.
7 Bryman, A. & Cramer, D. (1990). Quantitative data analysis for social scientists, Taylor & Frances: Routledge.
8 Cho, K., Schunn, C. D., & Wilson, R. W. (2006). Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives, Journal of Educational Psychology 98(4), 891-901.   DOI
9 Cho, K. & Schunn, C. D. (2007). Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system, Computers & Education 48(3), 409-426.   DOI
10 Cohen, L., Manion, L., & Morrison, K. (2011). Research methods in education, New York: Routledge.
11 Cronbach, L. J. (1951). Coefficient alpha and the internal structure of tests, Psychometrika 16(3), 297-334.   DOI
12 Dochy, F., Segers, M., & Sluijsmans, D. M. A. (1999). The use of self-, peer and co-assessment in higher education: A review, Studies in Higher Education 24(3), 331-350.   DOI
13 Gaillet, L. L. (1992). A Foreshadowing of Modern Theories and Practices of Collaborative Learning: The Work of Scottish Rhetorician George Jardine. Paper presented at the 43rd Annual Meeting of the Conference on College Composition and Communication, Cincinnati, OH.
14 Johnson, R., Penny, J., Gordon, B., Shumate, S., & Fisher, S. (2005). Resolving Score Differences in the Rating of Writing Samples: Does Discussion Improve the Accuracy of Scores? Language Assessment Quarterly 2(2), 117–146.   DOI
15 Harel, G. & Sowder, L. (2007). Toward comprehensive perspectives on the learning and teaching of proof, Second handbook of research on mathematics teaching and learning. Greenwich: Information Age.
16 Hunter, D. & Russ, M. (1996). Peer assessment in performance studies, British Journal of Music Education 13, 67-78.   DOI
17 Jeffery, D., Yankulov, K., Crerar, A., & Ritchie, K. (2016). How to achieve accurate peer assessment for high value written assignments in a senior undergraduate course, Assessment & Evaluation in Higher Education 41(1), 127-140.   DOI
18 Lavy, I. & Shriki, A. (2014). Engaging prospective teachers in peer assessment as both assessors and assessees: The case of geometrical proofs. International Journal for Mathematics Teaching and Learning. Retrieved from http://www.cimt.org.uk/journal/lavy2.pdf
19 Li, H., Xiong, Y., Zang, X., Kornhaber, M. L., Lyu, Y., Chung K. S., & Suen, H. K. (2016). Peer assessment in the digital age: A meta-analysis comparing peer and teacher ratings, Assessment & Evaluation in Higher Education 41(2), 245-264.   DOI
20 Moore, R. C. (1994). Making the transition to formal proof, Educational Studies in Mathematics 27(3), 249-266.   DOI
21 Moore, R. C. (2016). Mathematics professors' evaluation of students' proofs: A complex teaching practice, International Journal of Research in Undergraduate Mathematics Education 2(2), 246-278.   DOI
22 Nunnaly, J. (1978). Psychometric theory, New York: McGraw-Hill.
23 Park, J. (2017). ClassPrep: A peer review system for class preparation, British Journal of Educational Technology 48(2), 511-523.   DOI
24 O'Donnell, A. M. & Topping, K. J. (1998). Peers assessing peers: Possibilities and problems. In K. J. Topping & S. Ehly (Eds), Peer-assisted learning (255-278). Mahwah, NJ: Lawrence Erlbaum Associates.
25 Orsmond, P., Merry, S., & Reiling, K. (2000). The use of student derived marking criteria in peer and self-assessment, Assessment and Evaluation in Higher Education 25(1), 23-38.   DOI
26 Panadero, E., Romero, M., & Strijbos, J. W. (2013). The impact of a rubric and friendship on peer assessment: Effects on construct validity, performance, and perceptions of fairness and comfort, Studies in Educational Evaluation 39(4), 195-203.   DOI
27 Santos, J. R. A. (1999). Cronbach's alpha: A tool for assessing the reliability of scales, Journal of extension 37(2), 1-5.
28 Scruggs, T. E., & Mastropieri, M. A. (1998). Tutoring and students with special needs. In K. J. Topping & S. Ehly (Eds.), Peer-assisted learning (165-182). Mahwahm NJ: Lawrence Erlbaum Associates.
29 Smith, J. C. (2006). A sense-making approach to proof: Strategies of students in traditional and problem-based number theory courses, Journal of Mathematical Behavior 25, 73-90.   DOI
30 Smith, J. C., Nichols, S. R., Yoo, S., & Oehler, K. (2009). Building a community of inquiry in a problem-based undergraduate number theory course. In D. A. Stylianou, M. L. Blanton, & E. J. Knuth (Eds.), Teaching and learning proof across the grades: A K-16 perspective. New York: Routledge.
31 Stefani, L. A. (1994). Peer, self and tutor assessment: Relative reliabilities, Studies in Higher Education 19(1), 69-75.   DOI
32 Topping, J. K. & Ehly, S. (1998). Peer assisted learning, Mahwah, NJ: Lawrence Erlbaum Associates.
33 Stylianou, D. A., Blanton, M. L., & Knuth, E. J. (2009). Teaching and learning proof across the grades: A K-16 perspective. New York: Routledge.
34 Taylor, P. J. (2010). An introduction to intraclass correlation that resolves some common confusions. Unpublished manuscript, University of Massachusetts, Boston, USA. Retrieved from http://www.faculty.umb.edu/peter_taylor/09b.pdf.
35 Topping, J. K. (2009). Peer assessment. Theory Into Practice 48(1), 20-27.   DOI
36 Topping, J. K., Smith, F. F., Swanson, I., & Elliot, A. (2000). Formative peer assessment of academic writing between postgraduate students, Assessment and Evaluation in Higher Education 25(2), 149-169.   DOI
37 Weber, K. (2001). Student difficulty in constructing proofs: The need for strategic knowledge, Educational studies in mathematics 48(1), 101-119.   DOI
38 Weber, K., & Mejía-Ramos, J. P. (2014). Mathematics majors' beliefs about proof reading International Journal of Mathematical Education in Science and Technology 45(1), 89-103.   DOI