Korean Journal of Cognitive Science (인지과학)

The Korean Society for Cognitive Science (한국인지과학회)
권호 표지
DOI QR코드

DOI QR Code

The validity of using cumulative peer assessed scores for final grades in college courses

대학 수업에서 누적 동료평가 점수를 활용한 성적 산출 방법의 타당성

  • Bae, Soo Jung (Department of Psychology & Institute of Psychological Sciences Seoul National University) ;
  • Park, Joo Yong (Department of Psychology & Institute of Psychological Sciences Seoul National University)
  • 배수정 (서울대학교 심리학과 & 심리과학연구소) ;
  • 박주용 (서울대학교 심리학과 & 심리과학연구소)
  • Received : 2016.05.28
  • Accepted : 2016.05.30
  • Published : 2016.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

Peer assessment refers to having students, rather than the instructor, make assessments of one another's work. Peer assessment is often used as a tool to train writing skills or a tool to apply or extend learning in higher education. Park(2016) recently proposed a system which utilizes peer assessment as a part of preparatory activity for college courses. Before weekly class, students studied given material on their own, wrote a one page essay on a given question based on their reading, and assessed the essays of other students. In this study, the system was implemented in undergraduate courses at S University over 2 semesters and the results were analyzed. The reliability of weekly scores given by students was not very high, but the correlation was high between the cumulative scores given by students across weeks and the scores of the end of the term paper assessed by the instructor. Based on these findings, the possibility of utilizing the results of the peer assessments as part of the final grades was discussed.

동료평가 (peer assessment)란 통상 교수자가 주관하는 평가를 학생들로 하여금 서로에 대해 하도록 하는 활동을 가리킨다. 동료평가는 고등교육에서 글쓰기를 훈련시키기 위한 도구 혹은 배운 지식을 적용하거나 확장하는 학습 활동을 위한 도구로 사용된다. Park(2016)은 최근 동료평가를 예습 활동의 한 부분으로 확장하는 방안을 제안하였다. 학생들은 매주의 수업에 앞서 스스로 공부하고, 공부한 내용을 바탕으로 하여 대답할 수 있는 질문에 대해 한 페이지 글을 쓰고 다른 학생들의 글을 평가하였다. 본 연구에서는, 이 시스템을 S 대학의 학부수업에 두 학기에 걸쳐 적용한 결과가 분석되었다. 학생들 간의 채점 신뢰도와, 학생들의 채점 결과와 교수자에 의한 최종 보고서 평가 간의 상관이 분석되었다. 매주의 동료평가에서 학생들이 부여한 점수간의 신뢰도는 그리 높지 않았지만, 이들이 누적된 점수와 교수가 평가한 기말 보고서 점수간의 상관은 두 수업 모두에서 유의미하게 높았다. 논의에서는 이 결과를 바탕으로 대학에서 예습을 위한 동료평가의 결과를 성적에 반영하는 방안의 확장 가능성이 다루어졌다.

Keywords

References

  1. Boud, D. (1989). The role of self-assessment in student grading. Assessment in Higher Education, 14(1), 20-30. https://doi.org/10.1080/0260293890140103
  2. Brown, S., & Smith, B. M. (1997). Getting to grips with assessment. Birmingham: SEDA.
  3. Cheng, W., & Warren, M. (1999). Peer and teacher assessment of the oral and written tasks of a group project. Assessment & Evaluation in Higher Education, 24(3), 301-314. https://doi.org/10.1080/0260293990240304
  4. Cho, K., Chung, T. R., King, W. R., & Schunn, C. (2008). Peer-based computer-supported knowledge refinement: An empirical investigation. Communications of the ACM, 51(3), 83-88. https://doi.org/10.1145/1325555.1325571
  5. Cho, K., & MacArthur, C. (2011). Learning by reviewing. Journal of Educational Psychology, 103(1), 73. https://doi.org/10.1037/a0021950
  6. Cho, K., & Schunn, C. D. (2007). Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system. Computers & Education, 48(3), 409-426. https://doi.org/10.1016/j.compedu.2005.02.004
  7. Cho, K., Schunn, C. D., & Wilson, R. W. (2006). Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives. Journal of Educational Psychology, 98(4), 891. https://doi.org/10.1037/0022-0663.98.4.891
  8. Cho, Y. H., & Cho, K. (2011). Peer reviewers learn from giving comments. Instructional Science, 39(5), 629-643. https://doi.org/10.1007/s11251-010-9146-1
  9. Cohen, L., Manion, L., & Morrison, K. K. (2000). Research methods in education. London and New York: Falmer.
  10. Davies, P. (2000). Computerized peer assessment. Innovations in Education and Teaching International, 37(4), 346-355. https://doi.org/10.1080/135580000750052955
  11. Davis, A., & Rose, D. (2000). The experimental method in psychology. Research methods in psychology, 2, 42-58.
  12. Fabos, B., & Young, M. D. (1999). Telecommunication in the classroom: Rhetoric versus reality. Review of educational research, 69(3), 217-259. https://doi.org/10.3102/00346543069003217
  13. Falchikov, N. (1986). Product comparisons and process benefits of collaborative peer group and self assessments. Assessment and Evaluation in Higher Education, 11(2), 146-166. https://doi.org/10.1080/0260293860110206
  14. Falchikov, N. (1995). Improving feedback to and from students. Assessment for learning in higher education, 1.
  15. Falchikov, N. (2005). Improving assessment through student involvement: Practical solutions for aiding learning in higher and further education. Routledge.
  16. Falchikov, N., & Goldfinch, J. (2000). Student peer assessment in higher education: A meta-analysis comparing peer and teacher marks. Review of educational research, 70(3), 287-322. https://doi.org/10.3102/00346543070003287
  17. Freeman, M. (1995). Peer assessment by groups of group work. Assessment & Evaluation in Higher Education, 20(3), 289-300. https://doi.org/10.1080/0260293950200305
  18. Gielen, S., Peeters, E., Dochy, F., Onghena, P., & Struyven, K. (2010). Improving the effectiveness of peer feedback for learning. Learning and Instruction, 20(4), 304-315. https://doi.org/10.1016/j.learninstruc.2009.08.007
  19. Hamer, J., Purchase, H. C., Denny, P., & Luxton-Reilly, A. (2009). Quality of peer assessment in CS1. In Proceedings of the fifth international workshop on Computing education research workshop (pp. 27-36). ACM.
  20. Hamer, J., Purchase, H., Luxton-Reilly, A., & Denny, P. (2015). A comparison of peer and tutor feedback. Assessment & Evaluation in Higher Education, 40(1), 151-164. https://doi.org/10.1080/02602938.2014.893418
  21. Hunter, J. E. (1983). A causal analysis of cognitive ability, job knowledge, job performance, and supervisor ratings. Performance measurement and theory, 257, 266.
  22. Karpicke, J. D., & Roediger, H. L. (2008). The critical importance of retrieval for learning. science, 319(5865), 966-968. https://doi.org/10.1126/science.1152408
  23. Kwok, R. C., & Ma, J. (1999). Use of a group support system for collaborative assessment. Computers & Education, 32(2), 109-125. https://doi.org/10.1016/S0360-1315(98)00059-1
  24. Liu, E. Z. F., Lin, S. S., Chiu, C. H., & Yuan, S. M. (2001). Web-based peer review: the learner as both adapter and reviewer. Education, IEEE Transactions on, 44(3), 246-251. https://doi.org/10.1109/13.940995
  25. Lynch, D. H., & Golen, S. (1992). Peer evaluation of writing in business communication classes. Journal of Education for Business, 68(1), 44-48. https://doi.org/10.1080/08832323.1992.10117585
  26. Magin, D. (2001). Reciprocity as a source of bias in multiple peer assessment of group work. Studies in Higher Education, 26(1), 53-63. https://doi.org/10.1080/03075070020030715
  27. Moskal, B. M., Leydens, J. A., & Pavelich, M. J. (2002). Validity, reliability and the assessment of engineering education. Journal of Engineering Education, 91(3), 351-354. https://doi.org/10.1002/j.2168-9830.2002.tb00714.x
  28. Nunnally, J. C., Bernstein, I. H., & Berge, J. M. T. (1967). Psychometric theory (Vol. 226). New York: McGraw-Hill.
  29. Park, J. (2016). ClassPrep: A peer review system for class preparation. British Journal of Educational Technology.
  30. Rada, R. (1998). Efficiency and effectiveness in computer-supported peer-peer learning. Computers and Education, 30(3), 137-146. https://doi.org/10.1016/S0360-1315(97)00042-0
  31. Rosenthal, R., & Rosnow, R. L. (1991). Essentials of behavioral research: Methods and data analysis. New York: McGraw-Hill Book Company.
  32. Rushton, C. (1993). Peer Assessment in a Collaborative Hypermedia Environment: A Case Study. Journal of Computer-Based Instruction, 20(3), 75-80.
  33. Sadler, D. R. (1989). Formative assessment and the design of instructional systems. Instructional science, 18(2), 119-144. https://doi.org/10.1007/BF00117714
  34. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: uses in assessing rater reliability. Psychological bulletin, 86(2), 420. https://doi.org/10.1037/0033-2909.86.2.420
  35. Stefani, L. A. (1994). Peer, self and tutor assessment: relative reliabilities. Studies in Higher Education, 19(1), 69-75. https://doi.org/10.1080/03075079412331382153
  36. Stiggins, R. J. (2001). Student-involved classroom assessment. Prentice Hall.
  37. Strachan, I. B., & Wilcox, S. (1996). Peer and self assessment of group work: developing an effective response to increased enrolment in a third-year course in microclimatology. Journal of Geography in Higher Education, 20(3), 343-353. https://doi.org/10.1080/03098269608709377
  38. Suen, H. K. (2014). Peer assessment for massive open online courses (MOOCs). The International Review of Research in Open and Distributed Learning, 15(3).
  39. Swanson, D. B., Case, S. M., & van der Vleuten, C. P. (1991). Strategies for student assessment. The challenge of problem based learning, 260-273.
  40. Taylor, P. J. (2010). An introduction to intraclass correlation that resolves some common confusions. Unpublished manuscript, University of Massachusetts, Boston, USA. Retrieved from http://www.faculty.umb.edu/peter_taylor/09b.pdf.
  41. Topping, K. (1998). Peer assessment between students in colleges and universities. Review of educational Research, 68(3), 249-276. https://doi.org/10.3102/00346543068003249

Cited by

  1. 증명 동료평가의 신뢰도 및 타당도 분석: 대학 정수론 수업의 사례를 중심으로 vol.57, pp.3, 2016, https://doi.org/10.7468/mathedu.2018.57.3.215