Journal of The Korean Association For Science Education (한국과학교육학회지)

The Korean Association for Science Education (한국과학교육학회)
권호 표지
DOI QR코드

DOI QR Code

An Analysis of the Characteristics of the Processes of Pre-Service Chemistry Teachers in Making Written Test Items Using Think-Aloud Method

발성 사고법을 이용한 예비 화학 교사의 지필평가 문항 제작 과정의 특징 분석

  • Received : 2017.02.06
  • Accepted : 2017.04.15
  • Published : 2017.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this study, we investigated the characteristics of the individual processes of pre-service chemistry teachers in making written test items. For this, we analyzed the think-aloud processes of eight pre-service chemistry teachers while making six written test items about gas laws and their in-depth interview scripts. The characteristics of the processes were found to be divided into six categories; the lack of systemicity in planning stage for making written test items, the lack of awareness and consideration for the interrelationship between instruction and assessment, the diverse criteria in determining test item types, the difficulties in judging behavioral domains and difficulties of items in the table of specification, the limitation in using teacher's guides and misunderstanding of teacher's guides, and the limitation in point and/or content of review and differences depending on the time of review. These can provide some significant guidelines and implications in finding ways to improve pre-service chemistry teachers' ability to make written test items.

이 연구에서는 예비 화학 교사가 개별적으로 지필평가 문항을 제작하는 과정의 특징을 분석하였다. 이를 위하여 8명의 예비 화학 교사가 기체 법칙에 대한 6개의 지필평가 문항을 제작하는 발성 사고 과정과 개별 심층 면담 전사본을 분석하였다. 분석 결과, 예비교사들은 지필평가 문항 제작 과정에서 문항 제작 계획 단계의 체계성 부족, 수업과 평가의 상호관련성에 대한 인식 및 고려 부족, 문항 유형 결정 기준의 다양성, 문항의 행동 영역과 난이도 판별 곤란, 교사용 지도서 활용의 제한성과 교사용 지도서에 대한 오해, 검토 관점과 내용의 제한성 및 검토시기에 따른 차이점 등의 6가지 특징을 보였다. 이는 예비교사의 지필평가 문항 제작 능력을 향상시키는 방안을 모색하는 데 의미있는 시사점을 제공할 수 있다.

Keywords

References

  1. Abell, S. K., & Siegel, M. A. (2011). Assessment literacy: What science teachers need to know and be able to do. In D. Corrigan, J. Dillon, & R. Gunstone (Eds.), The professional knowledge base of science teaching (pp. 205-221). London, UK: Springer.
  2. Atjonen, P. (2014). Teachers' views of their assessment practice. The Curriculum Journal, 25(2), 238-259. https://doi.org/10.1080/09585176.2013.874952
  3. Baek, S.-G. (2000). Principles of performance assessment [수행평가의 원리]. Seoul: Kyoyookbook Publication Co.
  4. Bismack, A. S., Arias, A. M., Davis, E. A., & Palincsar, A. S. (2014). Connecting curriculum materials and teachers: Elementary science teachers' enactment of a reform-based curricular unit. Journal of Science Teacher Education, 25(4), 489-512. https://doi.org/10.1007/s10972-013-9372-x
  5. Black, P., Harrison, C., Lee, C., Marshall, B., & Wiliam, D. (2004). Working inside the black box: Assessment for learning in the classroom. Phi Delta Kappan, 86(1), 8-21. https://doi.org/10.1177/003172170408600105
  6. Brookhart, S. M. (2011). Educational assessment knowledge and skills for teachers. Educational Measurement: Issues and Practice, 30(1), 3–12. https://doi.org/10.1111/j.1745-3992.2010.00195.x
  7. Cho, H., & Ko, Y. (2008). Re-conceptualization of secondary science teacher's pedagogical content knowledge (PCK) and its application. Journal of the Korean Association for Research in Science Education, 28(6), 618-632.
  8. Cho, K.-W. (2004). A critical examination of secondary teacher education for secondary school teachers. Journal of Educational Studies, 35(1), 1-19. https://doi.org/10.1207/s15326993es3501_1
  9. Choi, H. S., & Kim, J. B. (2013). A study on performance level of pre-service physics teachers in constructing questions for classroom assessment - Focused on analysis of multiple choice question about physics conceptest for formative assessment. Journal of Science Education, 37(3), 458-475. https://doi.org/10.21796/jse.2013.37.3.458
  10. Choi, J.-I., & Paik, S.-H. (2016). An analysis of content validity of behavioral domain of descriptive tests and factors that affect content validity: Focus on the fifth and sixth grade science. Journal of the Korean Association for Science Education, 36(1), 87-101. https://doi.org/10.14697/jkase.2016.36.1.0087
  11. Choi, J.-Y. (2006). An qualitative case study of pre-service teacher's planning for the practical arts instruction in the elementary school. The Journal of Elementary Education, 19(1), 477-499.
  12. Davis, E. A., Palincsar, A. S., Arias, A. M., Bismack, A. S., Marulis, L., & Iwashyna, S. (2014). Designing educative curriculum materials: A theoretically and empirically driven process. Harvard Educational Review, 84(1), 24-52. https://doi.org/10.17763/haer.84.1.g48488u230616264
  13. Falk, A. (2012). Teachers learning from professional development in elementary science: Reciprocal relations between formative assessment and pedagogical content knowledge. Science Education, 96(2), 265-290. https://doi.org/10.1002/sce.20473
  14. Ferguson, L. E., Braten, I., & Stromso, H. I. (2012). Epistemic cognition when students read multiple documents containing conflicting scientific evidence: A think-aloud study. Learning and Instruction, 22(2), 103-120. https://doi.org/10.1016/j.learninstruc.2011.08.002
  15. Grossman, P., & Thompson, C. (2004). Curriculum materials: Scaffolds for new teacher learning? (A Research Report No. R-04-1). Seattle, WA: Center for the Study of Teaching and Policy.
  16. Han, K.-A., & Noh, S.-G. (2003). An analysis on the utilization of teacher's guides for science in elementary school. Journal of Korean Elementary Science Education, 22(1), 51-64.
  17. Hashweh, M. Z. (2005). Teacher pedagogical constructions: A reconfiguration of pedagogical content knowledge. Teachers and Teaching, 11(3), 273-292. https://doi.org/10.1080/13450600500105502
  18. Jang, S.-M., & Kim, J.-Y. (2002). Analysis on the status of performance assessment in science based on the elementary teachers' concerns. Journal of Korean Elementary Science Education, 21(2), 227-240.
  19. Jeong, E., & Choi, W. (2014). A survey on evaluation in science education at primary and secondary school in Korea. Journal of Science Education, 38(1), 168-181. https://doi.org/10.21796/jse.2014.38.1.168
  20. Jonassen, D. H. (1991), Evaluating constructivistic learning. Educational Technology, 31(9), 28-33.
  21. Kang, H., & Kang, S. (2015). Analysis of elementary school teachers' self-diagnosis on their competency for assessment in science. Journal of Korean Elementary Science Education, 34(2), 153-163. https://doi.org/10.15267/keses.2015.34.2.153
  22. Kim, H. J., & Yoo, J. (2012). An analysis on rater error in holistic scoring for performance assessments of middle school students' science investigation activities. Journal of the Korean Association for Research in Science Education, 32(1), 160-181. https://doi.org/10.14697/jkase.2012.32.1.160
  23. Kim, H.-J., Kwack, D.-O., & Sung, M.-W. (2000). An investigation on science teachers' evaluation practices in the secondary schools. Journal of the Korean Association for Science Education, 20(1), 101-111.
  24. Kim, K., Park, E., Song, M., Sang, K., Kim, S., Kim, H., Shin, J., Seo, J., Lee, C., Kim, J., Kim, K. & Choi, S. (2012). A study on management of standards-based assessment in secondary schools [중등학교의 성취평가제 운영 방안 연구]. Seoul: Korea Institute for Curriculum and Evaluation (CRE 2012-8).
  25. Kim, K.-M. & Kim, S.-W. (2002). A study on the weight of assessment domains in science education focused on the teacher's view points. Journal of the Korean Association for Science Education, 22(3), 540-549.
  26. Kim, S. (2002). A study on the teacher's competence for classroom assessment. Journal of Educational Evaluation, 15(1), 51-171.
  27. Kim, S.-W., & Hyun, M.-S. (2005). The study on the recognition of science teachers about the general matters of performance assessment and the appropriate performance assessment methods in middle school science curriculum. Journal of Research in Curriculum Instruction, 9(2),
  28. Kim, Y., Park, J., Park, J., Lee, H., & Kim, Y. (2010). Science teachers' perceptions and needs for courses in science education subjects for science teacher preparation program in Korea. Journal of the Korean Association for Research in Science Education, 30(6), 785-798.
  29. Ko, M.-S., Kim, E.-A., Heo, J.-M., & Y. I.-H. (2013). Elementary school teachers' beliefs of inquiry and practice of science performance assessment. Journal of the Korean Society of Earth Science Education, 6(2), 124-135.
  30. Kwon, J.-S., Kim, B. K., Choi, B.-S., Kim, H.-N., Paik, S.-H., Yang, I.-H., Kwon, Y.-J., Cha, H., Woo, J.-O., & Jeong, J.-W. (2012). Theories in science education [과학교육론]. Seoul: Kyoyookbook Publication Co.
  31. Lee, I., Lee, B., Kim, S., Park, J., Jin, J., Kim, O., & Seo, S. (2004). Analyzing realities of teacher's student assessment and educational needs for their professional development [교사의 학생 평가 실태 조사 및 전문성 신장에 대한 요구분석]. Seoul: Korea Institute for Curriculum and Evaluation (RRE 2004-5-1).
  32. Lee, K. H., & Choi, M. Y. (2013). Critical examination of 2009 revised teacher's guide book by an aspect of pedagogical content knowledge. Journal of Learner-Centered Curriculum and Instruction, 13(6), 667-689.
  33. Magnusson, S., Krajcik, J., & Borko, H. (1999). Nature, source, and development of pedagogical content knowledge for science teaching. In J. Gess-Newsome, & N. Lederman (Eds.), Examining pedagogical content knowledge. Dordrect, The Netherlands: Kluwer Academic Publishers.
  34. Marzano, R. J. (2006). Classroom assessment and grading that work. Alexandria, VA: Association for Supervision and Curriculum Development.
  35. McMillan, J. H. (2014). Classroom assessment: Principles and practice for effective standards-based instruction (6th Ed.). Boston, MA: Pearson.
  36. McTighe, J., & Wiggins, G. (2004). Understanding by design: Professional development workbook. Alexandria, Virginia: Association for Supervision and Curriculum Development.
  37. Min, H. J. (2012). Development of assessment expertise model through analyzing realities of science teacher's student assessment and teacher training. (Doctoral dissertation). Korea National University of Education, Cheongju.
  38. Mulholland, J., & Wallace, J. (2005). Growing the tree of teacher knowledge: Ten years of learning to teach elementary science. Journal of Research in Science Teaching, 42(7), 767–790. https://doi.org/10.1002/tea.20073
  39. Nam, M., Park, S., Song, M., Kim, K., Kim, S., Cho, I., Lim, W., Lee, K., Oh, S., Kang, M. & Kang, J. (2006). A study on improvement of teacher's competence in student assessment(III) [교사의 학생평가 전문성 신장 연구(III)]. Seoul: Korea Institute for Curriculum and Evaluation (RRE 2006-5).
  40. Noh, T., Lee, J., Kang, S., & Kang, H. (2015). Secondary school science teachers' actual and preferred types of assessment. Journal of the Korean Association for Science Education, 35(4), 725-733. https://doi.org/10.14697/jkase.2015.35.4.0725
  41. Noh, T., Yoon, J., & Kang, S. (2009). The investigation of elementary school teachers' perceptions toward constructivist science assessment and their relationship with related variables. Journal of Korean Elementary Science Education, 28(3), 352-360.
  42. Park, C. (2003). The relationship between item types and student's achievement. Korean Journal of Educational Research, 41(4), 71-86.
  43. Park, D. (2012). Educational evaluation: Understanding and application. Seoul: Kyoyookkwahaksa.
  44. Park, H. (2016). A survey on the conditions of middle school science evaluation. Teacher Education Research, 55(3), 389-398. https://doi.org/10.15812/ter.55.3.201609.389
  45. Park, H. J., Jeong, D. H., & Choi, W. H. (2011). Science teachers' perceptions of science practices. Journal of the Korean Association for Science Education, 31(1), 61-77.
  46. Park, K. (2008). A case study on instructional planning process of teachers. The Journal of Korean Teacher Education, 25(3), 379-405. https://doi.org/10.24211/tjkte.2008.25.3.379
  47. Park, S., & Oliver, J. S. (2008). Revisiting the conceptualization of pedagogical content knowledge (PCK): PCK as a conceptual tool to understand teachers as professionals. Research in Science Education, 38(3), 261-284. https://doi.org/10.1007/s11165-007-9049-6
  48. Park, S., Kang, C., Kim, K., Lee, G., Lee, M., Shon, M., Lee, H. (2003). An Evaluative Study on the 7th Primary Curriculum(III) [제7차 초.중등학교 교육과정 평가연구(III)]. Seoul: Korea Institute for Curriculum and Evaluation (RRC 2003-2).
  49. Sato, M., Coffey, J., & Moorthy, S. (2005). Two teachers making assessment for learning their own. Curriculum Journal, 16(2), 177-191. https://doi.org/10.1080/09585170500135996
  50. Schellings, G. L. M., & Broekkamp, H. (2011). Signaling task awareness in think-aloud protocols from students selecting relevant information from text. Metacognition and Learning, 6(1), 65-82. https://doi.org/10.1007/s11409-010-9067-z
  51. Shin, J., Ryu, S., & Yang, I. (2016). Teachers' needs and utilization of science performance assessment in the elementary schools. KNUE Journal of Research in Science Education, 22(2), 18-29.
  52. Song, S.-C., Lee, C.-H., & Shim, K.-C. (2014). A study on science teachers' perception about science teacher training curriculum. Teacher Education Research, 53(1), 15-27. https://doi.org/10.15812/ter.53.1.201403.15
  53. Strauss, A., & Corbin, J. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. Thousand Oaks, CA: Sage.
  54. Wiggins, G., & McTighe, J. (2005). Understanding by design. (2nd ed.). Alexandria, Virginia: Association for Supervision and Curriculum Development.
  55. Yang, I.-H., Na, J.-C., Lim, S.-M., Lim, J.-K., & Choi, H.-D. (2008). An analysis of elementary schools' science test items by Klopfer's taxonomy of educational objectives: Focusing on the first term of the 5th grade. Journal of Korean Elementary Science Education, 27(3), 221-232.