Browse > Article
http://dx.doi.org/10.14697/jkase.2011.31.6.887

The Analysis of Pre-Service Biology Teachers' Natural Selection Conceptions in Multiple-Choice and Open-Response Instruments  

Ha, Min-Su (The Ohio State University)
Lee, Jun-Ki (Chonbuk National University)
Publication Information
Journal of The Korean Association For Science Education / v.31, no.6, 2011 , pp. 887-900 More about this Journal
Abstract
Teachers use explanations to communicate important scientific ideas to students. Consequently, all biology teachers should be evaluated to determine how effective they are at constructing and communicating biological explanations. Open response questions are required to detect pre-service biology teachers' abilities to communicate robust and accurate scientific explanations. Nevertheless, multiple-choice questions are typically preferred by educators because of the common drawbacks of using open-response instruments, such as scoring time, inter-rater scoring disagreements, and delayed feedback to test takers. This study aims to measure pre-service biology teachers' competence in building scientific explanations and to investigate how accurately multiple-choice questions predict the results of open-response questions. One hundred twenty four pre-service biology teachers participated in the study and were administered 20 multiple-choice items and three open-response items designed to measure the accuracy and quality of their explanations of evolutionary change. The results demonstrated that pre-service teachers displayed higher competence when tested with multiple choice items than when tested with open response items. Moreover, scores derived from multiple-choice items poorly predicted the scores derived from open-response items. Multiple-choice items were also found to be poor measures of the consistency, purity and abundance of conceptual elements in teachers' evolutionary explanations. Additionally, many teachers held mixed-models composed of both scientific and naive ideas, which were difficult to detect using multiple-choice formats. Overall, the study indicates that multiple-choice formats are poorly suited to measuring several aspects of biology teachers' knowledge of evolution, including their ability to generate scientific explanations. This study suggests that open-response items should be used in teacher education programs to assess pre-service teachers' explanatory competency prior to being permitted to teach science to children.
Keywords
pre-service biology teachers; natural selection; multiple-choice; open-response; scientific explanation;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Vosniadou, S. (2007). Conceptual change and education. Human Development, 50, 47- 54.   DOI   ScienceOn
2 Vosniadou, S., & Brewer, W. F. (1992). Mental models of the earth: A study of conceptual change in childhood. Cognitive Psychology, 24, 535-585.   DOI
3 Wittmann, M. (1998). Making sense of how students come to an understanding of physics: An example from mechanical waves. Unpublished doctoral dissertation, University of Maryland.
4 Ha, M. & Cha, H. Y. (2009). Pre-service teachers'synthetic view on Darwinism and Lamarckism. Paper in the proceedings at 2009 International Conference of National Association for Research in Science Teaching, Garden Grove, CA.
5 Hammer, D. M., Elby, A., Scherr, R. E., & Redish, E. F. (2005). Resources, framing, and transfer. In J. Mestre (Ed.), Transfer of learning from a modern multidisciplinary perspective (pp. 89-120). Greenwich, CT: Information Age Publishing.
6 Kwon, Y. J., Lee, J. K., Shin, D. H., & Jeong, J. S. (2009). Changes in brain activation induced by the training of hypothesis generation skills: An fMRI study. Brain & Cognition, 69, 391-397.   DOI   ScienceOn
7 Larkin, J., McDermott, J., Simon, D. P., & Simon, H. A. (1980). Expert and novice performance in solving physics problems. Science, 208(4450), 1335-1342.   DOI
8 Lee, J. K. (2009). Dissociation of the brain activation network associated with hypothesis-generating and hypothesisunderstanding in biology learning: Evidence from an fMRI study. Unpublished Doctoral Dissertation. Cheongwon, Chungbuk: Korea National University of Education.
9 Lee, J. K., & Kwon, Y. J. (2011). Why traditional expository teaching-learning approaches may founder? An experimental examination of neural networks in biology learning. Journal of Biological Education, 45(2), 83-92.   DOI   ScienceOn
10 Nehm, R. H., & Ha, M. (2011). Item feature effects in evolution assessment. Journal of Research in Science Teaching, 48(3), 237-256.   DOI   ScienceOn
11 Nehm, R. H., & Reilly, L. (2007). Biology majors'knowledge and misconceptions of natural selection. BioScience, 57(3), 263-272.   DOI   ScienceOn
12 Nehm, R. H., & Schonfeld, I. S. (2007). Does increasing biology teacher knowledge of evolution and the nature of science lead to greater preference for the teaching of evolution in schools? Journal of Science Teacher Education, 18(5), 699-723.   DOI   ScienceOn
13 Nehm, R. H., & Schonfeld, I. S. (2008). Measuring knowledge of natural selection: A comparison of the CINS, an open-response instrument, and an oral interview. Journal of Research in Science Teaching, 45(10), 1131- 1160.   DOI   ScienceOn
14 Nehm, R. H., & Schonfeld, I. S. (2010). The future of natural selection knowledge measurement: A reply to Anderson et al.(2010). Journal of Research in Science Teaching, 47(3), 358-362.
15 박정, 홍미영 (2002). 문항 유형에 따른 과학 능력 추정의 효율성 비교. 한국과학교육학회지, 22(1), 122-131.
16 박정, 홍미영, 김성숙, 전현정 (2000). 제3차 수학.과학 성취도 국제 비교 연구 (TIMSS-R) 국내 평가 결과 분석 연구II. 한국교육과정평가원. 연구보고RRE 2000-7.
17 이안나, 권용주, 정진수, 양일호 (2007). 동물 행동학자의 연구 활동에서 나타나는 연구 단계, 사고 과정, 행동 양식 및 생성 지식에 관한 연구. 한국생물교육학회지, 35(3), 361-373.
18 Anderson, D. L., Fisher, K. M., & Norman, G. J. (2002). Development and evaluation of the conceptual inventory of natural selection. Journal of Research in Science Teaching, 39(10), 952-978.   DOI   ScienceOn
19 Chi, M. T. H. (1996). Constructing selfexplanations and scaffolded explanations in tutoring. Applied Cognitive Psychology, 10(7), 33-49.   DOI
20 Caleon, I. S., & Subramaniam, R. (2010). Do students know what they know and what they don't know? Using a four-tier diagnostic test to assess the nature of students alternative conceptions. Research in Science Education, 40(3), 313-337.   DOI   ScienceOn
21 Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5(2), 121-152.   DOI   ScienceOn
22 Choi, K, Lee, H, Shin, N., Kim, S. W., & Krajcik, J. (2011). Re-conceptualization of scientific literacy in South Korea for the 21st Century. Journal of Research in Science Teaching. 48(6), 670-697.   DOI   ScienceOn
23 diSessa, A. A. (1988). Knowledge in pieces. In G. Forman & P. Pufall (Eds.), Constructivism in the computer age (pp. 49- 70). Hillsdale, NJ: Erlbaum.
24 Dochy, F. (2001). A new assessment era: different needs, new challenges. Learning and Instruction, 10(1), 11-20.
25 Gitomer, D. H., & Duschl, R. A. (2007). Establishing multilevel coherence in assessment. In P. A. Moss (Ed.), Evidence and decision making. The 106 th yearbook of the National Society for the Study of Education, Part I (pp. 288-320 ). Chicago: National Society for the Study of Education.
26 Scouller, K. (1998). The influence of assessment method on students'learning approaches: Multiple choice question examination versus assignment essay. Higher Education, 35(4), 453-472.   DOI   ScienceOn
27 Nehm, R. H., Ha, M., Rector, M., Opfer, J., Perrin, L., Ridgway, J., Mollohan, K. (2010). Scoring Guide for the Open Response Instrument (ORI) and Evolutionary Gain and Loss Test (EGALT). Technical Report of National Science Foundation REESE Project 0909999. Accessed online January 10, 2011 at: http://evolutionassessment.org
28 Park, J. (2010). Constructive multiple choice testing system. British Journal of Educational Technology, 41(6), 1054-1064.   DOI   ScienceOn
29 Pyc, M. A., & Rawson, K. A. (2010). Why testing improves memory: Mediator effectiveness hypothesis. Science, 330(6002), 335.   DOI
30 Southerland, S. A., Abrams, E., Cummins, C. L., & Anzelmo, J. (2001). Understanding students' explanations of biological phenomena: Conceptual frameworks or pprims? Science Education, 85(4), 328-348.   DOI   ScienceOn
31 Tan, K. C. D., Goh, N. K., Chia, L. S., & Treagust, D. F. (2002). Development and application of a two tier multiple choice diagnostic instrument to assess high school students' understanding of inorganic chemistry qualitative analysis. Journal of Research in Science Teaching, 39(4), 283-301.   DOI   ScienceOn
32 Vosniadou, S. (1994). Capturing and modeling the process of conceptual change. Learning and Instruction, 4(1), 45-69.   DOI   ScienceOn