Browse > Article
http://dx.doi.org/10.14697/jkase.2020.40.3.321

Automated Scoring of Scientific Argumentation Using Expert Morpheme Classification Approaches  

Lee, Manhyoung (Korea National University of Education)
Ryu, Suna (Korea National University of Education)
Publication Information
Journal of The Korean Association For Science Education / v.40, no.3, 2020 , pp. 321-336 More about this Journal
Abstract
We explore automated scoring models of scientific argumentation. We consider how a new analytical approach using a machine learning technique may enhance the understanding of spoken argumentation in the classroom. We sampled 2,605 utterances that occurred during a high school student's science class on molecular structure and classified the utterances into five argumentative elements. Next, we performed Text Preprocessing for the classified utterances. As machine learning techniques, we applied support vector machines, decision tree, random forest, and artificial neural network. For enhancing the identification of rebuttal elements, we used a heuristic feature-engineering method that applies experts' classification of morphemes of scientific argumentation.
Keywords
scientific argumentation; automated scoring; machine learning; scientific language; experts system;
Citations & Related Records
Times Cited By KSCI : 17  (Citation Analysis)
연도 인용수 순위
1 Fernandez-Delgado, M., Cernadas, E., Barro, S., & Amorim, D. (2014). Do we need hundreds of classifiers to solve real world classification problems?. The journal of machine learning research, 15(1), 3133-3181.
2 Giarratano, J. C., & Riley, G. (1998). Expert systems: Principles and programming. Boston: PWS.
3 Breiman, L. (2001). Random forests. Machine learning, 45(1), 5-32.   DOI
4 Ha, M. (2016). Scoring Korean Written Responses Using English-Based Automated Computer Scoring Models and Machine Translation: A Case of Natural Selection Concept Test. Journal of The Korean Association For Science Education, 36(3), 389-397.   DOI
5 Ha, M., Lee, G.-G., Shin, S., Lee, J.-K., Choi, S., Choo, J., Kim, N., Lee, H., Lee, J., Lee, J., Jo, Y., Kang, K., & Park, J. (2019). Assessment as a Learning-Support Tool and Utilization of Artificial Intelligence: WA3I Project Case. School Science Journal, 13(3), 271-282.   DOI
6 Jimenez-Aleixandre, M. P., Bugallo Rodriguez, A., & Duschl, R. A. (2000). "Doing the lesson" or "doing science": Argument in high school genetics. Science Education, 84(6), 757-792.   DOI
7 Jun, C.-H. (2015). Data mining techniques. Seoul: Hannarea Publishing Co.
8 Kang, S. M., Kwak, K. H., & Nam, J. H. (2006). The effects of argumentation-based teaching and learning strategy on cognitive development, science concept understanding, science-related attitude, and argumentation in middle school science. Journal of the Korean Association for Science Education, 26(3), 450-461.
9 Kelly, G. J., Druker, S., & Chen, C. (1998). Students' reasoning about electricity: Combining performance assessments with argumentation analysis. International journal of science education, 20(7), 849-871.   DOI
10 Kevin P. Murphy. (2012). Machine Learning: A Probabilistic Perspective. Cambridge, MA: The MIT Press.
11 KICE(Korea Institute Of Curriculum & Evaluation). (2006). A Study on the Development and Introduction of an Automated Scoring Program(RRI 2006-6). Retrieved from http://kice.re.kr/resrchBoard/view.do?seq=19388&s=kice&m=030102
12 Kil, H.-h. (2018). The Study of Korean Stopwords list for Text mining. The Korean Language and Literature, 78, 1-25.
13 Kim, M., & Ryu, S. (2019). Development of Scientific Conceptual Understanding through Process-Centered Assessment that Visualizes the Process of Scientific Argumentation. Journal of the Korean Association for Science Education, 39(5), 651-668.
14 Kim, S. J. (2019). A Study on the Extraction and Validation of Automatic Argumentative Writing Scoring Feature Using Text Mining (Unpublished Master's Thesis). Korea national university of education, Chung-Buk.
15 KOFAC (Korea Foundation for the Advancement of Science & Creativity) (2019). Scientific Literacy for All Koreans, Korean Science Education Standards for the Next Generation. Seoul: KOFAC.
16 Kwon, J.-S., & Kim, H.-B. (2016). Exploring small group argumentation shown in designing an experiment: Focusing on students' epistemic goals and epistemic considerations for activities. Journal of the Korean Association for Science Education, 36(1), 45-61.   DOI
17 Lantz, B. (2013). Machine learning with R. Birmingham, UK: Packt Publishing Ltd.
18 Landis, J. R., & Koch, G. G. (1977). The Measurement of Observer Agreement for Categorical Data. Biometrics, 33(1), 159-174.   DOI
19 Lee, G.-G., Ha, H., Hong, H.-G., & Kim, H.-B. (2018). Exploratory Research on Automating the Analysis of Scientific Argumentation Using Machine Learning. Journal of The Korean Association For Science Education, 38(2), 219-234.   DOI
20 Lee, D., Yeon, J., Hwang, I., & Lee, S. (2010). KKMA: A tool for utilizing sejong corpus based on relational database. Journal of KIISE: Computing Practices and Letters, 16(11), 1046-1050.
21 Lee H.-J., & Park Y.-M. (2019). A Study on the Search for Automatic Scoring Variables by Comparison of Text Studies Using Natural Language Processing. The research in writing, 41, 255-287.
22 Lee, H.-S., Pallant, A., Pryputniewicz, S., Lord, T., Mulholland, M., & Liu, O. L. (2019). Automated text scoring and real-time adjustable feedback: Supporting revision of scientific arguments involving uncertainty. Science Education, 103(3), 590-622.   DOI
23 Lee, S., & Nam, J. (2016). Impact of Student Assessment Activities on Reflective Thinking in High School Argument-Based Inquiry. Journal of The Korean Association For Science Education, 36(2), 347-360.   DOI
24 Lee, S., & Nam, J. (2018). Impact of Student Assessment Activities on Claim and Evidence Formation in High School Argument-Based Inquiry. Journal of the Korean Chemical Society, 62(3), 203-213.   DOI
25 Linn, M. C., Gerard, L., Ryoo, K., McElhaney, K., Liu, O. L., & Rafferty, A. N. (2014). Computer-guided inquiry to improve science learning. Science, 344(6180), 155-156.   DOI
26 Liu, O. L., Rios, J. A., Heilman, M., Gerard, L., & Linn, M. C. (2016). Validation of automated scoring of science assessments. Journal of Research in Science Teaching, 53(2), 215-233.   DOI
27 McNeill, K. L., & Krajcik, J. (2007). Middle school students' use of appropriate and inappropriate evidence in writing scientific explanations. In M. Lovett, & P. Shah (Eds.), Thinking with data: The proceedings of 33rd Carnegie symposium on cognition. Mahwah, NJ: Erlbaum.
28 Mao, L., Liu, O. L., Roohr, K., Belur, V., Mulholland, M., Lee, H. S., & Pallant, A. (2018). Validation of automated scoring for a formative assessment that employs scientific argumentation. Educational Assessment, 23(2), 121-138.   DOI
29 Martin, T., & Sherin, B. (2013). Learning analytics and computational techniques for detecting and evaluating patterns in learning: An introduction to the special issue. Journal of the Learning Sciences, 22(4), 511-520.   DOI
30 McNeill, K. L., Lizotte, D. J., Krajcik, J., & Marx, R. W. (2006). Supporting students' construction of scientific explanations by fading scaffolds in instructional materials. The Journal of the Learning Sciences, 15(2), 153-191.   DOI
31 Negnevitsky, M. (2005). Artificial Intelligence: A Guide to Intelligent Systems. Harlow: Addison-Wesley.
32 Nehm, R. H., Ha, M., & Mayfield, E. (2012). Transforming biology assessment with machine learning: automated scoring of written evolutionary explanations. Journal of Science Education and Technology, 21(1), 183-196.   DOI
33 Ong, N., Litman, D., & Brusilovsky, A. (2014). Ontology-based argument mining and automatic essay scoring. In Proceedings of the First Workshop on Argumentation Mining (pp. 24-28). Baltimore, MD.
34 Osborne, J., Erduran, S., & Simon, S. (2004). Enhancing the quality of argumentation in school science. Journal of research in science teaching, 41(10), 994-1020.   DOI
35 Park, C., Kim, Y, Kim, J, Song, J., & Choi, H. (2015). R data mining. Seoul: Kyowoo.
36 Russell, S., & Norvig, P. (2016). Artificial intelligence: A modern approach. Harlow: Pearson.
37 Park, E. L., & Cho, S. (2014). KoNLPy: Korean natural language processing in Python. In Proceedings of the 26th Annual Conference on Human & Cognitive Language Technology (pp. 133-136). Chuncheon, Korea.
38 Park, J., & Kim, H.-B. (2018). Exploring Teachers' Responsive Teaching Practice in Argumentation-Based Science Classroom: Focus on Structural and Dialogical Aspects of Argument. Journal of The Korean Association For Science Education, 38(1), 69-85.   DOI
39 Ripley, B., & Venables, W. (2020). Package 'nnet'. R package version, 7.3-14. Retrieved April 28, 2020, from https://cran.r-project.org/web/packages/nnet
40 Sampson, V., & Clark, D. B. (2008). Assessment of the ways students generate arguments in science education: Current perspectives and recommendations for future directions. Science education, 92(3), 447-472.   DOI
41 Shen, L. (2019). A Study on Functions of Korean Discourse Markers (Unpublished Doctoral Dissertation). Yonsei University, Seoul.
42 Shin, H. S. & Kim, H.-J. (2012). Development of the Analytic Framework for Dialogic Argumentation Using the TAP and a Diagram in the Context of Learning the Circular Motion. Journal of the Korean Association for Science Education, 32(5), 1007-1026.   DOI
43 Simon, S., Erduran, S., & Osborne, J. (2006). Learning to teach argumentation: Research and development in the science classroom. International journal of science education, 28(2-3), 235-260.   DOI
44 Song, J., Kang, S. J., Kwak, Y., Kim, D., Kim, S., Na, J., ... & Son, Y. A. (2019). Contents and Features of' Korean Science Education Standards (KSES)'for the Next Generation. Journal of The Korean Association For Science Education, 39(3), 465-478.   DOI
45 Yoo, J. E. (2015). Random forests, an alternative data mining technique to decision tree. Journal of Educational Evaluation, 28(2), 427-448.
46 Song, M.-Y., Noh, E.-H., & Sung, K.-H. (2016). Analysis on the Accuracy of Automated Scoring for Korean Large-scale Assessments. The Journal of Curriculum and Evaluation, 19(1), 255-274.   DOI
47 Toulmin, S. (1958). The uses of argument. Cambridge, UK: Cambridge University Press.
48 Yang, I. H., Lee, H. J., Lee, H. N., & Cho, H. J. (2009). The development of rubrics to assess scientific argumentation. Journal of The Korean Association For Science Education, 29(2), 203-220.
49 Yoo. J. E. (2019). Machine Learning for Large-scale/Panel Data and Learning Analytics Data Analysis. Journal of Educational Technology, 35(2), 313-338.   DOI
50 Zohar, A., & Nemet, F. (2002). Fostering students' knowledge and argumentation skills through dilemmas in human genetics. Journal of Research in Science Teaching: The Official Journal of the National Association for Research in Science Teaching, 39(1), 35-62.   DOI
51 Kang, N.-H., & Lee, E. K. (2013). Argument and argumentation: A review of literature for clarification of translated words. Journal of The Korean Association For Science Education, 33(6), 1119-1138.   DOI
52 Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine learning, 20(3), 273-297.   DOI
53 Ah, H. Y. (2018). A Study of Counter-Arguments as Observed in Debate of High School Students (Unpublished Master's Thesis). Pusan national university, Pusan.
54 Baek, Y. K. (1989). The Educational Potentials of Expert Systems. Journal of Educational Technology, 5(1), 79-91.
55 Beggrow, E. P., Ha, M., Nehm, R. H., Pearl, D., & Boone, W. J. (2014). Assessing scientific practices using machine-learning methods: How closely do they match clinical interview performance?. Journal of Science education and Technology, 23(1), 160-182.   DOI
56 Bridgeman, B., Trapani, C., & Attali, Y. (2012). Comparison of human and machine scoring of essays: Differences by gender, ethnicity, and country. Applied Measurement in Education, 25(1), 27-40.   DOI
57 Buchanan, B. G., & Feigenbaum, E. A. (1980). The stanford heuristic programming project: Goals and activities. AI Magazine, 1(1), 25-30.   DOI
58 Cho, H. S., & Nam, J. (2014). The impact of the argument-based modeling strategy using scientific writing implemented in middle school science. Journal of the Korean Association for Science Education, 34(6), 583-592.   DOI
59 Driver, R., Newton, P., & Osborne, J. (2000). Establishing the norms of scientific argumentation in classrooms. Science education, 84(3), 287-312.   DOI
60 Duschl, R. (2008). Science education in three-part harmony: Balancing conceptual, epistemic, and social learning goals. Review of research in education, 32(1), 268-291.   DOI
61 Engin, G., Aksoyer, B., Avdagic, M., Bozanli, D., Hanay, U., Maden, D., & Ertek, G. (2014). Rule-based Expert Systems for Supporting University Students. Procedia Computer Science, 31, 22-31.   DOI
62 Erduran, S., Simon, S., & Osborne, J. (2004). TAPping into argumentation: Developments in the application of Toulmin's argument pattern for studying science discourse. Science education, 88(6), 915-933.   DOI