Browse > Article
http://dx.doi.org/10.9708/jksci.2013.18.8.017

Design and Implementation of an Automatic Scoring Model Using a Voting Method for Descriptive Answers  

Heo, Jeongman (Dept. of Digital Media, SangMyung University)
Park, So-Young (Dept. of Game Design & Development, SangMyung University)
Publication Information
Journal of the Korea Society of Computer and Information / v.18, no.8, 2013 , pp. 17-25 More about this Journal
Abstract
TIn this paper, we propose a model automatically scoring a student's answer for a descriptive problem by using a voting method. Considering the model construction cost, the proposed model does not separately construct the automatic scoring model per problem type. In order to utilize features useful for automatically scoring the descriptive answers, the proposed model extracts feature values from the results, generated by comparing the student's answer with the answer sheet. For the purpose of improving the precision of the scoring result, the proposed model collects the scoring results classified by a few machine learning based classifiers, and unanimously selects the scoring result as the final result. Experimental results show that the single machine learning based classifier C4.5 takes 83.00% on precision while the proposed model improve the precision up to 90.57% by using three machine learning based classifiers C4.5, ME, and SVM.
Keywords
Automatic Scoring Model; Machine Learning based Classifier; Voting Method;
Citations & Related Records
Times Cited By KSCI : 6  (Citation Analysis)
연도 인용수 순위
1 J. T. Lee, Y. I. Song, S. Y. Park, H. C. Rim, "Text-Confidence Feature Based Quality Evaluation Model for Knowledge Q&A Documents," Journal of KIISE : Software and Applications, Vol. 35, No. 10, pp.608-615, Nov. 2008.   과학기술학회마을
2 S. Y. Kim, "Incremental Supervised Learning based on SVM with Unlabeled Documents," The Graduate School of Computer Science. Yonsei University, pp.12-13, 2002.
3 W. S. Kang, "Design and Implementation of a Subjective-type Evaluation System Using Syntactic and Case-Role Information," THE JOURNAL OF KOREAN ASSOCIATION OF COMPUTER EDUCATION, Vol. 10, No. 5, pp.61-69, Sept. 2007.
4 C. Park, "Validating an Automated Scoring System of Constructed-response items for applications," Journal of Educational Evaluation. Vol. 22, No. 3, pp615-631, Sept. 2009.
5 W. J. Cho, "An Intelligent Marking System based on Semantic Kernel and Korean WordNet," Graduate School. Hallym University, pp. 3-33, Aug. 2006.
6 S. Z. Lee, "New Statistical Models for Automatic Part-of-Speech Tagging," Ph.D. Thesis, Korea University, 1999.
7 H. J. Jung, "A Maximum Entropy Approach to Korean Part-of-Speech Tagging," Information & Communications of Engineering, Myongji University, pp.11-18, 2005.
8 Vapnik, "The Nature of Statistical Learning Theory," Springer-Verlag, 1999
9 T. Joachims, "Estimating the Generalization Performance of an SVMEfficiently," Proceedings of the 17th International Conference on Machine Learning (ICML 2000), pp. 431-438, 2000.
10 S. Y. Kim, "Object of Interest Extraction Using Gabor Filters," THE JOURNAL OF KOREAN ASSOCIATION OF COMPUTER EDUCATION, Vol. 13, No. 2, pp.87-94, 2008   과학기술학회마을   DOI
11 J. S. Oh, W. J. Cho, Y. S. Kim, J. Y. Lee, "A Descriptive Question Marking System based on Semantic Kernels," Journal of Korean Institute of Information Technology, Vol. 3, No. 4, pp.95-104, Sept. 2005.
12 C. G. Jung, R. I. Choi, "A Developmental Plan on Public Education in Digital Times, 21c," Journal of The Korea Society of Computer and Information, Vol. 8, No. 1, pp.120-128, Mar. 2003.   과학기술학회마을
13 H. J. Kim, J. H. Choi, "Development of a Teaching and Learning Model for Educational Usage of Web 2.0 and Its Effect Analysis," Journal of The Korea Society of Computer and Information, Vol. 16, No. 10, pp.45-52, Oct. 2011.   과학기술학회마을   DOI   ScienceOn
14 K. A. Jin, "Development of automated scoring system for English writing." English Language & Literature Teaching, Vol. 13, No. 1, pp.235-259, Spring. 2007.
15 E. M. Jung, "Design and Implementation of Automatic Marking System for a Subjectivity Problem of the Program," Journal of Korea Multimedia Society, Vol. 12, No. 5, pp767-776, May. 2009.   과학기술학회마을
16 J. M. Cho, K. H. Kim, "A Study on design of The Internet-based scoring system for constructed responses," THE JOURNAL OF KOREAN ASSOCIATION OF COMPUTER EDUCATION, Vol. 10, No. 2, pp.89-100, Mar. 2007.
17 D. K. Chung, "Subjective Questions Scoring System Using Vector Similarity and Thesaurus," The Graduate School of Education. Dongguk University, pp.1-45, 2002.
18 J. H. Cho, H. K. Jung, C. Y. Park, Y. S. Kim, "An Autonomous Assessment of a Short Essay Answer by Using the BLEU," HCI2009, pp.606-605, Gangwon-do Phoenixpark Convention center, Korea, Feb. 2009.
19 S. H. Jang, "Planning and realizing the Automatic Grading System for Subjective Questions Through an Analysis of Question Types," The Graduate School of Education. Andong National University, 53p, 2008.
20 Van Halteren, H., Daelemans, W. and Zavrel, J., "Improving accuracy in word class tagging through the combination of machine learning systems," Computational linguistics, Vol. 27, No. 2, pp.199-229, Feb. 2001.   DOI   ScienceOn
21 E. H. No, "Developing an Automatic Content Scoring Program for Short Answer Korean Items in Large-Scalse Assessments," Korea Institute for Curriculum and Evaluation, pp.3-48(230), 2012.
22 K. B. Kim, J. H. Cho, "Performance Assessment System using Fuzzy Reasoning Rule," Journal of The Korea Society of Computer and Information, Vol. 10, No. 1, pp.209-216, Mar. 2005.   과학기술학회마을
23 O. Y. Kwon, "The Design and Implementation of Web-based Subjective questions Grading Algorithm," Department of Educational(Coumputer Scientce) The Graduate School of Education. Hanseo University, pp.1-3(52), Feb. 2004.
24 H. J. Park, W. S. Kang, "Design and Implementation of a Subjective-type Evaluation System Using Natural Language Processing Technique," THE JOURNAL OF KOREAN ASSOCIATION OF COMPUTER EDUCATION, Vol. 6, NO. 3, pp.207-216, Jul. 2003.
25 Y. R. Kim, B. Y. Cho, "(A)review on NAEA Korean test based on CURRV framework," Korea Institute for Curriculum and Evaluation, pp.1-12p, 2011.
26 Quinlan, J. Ross, "C4.5:Programs for Machine Learning," Morgan Kaufmann Publishers, 1993
27 Van Halteren, H., Zavrel, J. and Daelemans, W., "Improving data driven wordclass tagging by system combination," Proceedings of the 36th Annual Meeting of the Association for Computational Linguistics and 17th International Conference on Computational Linguistics, Vol. 1, pp.491-497, 1998.
28 J. T. Lee, Y. I. Song, H. C. Rim, "Predicting the quality of answers using surface linguistic features," In Proceedings of the 6th International Conference on Advanced Language Processing and Web Information Technology, pp.111-116, 2007.