정보과학회 논문지 (Journal of KIISE)

  • 제41권11호
  • /
  • Pages.911-920
  • /
  • 2014
  • /
  • 2383-630X(pISSN)
  • /
  • 2383-6296(eISSN)
한국정보과학회 (Korean Institute of Information Scientists and Engineers)
권호 표지
DOI QR코드

DOI QR Code

기계학습을 이용한 중등 수준의 단문형 영어 작문 자동 채점 시스템 구현

Developing an Automated English Sentence Scoring System for Middle-school Level Writing Test by Using Machine Learning Techniques

  • 이경호 (충남대학교 정보통신공학과) ;
  • 이공주 (충남대학교 정보통신공학과)
  • 투고 : 2014.05.15
  • 심사 : 2014.08.25
  • 발행 : 2014.11.15
⟨ 이전 논문 다음 논문 ⟩

초록

본 논문은 기계학습을 기반으로 하는 중등수준의 단문형 영어작문 자동채점시스템에 대해 제안한다. 본 논문에서는 기계학습을 이용한 영어 자동채점의 전반적인 수행 방법과 시스템의 구성 및 동작방식, 채점자질의 고려사항에 대해 논한다. 학생 답안의 내용 완성도를 평가하기 위하여 문서의 내용을 요약한 "개념답안"을 제안하여 사용하였다. 본 연구에서는 여러 개의 기계학습 알고리즘을 사용하여 자동평가를 수행한다. 자동평가의 성능을 향상시키기 위해 여러 개의 기계학습 알고리즘의 결과를 최적으로 결합하여 하나의 최종 결과를 도출할 수 있는 "최적조합" 결정과정을 제안한다. 실제 학생들의 작문 데이터를 이용하여 시스템을 구축하고 자동채점 시스템의 성능 평가를 수행하였다.

In this paper, we introduce an automatic scoring system for middle-school level writing test based on using machine learning techniques. We discuss overall process and features for building an automatic English writing scoring system. A "concept answer" which represents an abstract meaning of text is newly introduced in order to evaluate the elaboration of a student's answer. In this work, multiple machine learning algorithms are adopted for scoring English writings. We suggest a decision process "optimal combination" which optimally combines multiple outputs of machine learning algorithms and generates a final single output in order to improve the performance of the automatic scoring. By experiments with actual test data, we evaluate the performance of overall automated English writing scoring system.

키워드

과제정보

연구 과제 주관 기관 : 한국교육과정평가원

참고문헌

  1. K. Y. Jin, M. H. Nam, M. H Kim, S. C. O, M. J. Kim, H. M. Ju, H. P. Sin, J. C. Ban, and S. K. Kim, "Study on the development and introduction of an automated scoring program(I)," KICE, 2006. (in Korean)
  2. K. Y. Jin, B. C. Lee, H. M. Ju, D. G. Sin, J. Park, J.Y. Kim, K. J. Lee, and Y. S. Lee, "Study on the development and introduction of an automated scoring program(II) : Mainly in English composition scoring," KICE, 2007. (in Korean)
  3. K. Y. Jin, B. C. Lee, D. G. Sin, T. J. Park, and H. Y. Ju, "Study on the development and introduction of an automated scoring program(III)," KICE, 2008. (in Korean)
  4. T. J. S, G. S. Yang, H. J. Jun, Y. Y. Jung, Y. J. Lee, J. H. Park, S. G. Gu, and G. Y. Lee, "Academic achievement evaluation essay questions computer scoring of the proposed navigation," KICE, 2010. (in Korean)
  5. K. Y. Jin, G. J. Si, D. G. Sin, M. Y. Song, Y. S. Kim, Y. S. Lee, and Y. H. Kim, "KICE-Pearson English, speak, riting, research of the validity of the automatic scoring program," KICE, 2011. (in Korean)
  6. J. E. Kim and K. J. Lee, "Implementing Automated English Error Detecting and Scoring System for Junior High school Students," The Korea Contents Society, Vol. 7, No. 5, 2007. (in Korean)
  7. M. D Shermis and J. C. Burstein, "Automated essay scoring: A cross-disciplinary perspective," Routledge, 2002.
  8. J. Z. Sukkarieh and J. Blackmore, "c-rater: Automatic Content Scoring for Short Constructed Responses," FLAIRS Conference, 2009.
  9. J. Burstein, "The $E-rater^{(R)}$ scoring engine: Automated essay scoring with natural language processing," 2003.
  10. T. K. Landauer, D. Laham, and P. W. Foltz, "Automated scoring and annotation of essays with the Intelligent Essay Assessor," Automated essay scoring: A cross-disciplinary perspective, pp. 87-112, 2003.
  11. L. M. Rudner and T. Liang, "Automated essay scoring using Bayes' theorem," Journal of Technology, Learning, and Assessment, pp. 3-21. 2002.
  12. D. Pelleg and A. Moore, "X-means: Extending Kmeans with Efficient Estimation of the Number of Clusters," ICML, 2000.
  13. C. Fellbaum, "WordNet: An Electronic Lexical Database, Cambridge," MA: MIT Press, 1998.
  14. F. Jelinek, R. L. Mercer, L. R. Bahl, and J. K. Baker, "Perplexity-a measure of the difficulty of speech recognition tasks," The Journal of the Acoustical Society of America, pp. 63-63, 1977.
  15. J. Jia, P. Wang, and H. Zhao, "Grammatical Error Correction as Multiclass Classification with Single Model," CoNLL-2013, 2013.
  16. S. W. Lee and K. J. Lee, "A Comparison of Grammatical Error Detection Techniques for an Automated English Scoring System," Journal of the Korean Society of Marine Engineering, Vol. 37, No. 7, pp. 760-770, 2013. https://doi.org/10.5916/jkosme.2013.37.7.760
  17. J. Platt, "Sequential minimal optimization: A fast algorithm for training support vector machines," Techmical Report MSR-TR-94-14, Microsoft Research, 1998.
  18. N. Friedman, D. Geiger, and M. Goldszmidt, "Bayesian network classifiers," Machine learning 29.2-3, pp. 131-163. 1997. https://doi.org/10.1023/A:1007465528199
  19. S. Haykin, "Neural Networks: A Comprehensive Foundation," Mc Millan, New Jersey, 2010.
  20. T.G. Dietterich, "Ensemble methods in machine learning," Multiple classifier systems, Springer Berlin Heidelberg, pp. 1-15, 2000.
  21. J. Burstein and M. Chodorow, "Automated essay scoring for nonnative English speakers," Proc. of a Symposium on Computer Mediated Language Assessment and Evaluation in Natural Language Processing, Association for Computational Linguistics, pp. 68-75, 1999.

피인용 문헌

  1. Context-sensitive Word Error Detection and Correction for Automatic Scoring System of English Writing vol.4, pp.1, 2015, https://doi.org/10.3745/KTSDE.2015.4.1.45
  2. Effect of Application of Ensemble Method on Machine Learning with Insufficient Training Set in Developing Automated English Essay Scoring System vol.42, pp.9, 2015, https://doi.org/10.5626/JOK.2015.42.9.1124
  3. Automatic Detection of Off-topic Documents using ConceptNet and Essay Prompt in Automated English Essay Scoring vol.42, pp.12, 2015, https://doi.org/10.5626/JOK.2015.42.12.1522