Journal of The Korean Association of Information Education (정보교육학회논문지)

Korea Association of Information Education (한국정보교육학회)
권호 표지
DOI QR코드

DOI QR Code

Automatic Generation of Multiple-Choice Questions Based on Statistical Language Model

통계 언어모델 기반 객관식 빈칸 채우기 문제 생성

  • Park, Youngki (School of Computer Science and Engineering, Seoul National University)
  • Received : 2016.04.13
  • Accepted : 2016.04.21
  • Published : 2016.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

A fill-in-the-blank with choices are widely used in classrooms in order to check whether students' understand what is being taught. Although there have been proposed many algorithms for generating this type of questions, most of them focus on preparing sentences with blanks rather than generating multiple choices. In this paper, we propose a novel algorithm for generating multiple choices, given a sentence with a blank. Because the algorithm is based on a statistical language model, we can generate relatively unbiased result and adjust the level of difficulty with ease. The experimental results show that our approach automatically produces similar multiple-choices to those of the exam writers.

빈칸 채우기 문제는 학생들이 학습 내용을 제대로 이해했는지 확인하기 위해 널리 사용되어 왔다. 이런 유형의 문제를 컴퓨터 알고리즘에 의해 자동으로 생성하는 많은 방법들이 제안되어 왔지만, 대부분 어떤 부분을 빈칸으로 만들면 좋을지에 대해 집중했기 때문에 적절한 보기를 자동으로 생성하는 연구는 미흡했다. 본 논문에서는 빈칸이 주어졌다고 가정하고, 이에 어울리는 보기를 자동 생성하는 알고리즘을 제안한다. 본 알고리즘은 통계 언어 모델에 기반하여 보기를 생성하기 때문에, 사람이 생성하는 경우보다 출제자에 편향되지 않은 보기를 제공할 수 있다. 또, 확률값에 기반하여 난이도를 자동으로 조절하는 것이 가능하기 때문에, 직접 사람이 문제를 만드는 것에 비해 상당한 비용 절감 효과가 있다. TEPS 문법, 어휘 시험에 대해 적용하여 실험한 결과, 사람과 유사한 결과를 생성함을 확인하였다. 향후 스마트 교육 분야에서 높은 활용도를 보일 것으로 기대한다.

Keywords

References

  1. Acerbi, A., Lampos, V., Garnett P. & Bentley RA. (2013). The Expression of Emotions in 20th Century Books. PLoS ONE, 8(3).
  2. Alsubait, T., Parsia, B. & Sattler, U. Generating Multiple Choice Questions From Ontologies: Lessons Learnt. Proceedings of the 11th International Workshop on OWL: Experiences and Directions, 73-84.
  3. Bae, Y. & Do, J. (2013). Study on Smart Learning Contents Development using Storyline. Journal of the Korean Association of Computer Education, 17(2), 135-146.
  4. Google n-gram viewer. https://books.google.com/ngrams
  5. Google n-gram viewer (Wikipedia). https://en.wikipedia.org/wiki/Google_Ngram_Viewer
  6. Hoshino, A. & Nakagawa, H. (2005). A Real-time Multiple-choice Question Generation for Language Testing: a Preliminary Study. Proceedings of the 2nd Workshop on Building Educational Applications Using NLP, 17-20.
  7. Kim, S. & Kim, K. (2012). Design and Implementation learning English words Smart-phone application for Elementary school students on Android platform by Focus on form. Journal of the Korean Association of Computer Education, 16(2), 223-231.
  8. Lee, S. & Ryu, H. (2013). Suggestion on the Key Factors of Smart Education. Journal of the Korean Association of Computer Education, 17(2), 101-113.
  9. Lin, J., Quan, D., Sinha, V., Bakshi, K., Huynh, D., Katz, B. & Karger D. R. (2003). The role of context in question answering systems. Proceedings of the Extened Abstracts on Human Factors in Computing Systems, 1006-1007.
  10. Pintos. http://pintos-os.org/
  11. Roth, S. (2014). Fashionable functions. A Google ngram view of trends in functional differentiation(1800-2000). International Journal of Technology and Human Interaction, 10(2), 34-58.
  12. Soul M. & Son. C. (2012). A Survey on Teacher's Perceptions about the Current State of Using Smart Learning in Elementary Schools. Journal of the Korean Association of Computer Education, 16(3), 309-318.
  13. Sumita, E., Sugaya F. & Yamamoto, S. (2005). Measuring non-native speakers' proficiency of English by using a test with automatically-generated fill-in-the-blank questions. Proceedings of the second workshop on Building Educational Applications Using NLP, 61-68.
  14. Thompson, A. S. (2015). Are Your Participants Multilingual? The Role of Self-assessment in SLA Research. Language in Focus, 1(1), 51-65. https://doi.org/10.1515/lifijsal-2015-0004
  15. Wood, C. L., Mustian A. L. & Cooke N. L. (2010). Comparing Whole-Word and Morphograph Instruction During Computer-Assisted Peer Tutoring on Students' Acquisition and Generalization of Vocabulary. Remedial and Special Education, 33(1), 39-47. https://doi.org/10.1177/0741932510362515