KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Learning Source Code Context with Feature-Wise Linear Modulation to Support Online Judge System

온라인 저지 시스템 지원을 위한 Feature-Wise Linear Modulation 기반 소스코드 문맥 학습 모델 설계

  • Received : 2022.07.05
  • Accepted : 2022.08.09
  • Published : 2022.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

Evaluation learning based on code testing is becoming a popular solution in programming education via Online judge(OJ). In the recent past, many papers have been published on how to detect plagiarism through source code similarity analysis to support OJ. However, deep learning-based research to support automated tutoring is insufficient. In this paper, we propose Input & Output side FiLM models to predict whether the input code will pass or fail. By applying Feature-wise Linear Modulation(FiLM) technique to GRU, our model can learn combined information of Java byte codes and problem information that it tries to solve. On experimental design, a balanced sampling technique was applied to evenly distribute the data due to the occurrence of asymmetry in data collected by OJ. Among the proposed models, the Input Side FiLM model showed the highest performance of 73.63%. Based on result, it has been shown that students can check whether their codes will pass or fail before receiving the OJ evaluation which could provide basic feedback for improvements.

온라인 저지 시스템 지원하기 위한 표절 검사, 소스코드 분석 및 자동화된 튜터링 기법이 연구되고 있다. 최근 딥러닝 기술 기반의 소스코드 유사도 분석을 통한 표절 감지 기술들이 제안되었으나, 자동화된 튜터링을 지원하기 위한 딥러닝 기반의 연구는 미흡한 실정이다. 따라서 본 논문에서는 자바 바이트코드와 문제정보를 결합하여 학습하고, 학습자가 온라인 저지 시스템에 코드를 제출하기 전에 pass/fail 여부를 예측할 수 있는 GRU 기반의 Input / Output side FiLM 모델을 제안한다. 또한 온라인 저지에 수집되는 데이터의 특성상 비대칭이 발생하기 때문에 밸런스 샘플링 기법을 적용하여 데이터를 균등하게 분포시켜 두 상황을 제안한 모델로 학습하였다. 실험 결과 Input side FiLM 모델이 가장 높은 73.63%의 성능을 보였다. 이를 기반으로 학습자들이 온라인 저지의 평가를 받기 전에 pass/faill 여부를 확인하여 소스코드 개선에 대한 피드백 기능에 적용 가능할 것으로 예상된다.

Keywords

Acknowledgement

이 논문은 2019년도 한국방송통신대학교 학술연구비 지원을 받아 작성된 것임.

References

  1. Leetcode - The World's Leading Online Programming Learning Platform [Internet], https://leetcode.com/.
  2. Baekjoon Online Judge [Internet], https://www.acmicpc.net/.
  3. H. S. Mun, S. H. Kim, J. H. Kim, and Y. S. Lee, "A sourcecode similarity-based automatic tutoring method for online coding test service," Journal of KIISE, Vol.48, No.9, pp.1044-1051, 2021. https://doi.org/10.5626/JOK.2021.48.9.1044
  4. J. H. Ji, G. Woo, and H. G. Cho, "A plagiarism detection technique for Java program using bytecode analysis," Journal of KIISE Software and Applications, Vol.35, No.7, pp.442-451, 2008.
  5. J. W. Son, S. B. Park, and S. Y. Park, "Program plagiarism detection using parse tree kernels," Pacific Rim International Conference on Artificial Intelligence, Springer, Berlin, Heidelberg, pp.1000-1004, 2006.
  6. L. Prechelt, G. Malpohl, and M. Philippsen. "Finding plagiarisms among a set of programs with JPlag," Journal of Universal Computer Science, Vol.8, No.11, pp.1016-1038, 2002.
  7. R. Gupta, S. Pal, A. Kanade, and S. Shevade, "Deepfix: Fixing common c language errors by deep learning," Thirty-First AAAI Conference on Artificial Intelligence, Vol.31, No.1, 2017.
  8. U. Z. Ahmed, P. Kumar, A. Karkare, and S. Gulwani, "Compilation error repair: For the student programs, from the student programs," Proceedings of the 40th International Conference on Software Engineering: Software Engineering Education and Training, pp.78-87, 2018.
  9. S. Bhatia, P. Kohli, R. Singh, P. Kohli, and R. Singh, "Neurosymbolic program corrector for introductory programming assignments," 2018 IEEE/ACM 40th International Conference on Software Engineering, pp.60-70, 2018.
  10. S. Wang, D. Chollak, and D. Movshovitz-Attias, "Bugram: Bug detection with n-gram language models," Proceedings of the 31st IEEE/ACM International Conference on Automated Software Engineering, pp.708-719, 2016.
  11. Y. Pu, K. Narasumhan, A. Solar-lezama, and R. Barzilay "sk_p: A neural program corrector for MOOCs," Companion Proceedings of the 2016 ACM SIGPLAN International Conference on Systems, Programming, Languages and Applications: Software for Humanity, pp.39-40, 2016.
  12. M. Vasic, A. Kanade, P. Maniatis, D. Bieber, and R. Singh, "Neural program repair by jointly learning to localize and repair," arXiv preprint arXiv:1904.01720, 2019.
  13. R. Gupta, A. Kanade, and S. Shevade, "Neural attribution for semantic bug-localization in student programs," Advances in Neural Information Processing Systems, 2019.
  14. E. Perez, F, Strub, H. DE Vries, V. Dumoulin, and A. Courville, "Film: Visual reasoning with a general conditioning layer," Proceedings of the AAAI Conference on Artificial Intelligence, Vol.32, No.1, 2018.