Journal of the Korea Institute of Information and Communication Engineering (한국정보통신학회논문지)

The Korea Institute of Information and Commucation Engineering (한국정보통신학회)
권호 표지
DOI QR코드

DOI QR Code

Korean Sentiment Model Interpretation using LIME Algorithm

LIME 알고리즘을 이용한 한국어 감성 분류 모델 해석

  • Nam, Chung-Hyeon (Department of Computer Engineering, Korea University of Technology and Education) ;
  • Jang, Kyung-Sik (Department of Computer Engineering, Korea University of Technology and Education)
  • Received : 2021.09.07
  • Accepted : 2021.11.06
  • Published : 2021.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Korean sentiment classification task is used in real-world services such as chatbots and analysis of user's purchase reviews. And due to the development of deep learning technology, neural network models with high performance are being applied. However, the neural network model is not easy to interpret what the input sentences are predicting due to which words, and recently, model interpretation methods for interpreting these neural network models have been popularly proposed. In this paper, we used the LIME algorithm among the model interpretation methods to interpret which of the words in the input sentences of the models learned with the korean sentiment classification dataset. As a result, the interpretation of the Bi-LSTM model with 85.24% performance included 25,283 words, but 84.20% of the transformer model with relatively low performance showed that the transformer model was more reliable than the Bi-LSTM model because it contains 26,447 words.

한국어 감성 분류 작업은 챗봇, 사용자의 물건 구매 평 분석 등 실 서비스에서 사용되고 있으며, 현재 딥러닝 기술의 발달로 높은 성능을 가진 신경망 모델을 활발히 사용하여 감성 분류 작업을 수행하고 있다. 하지만 신경망 모델은 입력 문장이 어떤 단어들로 인해 결과가 예측되었는지 해석하는 것이 쉽지 않으며, 최근 신경망 모델의 해석을 위한 모델 해석 방법들이 활발히 제안되어지고 있다. 본 논문에서는 모델 해석 방법 중 LIME 알고리즘을 이용하여 한국어 감성 분류 데이터 셋으로 학습된 모델들의 입력 문장 내 단어들 중 어떤 단어가 결과에 영향을 미쳤는지 해석하고자 한다. 그 결과, 85.23%의 성능을 보인 양방향 순환 신경망 모델의 해석 결과, 총 25,283개의 긍정, 부정 단어를 포함했지만, 상대적으로 낮은 성능을 보인 84.20%의 Transformer 모델의 해석 결과, 총 26,447개의 긍정, 부정 단어가 포함되어 있어 양방향 순환 신경망 모델보다 Transformer 모델이 신뢰할 수 있는 모델임을 확인할 수 있었다.

Keywords

Acknowledgement

This paper was supported by Education and Research Promotion Program of KoreaTech.

References

  1. M. T. Ribeiro, S. Singh, and C. Guestrin, "Why Should I Trust You?: Explaining the Predictions of Any Classifier," in Proceeding of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Francisco, pp. 1135-1144, 2016.
  2. J. S. Bae and C. K. Lee, "Sentiment Analysis with Skip-Connected LSTM," in Proceeding of the 2017 Conference on Korea Software Congress, PyeongChang, pp. 633-635, 2017.
  3. E. J. You, J. H. Lee, and S. Y. Park, "The Sentiment Classification of News Articles using LSTM," in Proceeding of the 2018 Conference on Korea Software Congress, PyeongChang, pp. 1949-1951, 2018.
  4. Y. Kim, "Convolutional Neural Networks for Sentence Classification," in Proceeding of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, pp. 1746-1751, 2014.
  5. W. W. Kim and K. H. Park, "Design of Korean Text Emotion Classifier Using Convolution Neural Network," in Proceeding of the 2017 Conference on Korea Computer Congress, Jeju, pp. 642-644, 2017.
  6. M. Kim, J. H. Byun, C. H. Lee, and Y. S. Lee, "Multi-channel CNN for Korean Sentiment Analysis," in Proceeding of the 2018 Conference on Human and Language Technology, Seoul, pp. 79-83, 2018.
  7. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, "Attention Is All You Need," in Proceeding of the 31st Conference on Neural Information Processing Systems, Long Beach, pp. 5998-6008, 2017.
  8. C. E. Park and C. K. Lee, "Sentimental Analysis of Korean Movie Review using Variational Inference and RNN based on BERT," The KIISE Transactions on Computing Practices, vol. 25, no. 11, pp. 552-558, Aug. 2019. https://doi.org/10.5626/ktcp.2019.25.11.552
  9. A. Paszke, PyTorch [Internet]. Available: https://pytorch.org.
  10. E. J. Park, Naver Korean Sentiment Classification Dataset [Internet]. Available: https://github.com/e9t/nsmc.
  11. B. W. On, S. M. Park, and C. W. Na. KnuSentLex [Internet]. Available: https://github.com/park1200656/KnuSentiLex.