KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Deep Learning Based Semantic Similarity for Korean Legal Field

딥러닝을 이용한 법률 분야 한국어 의미 유사판단에 관한 연구

  • 김성원 (한국과학기술원 지식서비스공학대학원) ;
  • 박광렬 (인하대학교 법학전문대학원)
  • Received : 2021.11.23
  • Accepted : 2022.01.28
  • Published : 2022.02.28
Download PDF
⟨ Previous Next ⟩

Abstract

Keyword-oriented search methods are mainly used as data search methods, but this is not suitable as a search method in the legal field where professional terms are widely used. In response, this paper proposes an effective data search method in the legal field. We describe embedding methods optimized for determining similarities between sentences in the field of natural language processing of legal domains. After embedding legal sentences based on keywords using TF-IDF or semantic embedding using Universal Sentence Encoder, we propose an optimal way to search for data by combining BERT models to check similarities between sentences in the legal field.

기존의 데이터 검색 방법으로는 키워드 중심의 검색 방법이 주로 사용되나, 이는 전문적인 용어가 많이 쓰이는 법률 분야의 검색 방법으로는 적합하지 않다. 이에 대해 본 논문에서는 법률 분야의 효과적인 데이터 검색 방안을 제안한다. 법률 도메인의 자연어처리 분야에서 문장 간의 유사성을 판단하는 데 최적화된 임베딩 방법에 관하여 서술한다. 법률문장을 TF-IDF를 이용하여 키워드 기반으로 임베딩하거나 Universal Sentence Encoder를 이용하여 의미 기반으로 임베딩을 한 후, BERT모델을 결합하여 법률 분야에서 문장 간 유사성을 검사하여 데이터를 검색하는 최적의 방안을 제안한다.

Keywords

Acknowledgement

이 논문은 2021년도 서울시 산학연 협력사업(IC210005)의 재원으로 서울R&D지원센터의 지원을 받아 수행된 연구임.

References

  1. J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, "Bert: Pre-training of deep bidirectional transformers for language understanding," arXiv preprint arXiv:1810.04805, 2018.
  2. J. Ramos, "Using tf-idf to determine word relevance in document queries," In Proceedings of the First Instructional Conference on Machine Learning, Vol.242, No.1, pp.29-48, 2003.
  3. Cer, D. et al., "Universal sentence encoder," arXiv preprint arXiv:1803.11175, 2018.
  4. A. Radford, K. Narasimhan, T. Salimans, and I. Sutskever, "Improving language understanding by generative pretraining," 2018.
  5. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, "Language models are unsupervised multitask learners," OpenAI Blog, Vol.1, No.8, pp.9, 2019.
  6. T. B. Brown, et al., "Language models are few-shot learners," arXiv preprint arXiv:2005.14165, 2020.
  7. M. E. Peters, et al., "Deep contextualized word representations," arXiv preprint arXiv:1802.05365, 2018
  8. T. M. Mitchell, "Bayesian Learning," Machine Learning, McGraw-Hill, pp.154-200, 1997.
  9. H. Zhong, C. Xiao, C. Tu, T. Zhang, Z. Liu, and M. Sun, "How Does NLP benefit legal system: A summary of legal artificial intelligence," arXiv: 2004.12158, 2020.
  10. A. Joulin, E. Grave, P. Bojanowski, M. Douze, H. Jegou, and T. Mikolov, "Fasttext. zip: Compressing text classification models," arXiv preprint arXiv:1612.03651, 2016.
  11. M. E. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer, "Deep contextualized word representations," arXiv preprint arXiv:1802.05365, 2018.
  12. J. Pennington, R. Socher, and C. D. Manning. "Glove: Global vectors for word representation," In Proceedings of Empirical Methods in Natural Language Processing, pp.1532-1543, 2014.
  13. J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, "BERT: Pre-training of deep bidirectional transformers for language understanding," In Proceedings of Northern American Chapter of the Association for Computational Linguistics, 2019.
  14. A. Akbik, T. Bergmann, and R. Vollgraf, "Pooled contextualized embeddings for namedentity recognition," In Proceedings of Northern American Chapter of the Association for Computational Linguistics, 2019.
  15. "판결문 공개 과감히 확대하라," 법률신문, 2019.10.28. [Internet], https://m.lawtimes.co.kr/Content/Article?serial=156740
  16. "법률 지식 베이스 소개" AI허브, 2018.01.02. [Internet], https://aihub.or.kr/aidata/29
  17. "법제처 보도자료," pp.4-5, 2021.12.29.
  18. J. Pennington, R. Socher, and C. D. Manning, "Glove: Global vectors for word representation," In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing(EMNLP), pp.1532-1543, 2014.
  19. A. Vaswani, et al., "Attention is all you need," In Proceedings of Neural Information Processing Systems, 2017.
  20. M. Iyyer, V. Manjunatha, J. Boyd-Graber, and H. Daume III. "Deep unordered composition rivals syntactic methods for text classification," In Proceedings of Association for Computational Linguistics and the International Joint Conference on Natural Language Processing, 2015.
  21. Y. Yang, et al., "Multilingual universal sentence encoder for semantic retrieval," arXiv preprint arXiv:1907.04307, 2019.
  22. "universal-sentence-encoder-multilingual," TensorFlow hub. last modified Jan 05, 2022. accessed Oct. 21, 2021. [Internet], https://tfhub.dev/google/universal-sentence-encoder-multilingual/3
  23. "mecab-ko-dic," bitbucket. last modified Jul. 20, 2018. accessed Oct. 21, 2021. [Internet], https://bitbucket.org/eunjeon/mecab-ko-dic/src/master/