KIISE Transactions on Computing Practices (정보과학회 컴퓨팅의 실제 논문지)

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Korean Coreference Resolution using Stacked Pointer Networks based on Position Encoding

포지션 인코딩 기반 스택 포인터 네트워크를 이용한 한국어 상호참조해결

  • Received : 2017.09.06
  • Accepted : 2017.12.18
  • Published : 2018.03.15
⟨ Previous Next ⟩

Abstract

Position encoding is a method of applying weights according to position of words that appear in a sentence. Pointer networks is a deep learning model that outputs corresponding index with an input sequence. This model can be applied to coreference resolution using attribute. However, the pointer networks has a problem in that its performance is degraded when the length of input sequence is long. To solve this problem, we proposed two contributions to resolve the coreference. First, we applied position encoding and dynamic position encoding to pointer networks. Second, we stack deeply layers of encoder to make high-level abstraction. As results, the position encoding based stacked pointer networks model proposed in this paper had a CoNLL F1 performance of 71.78%, which was improved by 6.01% compared to vanilla pointer networks.

포지션 인코딩은 문장 내 등장하는 단어의 위치에 따라 가중치를 적용하는 방법이다. 포인터 네트워크는 입력열에 대응되는 위치를 출력하는 딥 러닝 모델이며, 상호참조해결에 적용될 수 있다. 그러나 포인터 네트워크는 입력열의 길이가 긴 경우에 성능이 저하되는 문제가 있다. 이러한 문제를 해결하기 위하여 본 논문에서는 포지션 인코딩과 동적 포지션 인코딩을 포인터 네트워크에 적용할 것을 제안하고, Encoder RNN의 레이어를 더 깊게 쌓아 높은 수준으로 추상화할 것을 제안하며, 이를 이용한 상호참조해결 모델을 제안한다. 실험 결과, 본 논문에서 제안한 포지션 인코딩 기반 스택 포인터 네트워크 모델이 기존의 포인터 네트워크 모델보다 6.01% 향상된 CoNLL F1 71.78%의 성능을 보였다.

Keywords

Acknowledgement

Grant : (엑소브레인-1세부) 휴먼 지식증강 서비스를 위한 지능진화형 WiseQA 플랫폼 기술 개발

Supported by : 정보통신기술진흥센터, 한국연구재단

References

  1. H. Lee, A. Chang, Y. Peirsman, N. Chambers, M. Surdeanu and D. Jurafsky, Deterministic coreference resolution based on entity-centric, precision-ranked rules, Computational Linguistics 39.4, pp. 885-916, 2013. https://doi.org/10.1162/COLI_a_00152
  2. C. Park, K.H. Choi, C. Lee and S. Lim, Korean Coreference Resolution with Guided Mention Pair Model using the Deep Learning, ETRI Journal, 38.6, pp. 1207-1217, 2016. https://doi.org/10.4218/etrij.16.0115.0896
  3. D. Bahdanau, K. Cho and Y. Bengio, Neural machine translation by jointly learning to align and translate, Proceedings of ICLR' 15, 2015.
  4. K. Cho, B. Van Merrienboer and C. Gulcehre, Learning phrase representation using RNN encoder-decoder for statistical machine translation, Proc. of EMNLP' 14, 2014.
  5. O. Vinyals, M. Fortunato and N. Jaitly, Pointer Networks, Advances in Neural Information Processing Systems, pp. 2674-2682, 2015.
  6. C. Park and C. Lee, Coreference Resolution for Pronoun with Pointer Networks, KCC 2016, pp. 699-701, 2016.
  7. S. Sukhbaatar, J. Weston and R. Fergus, End-to-end memory networks, Advances in neural information processing systems, pp. 2440-2448, 2015.
  8. A. Rahman, and V. Ng, Supervised models for coreference resolution, Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing: Volume 2-Volume 2. Association for Computational Linguistics, pp. 968-977, 2009.
  9. K. Lee, L. He, M. Lewis and L. Zettlemoyer, End-to-end Neural Coreference Resolution, arXiv preprint, arXiv:1707.07045, 2017.
  10. K. Clark and C. D. Manning, Deep Reinforcement Learning for Mention-Ranking Coreference Models, Proceedings of EMNLP' 16, 2016.
  11. C. Park and C. Lee, Korean Coreference Resolution using Pointer Networks based on Position Encoding, Proc. Of the KIISE and the KBS Joint Symposium, 2017.
  12. C. Lee, J. Kim and J. Kim, Korean Dependency Parsing using Deep Learning, The 26th Annual Conference on Human & Cognitive Language Technology, pp. 87-91, 2014.