Browse > Article
http://dx.doi.org/10.4218/etrij.2020-0282

Simple and effective neural coreference resolution for Korean language  

Park, Cheoneum (AIRS Company, Hyundai Motor Group)
Lim, Joonho (SW and Contents Research Laboratory, Electronics and Telecommunications Research Institute)
Ryu, Jihee (SW and Contents Research Laboratory, Electronics and Telecommunications Research Institute)
Kim, Hyunki (SW and Contents Research Laboratory, Electronics and Telecommunications Research Institute)
Lee, Changki (Computer Science, Kangwon National University)
Publication Information
ETRI Journal / v.43, no.6, 2021 , pp. 1038-1048 More about this Journal
Abstract
We propose an end-to-end neural coreference resolution for the Korean language that uses an attention mechanism to point to the same entity. Because Korean is a head-final language, we focused on a method that uses a pointer network based on the head. The key idea is to consider all nouns in the document as candidates based on the head-final characteristics of the Korean language and learn distributions over the referenced entity positions for each noun. Given the recent success of applications using bidirectional encoder representation from transformer (BERT) in natural language-processing tasks, we employed BERT in the proposed model to create word representations based on contextual information. The experimental results indicated that the proposed model achieved state-of-the-art performance in Korean language coreference resolution.
Keywords
coreference resolution; head-final language; Korean; pretrained language model; recurrent neural network;
Citations & Related Records
연도 인용수 순위
  • Reference
1 A. Bagga and B. Baldwin, Algorithms for scoring coreference chains, in Proc. Int. Conf. Lang. Resour. Eval. (Granada, Spain), May 1998, pp. 563-566.
2 R. K. Srivastava, K. Greff, and J. Schmidhuber, Highway networks, arXiv preprint, CoRR, 2015, arXiv: 1505.00387.
3 H. Lee et al., Stanford's multi-pass sieve coreference resolution system at the conll-2011 shared task, in Proc. Conf. Comput. Natural Lang. Learn.: Shared Task (Portland, OR, USA), June 2011, pp. 28-34.
4 W. M. Soon, H. T. Ng, and D. C. Y. Lim, A machine learning approach to coreference resolution of noun phrases, Comput. Linguistics 27 (2001), no. 4, 521-544.   DOI
5 A. Vaswani et al., Attention is all you need, in Proc. Conf. Neural Inf. Process. Syst. (Long Beach, CA, USA), Dec. 2017, pp. 5998-6008.
6 C. Park, K.-H. Choi, and C. Lee, Korean coreference resolution using the multi-pass sieve, J. KIISE 41 (2014), no. 11, 992-1005.   DOI
7 S. Hochreiter and J. Schmidhuber, Long short-term memory, Neural Comput. 9 (1997), no. 8, 1735-1780.   DOI
8 T. Lei, Y. Zhang, and Y. Artzi, Training RNNS as fast as CNNS, arXiv preprint, CoRR, 2017, arXiv: 1709.02755.
9 K. Clark and C. D. Manning, Improving coreference resolution by learning entity-level distributed representations, arXiv preprint, CoRR, 2016, arXiv: 1606.01323.
10 T. Dozat and C. D. Manning, Deep biaffine attention for neural dependency parsing, arXiv preprint, CoRR, 2016, arXiv: 1611.01734.
11 M. Vilain et al., A model-theoretic coreference scoring scheme, in Proc. Conf. Message Underst. (Columbia, Maryland), Nov. 1995, pp. 45-52.
12 S. Pradhan et al., Conll-2012 shared task: Modeling multilingual unrestricted coreference in ontonotes, in Proc. Joint Conf. EMNLP and CoNLL: Shared Task (Jeju Island, South Korea), July 2012, pp. 1-40.
13 J. Devlin et al., Bert: Pre-training of deep bidirectional transformers for language understanding, arXiv preprint, CoRR, 2018, arXiv: 1810.04805.
14 V. Ng and C. Cardie, Improving machine learning approaches to coreference resolution, in Proc. Annu. Meet. Assoc. Comput. Linguistics (Philadelphia, PA, USA), July 2002, pp. 104-111.
15 N. Kwon, M. Polinsky, and R. Kluender, Subject preference in Korean, in Proc. West Coast Conf. Form. Linguistics (Somerville, MA, USA), Sept. 2006, pp. 1-14.
16 D. Bahdanau, K. Cho, and Y. Bengio, Neural machine translation by jointly learning to align and translate, arXiv preprint, CoRR, 2014, arXiv: 1409.0473.
17 T. Mikolov et al., Recurrent neural network based language model, in Proc. Annu. Conf. Int. Speech Commun. Assoc. (Chiba, Japan), Sept. 2010, pp. 1045-1048.
18 C. Park et al., Korean coreference resolution with guided mention pair model using deep learning, ETRI J. 38 (2016), no. 6, 1207-1217.   DOI
19 K. Lee et al., End-to-end neural coreference resolution, CoRR, 2017, arXiv: 1707.07045.
20 R. Zhang et al., Neural coreference resolution with deep biaffine attention by joint mention detection and mention clustering, arXiv preprint, CoRR, 2018, arXiv: 1805.04893.
21 R. Sennrich, B. Haddow, and A. Birch, Neural machine translation of rare words with subword units, arXiv preprint, CoRR, 2015, arXiv: 1508.07909.
22 D. P. Kingma and J. Ba, Adam: A method for stochastic optimization, arXiv preprint, CoRR, 2014, arXiv: 1412.6980.
23 M. Schuster and K. K. Paliwal, Bidirectional recurrent neural networks, IEEE Trans. Signal Process. 45 (1997), no. 11, 2673-2681.   DOI
24 K. Lee, L. He, and L. Zettlemoyer, Higher-order coreference resolution with coarse-to-fine inference, arXiv preprint, 2018, arXiv: 1804.05392.
25 D.-A. Clevert, T. Unterthiner, and S. Hochreiter, Fast and accurate deep network learning by exponential linear units (ELUs), arXiv preprint, CoRR, 2015, arXiv: 1511.07289.
26 C. Park et al., Contextualized embedding- and character embedding-based pointer network for Korean coreference resolution, Annu. Conf. Hum. Lang. Technol. 2018 (2018), 239-242.
27 X. Luo, On coreference resolution performance metrics, in Proc. Conf. Hum. Lang. Technol. Empirical Methods Nat. Lang. Process. (Vancouver, Canada), Oct. 2005, pp. 25-32.
28 W. Wang et al., Gated self-matching networks for reading comprehension and question answering, in Proc. Annu. Meet. Assoc. Comput. Linguistics (Vancouver, Canada), July 2017, pp. 189-198.