• ETRI Journal
• /
• v.46 no.1
• /
• pp.35-47
• /
• 2024

This study introduces CR-M-SpanBERT, a coreference resolution (CR) model that utilizes multiple embedding-based span bidirectional encoder representations from transformers, for antecedent recognition in natural language (NL) text. Information extraction studies aimed to extract knowledge from NL text autonomously and cost-effectively. However, the extracted information may not represent knowledge accurately owing to the presence of ambiguous entities. Therefore, we propose a CR model that identifies mentions referring to the same entity in NL text. In the case of CR, it is necessary to understand both the syntax and semantics of the NL text simultaneously. Therefore, multiple embeddings are generated for CR, which can include syntactic and semantic information for each word. We evaluate the effectiveness of CR-M-SpanBERT by comparing it to a model that uses SpanBERT as the language model in CR studies. The results demonstrate that our proposed deep neural network model achieves high-recognition accuracy for extracting antecedents from NL text. Additionally, it requires fewer epochs to achieve an average F1 accuracy greater than 75% compared with the conventional SpanBERT approach.

• KIISE Transactions on Computing Practices
• /
• v.21 no.4
• /
• pp.333-337
• /
• 2015

In this paper, we suggest a Coreference Resolution system for Korean using Mention Pair with SVM. The system introduced in this paper, also be able to extract Mention from document which is including automatically tagged name entity information, dependency trees and POS tags. We also built a corpus, including 214 documents with Coreference tags, referencing online news and Wikipedia for training the system and testing the system's performance. The corpus had 14 documents from online news, along with 200 question-and-answer documents from Wikipedia. When we tested the system by corpus, the performance of the system was extracted by MUC-F1 55.68%, B-cube-F1 57.19%, and CEAFE-F1 61.75%.

• KIISE Transactions on Computing Practices
• /
• v.24 no.3
• /
• pp.113-121
• /
• 2018

Position encoding is a method of applying weights according to position of words that appear in a sentence. Pointer networks is a deep learning model that outputs corresponding index with an input sequence. This model can be applied to coreference resolution using attribute. However, the pointer networks has a problem in that its performance is degraded when the length of input sequence is long. To solve this problem, we proposed two contributions to resolve the coreference. First, we applied position encoding and dynamic position encoding to pointer networks. Second, we stack deeply layers of encoder to make high-level abstraction. As results, the position encoding based stacked pointer networks model proposed in this paper had a CoNLL F1 performance of 71.78%, which was improved by 6.01% compared to vanilla pointer networks.

• Journal of KIISE
• /
• v.44 no.8
• /
• pp.774-781
• /
• 2017

Mention detection systems use nouns or noun phrases as a head and construct a chunk of text that defines any meaning, including a modifier. The term "mention detection" relates to the extraction of mentions in a document. In the mentions, a coreference resolution pertains to finding out if various mentions have the same meaning to each other. A pointer network is a model based on a recurrent neural network (RNN) encoder-decoder, and outputs a list of elements that correspond to input sequence. In this paper, we propose the use of mention detection using pointer networks. Our proposed model can solve the problem of overlapped mention detection, an issue that could not be solved by sequence labeling when applying the pointer network to the mention detection. As a result of this experiment, performance of the proposed mention detection model showed an F1 of 80.07%, a 7.65%p higher than rule-based mention detection; a co-reference resolution performance using this mention detection model showed a CoNLL F1 of 52.67% (mention boundary), and a CoNLL F1 of 60.11% (head boundary) that is high, 7.68%p, or 1.5%p more than coreference resolution using rule-based mention detection.

• Proceedings of the Korean Information Science Society Conference
• /
• 2001.10b
• /
• pp.193-195
• /
• 2001

정보추출과 정보검색 시스템에서 문서의 내용을 보다 정확히 분석하기 위해 3인칭 대명사 \"그/그녀/그들/그녀들\"의 선행사를 결정하는 방법을 제안한다. 일반적으로 3인칭 대명사의 선행사는 현재문장 또는 이전문장의 주어인 경우가 많고, 또한 3인칭 대명사가 2회 이상 반복되는 경우가 자주 발생한다. 이러한 특성을 이용하여 현재 문장과 이전 문장에 출현한 인칭명사들 중에서 선행사로 사용되는 경우를 조사하여 경험적인 방법으로 선행사 결정 규칙을 발견하였다. 이 경험 규칙은 3인칭 대명사의 격에 따라 조금씩 달라지기 때문에 대명사의 격에 따라 \"주격/목적격/소유격\"으로 구분하여 기술하였다. 실험 결과, 3인칭 대명사의 선행사 결정 정확도는 주격, 소유격, 목적격에 대해 각각 88.6%, 90.3%, 81.5%로 나타났다. 90.3%, 81.5%로 나타났다.

• The KIPS Transactions:PartB
• /
• v.11B no.2
• /
• pp.227-232
• /
• 2004

When we extract a representative term from text for information retrieval system or a special information for information retrieval and text milling system, we often need to solve the anaphora resolution problem. The antecedent decision problem of a pronoun is one of the major issues for anaphora resolution. In this paper, we are suggesting a method of deciding an antecedent of the third personal pronouns, such as “he/she/they” to analyze the contents of documents precisely. Generally, the antecedent of the third personal Pronouns seem to be the subject of the current statement or previous statement, and also it occasionally happens more than twice. Based on these characteristics, we have found rules for deciding an antecedent, by investigating a case of being an antecedent from the personal pronouns, which appears in the current statement and the previous statements. Since the heuristic rule differs on the case of the third personal pronouns, we described it as subjective case, objective case, and possessive case based on the case of the pronouns. We collected 300 sentences that include a pronoun from the newspaper articles on political issues. The result of our experiment shows that the recall and precision ratio on deciding the antecedent of the third personal pronouns are 79.0% and 86.8%, respectively.

• KIISE Transactions on Computing Practices
• /
• v.22 no.12
• /
• pp.625-631
• /
• 2016

Pointer Networks is a model which generates an output sequence with elements that correspond to an input sequence, based on the attention mechanism. A time complexity of the pointer networks is $O(N^2)$ resulting in longer decoding time of the model. This is because the model calculates attention for each input, if size of the input sequence is N. In this paper, we propose the pointer networks based on skip pointing model, which confirms the necessary input vector at decoding for reducing the decoding time of the pointer networks. Furthermore, experiments were conducted for the pronouns coreference resolution, which uses the method proposed in this paper. Our results show that the processing time per sentence was approximately 1.15 times faster, and the MUC F1 was 83.60%; this was approximately 2.17% improvement and a better performance than the original pointer networks.

• Annual Conference on Human and Language Technology
• /
• 2018.10a
• /
• pp.239-242
• /
• 2018

문맥 표현은 Recurrent neural network (RNN)에 기반한 언어 모델을 학습하여 얻은 여러 층의 히든 스테이트(hidden state)를 가중치 합(weighted sum)을 하여 얻어낸 벡터이다. Convolution neural network (CNN)를 이용하여 음절 표현을 학습하는 경우, 데이터 내에서 발생하는 미등록어를 처리할 수 있다. 본 논문에서는 음절 표현 CNN 기반의 포인터 네트워크와 문맥 표현을 함께 이용하는 방법을 제안하고, 이를 상호참조해결에 적용한다. 실험 결과, 질의응답 데이터셋에서 CoNLL F1 57.88%로 규칙기반에 비하여 11.09% 더 좋은 성능을 보였다.

• Annual Conference on Human and Language Technology
• /
• 2014.10a
• /
• pp.61-64
• /
• 2014

본 논문은 Stanford의 다 단계 시브(Multi-pass Sieve) 상호참조해결을 기반으로, 한국어에 적용한 한국어 상호참조해결(선행 연구)을 이용하여 한정 명사구에 대한 처리와 확장된 대명사 상호참조해결 방법을 제안한다. 지시 관형사와 명사가 결합하여 형성되는 한정 명사구는 일반 멘션(mention)의 특징과 대명사 속성을 한 번에 갖게 된다. 이렇게 되면, 한정 명사구는 모든 시브(sieve)에서 상호참조를 진행할 수 있게 된다. 따라서 이런 특징으로 한정 명사구를 어떤 관점(멘션 또는 대명사)에서 상호참조해결하는 것이 좋은지 보인다. 또한 이런 한정 명사구의 대명사 속성을 이용하기 위해 문법적 의미적 규칙을 적용할 것을 제안한다. 그 결과, 본 논문의 선행 연구인 한국어 상호참조해결에 비하여 CoNLL 값이 약 0.8%만큼 향상되어 61.45%를 측정하였다.

• Proceedings of the Korea Information Processing Society Conference
• /
• 2015.04a
• /
• pp.824-827
• /
• 2015

최근 자연어처리에 딥 러닝이 적용되고 있다. 딥 러닝은 기존의 기계학습 방법들과 달리, 자질 추출 및 조합 등과 같이 사람이 직접 수행해야 했던 부분들을 자동으로 처리할 수 있는 장점이 있다. 본 논문에서는 기존 상호참조해결에 적용했던 SVM 대신 딥 러닝을 이용할 것을 제안한다. 실험결과, 딥 러닝을 이용한 시스템의 성능이 57.96%로 SVM을 이용한 것보다 약 9.6%만큼 높았다.