• Journal of the Korean Institute of Telematics and Electronics C
• /
• v.35C no.10
• /
• pp.38-47
• /
• 1998

The syntactic analysis for the practical machine translation should be able to analyze a long sentence, but the long sentence analysis is a critical problem because of its high analysis complexity. In this paper a sentence segmentation method is proposed for an efficient analysis of a long sentence and the method of determining optimal sentence segmentation positions using statistical information and genetic learning is introduced. It consists of two modules: (1) decomposable position determination which uses lexical contextual constraints acquired from a training data tagged with segmentation positions. (2) segmentation position selection by the selection function of which the weights of parameters are determined through genetic learning, which selects safe segmentation positions with enhancing the analysis efficiency as much as possible. The safe segmentation by the proposed sentence segmentation method and the efficiency enhancement of the analysis are presented through experiments.

• Journal of KIISE:Software and Applications
• /
• v.33 no.5
• /
• pp.470-480
• /
• 2006

The longer the input sentences, the worse the parsing results. To improve the parsing performance, many methods about long sentence segmentation have been reserarched. As an isolating language, Chinese sentence has fewer cues for sentence segmentation. However, the average frequency of comma usage in Chinese is higher than that of other languages. The syntactic information that the comma conveys can play an important role in long sentence segmentation of Chinese languages. This paper proposes a method for classifying commas in Chinese sentences according to the context where the comma occurs. Then, sentences are segmented using the classification result. The experimental results show that the accuracy of the comma classification reaches 87.1%, and with our segmentation model, the dependency parsing accuracy of our parser is improved by 5.6%.

• Journal of KIISE:Software and Applications
• /
• v.32 no.5
• /
• pp.385-395
• /
• 2005

Long sentence analysis has been a critical problem in machine translation because of high complexity. The methods of intra-sentence segmentation have been proposed to reduce parsing complexity. This paper presents the intra-sentence segmentation method based on maximum entropy probability model to increase the coverage and accuracy of the segmentation. We construct the rules for choosing candidate segmentation positions by a teaming method using the lexical context of the words tagged as segmentation position. We also generate the model that gives probability value to each candidate segmentation positions. The lexical contexts are extracted from the corpus tagged with segmentation positions and are incorporated into the probability model. We construct training data using the sentences from Wall Street Journal and experiment the intra-sentence segmentation on the sentences from four different domains. The experiments show about $88\%$ accuracy and about $98\%$ coverage of the segmentation. Also, the proposed method results in parsing efficiency improvement by 4.8 times in speed and 3.6 times in space.

• The Transactions of the Korea Information Processing Society
• /
• v.7 no.9
• /
• pp.2876-2884
• /
• 2000

Clausal segmentation is the method that parses Korean sentences by segmenting one long sentence into several phrases according to the predicates. So far most of researches could be useful for literary sentences, but long sentences increase complexities of the syntax analysis. Thus this paper proposed Two-Level Clausal Segmentation using sense information which was designed and implemented to solve this problem. Analysis of clausal segmentation and understanding of word senses can reduce syntactic and semantic ambiguity. Clausal segmentation using Sense Information is necessary because there are structural ambiguity of sentences and a frequent abbreviation of auxiliary word in common sentences. Two-Level Clausal Segmentation System(TLCSS) consists of Complement Selection Process(CSP) and Noncomplement Expansion Process(NEP). CSP matches sentence elements to subcategorization dictionary and noun thesaurus. As a result of this step, we can find the complement and subcategorization pattern. Secondly, NEP is the method that uses syntactic property and the others methods for noncomplement increase of growth. As a result of this step, we acquire segmented sentences. We present a technique to estimate the precision of Two-Level Clausal Segmentation System, and shows a result of Clausal Segmentation with 25,000 manually sense tagged corpus constructed by ETRl-KONAN group. An Two-Level Clausal Segmentation System shows clausal segmentation precision of 91.8%.

• Journal of KIISE:Software and Applications
• /
• v.32 no.9
• /
• pp.936-947
• /
• 2005

In dependency parsing of long sentences with fewer subjects than predicates, it is difficult to recognize which predicate governs which subject. To handle such syntactic ambiguity between subjects and predicates, this paper proposes an 'S-clause' segmentation method, where an S(ubject)-clause is defined as a group of words containing several predicates and their common subject. We propose an automatic S -clause segmentation method using decision trees. The S-clause information was shown to be very effective in analyzing long sentences, with an improved parsing performance of 5 percent. In addition, the performance in detecting the governor of subjects was improved by $32\%$.

• The KIPS Transactions:PartB
• /
• v.13B no.3 s.106
• /
• pp.345-352
• /
• 2006

The practical English-Korean machine translation system should be able to translate long sentences quickly and accurately. The intra-sentence segmentation method has been proposed and contributed to speeding up the syntactic analysis. This paper proposes the syntactic category prediction method using decision trees for getting accurate parsing results. In parsing with segmentation, the segment is separately parsed and combined to generate the sentence structure. The syntactic category prediction would facilitate to select more accurate analysis structures after the partial parsing. Thus, we could improve the parsing accuracy by the prediction. We construct features for predicting syntactic categories from the parsed corpus of Wall Street Journal and generate decision trees. In the experiments, we show the performance comparisons with the predictions by human-built rules, trigram probability and neural networks. Also, we present how much the category prediction would contribute to improving the translation quality.

• KIPS Transactions on Software and Data Engineering
• /
• v.5 no.1
• /
• pp.21-28
• /
• 2016

The performance of an English-Korean machine translation system depends heavily on its English parser. The parser in this paper is a part of the rule-based English-Korean MT system, which includes many syntactic rules and performs the chart-based parsing. The parser generates too many structures due to many syntactic rules, so much time and memory are required. The rule-based parser has difficulty in analyzing and translating the long sentences including the commas because they cause high parsing complexity. In this paper, we propose the 3-phase parsing method with sentence segmentation to efficiently translate the long sentences appearing in usual. Each phase of the syntactic analysis applies its own independent syntactic rules in order to reduce parsing complexity. For the purpose, we classify the syntactic rules into 3 classes and design the 3-phase parsing algorithm. Especially, the syntactic rules in the 3rd class are for the sentence structures composed with commas. We present the automatic rule acquisition method for 3rd class rules from the syntactic analysis of the corpus, with which we aim to continuously improve the coverage of the parsing. The experimental results shows that the proposed 3-phase parsing method is superior to the prior parsing method using only intra-sentence segmentation in terms of the parsing speed/memory efficiency with keeping the translation quality.

• Annual Conference on Human and Language Technology
• /
• 2018.10a
• /
• pp.559-560
• /
• 2018

순환 신경망(RNN) 기반의 Long Short-Term Memory(LSTM)는 자연어처리 분야에서 우수한 성능을 보이는 모델이다. 음성을 문자로 변환해주는 Speech to Text (STT)를 이용해 자막을 생성하고, 생성된 자막을 다른 언어로 동시에 번역을 해주는 서비스가 활발히 진행되고 있다. STT를 사용하여 자막을 추출하는 경우에는 마침표가 없이 전부 연결된 문장이 생성되기 때문에 정확한 번역이 불가능하다. 본 논문에서는 영어자막의 자동 번역 시, 정확도를 높이기 위해 텍스트를 문장으로 분할하여 마침표를 생성해주는 방법을 제안한다. 이 때, LSTM을 이용하여 데이터를 학습시킨 후 테스트한 결과 62.3%의 정확도로 마침표의 위치를 예측했다.

• Annual Conference on Human and Language Technology
• /
• 2003.10d
• /
• pp.261-266
• /
• 2003

문장이 길면 구문분석의 정확률이 크게 낮아진다. 따라서 장문을 분할하여 분석하면 구문분석의 복잡도를 크게 줄일 수 있어 정확률 향상에 크게 기여할 수 있다. 특히, 중국어는 고립어로서, 교착어나 융합어와 비교할 때 자연어처리에 도움을 줄 수 있는 굴절이나 어미정보가 없어 구문분석에 어려움이 더욱 많다. 반면, 중국어 문자에서는 쉼표를 비교적 많이 사용하고 있고 또한 쉼표의 쓰임이 정확하므로 구문 분석에 도움을 줄 수 있다. 본 논문에서는 쉼표가 많이 쓰이고 있는 중국어 문장에서 해당 쉼표위치 문장 분할가능여부를 Support Vector Machine을 이용 판단하여 정확률 88.61%의 높은 분할 성능을 보였다.

• Annual Conference on Human and Language Technology
• /
• 2005.10a
• /
• pp.89-96
• /
• 2005

본 연구에서는 영한 기계 번역에서 20단어 이상의 긴 문장을 보다 정확히 분석하기 위하여 문장을 복수개의 의미 있는 절로 분할하고자 한다. 긴 문장은 구문 분석을 시도할 때, 시간적으로 또는 공간적으로 급격히 증가하는 자원을 소모시킨다. 이러한 문제를 해결하기 위하여, 본 연구에서는 긴 문장에서 분할 가능한 지점을 인식하여 이러한 지점을 중심으로 여러 개의 절을 생성한 후, 이 절을 개별적으로 분석하고자 하였다. 문장을 분할하기 위해서 일단 문장 내부에 존재하고 있는 분할이 가능한 지점을 선택하고, 선택된 지점을 중심으로 문맥 정보를 표현하는 입력 벡터를 생성하였다. 그리고 Support Vector Machine (SVM)을 이용하여 이러한 후보 지점의 특성을 학습하여 향후 긴 문장이 입력되었을 때 보다 정확하게 분할점을 찾고자 하였다. 본 논문에서는 SVM의 보다 좋은 학습과 분류를 위하여 내부 커널로써 다항 커널 (polynomial kernel)을 사용하였다. 그리고 실험을 통하여 약 0.97의 f-measure 값을 얻을 수 있었다.