• The Journal of the Korea Contents Association
• /
• v.17 no.4
• /
• pp.576-584
• /
• 2017

Unknown nouns which are not in a dictionary make problems not only morphological analysis but also almost all natural language processing area. This paper describes a recognition method for Korean unknown nouns using strings following nouns such as postposition, suffix and postposition, suffix and eomi, etc. We collect and sort words including nouns from documents and divide a word including unknown noun into two parts, candidate noun and string following the noun, by finding same prefix morphemes from more than two unknown words. We use information of strings following nouns extracted from Sejong corpus and decide unknown noun finally. We obtain 99.64% precision and 99.46% recall for unknown nouns occurred more than two forms in news of two portal sites.

• Journal of KIISE
• /
• v.43 no.4
• /
• pp.430-436
• /
• 2016

Processing of unknown words such as proper nouns and newly coined words is important for a morphological analyzer to process documents in various domains. In this study, a segmentation and tagging method for unknown Korean words is proposed for the 3-step probabilistic morphological analysis. For guessing unknown word, it uses rich suffixes that are attached to open class words, such as general nouns and proper nouns. We propose a method to learn the suffix patterns from a morpheme tagged corpus, and calculate their probabilities for unknown open word segmentation and tagging in the probabilistic morphological analysis model. Results of the experiment showed that the performance of unknown word processing is greatly improved in the documents containing many unregistered words.

• Journal of Korea Multimedia Society
• /
• v.9 no.2
• /
• pp.151-159
• /
• 2006

The ratio of a segmentation algorithm of compound nouns causes an effect a lot in nouns which are not in the dictionary. The structure of Korean compound nouns are mostly derived from the Chinese characters and it includes some preference ratio. So it will be able to use segmentation rule of compound nouns. This paper suggests a segmentation algorithm using some preference ratio of Korean compound nouns which are not in the dictionary. The experiment resulted in getting 88.49% of correct segmentation and showed effective result from the comparative experimentation with other algorithm.

• Journal of Korea Multimedia Society
• /
• v.7 no.3
• /
• pp.418-426
• /
• 2004

This paper suggests a reverse segmentation Algorithm using affix information and some preference pattern information of Korean compound nouns. The structure of Korean compound nouns are mostly derived from the Chinese characters and it includes some preference patterns, which are going to be utilized as a segmentation rule in this paper. To evaluate the accuracy of the proposed algorithm, an experiment was performed with 36061 compound nouns. The experiment resulted in getting 99.3% of correct segmentation and showed excellent satisfactory result from the comparative experimentation with other algorithm, especially most of the four or five-syllable compound nouns were successfully segmented without fail.

• Journal of KIISE:Software and Applications
• /
• v.30 no.1_2
• /
• pp.173-183
• /
• 2003

Morphological analysis is the most widely used method for extracting nouns from Korean texts. For every Eojeol, in order to extract nouns from it, a morphological analyzer performs frequent dictionary lookup and applies many morphonological rules, therefore it requires many operations. Moreover, a morphological analyzer generates all the possible morphological interpretations (sequences of morphemes) of a given Eojeol, which may by unnecessary from the noun extraction`s point of view. To reduce unnecessary computation of morphological analysis from the noun extraction`s point of view, this paper proposes a method for Korean noun extraction considering noun occurrence characteristics. Noun patterns denote conditions on which nouns are included in an Eojeol or not, which are positive cues or negative cues, respectively. When using the exclusive information as the negative cues, it is possible to reduce the search space of morphological analysis by ignoring Eojeols not including nouns. Post-noun syllable sequences(PNSS) as the positive cues can simply extract nouns by checking the part of the Eojeol preceding the PNSS and can guess unknown nouns. In addition, morphonological information is used instead of many morphonological rules in order to recover the lexical form from its altered surface form. Experimental results show that the proposed method can speed up without losing accuracy compared with other systems based on morphological analysis.

• Journal of KIISE:Software and Applications
• /
• v.37 no.12
• /
• pp.919-927
• /
• 2010

Since the performance of mobile devices is recently improved, the requirement of information retrieval is increased in the mobile devices as well as PCs. If a mobile device with small memory uses a tradition language analysis tool to extract nouns from korean texts, it will impose a burden of analysing language. As a result, the need for the language analysis tools adequate to the mobile devices is increasing. Therefore, this paper proposes a new method for noun extraction using post-noun morpheme sequences and noun patterns from a large corpus. The proposed noun extractor has only the dictionary capacity of 146KB and its performance shows 0.86 $F_1$-measure; the capacity of noun dictionary corresponds to only the 4% capacity of the existing noun extractor with a POS tagger. In addition, it easily extract nouns for unknown word because its dependence for noun dictionaries is low.

• Language and Information
• /
• v.7 no.1
• /
• pp.23-40
• /
• 2003

In this paper, we have proposed Auto-TUS, an automatic transcription module of three union symbols-hyphen, dash and tilde (‘­’, ‘―’, ‘∼’)-using their linguistic contexts. Few previous studies have discussed the problems of ambiguities in transcribing symbols into Korean alphabetic letters. We have classified six different reading formulae of the union symbols, analyzed the left and right contexts of the symbols, and investigated selection rules and distributions between the symbols and their contexts. Based on these linguistic features, 86 stereotyped patterns, 78 rules and 8 heuristics determining the types of reading formulae are suggested for Auto-TUS. This module works modularly in three steps. The pilot test was conducted with three test suites, which contains respectively 418, 987 and 1,014 clusters of words containing a union symbol. Encouraging results of 97.36%, 98.48%, 96.55% accuracy were obtained for three test suites. Our next phases are to develop a guessing routine for unknown contexts of the union symbols by using statistical information; to refine the proper nouns and terminology detecting module; and to apply Auto-TUS on a larger scale.

• Proceedings of the Korean Society for Cognitive Science Conference
• /
• 2006.06a
• /
• pp.73-79
• /
• 2006

본 논문에서는 미등록어 처리가 강화된 복합명사 분석 기법을 제시한다. 기본적으로 모든 복합명사 내에 한국어나 외래어의 미등록어가 포함되어 있을 수 있다는 가정하에 분석을 시도한다. 따라서 등록어로 구성된 복합명사에 대해서도 미등록어가 포함된 분해 후보가 생성될 수도 있다. 이는 분해 후보의 수를 크게 증가시키는 문제를 일으킨다. 이 문제에 대처하기 위하여 미등록어의 분류에 따라 미등록어로서의 가능성 여부의 판별 및 제거, 분해 후보 상호간의 견제에 의한 제거 등을 이용하였다. 이러한 과정은 정답 후보 선택시에도 영향을 미쳐 정답이 아닌 분해 후보가 선택되는 것을 방지할 수 있으며, 처리 시간을 줄일 수 있는 이점이 있다. 실험 결과 제시된 기법들이 매우 효과적임을 확인할 수 있었다.

• Proceedings of the IEEK Conference
• /
• 2003.11b
• /
• pp.145-148
• /
• 2003

The agglutinative nature of Korean language makes the problem of automatic indexing of Korean much different from that of Indo-Eroupean languages. Especially, indexing with compound nouns in Korean is very problematic because of the exponential number of possible analysis and the existence of unknown words. To deal with this compound noun indexing problem, we propose a new indexing methods which combines the merits of the morpheme-based indexing methods and the n-gram based indexing methods. Through the experiments, we also find that the best performance of n-gram indexing methods can be achieved with 1.75-gram which is never considered in the previous researches.

• The KIPS Transactions:PartB
• /
• v.8B no.4
• /
• pp.357-364
• /
• 2001

본 논문에서는 단위명사 사전과 접사 사전을 이용하여 한국어 복합명사를 분해하는 새로운 알고리즘을 제안한다. 한국어 복합명사는 그 구조에 있어서 중심어가 뒤에 나타난다는 점에 착안하여 본 논문에서 제안한 분해 알고리즘은 복합명사를 끝음절에서 첫음절 방향 즉 역방향으로 분해를 시도한다. ETRI의 태깅된 코퍼스로부터 추출한 복합명사 3,230개에 대해 실험한 결과 약 96.6%의 분해 정확도를 얻었다. 미등록어를 포함한 복합명사의 경우는 77.5%의 분해 정확도를 나타냈다. 실험에 사용된 데이터중의 미등록어는 대부분 접사를 포함한 파행어로서, 제안한 복합명사 분해 알고리즘은 접사가 부착된 미등록어 분석에 있어서 보다 높은 분석 정확도를 나타냄을 알 수 있었다.