• The KIPS Transactions:PartB
• /
• v.10B no.2
• /
• pp.229-236
• /
• 2003

Named entity recognition is the process indispensable to question answering and information extraction systems. This paper presents an HMM based named entity (m) recognition method using the construction principles of compound words. In Korean, many named entities can be decomposed into more than one word. Moreover, there are contextual relationships among nouns in an NE, and among an NE and its surrounding words. In this paper, we classify words into a word as an NE in itself, a word in an NE, and/or a word adjacent to an n, and train an HMM based on NE-related word types and parts of speech. Proposed named entity recognition (NER) system uses trigram model of HMM for considering variable length of NEs. However, the trigram model of HMM has a serious data sparseness problem. In order to solve the problem, we use multi-level back-offs. Experimental results show that our NER system can achieve an F-measure of 87.6% in the economic articles.

• Proceedings of the Korean Society of Computer Information Conference
• /
• 2014.01a
• /
• pp.467-468
• /
• 2014

본 논문에서는 파일 내용 기반 검색 프로그램을 설계하고 구현하였다. 역 인덱스 구조를 이용하여 설계하였으며 별도의 정보 검색 라이브러리 사용 없이 구현하였다. 인덱스 파일은 트라이 자료 구조를 직접 설계 및 구현 하였으며 자바 언어의 HashMap 구조를 중첩 형태로 구현하였다. 개발 시스템의 유용성을 테스트하기 위해 GRE 단어집에 수록된 약 3,300개의 단어를 사용하여 임의 생성한 텍스트 파일 집합을 사용하였다.

• Journal of the Korea Society of Computer and Information
• /
• v.14 no.8
• /
• pp.65-71
• /
• 2009

Natural language processing is necessary in order to efficiently perform filtering tremendous information produced in information retrieval of world wide web. This paper suggested an algorithm for meaning of numerals in the text. The algorithm for meaning of numerals utilized context-free grammars with the chart parsing technique, interpreted affixes connected with the numerals and was designed to disambiguate their meanings systematically supported by the n-gram based words. And the algorithm was designed to use POS (part-of-speech) taggers, to automatically recognize restriction conditions of trigram words, and to gradually disambiguate the meaning of the numerals. This research performed experiment for the suggested system of the numeral interpretation. The result showed that the frequency-proportional method recognized the numerals with 86.3% accuracy and the condition-proportional method with 82.8% accuracy.

• Annual Conference on Human and Language Technology
• /
• 2002.10e
• /
• pp.111-117
• /
• 2002

개체명 인식은 질의응답(QA), 정보 주줄(IE), 텍스트 마이닝 시스템의 성능 향상에 중요한 역할을 담당한다. 이 논문에서는 교사학습 기반의 한국어 개체명 인식에 대해 설명한다. 한국어에서 많은 개체명들이 하나 이상의 단어로 구성되어 있으며, 개체명을 구성하는 단어 사이에는 의존 관계가 존재하고, 개체명과 개체명 주위의 단어 사이에도 문맥적 의존관계를 가지고 있다. 본 논문에서는 가변길이의 개체명과 주변 문맥의 학습을 위해 트라이그램을 이용한 HMM을 사용하였으며, 자료 부족 문제를 해소하기 위해 어휘 기반이 아닌 부개체 유형 기반의 학습을 수행하였다. 학습된 개체명 인식 시스템을 이용하여 경제 분야의 신문 기사에 대한 실험 결과, 84.4%의 정확률과 90.9%의 재현률을 보였다.

• The Journal of the Acoustical Society of Korea
• /
• v.21 no.3
• /
• pp.274-284
• /
• 2002

Word segmentation errors occurring in text preprocessing often insert incorrect words into recognition vocabulary and cause poor language models for Korean large vocabulary continuous speech recognition. We propose an automatic word segmentation algorithm using Markov chains and syllable-based n-gram language models in order to correct word segmentation error in teat corpora. We assume that a sentence is generated from a Markov chain. Spaces and non-space characters are generated on self-transitions and other transitions of the Markov chain, respectively Then word segmentation of the sentence is obtained by finding the maximum likelihood path using syllable n-gram scores. In experimental results, the algorithm showed 91.58% word accuracy and 96.69% syllable accuracy for word segmentation of 254 sentence newspaper columns without any spaces. The algorithm improved the word accuracy from 91.00% to 96.27% for word segmentation correction at line breaks and yielded the decomposition accuracy of 96.22% for compound-noun decomposition.