• Title/Summary/Keyword: Korean morphological analyzer

Search Result 116, Processing Time 0.025 seconds

A Natural Language Retrieval System for Entertainment Data (엔터테인먼트 데이터를 위한 자연어 검색시스템)

  • Kim, Jung-In
    • Journal of Korea Multimedia Society
    • /
    • v.18 no.1
    • /
    • pp.52-64
    • /
    • 2015
  • Recently, as the quality of life has been improving, search items in the area of entertainment represent an increasing share of the total usage of Internet portal sites. Information retrieval in the entertainment area is mainly depending on keywords that users are inputting, and the results of information retrieval are the contents that contain those keywords. In this paper, we propose a search method that takes natural language inputs and retrieves the database pertaining to entertainment. The main components of our study are the simple Korean morphological analyzer using case particle information, predicate-oriented token generation, standardized pattern generation coherent to tokens, and automatic generation of the corresponding SQL queries. We also propose an efficient retrieval system that searches the most relevant results from the database in terms of natural language querying, especially in the restricted domain of music, and shows the effectiveness of our system.

A Plug-In Component-based Korean Morphological Analyzer (플러그인 컴포넌트 기반의 한국어 형태소 분석기)

  • Park, Sangwon;Choi, Donghyun;Kim, Eunkyung;Choi, Keysun
    • Annual Conference on Human and Language Technology
    • /
    • 2010.10a
    • /
    • pp.197-201
    • /
    • 2010
  • 지식/정보 서비스의 수준이 급격히 발전함에 따라 기반 기술인 자연언어처리의 중요성이 증가했고, 또 여러 분야에서 다양한 목적에 부합하는 자연언어처리 도구를 필요로 하고 있다. 한국어 자연언어처리 기반 기술 중에서 형태소 분석은 기초적이면서도 중요한 분석 단계이다. 하지만 현재까지 개발된 한국어 형태소 분석기들은 대부분 특정 시스템에 맞게 최적화되어 실행 효율성과 정확성을 높이는데 중점을 두고 있다. 이러한 도구들은 접근성과 확장성이 떨어지고 다양한 요구에 유언하게 대처할 수 없는 단점이 있다. 따라서 본 논문에서는 플러그인 형태의 한국어 형태소 분석 컴포넌트들을 이용하여 유연하게 워크플로를 구성하고 다양한 목적에 맞게 활용할 수 있도록 지원하는 한국어 형태소 분석 시스템을 제안한다. 이번 연구를 통해서 보다 많은 사람들이 형태소 분석 기술에 쉽게 접근하고 활용하여, 한국어 분석 기반 기술 및 용용 기술이 더욱 발전 할 수 있을 것으로 기대한다.

  • PDF

Evaluation of Hormone Deficiency in Vertebral Body: Analysis of Bone Structure and Quality (호르몬 결핍이 척추체에 미치는 영향 평가: 골의 구조학적 및 질적 요소 분석)

  • Kim, Chi-Hoon;Woo, Dae-Gon;Park, Ji-Hyung;Lee, Beob-Yi;Kim, Chi-Hyun;Kim, Han-Sung
    • Journal of the Korean Society for Precision Engineering
    • /
    • v.27 no.5
    • /
    • pp.92-101
    • /
    • 2010
  • This study evaluated the structure and quality of osteoporotic vertebral bone. To induce osteoporosis, eight rats were ovariectomized (OVX). All rats were divided into two groups (Normal group: 4, OVX group: 4). Total lumbar vertebrae for each rat were scanned by in-vivo ${\mu}CT$ at 0, 4 and 8 weeks. Morphological characteristics (BV/TV, Tb.Th, Tb.N, Tb.Sp and SMI) were calculated by in-vivo ${\mu}CT$ image analyzer. Three dimensional finite element models were analyzed to investigate bone strength of OVX and Normal groups. Moreover, the elastic modulus was quantitatively analyzed to evaluate the quality changes of osteoporotic bone. In the OVX group, BV/TV, Tb.Th and Tb.N were significantly decreased at all the lumbar over time (p<0.05). We also investigated a contrary tendency in Tb.Sp and SMI, compared to the above results in each group. A degree of alteration of mechanical characteristics in OVX group was decreased over measuring time (p<0.05). Bone quality presented by distribution of elastic modulus was improved in the Normal group more than OVX group. The findings of the present study indicated that both bone structure and quality of whole lumbar could be tracked and detected by analyzing the morphological and biomechanical characteristics of bones, based on a nondestructive method.

A Corpus-based Hybrid Model for Morphological Analysis and Part-of-Speech Tagging (형태소 분석 및 품사 부착을 위한 말뭉치 기반 혼합 모형)

  • Lee, Seung-Wook;Lee, Do-Gil;Rim, Hae-Chang
    • Journal of the Korea Society of Computer and Information
    • /
    • v.13 no.7
    • /
    • pp.11-18
    • /
    • 2008
  • Korean morphological analyzer generally generates multiple candidates, and then selects the most likely one among multiple candidates. As the number of candidates increases, the chance that the correctly analyzed candidate is included in the candidate list also grows. This process, however, increases ambiguity and then deteriorates the performance. In this paper, we propose a new rule-based model that produces one best analysis. The analysis rules are automatically extracted from large amount of Part-of-Speech tagged corpus, and the proposed model does not require any manual construction cost of analysis rules, and has shown high success rate of analysis. Futhermore, the proposed model can reduce the ambiguities and computational complexities in the candidate selection phase because the model produces one analysis when it can successfully analyze the given word. By combining the conventional probability-based model. the model can also improve the performance of analysis when it does not produce a successful analysis.

  • PDF

Rule-based Speech Recognition Error Correction for Mobile Environment (모바일 환경을 고려한 규칙기반 음성인식 오류교정)

  • Kim, Jin-Hyung;Park, So-Young
    • Journal of the Korea Society of Computer and Information
    • /
    • v.17 no.10
    • /
    • pp.25-33
    • /
    • 2012
  • In this paper, we propose a rule-based model to correct errors in a speech recognition result in the mobile device environment. The proposed model considers the mobile device environment with limited resources such as processing time and memory, as follows. In order to minimize the error correction processing time, the proposed model removes some processing steps such as morphological analysis and the composition and decomposition of syllable. Also, the proposed model utilizes the longest match rule selection method to generate one error correction candidate per point, assumed that an error occurs. For the purpose of deploying memory resource, the proposed model uses neither the Eojeol dictionary nor the morphological analyzer, and stores a combined rule list without any classification. Considering the modification and maintenance of the proposed model, the error correction rules are automatically extracted from a training corpus. Experimental results show that the proposed model improves 5.27% on the precision and 5.60% on the recall based on Eojoel unit for the speech recognition result.

DaHae: Japanese Morphological Analyzer for Japanese to Korean Machine Translation (DaHae: 일한 기계번역을 위한 일본어 형태소 분석기)

  • Yuh, Sang-Hwa;Jung, Han-Min;Chang, Won;Kim, Tae-Wan;Hwang, Do-Sam;Park, Dong-In
    • Annual Conference on Human and Language Technology
    • /
    • 1995.10a
    • /
    • pp.195-207
    • /
    • 1995
  • 일본어는 한자, 히라가나, 가다가나 등 다양한 종류의 문자를 사용하며 이들의 혼용 비율이 매우 높아 띄어쓰기를 하지 않아도 문서의 가독성을 유지한다. ICOT 사전, EDR 사전, ATLAS I/JK사전 등 기존의 전자 사전에서 복합 자종의 표제어가 차지하는 비율(한자+히라가나의 표제어 제외)은 평균 8.8%로 그 수가 매우 작다. 따라서, 문장 내에서 자종의 변화는 단어를 구분하는 하나의 delimiter로 이용될 수 있다. 본 시스템에서는 형태소 분석의 전단계로 전처리기를 두어 자종정보(character type information)에 의한 fragment 분리 및 예외 단어, 정형표현 처리를 수행하며 각 fragment 의 형태소 분석 방법을 제시한다. 형태소 분석기는 전처리기의 처리 결과를 입력받아 각각의 fragment를 전처리기가 제시한 분석 방법에 따라 분석하여 입력 문장의 가능한 모든 분석을 추출한다. 이 방법은 불필요한 사전 탐색과 접속 체크 회수를 줄여 분석 성능을 향상시킨다.

  • PDF

Analysis and Localization of freeWAIS-sf (FreeWAIS-sf의 분석 및 한글화)

  • O, Jeong-Seok;Kim, Ji-Seung;Lee, Jun-Ho;Lee, Sang-Ho
    • Journal of KIISE:Computing Practices and Letters
    • /
    • v.5 no.5
    • /
    • pp.611-618
    • /
    • 1999
  • An efficient and effective access to needed information becomes an important factor in the modern information society. Many people have developed information retrieval (IR) systems that retrieve needed information from a large amount of data at a given time. However, most freely available IR systems have been developed for English text rather than for Korean text. In this research, we have analyzed the IR system freeWAIS-sf, and localized it with the Korean morphological analyzer, namely HAM. The localized freeWAIS-sf can handle both English text and Korean text simultaneously. We have also modified the weighting scheme of freeWAIS-sf. The experimental result shows that the modified weighting scheme outperforms the original one in terms of retrieval effectiveness.

A Study on Improved Comments Generation Using Transformer (트랜스포머를 이용한 향상된 댓글 생성에 관한 연구)

  • Seong, So-yun;Choi, Jae-yong;Kim, Kyoung-chul
    • Journal of Korea Game Society
    • /
    • v.19 no.5
    • /
    • pp.103-114
    • /
    • 2019
  • We have been studying a deep-learning program that can communicate with other users in online communities since 2017. But there were problems with processing a Korean data set because of Korean characteristics. Also, low usage of GPUs of RNN models was a problem too. In this study, as Natural Language Processing models are improved, we aim to make better results using these improved models. To archive this, we use a Transformer model which includes Self-Attention mechanism. Also we use MeCab, korean morphological analyzer, to address a problem with processing korean words.

A Study on Lexical Ambiguity Resolution of Korean Morphological Analyzer (형태소 분석기의 어휘적 중의성 해결에 관한 연구)

  • Park, Yong-Uk
    • The Journal of the Korea institute of electronic communication sciences
    • /
    • v.7 no.4
    • /
    • pp.783-787
    • /
    • 2012
  • It is not easy to find out syntactic error in a spelling checker systems of Korean, because the spelling checker is generally to correct each phrase and it cannot check the errors of contextual ill-matched words. Spelling checker system tests errors based on a words. Disambiguation of lexical ambiguities is important in natural language processing. Its outputs is used in syntactic analysis. For accurate analysis of a sentence, syntactic analysis system must find out the ambiguity of morphemes in a word. In this paper, we suggest several rules to resolve the ambiguities of morphemes in a word. Using these methods, we can reduce many lexical ambiguities in Korean.

Korean Homograph Tagging Model based on Sub-Word Conditional Probability (부분어절 조건부확률 기반 동형이의어 태깅 모델)

  • Shin, Joon Choul;Ock, Cheol Young
    • KIPS Transactions on Software and Data Engineering
    • /
    • v.3 no.10
    • /
    • pp.407-420
    • /
    • 2014
  • In general, the Korean morpheme analysis procedure is divided into two steps. In the first step as an ambiguity generation step, an Eojeol is analyzed into many morpheme sequences as candidates. In the second step, one appropriate candidate is chosen by using contextual information. Hidden Markov Model(HMM) is typically applied in the second step. This paper proposes Sub-word Conditional Probability(SCP) model as an alternate algorithm. SCP uses sub-word information of adjacent eojeol first. If it failed, then SCP use morpheme information restrictively. In the accuracy and speed comparative test, HMM's accuracy is 96.49% and SCP's accuracy is just 0.07% lower. But SCP reduced processing time 53%.