• Title/Summary/Keyword: morpheme

Search Result 238, Processing Time 0.03 seconds

An Efficient Korean Morpheme Analyzer and Synthesizer using Dictionary Information and Chart Data Structure (사전 정보와 차트 자료 구조를 이용한 효율적인 형태소 분석기 및 합성기(KoMAS))

  • 김정해;이상조
    • Journal of the Korean Institute of Telematics and Electronics B
    • /
    • v.31B no.3
    • /
    • pp.123-131
    • /
    • 1994
  • This paper describes on the analysis of morphemes and it's synthesis being constituted of Korean word phrases. To analyze morphemes, we propose the introduction of "morph" for morpheme features in lexicon and the usage of chart data structures. it controls over the generation of unnecessary morpheme, and extracts every possible morpheme unit in a word phrase which minimized lexicon investigation by using heuristic information. Moreover, to synthesize morphemes, it is composed of every possible analyzed morphemes in word phrases to take advantage of speech and union information which can be obtained for program. Therefore, the systhesis of analyzed morphemes were designed to aid a syntactic analysis next step of natural language processing. This system for analyzing and systhesizing morpheme was to generate a word phrase by unifying syntactic and semantic features of analyzed morphemes in lexicon, and then established by C language of the personal computer.

  • PDF

Morphological analysis of spoken Korean using Viterbi search (Viterbi 검색 기법을 이용한 한국어 음성 언어의 형태소 분석)

  • 김병창
    • Proceedings of the Acoustical Society of Korea Conference
    • /
    • 1995.06a
    • /
    • pp.200-203
    • /
    • 1995
  • This paper proposes a spoken Korean processing model which is extensible to large vocabulary continuous spoken Korean system. The integration of phoneme level speech recognition with natural language processing can support a sophisticated phonological/morphological analysis. The model consists of a diphone speech recognizer, a viterbi dictionaly searcher and a morpheme connectivity information checker. Two-level hierarchical TDNNs recognize newly defined Korean diphones. The diphone sequences are segmented and converted to the most probable morpheme sequences by the Viterbi dictionary searcher. The morpheme sequency are then examined by the morpheme connectivity information checker and the correct morpheme sequence which has the greatest probability is collected. The experiments show that the morphological analysis for spoken Korean can be achieved for 328 Eojeols with 80.6% success rate.

  • PDF

Sentiment Analysis of Korean Reviews Using CNN: Focusing on Morpheme Embedding (CNN을 적용한 한국어 상품평 감성분석: 형태소 임베딩을 중심으로)

  • Park, Hyun-jung;Song, Min-chae;Shin, Kyung-shik
    • Journal of Intelligence and Information Systems
    • /
    • v.24 no.2
    • /
    • pp.59-83
    • /
    • 2018
  • With the increasing importance of sentiment analysis to grasp the needs of customers and the public, various types of deep learning models have been actively applied to English texts. In the sentiment analysis of English texts by deep learning, natural language sentences included in training and test datasets are usually converted into sequences of word vectors before being entered into the deep learning models. In this case, word vectors generally refer to vector representations of words obtained through splitting a sentence by space characters. There are several ways to derive word vectors, one of which is Word2Vec used for producing the 300 dimensional Google word vectors from about 100 billion words of Google News data. They have been widely used in the studies of sentiment analysis of reviews from various fields such as restaurants, movies, laptops, cameras, etc. Unlike English, morpheme plays an essential role in sentiment analysis and sentence structure analysis in Korean, which is a typical agglutinative language with developed postpositions and endings. A morpheme can be defined as the smallest meaningful unit of a language, and a word consists of one or more morphemes. For example, for a word '예쁘고', the morphemes are '예쁘(= adjective)' and '고(=connective ending)'. Reflecting the significance of Korean morphemes, it seems reasonable to adopt the morphemes as a basic unit in Korean sentiment analysis. Therefore, in this study, we use 'morpheme vector' as an input to a deep learning model rather than 'word vector' which is mainly used in English text. The morpheme vector refers to a vector representation for the morpheme and can be derived by applying an existent word vector derivation mechanism to the sentences divided into constituent morphemes. By the way, here come some questions as follows. What is the desirable range of POS(Part-Of-Speech) tags when deriving morpheme vectors for improving the classification accuracy of a deep learning model? Is it proper to apply a typical word vector model which primarily relies on the form of words to Korean with a high homonym ratio? Will the text preprocessing such as correcting spelling or spacing errors affect the classification accuracy, especially when drawing morpheme vectors from Korean product reviews with a lot of grammatical mistakes and variations? We seek to find empirical answers to these fundamental issues, which may be encountered first when applying various deep learning models to Korean texts. As a starting point, we summarized these issues as three central research questions as follows. First, which is better effective, to use morpheme vectors from grammatically correct texts of other domain than the analysis target, or to use morpheme vectors from considerably ungrammatical texts of the same domain, as the initial input of a deep learning model? Second, what is an appropriate morpheme vector derivation method for Korean regarding the range of POS tags, homonym, text preprocessing, minimum frequency? Third, can we get a satisfactory level of classification accuracy when applying deep learning to Korean sentiment analysis? As an approach to these research questions, we generate various types of morpheme vectors reflecting the research questions and then compare the classification accuracy through a non-static CNN(Convolutional Neural Network) model taking in the morpheme vectors. As for training and test datasets, Naver Shopping's 17,260 cosmetics product reviews are used. To derive morpheme vectors, we use data from the same domain as the target one and data from other domain; Naver shopping's about 2 million cosmetics product reviews and 520,000 Naver News data arguably corresponding to Google's News data. The six primary sets of morpheme vectors constructed in this study differ in terms of the following three criteria. First, they come from two types of data source; Naver news of high grammatical correctness and Naver shopping's cosmetics product reviews of low grammatical correctness. Second, they are distinguished in the degree of data preprocessing, namely, only splitting sentences or up to additional spelling and spacing corrections after sentence separation. Third, they vary concerning the form of input fed into a word vector model; whether the morphemes themselves are entered into a word vector model or with their POS tags attached. The morpheme vectors further vary depending on the consideration range of POS tags, the minimum frequency of morphemes included, and the random initialization range. All morpheme vectors are derived through CBOW(Continuous Bag-Of-Words) model with the context window 5 and the vector dimension 300. It seems that utilizing the same domain text even with a lower degree of grammatical correctness, performing spelling and spacing corrections as well as sentence splitting, and incorporating morphemes of any POS tags including incomprehensible category lead to the better classification accuracy. The POS tag attachment, which is devised for the high proportion of homonyms in Korean, and the minimum frequency standard for the morpheme to be included seem not to have any definite influence on the classification accuracy.

Implementation of A Morphological Analyzer Based on Pseudo-morpheme for Large Vocabulary Speech Recognizing (대어휘 음성인식을 위한 의사형태소 분석 시스템의 구현)

  • 양승원
    • Journal of Korea Society of Industrial Information Systems
    • /
    • v.4 no.2
    • /
    • pp.102-108
    • /
    • 1999
  • It is important to decide processing unit in the large vocabulary speech recognition system we propose a Pseudo-Morpheme as the recognition unit to resolve the problems in the recognition systems using the phrase or the general morpheme. We implement a morphological analysis system and tagger for Pseudo-Morpheme. The speech processing system using this pseudo-morpheme can get better result than other systems using the phrase or the general morpheme. So, the quality of the whole spoken language translation system can be improved. The analysis-ratio of our implemented system is similar to the common morphological analysis systems.

  • PDF

Performance of speech recognition unit considering morphological pronunciation variation (형태소 발음변이를 고려한 음성인식 단위의 성능)

  • Bang, Jeong-Uk;Kim, Sang-Hun;Kwon, Oh-Wook
    • Phonetics and Speech Sciences
    • /
    • v.10 no.4
    • /
    • pp.111-119
    • /
    • 2018
  • This paper proposes a method to improve speech recognition performance by extracting various pronunciations of the pseudo-morpheme unit from an eojeol unit corpus and generating a new recognition unit considering pronunciation variations. In the proposed method, we first align the pronunciation of the eojeol units and the pseudo-morpheme units, and then expand the pronunciation dictionary by extracting the new pronunciations of the pseudo-morpheme units at the pronunciation of the eojeol units. Then, we propose a new recognition unit that relies on pronunciation by tagging the obtained phoneme symbols according to the pseudo-morpheme units. The proposed units and their extended pronunciations are incorporated into the lexicon and language model of the speech recognizer. Experiments for performance evaluation are performed using the Korean speech recognizer with a trigram language model obtained by a 100 million pseudo-morpheme corpus and an acoustic model trained by a multi-genre broadcast speech data of 445 hours. The proposed method is shown to reduce the word error rate relatively by 13.8% in the news-genre evaluation data and by 4.5% in the total evaluation data.

A Comparative Study on Korean Connective Morpheme '-myenseo' to the Chinese expression - based on Korean-Chinese parallel corpus (한국어 연결어미 '-면서'와 중국어 대응표현의 대조연구 -한·중 병렬 말뭉치를 기반으로)

  • YI, CHAO
    • Cross-Cultural Studies
    • /
    • v.37
    • /
    • pp.309-334
    • /
    • 2014
  • This study is based on the Korean-Chinese parallel corpus, utilizing the Korean connective morpheme '-myenseo' and contrasting with the Chinese expression. Korean learners often struggle with the use of Korean Connective Morpheme especially when there is a lexical gap between their mother language. '-myenseo' is of the most use Korean Connective Morpheme, it usually contrast to the Chinese coordinating conjunction. But according to the corpus, the contrastive Chinese expression to '-myenseo' is more than coordinating conjunction. So through this study, can help the Chinese Korean language learners learn easier while studying '-myenseo', because the variety Chinese expression are found from the parallel corpus that related to '-myenseo'. In this study, firstly discussed the semantic features and syntactic characteristics of '-myenseo'. The significant semantic features of '-myenseo' are 'simultaneous' and 'conflict'. So in this chapter the study use examples of usage to analyse the specific usage of '-myenseo'. And then this study analyse syntactic characteristics of '-myenseo' through the subject constraint, predicate constraints, temporal constraints, mood constraints, negatives constraints. then summarize them into a table. And the most important part of this study is Chapter 4. In this chapter, it contrasted the Korean connective morpheme '-myenseo' to the Chinese expression by analysing the Korean-Chinese parallel corpus. As a result of the analysis, the frequency of the Chinese expression that contrasted to '-myenseo' is summarized into

    . It can see from the table that the most common Chinese expression comparative to '-myenseo' is non-marker patterns. That means the connection of sentence in Korean can use connective morpheme what is a clarifying linguistic marker, but in Chinese it often connect the sentence by their intrinsic logical relationships. So the conclusion of this chapter is that '-myenseo' can be comparative to Chinese conjunction, expression, non-marker patterns and liberal translation patterns, which are more than Chinese conjunction that discovered before. In the last Chapter, as the conclusion part of this study, it summarized and suggest the limitations and the future research direction.

  • Error Correction in Korean Morpheme Recovery using Deep Learning (딥 러닝을 이용한 한국어 형태소의 원형 복원 오류 수정)

    • Hwang, Hyunsun;Lee, Changki
      • Journal of KIISE
      • /
      • v.42 no.11
      • /
      • pp.1452-1458
      • /
      • 2015
    • Korean Morphological Analysis is a difficult process. Because Korean is an agglutinative language, one of the most important processes in Morphological Analysis is Morpheme Recovery. There are some methods using Heuristic rules and Pre-Analyzed Partial Words that were examined for this process. These methods have performance limits as a result of not using contextual information. In this study, we built a Korean morpheme recovery system using deep learning, and this system used word embedding for the utilization of contextual information. In '들/VV' and '듣/VV' morpheme recovery, the system showed 97.97% accuracy, a better performance than with SVM(Support Vector Machine) which showed 96.22% accuracy.

    A study on the correlation between the introduction order of English morphemes in the English textbook for the 7th graders and the natural order hypothesis (중학교 1학년 영어 교과서의 영어 형태소 도입 순위와 자연적 순서 가설과의 상관관계 연구)

    • Sohng, Hae-Sung
      • English Language & Literature Teaching
      • /
      • v.9 no.1
      • /
      • pp.131-152
      • /
      • 2003
    • The purpose of this study is to investigate the correlation between the introduction order of 9 English morphemes in the English textbook used in the middle school and the learning order of the morphemes by the 7th graders learning English as a foreign language. The subjects are 139 students in two middle schools, who learn English with different textbooks. The introduction order of each morpheme in two textbooks was examined according to its quantity and frequency. Data on the real learning order were collected through the written SLOPE test, and each morpheme was ranked by its group score. The introduction order of each morpheme in the textbook and the real learning order were analyzed by Spearman rank order correlation. It was shown that the correlation between the two was very low. This means that those textbooks do not take the learning order of English morphemes into account. Also it was shown that in the earlier stage of learning English the introduction order of each morpheme in the textbook had much influence on its learning order, but in the later stage such influence reduced gradually. This means that the learning order of English morphemes approaches the natural order as time passes by.

    • PDF

    Morpheme Conversion for korean Text-to-Sign Language Translation System (한국어-수화 번역시스템을 위한 형태소 변환)

    • Park, Su-Hyun;Kang, Seok-Hoon;Kwon, Hyuk-Chul
      • The Transactions of the Korea Information Processing Society
      • /
      • v.5 no.3
      • /
      • pp.688-702
      • /
      • 1998
    • In this paper, we propose sign language morpheme generation rule corresponding to morpheme analysis for each part of speech. Korean natural sign language has extremely limited vocabulary, and the number of grammatical components eing currently used are limited, too. In this paper, therefore, we define natural sign language grammar corresponding to Korean language grammar in order to translate natural Korean language sentences to the corresponding sign language. Each phrase should define sign language morpheme generation grammar which is different from Korean language analysis grammar. Then, this grammar is applied to morpheme analysis/combination rule and sentence structure analysis rule. It will make us generate most natural sign language by definition of this grammar.

    • PDF

    Classification of Education Video by Subtitle Analysis (자막 분석을 통한 교육 영상의 카테고리 분류 방안)

    • Lee, Ji-Hoon;Lee, Hyeon Sup;Kim, Jin-Deog
      • Proceedings of the Korean Institute of Information and Commucation Sciences Conference
      • /
      • 2021.05a
      • /
      • pp.88-90
      • /
      • 2021
    • This paper introduces a method for extracting subtitles from lecture videos through a Korean morpheme analyzer and classifying video categories according to the extracted morpheme information. In some cases incorrect information is entered due to human error and reflected in the characteristics of the items, affecting the accuracy of the recommendation system. To prevent this, we generate a keyword table for each category using morpheme information extracted from pre-classified videos, and compare the similarity of morpheme in each category keyword table to classify categories of Lecture videos using the most similar keyword table. These human intervention reduction systems directly classify videos and aim to increase the accuracy of the system.

    • PDF

    (34141) Korea Institute of Science and Technology Information, 245, Daehak-ro, Yuseong-gu, Daejeon
    Copyright (C) KISTI. All Rights Reserved.