• Title/Summary/Keyword: 단어 분리

Search Result 113, Processing Time 0.025 seconds

Online Korean Character Recognition for Intelligent Multimedia Terminal (인텔리젼트 멀티미디어 단말기를 위한 온라인 한글 인식)

  • 오준택;이우범;김욱현
    • Proceedings of the Korea Institute of Convergence Signal Processing
    • /
    • 2000.08a
    • /
    • pp.229-232
    • /
    • 2000
  • 문자인식은 멀티 모달 인터페이스의 핵심요소로서 이동 환경에서 사용자의 다양한 요구사항을 처리하는 지능형 단말기의 구현을 위해 필수적으로 개발되어야 할 과제이다. 그러나 대부분의 기존 연구는 인식률의 향상만을 위해서 복잡한 획 해석과 백트래킹을 사용하기 때문에 멀티미디어 단말기에 적합하지 못하다. 따라서 본 논문은 멀티미디어 단말기로의 적용을 목적으로 한 새로운 온라인 한글 문자 인식 방법을 제안한다. 제안된 방법은 한글 문자의 특성정보와 획 정보를 기반으로 구축된 한글 데이터 베이스를 사용한다. 또한 획간의 위치관계를 이용한 순차적 자소 분리와 향상된 백트래킹 기법에 의해서 보다 빠른 처리 시간을 보장한다. 제안된 시스템의 성능 평가는 상용 1,200 단어를 이용하여 다수의 필기자가 필기한 한글 600문자를 대상으로 실험한 결과 95% 이상의 인식률을 얻었다.

  • PDF

Korean Morpheme Restoration and Segmentation based on Transformer (트랜스포머 기반 한국어 형태소 원형복원 및 분리)

  • Hyeong Jin Shin;Jeongyeon Park;Jae Sung Lee
    • Annual Conference on Human and Language Technology
    • /
    • 2022.10a
    • /
    • pp.403-406
    • /
    • 2022
  • 최근 한국어 언어 모델이나 단어 벡터 생성 등에서는 효과적인 토큰을 만들기 위해 품사 태그 없이 형태소 열만을 사용하고 있다. 본 논문에서는 입력 문장에 대해 품사 태그열 생성없이 형태소 열만을 직접 출력하는 효율적인 모델을 제안한다. 특히, 자연어처리에서 적합한 트랜스포머를 활용하기 위해, 입력 음절과 원형 복원된 형태소 조각이 1:1로 대응되는 새로운 형태소 태깅 방법을 제안한다. 세종 품사 부착 말뭉치를 대상으로 평가해 본 결과 공개 배포되어 있는 기존 형태소 분석 모델들보다 형태소 단위 F1 기준으로 약 7%에서 14% 포인트 높은 성능을 보였다.

  • PDF

KoRIBES : A Study on the Problems of RIBES in Automatic Evaluation English-Korean Patent Machine Translation (특허 기계 번역에 대한 RIBES 한국어 자동평가 문제에 대한 고찰)

  • Jang, Hyeon-Jin;Jang, Moon-Seok;Noh, Han-Sung
    • Annual Conference on Human and Language Technology
    • /
    • 2020.10a
    • /
    • pp.543-547
    • /
    • 2020
  • 자연어 처리에서 기계번역은 가장 많이 사용되고 빠르게 발전하고 있다. 기계번역에 있어서 사람의 평가가 가장 정확하고 중요하지만 많은 시간과 비용이 발생된다. 이에 기계번역을 자동 평가하는 방법들이 많이 제안되어 사용되고 있지만, 한국어 특성을 잘 반영한 자동평가 방법은 연구되지 않고 있다. BLEU와 같은 자동평가 방법을 많이 사용하고 있지만 언어의 특성 차이로 인해 원하는 평가결과를 얻지 못하는 경우가 발생하며, 특히 특허나 논문과 같은 기술문서의 번역에서는 더 많이 발생한다. 이에 본 논문에서는 단어의 정밀도와 어순이 평가에 영향이 있는 RIBES를 가지고 특허 기계 번역에서 영어→한국어로 기계 번역된 결과물의 자동평가에 대해 사람의 평가와 유사한 결과를 얻기 위해 tokenization 과정에서 복합 형태소 분리를 통한 평가방법을 제안하고자 한다.

  • PDF

Korean Word Segmentation and Compound-noun Decomposition Using Markov Chain and Syllable N-gram (마코프 체인 밀 음절 N-그램을 이용한 한국어 띄어쓰기 및 복합명사 분리)

  • 권오욱
    • The Journal of the Acoustical Society of Korea
    • /
    • v.21 no.3
    • /
    • pp.274-284
    • /
    • 2002
  • Word segmentation errors occurring in text preprocessing often insert incorrect words into recognition vocabulary and cause poor language models for Korean large vocabulary continuous speech recognition. We propose an automatic word segmentation algorithm using Markov chains and syllable-based n-gram language models in order to correct word segmentation error in teat corpora. We assume that a sentence is generated from a Markov chain. Spaces and non-space characters are generated on self-transitions and other transitions of the Markov chain, respectively Then word segmentation of the sentence is obtained by finding the maximum likelihood path using syllable n-gram scores. In experimental results, the algorithm showed 91.58% word accuracy and 96.69% syllable accuracy for word segmentation of 254 sentence newspaper columns without any spaces. The algorithm improved the word accuracy from 91.00% to 96.27% for word segmentation correction at line breaks and yielded the decomposition accuracy of 96.22% for compound-noun decomposition.

The Analysis and Recognition of Korean Speech Signal using the Phoneme (음소에 의한 한국어 음성의 분석과 인식)

  • Kim, Yeong-Il;Lee, Geon-Gi;Lee, Mun-Su
    • The Journal of the Acoustical Society of Korea
    • /
    • v.6 no.2
    • /
    • pp.38-47
    • /
    • 1987
  • As Korean language can be phonemically classified according to the characteristic and structure of its pronunciation, Korean syllables can be divided into the phonemes such as consonant and vowel. The divided phonemes are analyzed by using the method of partial autocorrelation, and the order of partial autocorelation coefficient is 15. In analysis, it is shown that each characteristic of the same consonants, vowels, and end consonant in syllables in similar. The experiments is carried out by dividing 675 syllables into consonants, vowels, and end consonants. The recognition rate of consonants, vowels, end-consonants, and syllables are $85.0(\%)$, $90.7(\%)$, $85.5(\%)$and $72.1(\%)$ respectively. In conclusion, it is shown that Korean syllables, divided by the phonemes, are analyzed and recognized with minimum data and short processing time. Furthermore, it is shown that Korean syllables, words and sentences are recognized in the same way.

  • PDF

Application of Improved Variational Recurrent Auto-Encoder for Korean Sentence Generation (한국어 문장 생성을 위한 Variational Recurrent Auto-Encoder 개선 및 활용)

  • Hahn, Sangchul;Hong, Seokjin;Choi, Heeyoul
    • Journal of KIISE
    • /
    • v.45 no.2
    • /
    • pp.157-164
    • /
    • 2018
  • Due to the revolutionary advances in deep learning, performance of pattern recognition has increased significantly in many applications like speech recognition and image recognition, and some systems outperform human-level intelligence in specific domains. Unlike pattern recognition, in this paper, we focus on generating Korean sentences based on a few Korean sentences. We apply variational recurrent auto-encoder (VRAE) and modify the model considering some characteristics of Korean sentences. To reduce the number of words in the model, we apply a word spacing model. Also, there are many Korean sentences which have the same meaning but different word order, even without subjects or objects; therefore we change the unidirectional encoder of VRAE into a bidirectional encoder. In addition, we apply an interpolation method on the encoded vectors from the given sentences, so that we can generate new sentences which are similar to the given sentences. In experiments, we confirm that our proposed method generates better sentences which are semantically more similar to the given sentences.

Automatic Construction of Foreign Word Transliteration Dictionary from English-Korean Parallel Corpus (영-한 병렬 코퍼스로부터 외래어 표기 사전의 자동 구축)

  • Lee, Jae Sung
    • The Journal of Korean Association of Computer Education
    • /
    • v.6 no.2
    • /
    • pp.9-21
    • /
    • 2003
  • This paper proposes an automatic construction system for transliteration dictionary from English-Korean parallel corpus. The system works in 3 steps: it extracts all nouns from Korean documents as the first step, filters transliterated foreign word nouns out of them with the language identification method as the second step, and extracts the corresponding English words by using a probabilistic alignment method as the final step. Specially, the fact that there is a corresponding English word in most cases, is utilized to extract the purely transliterated part from a Koreans word phrase, which is usually used in combined forms with Korean endings(Eomi) or particles(Josa). Moreover, the direct phonetic comparison is done to the words in two different alphabet systems without converting them to the same alphabet system. The experiment showed that the performance was influenced by the first and the second preprocessing steps; the most efficient model among manually preprocessed ones showed 85.4% recall, 91.0% precision and the most efficient model among fully automated ones got 68.3% recall, 89.2% precision.

  • PDF

English Conversation System Using Artificial Intelligent of based on Virtual Reality (가상현실 기반의 인공지능 영어회화 시스템)

  • Cheon, EunYoung
    • Journal of the Korea Convergence Society
    • /
    • v.10 no.11
    • /
    • pp.55-61
    • /
    • 2019
  • In order to realize foreign language education, various existing educational media have been provided, but there are disadvantages in that the cost of the parish and the media program is high and the real-time responsiveness is poor. In this paper, we propose an artificial intelligence English conversation system based on VR and speech recognition. We used Google CardBoard VR and Google Speech API to build the system and developed artificial intelligence algorithms for providing virtual reality environment and talking. In the proposed speech recognition server system, the sentences spoken by the user can be divided into word units and compared with the data words stored in the database to provide the highest probability. Users can communicate with and respond to people in virtual reality. The function provided by the conversation is independent of the contextual conversations and themes, and the conversations with the AI assistant are implemented in real time so that the user system can be checked in real time. It is expected to contribute to the expansion of virtual education contents service related to the Fourth Industrial Revolution through the system combining the virtual reality and the voice recognition function proposed in this paper.

A minimal pair searching tool based on dictionary (사전 기반 최소대립쌍 검색 도구)

  • Kim, Tae-Hoon;Lee, Jae-Ho;Chang, Moon-Soo
    • Journal of the Korean Institute of Intelligent Systems
    • /
    • v.24 no.2
    • /
    • pp.117-122
    • /
    • 2014
  • The minimal pairs mean the pairs that have same phonotactics except just one sound in the sequences cause different lexical items. This paper proposes the searching tool of minimal pairs for efficiency of phonological researches with minimal pairs. We suggest a guide to develop Korean minimal pair searching programs by comparing to other programs. Proposing tool has user-friendly interface, minimizing key inputs, for linguistics who are not fluent in computer programs. And it serves the function which classifies the words in dictionary for the detailed researches. And for efficiency, it increases speed of dictionary loading by separating syllables through Unicode analysis, and optimizes dictionary structure for searching efficiency. The searching algorithm gains in speed by hashing algorithm using syllable counts. In our tool, the speed is improved more than earlier version about 5 times at converting dictionary and about 3 times at searching.

A Study on the Natural Language Generation by Machine Translation (영한 기계번역의 자연어 생성 연구)

  • Hong Sung-Ryong
    • Journal of Digital Contents Society
    • /
    • v.6 no.1
    • /
    • pp.89-94
    • /
    • 2005
  • In machine translation the goal of natural language generation is to produce an target sentence transmitting the meaning of source sentence by using an parsing tree of source sentence and target expressions. It provides generator with linguistic structures, word mapping, part-of-speech, lexical information. The purpose of this study is to research the Korean Characteristics which could be used for the establishment of an algorism in speech recognition and composite sound. This is a part of realization for the plan of automatic machine translation. The stage of MT is divided into the level of morphemic, semantic analysis and syntactic construction.

  • PDF