• Title/Summary/Keyword: 어휘학습

Search Result 361, Processing Time 0.025 seconds

Corpus-Based Ambiguity-Driven Learning of Context- Dependent Lexical Rules for Part-of-Speech Tagging (품사태킹을 위한 어휘문맥 의존규칙의 말뭉치기반 중의성주도 학습)

  • 이상주;류원호;김진동;임해창
    • Journal of KIISE:Software and Applications
    • /
    • v.26 no.1
    • /
    • pp.178-178
    • /
    • 1999
  • Most stochastic taggers can not resolve some morphological ambiguities that can be resolved only by referring to lexical contexts because they use only contextual probabilities based ontag n-grams and lexical probabilities. Existing lexical rules are effective for resolving such ambiguitiesbecause they can refer to lexical contexts. However, they have two limitations. One is that humanexperts tend to make erroneous rules because they are deterministic rules. Another is that it is hardand time-consuming to acquire rules because they should be manually acquired. In this paper, wepropose context-dependent lexical rules, which are lexical rules based on the statistics of a taggedcorpus, and an ambiguity-driven teaming method, which is the method of automatically acquiring theproposed rules from a tagged corpus. By using the proposed rules, the proposed tagger can partiallyannotate an unseen corpus with high accuracy because it is a kind of memorizing tagger that canannotate a training corpus with 100% accuracy. So, the proposed tagger is useful to improve theaccuracy of a stochastic tagger. And also, it is effectively used for detecting and correcting taggingerrors in a manually tagged corpus. Moreover, the experimental results show that the proposed methodis also effective for English part-of-speech tagging.

Improve Performance of Phrase-based Statistical Machine Translation through Standardizing Korean Allomorph (한국어의 이형태 표준화를 통한 구 기반 통계적 기계 번역 성능 향상)

  • Lee, Won-Kee;Kim, Young-Gil;Lee, Eui-Hyun;Kwon, Hong-Seok;Jo, Seung-U;Cho, Hyung-Mi;Lee, Jong-Hyeok
    • Annual Conference on Human and Language Technology
    • /
    • 2016.10a
    • /
    • pp.285-290
    • /
    • 2016
  • 한국어는 형태론적으로 굴절어에 속하는 언어로서, 어휘의 형태가 문장 속에서 문법적인 기능을 하게 되고, 형태론적으로 풍부한 언어라는 특징 때문에 조사나 어미와 같은 기능어들이 다양하게 내용어들과 결합한다. 이와 같은 특징들은 한국어를 대상으로 하는 구 기반 통계적 기계번역 시스템에서 데이터 부족 문제(Data Sparseness problem)를 더욱 크게 부각시킨다. 하지만, 한국어의 몇몇 조사와 어미는 함께 결합되는 내용어에 따라 의미는 같지만 두 가지의 형태를 가지는 이형태로 존재한다. 따라서 본 논문에서 이러한 이형태들을 하나로 표준화하여 데이터부족 문제를 완화하고, 베트남-한국어 통계적 기계 번역에서 성능이 개선됨을 보였다.

  • PDF

A Study on the Artificial Neural Networks for the Sentence-level Prosody Generation (문장단위 운율발생용 인공신경망에 관한 연구)

  • 신동엽;민경중;강찬구;임운천
    • Proceedings of the Acoustical Society of Korea Conference
    • /
    • autumn
    • /
    • pp.53-56
    • /
    • 2000
  • 무제한 어휘 음성합성 시스템의 문-음성 합성기는 합성음의 자연감을 높이기 위해 여러 가지 방법을 사용하게되는데 그중 하나가 자연음에 내재하는 운을 법칙을 정확히 구현하는 것이다. 합성에 필요한 운율법칙은 언어학적 정보를 이용해 구현하거나, 자연음을 분석해 구한 운을 정보로부터 운율 법칙을 추출하여 합성에 이용하고 있다. 이와 같이 구한 운을 법칙이 자연음에 존재하는 운율 법칙을 전부 반영하지 못했거나, 잘못 구현되는 경우에는 합성음의 자연성이 떨어지게 된다. 이런 점을 고려하여 우리는 자연음의 운율 정보를 이용해 인공 신경망을 훈련시켜, 문장단위 운율을 발생시킬 수 있는 방식을 제안하였다. 운율의 세 가지 요소는 피치, 지속시간, 크기 변화가 있는데, 인공 신경망은 문장이 입력되면, 각 해당 음소의 지속시간에 따른 피치 변화와 크기 변화를 학습할 수 있도록 설계하였다. 신경망을 훈련시키기 위해 고립 단어 군과 음소균형 문장 군을 화자로 하여금 발성하게 하여, 녹음하고, 분석하여 구한 운을 정보를 데이터베이스로 구축하였다. 문장 내의 각 음소에 대해 지속시간과 피치 변화 그리고 크기 변화를 구하고, 곡선적응 방법을 이용하여 각 변화 곡선에 대한 다항식 계수와 초기치를 구해 운을 데이터베이스를 구축한다. 이 운을 데이터베이스의 일부를 인공 신경망을 훈련시키는데 이용하고, 나머지를 이용해 인공 신경망의 성능을 평가한 결과 운을 데이터베이스를 계속 확장하면 좀더 자연스러운 운율을 발생시킬 수 있음을 관찰하였다.

  • PDF

In Out-of Vocabulary Rejection Algorithm by Measure of Normalized improvement using Optimization of Gaussian Model Confidence (미등록어 거절 알고리즘에서 가우시안 모델 최적화를 이용한 신뢰도 정규화 향상)

  • Ahn, Chan-Shik;Oh, Sang-Yeob
    • Journal of the Korea Society of Computer and Information
    • /
    • v.15 no.12
    • /
    • pp.125-132
    • /
    • 2010
  • In vocabulary recognition has unseen tri-phone appeared when recognition training. This system has not been created beginning estimation figure of model parameter. It's bad points could not be created that model for phoneme data. Therefore it's could not be secured accuracy of Gaussian model. To improve suggested Gaussian model to optimized method of model parameter using probability distribution. To improved of confidence that Gaussian model to optimized of probability distribution to offer by accuracy and to support searching of phoneme data. This paper suggested system performance comparison as a result of recognition improve represent 1.7% by out-of vocabulary rejection algorithm using normalization confidence.

Spam Filter by Using X2 Statistics and Support Vector Machines (카이제곱 통계량과 지지벡터기계를 이용한 스팸메일 필터)

  • Lee, Song-Wook
    • The KIPS Transactions:PartB
    • /
    • v.17B no.3
    • /
    • pp.249-254
    • /
    • 2010
  • We propose an automatic spam filter for e-mail data using Support Vector Machines(SVM). We use a lexical form of a word and its part of speech(POS) tags as features and select features by chi square statistics. We represent each feature by TF(text frequency), TF-IDF, and binary weight for experiments. After training SVM with the selected features, SVM classifies each e-mail as spam or not. In experiment, the selected features improve the performance of our system and we acquired overall 98.9% of accuracy with TREC05-p1 spam corpus.

SRLev-BIH: An Evaluation Metric for Korean Generative Commonsense Reasoning (SRLev-BIH: 한국어 일반 상식 추론 및 생성 능력 평가 지표)

  • Jaehyung Seo;Yoonna Jang;Jaewook Lee;Hyeonseok Moon;Sugyeong Eo;Chanjun Park;Aram So;Heuiseok Lim
    • Annual Conference on Human and Language Technology
    • /
    • 2022.10a
    • /
    • pp.176-181
    • /
    • 2022
  • 일반 상식 추론 능력은 가장 사람다운 능력 중 하나로써, 인공지능 모델이 쉽게 모사하기 어려운 영역이다. 딥러닝 기반의 언어 모델은 여전히 일반 상식에 기반한 추론을 필요로 하는 분야에서 부족한 성능을 보인다. 특히, 한국어에서는 일반 상식 추론과 관련한 연구가 상당히 부족한 상황이다. 이러한 문제 완화를 위해 최근 생성 기반의 일반 상식 추론을 위한 한국어 데이터셋인 Korean CommonGen [1]이 발표되었다. 그러나, 해당 데이터셋의 평가 지표는 어휘 단계의 유사성과 중첩에 의존하는 한계를 지니며, 생성한 문장이 일반 상식에 부합한 문장인지 측정하기 어렵다. 따라서 본 논문은 한국어 일반 상식 추론 및 생성 능력에 대한 평가 지표를 개선하기 위해 문장 성분의 의미역과 자모의 형태 변화를 바탕으로 생성 결과를 평가하는 SRLev, 사람의 평가 결과를 학습한 BIH, 그리고 두 평가 지표의 장점을 결합한 SRLev-BIH를 제안한다.

  • PDF

Construction of Evaluation-Annotated Datasets for EA-based Clothing Recommendation Chatbots (패션앱 후기글 평가분석에 기반한 의류 검색추천 챗봇 개발을 위한 학습데이터 EVAD 구축)

  • Choi, Su-Won;Hwang, Chang-Hoe;Yoo, Gwang-Hoon;Nam, Jee-Sun
    • Annual Conference on Human and Language Technology
    • /
    • 2021.10a
    • /
    • pp.467-472
    • /
    • 2021
  • 본 연구는 패션앱 후기글에 나타나는 구매자의 의견에 대한 '평가분석(Evaluation Analysis: EA)'을 수행하여, 이를 기반으로 상품의 검색 및 추천을 수행하는 의류 검색추천 챗봇을 개발하는 LICO 프로젝트의 언어데이터 구축의 일환으로 수행되었다. '평가분석 트리플(EAT)'과 '평가기반요청 쿼드러플(EARQ)'의 구성요소들에 대한 주석작업은, 도메인 특화된 단일형 핵심어휘와 다단어(MWE) 핵심패턴들을 FST 방식으로 구조화하는 DECO-LGG 언어자원에 기반하여 반자동 언어데이터 증강(SSP) 방식을 통해 진행되었다. 이 과정을 통해 20여만 건의 후기글 문서(230만 어절)로 구성된 EVAD 평가주석데이터셋이 생성되었다. 여성의류 도메인의 평가분석을 위한 '평가속성(ASPECT)' 성분으로 14가지 유형이 분류되었고, 각 '평가속성'에 연동된 '평가내용(VALUE)' 쌍으로 전체 35가지의 {ASPECT-VALUE} 카테고리가 분류되었다. 본 연구에서 구축된 EVAD 평가주석 데이터의 성능을 평가한 결과, F1-Score 0.91의 성능 평가를 획득하였으며, 이를 통해 향후 다른 도메인으로의 확장된 적용 가능성이 유효함을 확인하였다.

  • PDF

Automatic Target Recognition Study using Knowledge Graph and Deep Learning Models for Text and Image data (지식 그래프와 딥러닝 모델 기반 텍스트와 이미지 데이터를 활용한 자동 표적 인식 방법 연구)

  • Kim, Jongmo;Lee, Jeongbin;Jeon, Hocheol;Sohn, Mye
    • Journal of Internet Computing and Services
    • /
    • v.23 no.5
    • /
    • pp.145-154
    • /
    • 2022
  • Automatic Target Recognition (ATR) technology is emerging as a core technology of Future Combat Systems (FCS). Conventional ATR is performed based on IMINT (image information) collected from the SAR sensor, and various image-based deep learning models are used. However, with the development of IT and sensing technology, even though data/information related to ATR is expanding to HUMINT (human information) and SIGINT (signal information), ATR still contains image oriented IMINT data only is being used. In complex and diversified battlefield situations, it is difficult to guarantee high-level ATR accuracy and generalization performance with image data alone. Therefore, we propose a knowledge graph-based ATR method that can utilize image and text data simultaneously in this paper. The main idea of the knowledge graph and deep model-based ATR method is to convert the ATR image and text into graphs according to the characteristics of each data, align it to the knowledge graph, and connect the heterogeneous ATR data through the knowledge graph. In order to convert the ATR image into a graph, an object-tag graph consisting of object tags as nodes is generated from the image by using the pre-trained image object recognition model and the vocabulary of the knowledge graph. On the other hand, the ATR text uses the pre-trained language model, TF-IDF, co-occurrence word graph, and the vocabulary of knowledge graph to generate a word graph composed of nodes with key vocabulary for the ATR. The generated two types of graphs are connected to the knowledge graph using the entity alignment model for improvement of the ATR performance from images and texts. To prove the superiority of the proposed method, 227 documents from web documents and 61,714 RDF triples from dbpedia were collected, and comparison experiments were performed on precision, recall, and f1-score in a perspective of the entity alignment..

The Effect of Task-Oriented Multi-Sensory Movement Program on Self-efficacy and Writing Ability of Children with ADHD Tendency Accompanied by Learning Delays (과제 중심 다감각운동 프로그램이 학습지연을 동반한 ADHD성향 아동의 자아효능감과 쓰기능력에 미치는 변화)

  • Roh, Heo-Lyun;Kwag, Sung-Won
    • The Journal of Korean society of community based occupational therapy
    • /
    • v.8 no.2
    • /
    • pp.1-14
    • /
    • 2018
  • Objective : The purpose of this study was to investigate the change in self-efficacy and writing ability after applying a Task-Oriented Multi-Sensory Movement Program to children with ADHD tendency accompanied by learning delays. Methods : A Task-Oriented Multi-Sensory Movement Program was implemented to children with ADHD tendency accompanied by learning delays attending S elementary school. The research proceeded in the order of a pre-test, Task-Oriented Multi-Sensory Movement intervention, and a post-test. The first session involved a pre-test, in which the children's self-efficacy and writing ability were examined using self-efficacy test and type 'A' KNISE-BAAT writing test. The multisensory group activity program intervention was conducted for a total of 8 sessions. In the last session, a post-test was conducted using self-efficacy test and type 'B' KNISE-BAAT writing test. Data collected from the tests were analyzed using SPSS Statistics 18. Results : According to the tests taken before and after implementing the Task-Oriented Multi-Sensory Movement Program, there was a significant improvement in self-efficacy (school, society), writing ability(command of vocabulary and sentence). Conclusion : Task-Oriented Multi-Sensory Movement Program may be used as a beneficial measure to improve the self-efficacy and writing abilities of children with ADHD tendency accompanied by learning delays. It is necessary to design various intervention models by combining educational media based on a multisensory approach.

Building and Analyzing Panic Disorder Social Media Corpus for Automatic Deep Learning Classification Model (딥러닝 자동 분류 모델을 위한 공황장애 소셜미디어 코퍼스 구축 및 분석)

  • Lee, Soobin;Kim, Seongdeok;Lee, Juhee;Ko, Youngsoo;Song, Min
    • Journal of the Korean Society for information Management
    • /
    • v.38 no.2
    • /
    • pp.153-172
    • /
    • 2021
  • This study is to create a deep learning based classification model to examine the characteristics of panic disorder and to classify the panic disorder tendency literature by the panic disorder corpus constructed for the present study. For this purpose, 5,884 documents of the panic disorder corpus collected from social media were directly annotated based on the mental disease diagnosis manual and were classified into panic disorder-prone and non-panic-disorder documents. Then, TF-IDF scores were calculated and word co-occurrence analysis was performed to analyze the lexical characteristics of the corpus. In addition, the co-occurrence between the symptom frequency measurement and the annotated symptom was calculated to analyze the characteristics of panic disorder symptoms and the relationship between symptoms. We also conducted the performance evaluation for a deep learning based classification model. Three pre-trained models, BERT multi-lingual, KoBERT, and KcBERT, were adopted for classification model, and KcBERT showed the best performance among them. This study demonstrated that it can help early diagnosis and treatment of people suffering from related symptoms by examining the characteristics of panic disorder and expand the field of mental illness research to social media.