• ETRI Journal
• /
• v.39 no.4
• /
• pp.455-466
• /
• 2017

This study focuses on a method for sequential data augmentation in order to alleviate data sparseness problems. Specifically, we present corpus expansion techniques for enhancing the coverage of a language model. Recent recurrent neural network studies show that a seq2seq model can be applied for addressing language generation issues; it has the ability to generate new sentences from given input sentences. We present a method of corpus expansion using a sentence-chain based seq2seq model. For training the seq2seq model, sentence chains are used as triples. The first two sentences in a triple are used for the encoder of the seq2seq model, while the last sentence becomes a target sequence for the decoder. Using only internal resources, evaluation results show an improvement of approximately 7.6% relative perplexity over a baseline language model of Korean text. Additionally, from a comparison with a previous study, the sentence chain approach reduces the size of the training data by 38.4% while generating 1.4-times the number of n-grams with superior performance for English text.

• KIPS Transactions on Software and Data Engineering
• /
• v.7 no.4
• /
• pp.121-128
• /
• 2018

A Korean morphological analyzer adopts sequence-to-sequence (seq2seq) model, which can generate an output sequence of different length from an input. In general, a seq2seq based Korean morphological analyzer takes a syllable-unit based sequence as an input, and output a syllable-unit based sequence. Syllable-based morphological analysis has the advantage that unknown words can be easily handled, but has the disadvantages that morpheme-based information is ignored. In this paper, we propose a reranking model as a post-processor of seq2seq model that can improve the accuracy of morphological analysis. The seq2seq based morphological analyzer can generate K results by using a beam-search method. The reranking model exploits morpheme-unit embedding information as well as n-gram of morphemes in order to reorder K results. The experimental results show that the reranking model can improve 1.17% F1 score comparing with the original seq2seq model.

• 한국어정보학회:학술대회논문집
• /
• 2016.10a
• /
• pp.217-219
• /
• 2016

본 논문은 형태소 분석 및 품사 태깅을 위해 seq2seq 주의집중 모델을 이용하는 접근 방법에 대하여 기술한다. seq2seq 모델은 인코더와 디코더로 분할되어 있고, 일반적으로 RNN(recurrent neural network)를 기반으로 한다. 형태소 분석 및 품사 태깅을 위해 seq2seq 모델의 학습 단계에서 음절 시퀀스는 인코더의 입력으로, 각 음절에 해당하는 품사 태깅 시퀀스는 디코더의 출력으로 사용된다. 여기서 음절 시퀀스와 품사 태깅 시퀀스의 대응관계는 주의집중(attention) 모델을 통해 접근하게 된다. 본 연구는 사전 정보나 자질 정보와 같은 추가적 리소스를 배제한 end-to-end 접근 방법의 실험 결과를 제시한다. 또한, 디코딩 단계에서 빔(beam) 서치와 같은 추가적 프로세스를 배제하는 접근 방법을 취한다.

• Annual Conference on Human and Language Technology
• /
• 2016.10a
• /
• pp.217-219
• /
• 2016

본 논문은 형태소 분석 및 품사 태깅을 위해 seq2seq 주의집중 모델을 이용하는 접근 방법에 대하여 기술한다. seq2seq 모델은 인코더와 디코더로 분할되어 있고, 일반적으로 RNN(recurrent neural network)를 기반으로 한다. 형태소 분석 및 품사 태깅을 위해 seq2seq 모델의 학습 단계에서 음절 시퀀스는 인코더의 입력으로, 각 음절에 해당하는 품사 태깅 시퀀스는 디코더의 출력으로 사용된다. 여기서 음절 시퀀스와 품사 태깅 시퀀스의 대응관계는 주의집중(attention) 모델을 통해 접근하게 된다. 본 연구는 사전 정보나 자질 정보와 같은 추가적 리소스를 배제한 end-to-end 접근 방법의 실험 결과를 제시한다. 또한, 디코딩 단계에서 빔(beam) 서치와 같은 추가적 프로세스를 배제하는 접근 방법을 취한다.

• KIPS Transactions on Software and Data Engineering
• /
• v.6 no.5
• /
• pp.267-278
• /
• 2017

As the same letter can be pronounced differently depending on word contexts, one should refer to a lexicon in order to pronounce a word correctly. Phonetic alphabets that lexicons adopt as well as pronunciations that lexicons describe for the same word can be different from lexicon to lexicon. In this paper, we use a sequence-to-sequence model that is widely used in deep learning research area in order to convert automatically from one pronunciation to another. The 12 seq2seq models are implemented based on pronunciation training data collected from 4 different lexicons. The exact accuracy of the models ranges from 74.5% to 89.6%. The aim of this study is the following two things. One is to comprehend a property of phonetic alphabets and pronunciations used in various lexicons. The other is to understand characteristics of seq2seq models by analyzing an error.

• The Journal of Korea Institute of Information, Electronics, and Communication Technology
• /
• v.12 no.3
• /
• pp.242-250
• /
• 2019

In this paper, we propose a method to predict the failure of industrial robot using Seq2Seq (Sequence to Sequence) model, which is a model for transforming time series data among Artificial Neural Network models. The proposed method uses the data of the joint current and angular value, which can be measured by the robot itself, without additional sensor for fault diagnosis. After preprocessing the measured data for the model to learn, the Seq2Seq model was trained to convert the current to angle. Abnormal degree for fault diagnosis uses RMSE (Root Mean Squared Error) during unit time between predicted angle and actual angle. The performance evaluation of the proposed method was performed using the test data measured under different conditions of normal and defective condition of the robot. When the Abnormal degree exceed the threshold, it was classified as a fault, and the accuracy of the fault diagnosis was 96.67% from the experiment. The proposed method has the merit that it can perform fault prediction without additional sensor, and it has been confirmed from the experiment that high diagnostic performance and efficiency are available without requiring deep expert knowledge of the robot.

• KSII Transactions on Internet and Information Systems (TIIS)
• /
• v.14 no.5
• /
• pp.1909-1928
• /
• 2020

Predicting human mobility has always been an important task in Location-based Social Network. Previous efforts fail to capture spatial dependence effectively, mainly reflected in weakening the location topology information. In this paper, we propose a neural network-based method which can capture spatial-temporal dependence to predict the next location of a person. Specifically, we involve a graph convolutional network (GCN) based on a seq2seq framework to capture the location topology information and temporal dependence, respectively. The encoder of the seq2seq framework first generates the hidden state and cell state of the historical trajectories. The GCN is then used to generate graph embeddings of the location topology graph. Finally, we predict future trajectories by aggregated temporal dependence and graph embeddings in the decoder. For evaluation, we leverage two real-world datasets, Foursquare and Gowalla. The experimental results demonstrate that our model has a better performance than the compared models.

• Proceedings of the Korean Society of Broadcast Engineers Conference
• /
• 2020.11a
• /
• pp.325-327
• /
• 2020

최근 비대면 원격강의의 비율이 증가하였지만 비대면 상황에서 원격으로 진행하는 강의로 인해 강의를 수강하는 학생들의 강의를 진행하는 교수와의 질문에 대한 즉각적인 상호작용과 피드백이 부족하고 교수 또한 비대면 상황에서 학생들과의 소통의 어려움으로 인해 질문에 대한 답변을 하는 것에 어려움 있다. 본 논문에서는 이러한 문제를 해결하기 위해 학생들에게 질문에 대한 즉각적인 답변을 해주고 교수에게는 질문-답변을 관리할 수 있는 인공지능 챗봇 웹 서비스를 제안한다. 웹 서비스는 강의를 수강하는 학생과 강의를 진행하는 교수로 나눠져 제공된다. 구현을 위해 Seq2Seq 모델을 활용하였고 질문-답변 데이터셋으로 학습을 하여 테스트 하였다.

• Journal of The Korean Association of Information Education
• /
• v.26 no.3
• /
• pp.197-207
• /
• 2022

In this paper, we develop a machine learning based block code generation and recommendation model for the purpose of reducing cognitive load of learners during coding education that learns the learner's block that has been made in the block programming environment using natural processing model and fine-tuning and then generates and recommends the selectable blocks for the next step. To develop the model, the training dataset was produced by pre-processing 50 block codes that were on the popular block programming language web site 'Entry'. Also, after dividing the pre-processed blocks into training dataset, verification dataset and test dataset, we developed a model that generates block codes based on LSTM, Seq2Seq, and GPT-2 model. In the results of the performance evaluation of the developed model, GPT-2 showed a higher performance than the LSTM and Seq2Seq model in the BLEU and ROUGE scores which measure sentence similarity. The data results generated through the GPT-2 model, show that the performance was relatively similar in the BLEU and ROUGE scores except for the case where the number of blocks was 1 or 17.

• Asian-Australasian Journal of Animal Sciences
• /
• v.31 no.6
• /
• pp.791-797
• /
• 2018

Objective: Trimethylation of histone 3 (H3) at 4th lysine N-termini (H3K4me3) in gene promoter region was the universal marker of active genes specific to cell lineage. On the contrary, coexistence of trimethylation at 27th lysine (H3K27me3) in the same loci-the bivalent H3K4m3/H3K27me3 was known to suspend the gene transcription in germ cells, and could also be inherited to the developed stem cell. In galline species, throughout example of H3K4m3 and H3K27me3 ChIP-seq analysis was still not provided. We therefore designed and demonstrated such procedures using ChIP-seq and mRNA-seq data of chicken follicular mesenchymal cells and male germ cells. Methods: Analytical workflow was designed and provided in this study. ChIP-seq and RNA-seq datasets of follicular mesenchymal cells and male germ cells were acquired and properly preprocessed. Peak calling by Model-based analysis of ChIP-seq 2 was performed to identify H3K4m3 or H3K27me3 enriched regions ($Fold-change{\geq}2$, $FDR{\leq}0.01$) in gene promoter regions. Integrative genomics viewer was utilized for cellular retinoic acid binding protein 1 (CRABP1), growth differentiation factor 10 (GDF10), and gremlin 1 (GREM1) gene explorations. Results: The acquired results indicated that follicular mesenchymal cells and germ cells shared several unique gene promoter regions enriched with H3K4me3 (5,704 peaks) and also unique regions of bivalent H3K4m3/H3K27me3 shared between all cell types and germ cells (1,909 peaks). Subsequent observation of follicular mesenchyme-specific genes-CRABP1, GDF10, and GREM1 correctly revealed vigorous transcriptions of these genes in follicular mesenchymal cells. As expected, bivalent H3K4m3/H3K27me3 pattern was manifested in gene promoter regions of germ cells, and thus suspended their transcriptions. Conclusion: According the results, an example of chicken H3K4m3/H3K27me3 ChIP-seq data analysis was successfully demonstrated in this study. Hopefully, the provided methodology should hereby be useful for galline ChIP-seq data analysis in the future.