• Annual Conference on Human and Language Technology
• /
• 2022.10a
• /
• pp.295-299
• /
• 2022

자원이 부족한 언어 환경에서 사전학습 언어모델 학습을 위한 대용량의 코퍼스를 구축하는데는 한계가 존재한다. 본 논문은 이러한 한계를 극복할 수 있는 Cross-lingual Post-Training (XPT) 방법론을 적용하여 비교적 자원이 부족한 한국어에서 해당 방법론의 효율성을 분석한다. 적은 양의 한국어 코퍼스인 400K와 4M만을 사용하여 다양한 한국어 사전학습 모델 (KLUE-BERT, KLUE-RoBERTa, Albert-kor)과 mBERT와 전반적인 성능 비교 및 분석 연구를 진행한다. 한국어의 대표적인 벤치마크 데이터셋인 KLUE 벤치마크를 사용하여 한국어 하위태스크에 대한 성능평가를 진행하며, 총 7가지의 태스크 중에서 5가지의 태스크에서 XPT-4M 모델이 기존 한국어 언어모델과의 비교에서 가장 우수한 혹은 두번째로 우수한 성능을 보인다. 이를 통해 XPT가 훨씬 더 많은 데이터로 훈련된 한국어 언어모델과 유사한 성능을 보일 뿐 아니라 학습과정이 매우 효율적임을 보인다.

• Phonetics and Speech Sciences
• /
• v.13 no.1
• /
• pp.45-51
• /
• 2021

End-to-end (E2E) automatic speech recognition (ASR) has achieved promising performance gains with the introduced self-attention network, Transformer. However, due to training time and the number of hyperparameters, finding the optimal hyperparameter set is computationally expensive. This paper investigates the impact of hyperparameters in the Transformer network to answer two questions: which hyperparameter plays a critical role in the task performance and training speed. The Transformer network for training has two encoder and decoder networks combined with Connectionist Temporal Classification (CTC). We have trained the model with Wall Street Journal (WSJ) SI-284 and tested on devl93 and eval92. Seventeen hyperparameters were selected from the ESPnet training configuration, and varying ranges of values were used for experiments. The result shows that "num blocks" and "linear units" hyperparameters in the encoder and decoder networks reduce Word Error Rate (WER) significantly. However, performance gain is more prominent when they are altered in the encoder network. Training duration also linearly increased as "num blocks" and "linear units" hyperparameters' values grow. Based on the experimental results, we collected the optimal values from each hyperparameter and reduced the WER up to 2.9/1.9 from dev93 and eval93 respectively.

• English Language & Literature Teaching
• /
• v.16 no.1
• /
• pp.123-142
• /
• 2009

The present study used multi-faceted Rasch measurement to explore the characteristics and bias patterns of non-native raters when they scored L2 writing tasks. Three raters scored 254 writing tasks written by Korean university students on two topics adapted from the TOEFL Test of Written English (TWE). The written products were assessed using a five-category rating scale (Content, Organization, Language in Use, Grammar, and Mechanics). The raters only showed a difference in severity with regard to rating categories but not in task types. Overall, the raters scored Grammar most harshly and Organization most leniently. The results also indicated several bias patterns of ratings with regard to the rating categories and task types. In rater-task bias interactions, each rater showed recurring bias patterns in their rating between two writing tasks. Analysis of rater-category bias interaction showed that the three raters revealed biased patterns across all the rating categories though they were relatively consistent in their rating. The study has implications for the importance of rater training and task selection in L2 writing assessment.

• ETRI Journal
• /
• v.39 no.4
• /
• pp.455-466
• /
• 2017

This study focuses on a method for sequential data augmentation in order to alleviate data sparseness problems. Specifically, we present corpus expansion techniques for enhancing the coverage of a language model. Recent recurrent neural network studies show that a seq2seq model can be applied for addressing language generation issues; it has the ability to generate new sentences from given input sentences. We present a method of corpus expansion using a sentence-chain based seq2seq model. For training the seq2seq model, sentence chains are used as triples. The first two sentences in a triple are used for the encoder of the seq2seq model, while the last sentence becomes a target sequence for the decoder. Using only internal resources, evaluation results show an improvement of approximately 7.6% relative perplexity over a baseline language model of Korean text. Additionally, from a comparison with a previous study, the sentence chain approach reduces the size of the training data by 38.4% while generating 1.4-times the number of n-grams with superior performance for English text.

• ETRI Journal
• /
• v.38 no.3
• /
• pp.487-493
• /
• 2016

Two new methods are proposed for an unsupervised adaptation of a language model (LM) with a single sentence for automatic transcription tasks. At the training phase, training documents are clustered by a method known as Latent Dirichlet allocation (LDA), and then a domain-specific LM is trained for each cluster. At the test phase, an adapted LM is presented as a linear mixture of the now trained domain-specific LMs. Unlike previous adaptation methods, the proposed methods fully utilize a trained LDA model for the estimation of weight values, which are then to be assigned to the now trained domain-specific LMs; therefore, the clustering and weight-estimation algorithms of the trained LDA model are reliable. For the continuous speech recognition benchmark tests, the proposed methods outperform other unsupervised LM adaptation methods based on latent semantic analysis, non-negative matrix factorization, and LDA with n-gram counting.

• Annual Conference on Human and Language Technology
• /
• 2019.10a
• /
• pp.165-169
• /
• 2019

자연어 이해 모델은 대화 시스템의 핵심적인 구성 요소로서 자연어 문장에 대해 그 의도와 정보를 파악하여 의도(intent)와 슬롯(slot)의 형태로 분석하는 모델이다. 최근 연구에서 의도와 슬롯의 추정을 단일 합동 모델(joint model)을 이용하여 합동 학습(joint training)을 하는 연구들이 진행되고 있다. 합동 모델을 이용한 합동 학습은 의도와 슬롯의 추정 정보가 모델 내에서 암시적으로 교류 되도록 하여 의도와 슬롯 추정 성능이 향상된다. 본 논문에서는 기존 합동 모델이 암시적으로 추정 정보를 교류하는 데서 더 나아가 모델 내의 의도와 슬롯 추정 정보를 명시적으로 교류하도록 모델링하여 의도와 슬롯 추정 성능을 높일 수 있는 교차 게이트 메커니즘(Cross Gated Mechanism)을 제안한다.

• The Journal of Korean Association of Computer Education
• /
• v.20 no.5
• /
• pp.35-47
• /
• 2017

The purpose of this study was to develop and apply the elementary teacher training course for educational programming language based on TPACK in order to make elementary school teachers fully equipped with teaching efficacy for SW education. As a result, the informatics teaching efficacy of the teachers in the experimental group who participated in EPL training course developed based on TPACK was statistically more significant than the teachers in the control group(t=4.13, p<.001). The dependent sample t-test of the experimental group showed a statistically significant increase with t=4.57 (p< .001). It proved that TPACK-based teachers' training course is effective to improve teachers' informatics teaching efficacy. It is suggested that the development of SW education teacher training course should be systematically structured considering TPACK framework.

• Journal of the Korea Convergence Society
• /
• v.12 no.11
• /
• pp.457-466
• /
• 2021

This study sought to identify the effectiveness of training programs by developing mobile app-based training programs to enhance memory, attention, and language function, which is known to be vulnerable to menopause women. It was conducted on 40 Climacteric woman between 40 to 60 years complaining about cognitive function decline. The mobile app-based cognitive training was an 8 week program. There were a total of 24 sessions and each session took 20-30 minutes, three times per week. The survey was carried out including a baseline study pre and post intervention study. The research variables were objective cognitive function (overall cognitive function, memory, attention, and language function), subjective cognitive function and quality of life. The cognitive training program showed a significant increase in overall cognitive function(t=-8.688, p<.001), memory(t=-4.765, p<.001), attention : number of correct answers(t=-7.293, p<.001), language high frequency response speed(Z=-2.179, p=.036), language low frequency response speed(Z=-2.737, p=.009) and quality of life (t=-3.358, p=.002). However, there was no significant difference in the scores for subjective cognitive function. The cognitive training program was found to be an effective intervention for improving the cognitive function of Climacteric women. It could be used as a cognitive intervention tool that is accessible at home without expert help.

• Proceedings of KOSOMES biannual meeting
• /
• 2003.05a
• /
• pp.77-87
• /
• 2003

This paper describes recently developed PC based Aids to Navigation Training Simulator (AtoN-TS) using Virtual Reality Modeling language (VRML). The purpose of AtoN-TS is to train entry-level cadets to reduce the amount of sea-time training. The practical application procedure of VR technology to implement AtoN-TS is represented. The construction method of virtual waterway world, according to the guidelines of International Association of Lighthouse Authorities (IALA) is proposed. Design concepts and simulation experiments are also discussed. Results from trial tests and evaluations by subject assessment, provide practical insight on the importance of AtoN-TS.

• Phonetics and Speech Sciences
• /
• v.1 no.1
• /
• pp.77-86
• /
• 2009

This study observes how the combined intervention program effects on the acquisition reading pronunciation of Korean phonological process words and the acquisition aspects of each phonological process rules to four Korean learners whose first language is Chinese. The training program is the combination of multisensory Auditory, Visual and Kinethetic (AVK) approach, wholistic approach, and metalinguistic approach. The training purpose is to evaluate how accurately they read the words of the phonological process which have fortisization, nasalization, lateralization, intermediate sound /ㅅ/ (/${\int}iot"$/). We access how they read the untrained words which include the four factors above. The intervention effects are analyzed by the multiple probe across subjects design. The results indicate that the combined phonological process rule explanation and the words activity intervention affects the four Chinese subjects in every type of word. The implications of the study are these: First, it suggests the effect of Korean pronunciation intervention in a concrete way. Second, it offers how to evaluate the phonological process and how to train people who are learning Korean language.