• Smart Media Journal
• /
• v.10 no.1
• /
• pp.16-24
• /
• 2021

The exiting word embedding methodology such as word2vec represents words, which only occur in the raw training corpus, as a fixed-length vector into a continuous vector space, so when mapping the words incorporated in the raw training corpus into a fixed-length vector in morphologically rich language, out-of-vocabulary (OOV) problem often happens. Even for sentence embedding, when representing the meaning of a sentence as a fixed-length vector by synthesizing word vectors constituting a sentence, OOV words make it challenging to meaningfully represent a sentence into a fixed-length vector. In particular, since the agglutinative language, the Korean has a morphological characteristic to integrate lexical morpheme and grammatical morpheme, handling OOV words is an important factor in improving performance. In this paper, we propose parallel Tri-LSTM sentence embedding that is robust to the OOV problem by extending utilizing the morphological information of words into sentence-level. As a result of the sentiment analysis task with corpus in Korean, we empirically found that the character unit is better than the morpheme unit as an embedding unit for Korean sentence embedding. We achieved 86.17% accuracy on the sentiment analysis task with the parallel bidirectional Tri-LSTM sentence encoder.

• Proceedings of the Korean Society for Language and Information Conference
• /
• 2002.02a
• /
• pp.147-156
• /
• 2002

Aligned parallel corpora have proved very useful in many natural language processing tasks, including statistical machine translation and word sense disambiguation. In this paper, we describe an alignment technique for extracting transfer mapping from the parallel corpus. During building our system and data collection, we observe that there are three types of translation approaches can be used. We especially focuses on Traditional Chinese and Simplified Chinese text lexical translation and a method for extracting transfer mappings for machine translation.

• Annual Conference of KIPS
• /
• 2006.11a
• /
• pp.161-164
• /
• 2006

인터넷이 발전하면서 웹에는 같은 내용을 다양한 언어로 표현한 문서들이 많이 존재한다. 이와 같은 웹 문서의 성질을 이용하여, 이 논문은 웹으로부터 수집된 병렬문서(parallel document)를 이용하여 한영 병렬말뭉치 구축 시스템을 설계하고 구현한다. 이 논문에서 구축과정을 요약하면 다음과 같다. 첫째, 웹 문서수집기를 이용해서 웹으로부터 한영 웹문서(html 문서)를 각각 수집한다. 둘째, 수집된 각 언어의 웹 문서에서 불필요한 내용(태그와 광고 문구 등)을 제거하여 문장을 추출하고, 추출된 문장을 단락단위로 정렬한다. 셋째, 단락단위로 정렬된 문서를 문장정렬(sentence alignment) 방법을 이용해서 문장을 정렬한다. 끝으로 정렬된 병렬문장을 단어 단위로 분리하여 병렬말뭉치를 구축한다. 이와 같은 방법으로 이 논문에서는 약 42만 5천 문장의 한영 병렬말뭉치를 구축하였다.

• The Journal of Korean Association of Computer Education
• /
• v.6 no.2
• /
• pp.9-21
• /
• 2003

This paper proposes an automatic construction system for transliteration dictionary from English-Korean parallel corpus. The system works in 3 steps: it extracts all nouns from Korean documents as the first step, filters transliterated foreign word nouns out of them with the language identification method as the second step, and extracts the corresponding English words by using a probabilistic alignment method as the final step. Specially, the fact that there is a corresponding English word in most cases, is utilized to extract the purely transliterated part from a Koreans word phrase, which is usually used in combined forms with Korean endings(Eomi) or particles(Josa). Moreover, the direct phonetic comparison is done to the words in two different alphabet systems without converting them to the same alphabet system. The experiment showed that the performance was influenced by the first and the second preprocessing steps; the most efficient model among manually preprocessed ones showed 85.4% recall, 91.0% precision and the most efficient model among fully automated ones got 68.3% recall, 89.2% precision.

• Journal of Computing Science and Engineering
• /
• v.10 no.3
• /
• pp.95-101
• /
• 2016

In this study, we propose a new system for constructing parallel corpora for sign languages, which are generally under-resourced in comparison to spoken languages. In order to achieve scalability and accessibility regarding data collection and corpus construction, our system utilizes deep learning-based techniques and predicts depth information to perform pose estimation on hand information obtainable from video recordings by a single RGB camera. These estimated poses are then transcribed into expressions in SignWriting. We evaluate the accuracy of hand tracking and hand pose estimation modules of our system quantitatively, using the American Sign Language Image Dataset and the American Sign Language Lexicon Video Dataset. The evaluation results show that our transcription system has a high potential to be successfully employed in constructing a sizable sign language corpus using various types of video resources.

• Journal of Korean language education
• /
• v.25 no.1
• /
• pp.1-27
• /
• 2014

This study investigates how common English hedge expressions such as 'I think' and 'I guess' appear in Korean, with the aim of providing explicit explanation for English-speaking leaners of Korean. Based on a contrastive analysis of spoken English and Korean corpus, this study argues three points: Firstly, 'I guess' appears with a wider variety of modalities in Korean than 'I think'. Secondly, this study has found that Korean textbooks contain inappropriate use of registers regarding the English translations of '-geot -gat-': although these markers are used in spoken Korean, they were translated into written English. Therefore, this study suggests that '-geot -gat-' be translated into 'I think' in spoken English, and into 'it seems' in the case of written English and narratives. Lastly, the contrastive analysis has shown that when 'I think' is used with deontic modalities such as 'I think I have to', Korean use '-a-ya-get-': the use of hedge marker 'I think' with 'I have to', which shows obligation or speaker's volition turns the deontic modalities into expressions of speaker's opinion.

• Genomics & Informatics
• /
• v.19 no.3
• /
• pp.24.1-24.7
• /
• 2021

Tracking the most recent advances in Coronavirus disease 2019 (COVID-19)-related research is essential, given the disease's novelty and its impact on society. However, with the publication pace speeding up, researchers and clinicians require automatic approaches to keep up with the incoming information regarding this disease. A solution to this problem requires the development of text mining pipelines; the efficiency of which strongly depends on the availability of curated corpora. However, there is a lack of COVID-19-related corpora, even more, if considering other languages besides English. This project's main contribution was the annotation of a multilingual parallel corpus and the generation of a recommendation dataset (EN-PT and EN-ES) regarding relevant entities, their relations, and recommendation, providing this resource to the community to improve the text mining research on COVID-19-related literature. This work was developed during the 7th Biomedical Linked Annotation Hackathon (BLAH7).

• KIPS Transactions on Software and Data Engineering
• /
• v.9 no.2
• /
• pp.53-60
• /
• 2020

Sentiment analysis is the automated process of understanding attitudes and opinions about a given topic from written or spoken text. One of the sentiment analysis approaches is a dictionary-based approach, in which a sentiment dictionary plays an much important role. In this paper, we propose a method to automatically generate Korean sentiment lexicon from the well-known English sentiment lexicon called VADER (Valence Aware Dictionary and sEntiment Reasoner). The proposed method consists of three steps. The first step is to build a Korean-English bilingual lexicon using a Korean-English parallel corpus. The bilingual lexicon is a set of pairs between VADER sentiment words and Korean morphemes as candidates of Korean sentiment words. The second step is to construct a bilingual words graph using the bilingual lexicon. The third step is to run the label propagation algorithm throughout the bilingual graph. Finally a new Korean sentiment lexicon is generated by repeatedly applying the propagation algorithm until the values of all vertices converge. Empirically, the dictionary-based sentiment classifier using the Korean sentiment lexicon outperforms machine learning-based approaches on the KMU sentiment corpus and the Naver sentiment corpus. In the future, we will apply the proposed approach to generate multilingual sentiment lexica.

• Journal of the Korea Convergence Society
• /
• v.12 no.11
• /
• pp.109-117
• /
• 2021

Recently, the use of speech-based interfaces is increasing as a means for human-computer interaction (HCI). Accordingly, interest in post-processors for correcting errors in speech recognition results is also increasing. However, a lot of human-labor is required for data construction. in order to manufacture a sequence to sequence (S2S) based speech recognition post-processor. To this end, to alleviate the limitations of the existing construction methodology, a new data construction method called Back TranScription (BTS) was proposed. BTS refers to a technology that combines TTS and STT technology to create a pseudo parallel corpus. This methodology eliminates the role of a phonetic transcriptor and can automatically generate vast amounts of training data, saving the cost. This paper verified through experiments that data should be constructed in consideration of text style and domain rather than constructing data without any criteria by extending the existing BTS research.

• The KIPS Transactions:PartB
• /
• v.14B no.3 s.113
• /
• pp.145-152
• /
• 2007

In this paper, we propose an automatic extraction model for unknown translations and implement an unknown translation extraction system using the proposed model. The proposed model as a phrase-alignment model is incorporated with three models: a phrase-boundary model, a language model, and a translation model. Using the proposed model we implement the system for extracting unknown translations, which consists of three parts: construction of parallel corpora, alignment of Korean and English words, extraction of unknown translations. To evaluate the performance of the proposed system, we have established the reference corpus for extracting unknown translation, which comprises of 2,220 parallel sentences including about 1,500 unknown translations. Through several experiments, we have observed that the proposed model is very useful for extracting unknown translations. In the future, researches on objective evaluation and establishment of parallel corpora with good quality should be performed and studies on improving the performance of unknown translation extraction should be kept up.