• The KIPS Transactions:PartB
• /
• v.14B no.3 s.113
• /
• pp.145-152
• /
• 2007

In this paper, we propose an automatic extraction model for unknown translations and implement an unknown translation extraction system using the proposed model. The proposed model as a phrase-alignment model is incorporated with three models: a phrase-boundary model, a language model, and a translation model. Using the proposed model we implement the system for extracting unknown translations, which consists of three parts: construction of parallel corpora, alignment of Korean and English words, extraction of unknown translations. To evaluate the performance of the proposed system, we have established the reference corpus for extracting unknown translation, which comprises of 2,220 parallel sentences including about 1,500 unknown translations. Through several experiments, we have observed that the proposed model is very useful for extracting unknown translations. In the future, researches on objective evaluation and establishment of parallel corpora with good quality should be performed and studies on improving the performance of unknown translation extraction should be kept up.

• Genomics & Informatics
• /
• v.19 no.3
• /
• pp.24.1-24.7
• /
• 2021

Tracking the most recent advances in Coronavirus disease 2019 (COVID-19)-related research is essential, given the disease's novelty and its impact on society. However, with the publication pace speeding up, researchers and clinicians require automatic approaches to keep up with the incoming information regarding this disease. A solution to this problem requires the development of text mining pipelines; the efficiency of which strongly depends on the availability of curated corpora. However, there is a lack of COVID-19-related corpora, even more, if considering other languages besides English. This project's main contribution was the annotation of a multilingual parallel corpus and the generation of a recommendation dataset (EN-PT and EN-ES) regarding relevant entities, their relations, and recommendation, providing this resource to the community to improve the text mining research on COVID-19-related literature. This work was developed during the 7th Biomedical Linked Annotation Hackathon (BLAH7).

• Proceedings of the Korean Society for Language and Information Conference
• /
• 2002.02a
• /
• pp.147-156
• /
• 2002

Aligned parallel corpora have proved very useful in many natural language processing tasks, including statistical machine translation and word sense disambiguation. In this paper, we describe an alignment technique for extracting transfer mapping from the parallel corpus. During building our system and data collection, we observe that there are three types of translation approaches can be used. We especially focuses on Traditional Chinese and Simplified Chinese text lexical translation and a method for extracting transfer mappings for machine translation.

• Annual Conference on Human and Language Technology
• /
• 2018.10a
• /
• pp.470-473
• /
• 2018

최근 제안된 순환 신경망 기반 Encoder-Decoder 모델은 기계번역에서 좋은 성능을 보인다. 하지만 이는 대량의 병렬 코퍼스를 전제로 하며 병렬 코퍼스가 소량일 경우 데이터 희소성 문제가 발생하며 번역의 품질은 다소 제한적이다. 본 논문에서는 기계번역의 이러한 문제를 해결하기 위하여 단일-언어(Monolingual) 데이터를 학습과정에 사용하였다. 즉, 역-번역(Back-translation)을 이용하여 단일-언어 데이터를 가상 병렬(Pseudo Parallel) 데이터로 변환하는 방식으로 기존 병렬 코퍼스를 확장하여 번역 모델을 학습시켰다. 역-번역 방법을 이용하여 영-한 번역 실험을 수행한 결과 +0.48 BLEU 점수의 성능 향상을 보였다.

• Journal of the Korea Convergence Society
• /
• v.12 no.5
• /
• pp.23-29
• /
• 2021

Recent deep learning-based natural language processing studies are conducting research to improve performance by training large amounts of data from various sources together. However, there is a possibility that the methodology of learning by combining data from various sources into one may prevent performance improvement. In the case of machine translation, data deviation occurs due to differences in translation(liberal, literal), style(colloquial, written, formal, etc.), domains, etc. Combining these corpora into one for learning can adversely affect performance. In this paper, we propose a new Corpus Weight Balance(CWB) method that considers the balance between parallel corpora in machine translation. As a result of the experiment, the model trained with balanced corpus showed better performance than the existing model. In addition, we propose an additional corpus construction process that enables coexistence with the human translation market, which can build high-quality parallel corpus even with a monolingual corpus.

• Proceedings of the IEEK Conference
• /
• summer
• /
• pp.103-109
• /
• 2004

In NLP (Natural Language Processing) tasks, the highest difficulty which computers had to face with, is the built-in ambiguity of Natural Languages. To disambiguate it, formerly, they based on human-devised rules. Building such a complete rule-set is time-consuming and labor-intensive task whilst it doesn't cover all the cases. Besides, when the scale of system increases, it is very difficult to control that rule-set. So, recently, many NLP tasks have changed from rule-based approaches into corpus-based approaches with large annotated corpora. Corpus-based NLP tasks for such popular languages as English, French, etc. have been well studied with satisfactory achievements. In contrast, corpus-based NLP tasks for Vietnamese are at a deadlock due to absence of annotated training data. Furthermore, hand-annotation of even reasonably well-determined features such as part-of-speech (POS) tags has proved to be labor intensive and costly. In this paper, we present our building an annotated English-Vietnamese parallel aligned corpus named EVC to train for Vietnamese-related NLP tasks such as Word Segmentation, POS-tagger, Word Order transfer, Word Sense Disambiguation, English-to-Vietnamese Machine Translation, etc.

• Asia Pacific Journal of Corpus Research
• /
• v.3 no.2
• /
• pp.1-16
• /
• 2022

This study investigates the cooccurrence between temporal adverbs and grammatical tenses in Spanish and contrasts temporal specifications across Spanish, English, and Chinese. Based on a monolingual Spanish corpus and a trilingual parallel corpus, the study identified the top ten frequent single-word temporal adverbs collocating with grammatical tenses in Spanish. It also contrasted the cooccurrence of temporal adverbs and verb tenses in three languages. The results show that aun 'still', hoy 'today', and ahora 'now' collocate with the present tense at more than 80%. Ayer 'yesterday' and finalmente 'finally' cooccurring with the simple past tense are at 84% and 69%, respectively. Then, mientras 'meanwhile' collocates with the past imperfect at 55%, the highest of all. Mañana 'tomorrow' cooccurs with the future and present tenses at 34%. Other adverbs, ya 'already', siempre 'always', and nuevamete 'again', do not present a strong cooccurrence tendency with a tense overall. The contrastive analysis of the trilingual parallel corpus shows a comprehensive view of temporal specifications in the three languages. However, no clear one-to-one mapping pattern of the cooccurrence across the three languages can be concluded, which provides helpful insights for second language instruction with natural language data rather than intuition. Future research with larger corpora is needed.

• Annual Conference on Human and Language Technology
• /
• 1999.10e
• /
• pp.150-155
• /
• 1999

다국어를 이용한 통계적 자연어 처리의 연구가 진행됨에 따라 병렬 말뭉치의 중요성이 대두되고 있다. 그러나 여러 가지 제약점으로 인하여 현재 이용 가능한 한국어 병렬 말뭉치가 드문 상황이다. 월드 와이드 웹 상에는 다양한 언어로 번역된 문서들이 있으며 이를 병렬 말뭉치로 구축, 활용한다면 말뭉치의 희소성으로 인한 문제를 해결할 수 있다. 본 논문에서는 웹 상에서 번역문서 후보를 추출한 다음 HTML 문서 구조를 비교하여 번역문서인지를 판별하고 문장 단위 정렬을 이용하여 병렬 말뭉치로 구축하는 방법을 제시한다.

• Language and Information
• /
• v.9 no.2
• /
• pp.49-68
• /
• 2005

Application of chunking to English and some other European languages has shown that it is a viable parsing mechanism for natural languages. Although a small number of attempts have been made to apply chunking to the analysis of the Korean language, it still is not clear enough what criteria there are to identify appropriate units of chunking, and how efficient and valid the chunking algorithms would be when applied to some authentic Korean texts. The purpose of this research is to provide an alternative set of algorithms for chunking Korean, and to implement them, and to test them against some English-Korean parallel corpora, which is English and Korean bibles matched sentence by sentence. It is shown in the paper that aligning related texts and identifying matched phrases between the two languages can be achieved through appropriate chunking and matching algorithms defined on the morphologically-tagged parallel corpus. Chunking and matching processes are based on the content words rather than the function words, and the matching itself is done in terms of the transfer dictionary. The implementation is done in C and XML, and can be accessed through the Internet.

• Journal of Computing Science and Engineering
• /
• v.10 no.3
• /
• pp.95-101
• /
• 2016

In this study, we propose a new system for constructing parallel corpora for sign languages, which are generally under-resourced in comparison to spoken languages. In order to achieve scalability and accessibility regarding data collection and corpus construction, our system utilizes deep learning-based techniques and predicts depth information to perform pose estimation on hand information obtainable from video recordings by a single RGB camera. These estimated poses are then transcribed into expressions in SignWriting. We evaluate the accuracy of hand tracking and hand pose estimation modules of our system quantitatively, using the American Sign Language Image Dataset and the American Sign Language Lexicon Video Dataset. The evaluation results show that our transcription system has a high potential to be successfully employed in constructing a sizable sign language corpus using various types of video resources.