• Title/Summary/Keyword: Korean corpus

Search Result 1,186, Processing Time 0.027 seconds

Korean Word Sense Disambiguation using Dictionary and Corpus (사전과 말뭉치를 이용한 한국어 단어 중의성 해소)

  • Jeong, Hanjo;Park, Byeonghwa
    • Journal of Intelligence and Information Systems
    • /
    • v.21 no.1
    • /
    • pp.1-13
    • /
    • 2015
  • As opinion mining in big data applications has been highlighted, a lot of research on unstructured data has made. Lots of social media on the Internet generate unstructured or semi-structured data every second and they are often made by natural or human languages we use in daily life. Many words in human languages have multiple meanings or senses. In this result, it is very difficult for computers to extract useful information from these datasets. Traditional web search engines are usually based on keyword search, resulting in incorrect search results which are far from users' intentions. Even though a lot of progress in enhancing the performance of search engines has made over the last years in order to provide users with appropriate results, there is still so much to improve it. Word sense disambiguation can play a very important role in dealing with natural language processing and is considered as one of the most difficult problems in this area. Major approaches to word sense disambiguation can be classified as knowledge-base, supervised corpus-based, and unsupervised corpus-based approaches. This paper presents a method which automatically generates a corpus for word sense disambiguation by taking advantage of examples in existing dictionaries and avoids expensive sense tagging processes. It experiments the effectiveness of the method based on Naïve Bayes Model, which is one of supervised learning algorithms, by using Korean standard unabridged dictionary and Sejong Corpus. Korean standard unabridged dictionary has approximately 57,000 sentences. Sejong Corpus has about 790,000 sentences tagged with part-of-speech and senses all together. For the experiment of this study, Korean standard unabridged dictionary and Sejong Corpus were experimented as a combination and separate entities using cross validation. Only nouns, target subjects in word sense disambiguation, were selected. 93,522 word senses among 265,655 nouns and 56,914 sentences from related proverbs and examples were additionally combined in the corpus. Sejong Corpus was easily merged with Korean standard unabridged dictionary because Sejong Corpus was tagged based on sense indices defined by Korean standard unabridged dictionary. Sense vectors were formed after the merged corpus was created. Terms used in creating sense vectors were added in the named entity dictionary of Korean morphological analyzer. By using the extended named entity dictionary, term vectors were extracted from the input sentences and then term vectors for the sentences were created. Given the extracted term vector and the sense vector model made during the pre-processing stage, the sense-tagged terms were determined by the vector space model based word sense disambiguation. In addition, this study shows the effectiveness of merged corpus from examples in Korean standard unabridged dictionary and Sejong Corpus. The experiment shows the better results in precision and recall are found with the merged corpus. This study suggests it can practically enhance the performance of internet search engines and help us to understand more accurate meaning of a sentence in natural language processing pertinent to search engines, opinion mining, and text mining. Naïve Bayes classifier used in this study represents a supervised learning algorithm and uses Bayes theorem. Naïve Bayes classifier has an assumption that all senses are independent. Even though the assumption of Naïve Bayes classifier is not realistic and ignores the correlation between attributes, Naïve Bayes classifier is widely used because of its simplicity and in practice it is known to be very effective in many applications such as text classification and medical diagnosis. However, further research need to be carried out to consider all possible combinations and/or partial combinations of all senses in a sentence. Also, the effectiveness of word sense disambiguation may be improved if rhetorical structures or morphological dependencies between words are analyzed through syntactic analysis.

Studying the frequencies of sentence pattern for a entence patterns dictionary (문형 사전을 위한 문형 빈도 조사)

  • Kim Yu-Mi
    • Korean Journal of Cognitive Science
    • /
    • v.16 no.2
    • /
    • pp.123-140
    • /
    • 2005
  • The purpose of this paper is to examine the frequency and usage of sentence patterns appearing in electronic dictionaries used in Korean language education in order to design an automatic sentence patterns checking. First, the concept of sentence patterns is defined and it is classified into sentence structure patterns and sentencial expression patterns. Sentence structure patterns and sentencial expression patterns are analyzed how they are expressed in the Korean Learner's Corpus. learner's Corpus is built into the Standard Corpus, which all Korean Learners must learn, and the Errors Corpus made by learners. From these research, we will find out how frequently the Sentential Patterns are being used in the Standard Corpus which has been made of Korean Texts and how the Sentential Pattern are being used in the Errors Corpus which were constructed from Korean learner's writings. Finally, having described the Sentential Patterns on the Sentential Electric Dictionary, we determine the optimum speed in the search for the Sentential Pattern.

  • PDF

A Corpus-based Analysis of EFL Learners' Use of Discourse Markers in Cross-cultural Communication

  • Min, Sujung
    • English Language & Literature Teaching
    • /
    • v.17 no.3
    • /
    • pp.177-194
    • /
    • 2011
  • This study examines the use of discourse markers in cross-cultural communication between EFL learners in an e-learning environment. The study analyzes the use of discourse markers in a corpus of an interactive web with a bulletin board system through which college students of English at Japanese and Korean universities interacted with each other discussing the topics of local and global issues. It compares the use of discourse markers in the learners' corpus to that of a native English speakers' corpus. The results indicate that discourse markers are useful interactional devices to structure and organize discourse. EFL learners are found to display more frequent use of referentially and cognitively functional discourse markers and a relatively rare use of other markers. Native speakers are found to use a wider variety of discourse markers for different functions. Suggestions are made for using computer corpora in understanding EFL learners' language difficulties and helping them become more interactionally competent speakers.

  • PDF

Corpus-based evaluation of French text normalization (코퍼스 기반 프랑스어 텍스트 정규화 평가)

  • Kim, Sunhee
    • Phonetics and Speech Sciences
    • /
    • v.10 no.3
    • /
    • pp.31-39
    • /
    • 2018
  • This paper aims to present a taxonomy of non-standard words (NSW) for developing a French text normalization system and to propose a method for evaluating this system based on a corpus. The proposed taxonomy of French NSWs consists of 13 categories, including 2 types of letter-based categories and 9 types of number-based categories. In order to evaluate the text normalization system, a representative test set including NSWs from various text domains, such as news, literature, non-fiction, social-networking services (SNSs), and transcriptions, is constructed, and an evaluation equation is proposed reflecting the distribution of the NSW categories of the target domain to which the system is applied. The error rate of the test set is 1.64%, while the error rate of the whole corpus is 2.08%, reflecting the NSW distribution in the corpus. The results show that the literature and SNS domains are assessed as having higher error rates compared to the test set.

Error Correction and Praat Script Tools for the Buckeye Corpus of Conversational Speech (벅아이 코퍼스 오류 수정과 코퍼스 활용을 위한 프랏 스크립트 툴)

  • Yoon, Kyu-Chul
    • Phonetics and Speech Sciences
    • /
    • v.4 no.1
    • /
    • pp.29-47
    • /
    • 2012
  • The purpose of this paper is to show how to convert the label files of the Buckeye Corpus of Spontaneous Speech [1] into Praat format and to introduce some of the Praat scripts that will enable linguists to study various aspects of spoken American English present in the corpus. During the conversion process, several types of errors were identified and corrected either manually or automatically by the use of scripts. The Praat script tools that have been developed can help extract from the corpus massive amounts of phonetic measures such as the VOT of plosives, the formants of vowels, word frequency information and speech rates that span several consecutive words. The script tools can extract additional information concerning the phonetic environment of the target words or allophones.

Algorithm for automatic recognition of corpus callosum from saggital brain MR images (두뇌 자기공명영상에서의 corpus callosum의 자동인식 알고리즘)

  • Huh, S.;Lee, C.H.
    • Proceedings of the KOSOMBE Conference
    • /
    • v.1998 no.11
    • /
    • pp.62-63
    • /
    • 1998
  • In this paper, a new method to find the corpus callosum from sagittal brain MR images is proposed, which uses the statistical characteristics and shape information of corpus callosum. First, we extract regions satisfying the statistical characteristics of the corpus callosum and then find a region matching the shape information. In order to match the shape information, a new directed window region growing algorithm is proposed instead of using conventional contour matching algorithms. Using the proposed algorithm, we adaptively relax the statistical requirement until we find a region matching the shape information. Experiments show very promising results.

  • PDF

Prostaglandin $F_2{\alpha}$ Controls Reactive Oxygen Species in Bovine Corpus Luteum

  • Lee, Seunghyung;Yang, Boo-Keun;Park, Choon-Keun
    • Reproductive and Developmental Biology
    • /
    • v.39 no.1
    • /
    • pp.1-6
    • /
    • 2015
  • Luteolysis is a cyclical regression of the corpus luteum in many non-primate mammalian species. Prostaglandin $F_2{\alpha}$($PGF_2{\alpha}$) from the uterus and ovary induces functional and structural luteolysis in bovine. The action of $PGF_2{\alpha}$ is mediated by $PGF_2{\alpha}$ receptor located on the luteal steroidogenic and endothelial cell membranes. $PGF_2{\alpha}$ plays an important role in regulating nitric oxide production in endothelial cells of the bovine corpus luteum. Nitric oxide production and nitric oxide synthase activity are stimulated and induced by $PGF_2{\alpha}$ in luteal endothelial cells. Moreover, the reactive oxygen species inhibits progesterone secretion in bovine luteal cells and induces apoptosis. Thus, the interaction between $PGF_2{\alpha}$ and reactive oxygen species provides important aspects in physiology of the corpus luteum forfunctional and structural luteolysis.

A Study on the Voice Onset Time of English Voiceless Stops in the Buckeye Corpus (벅아이 코퍼스를 이용한 영어 무성파열음의 VOT 연구)

  • Yoon, Kyu-Chul
    • Phonetics and Speech Sciences
    • /
    • v.4 no.2
    • /
    • pp.33-40
    • /
    • 2012
  • The purpose of this paper is to investigate the voice onset time (VOT) of the English voiceless stops [p, t, k] found in the Buckeye Corpus of Conversational Speech [1]. Three young female speakers were chosen for this study and their VOT values were semi-automatically extracted along with other factors. The factors used for the analysis were place of articulation, location in word, syllabic stress, content word or not, word frequency calculated from the corpus, and the speech rate expressed in syllables per second. Results showed that, for the three places of articulation of each speaker, all the factors had a statistically significant effect on the VOT values. This paper has significance in that the materials used for the analysis were from a corpus of spontaneous natural English speech.

An Analysis of the Vowel Formants of the Young Males in the Buckeye Corpus (벅아이 코퍼스에서의 젊은 성인 남성의 모음 포먼트 분석)

  • Yoon, Kyu-Chul;Noh, Hye-Uk
    • Phonetics and Speech Sciences
    • /
    • v.4 no.2
    • /
    • pp.41-49
    • /
    • 2012
  • The purpose of this paper is to extract the vowel formants of the ten young male speakers from the Buckeye Corpus of Conversational Speech [1] and to analyze them in comparison to earlier works in terms of various phonetic factors that are expected to affect the realization of the formant distribution. The first two formant frequency values were automatically extracted with a Praat script along with such factors as the place of articulation, the content versus function word information, syllabic stress information, the location in a word, location in utterance, speech rate of three consecutive words, and the word frequency in the corpus. The results indicated that the formant patterns from the corpus were very different from those of earlier works although the overall pattern was similar and that the factors were strongly responsible for the realization of the two formants. The purpose of this paper is to extract the vowel formants of the ten young male speakers from the Buckeye Corpus of Conversational Speech [1] and to analyze them in comparison to earlier works in terms of various phonetic factors that are expected to affect the realization of the formant distribution. The first two formant frequency values were automatically extracted with a Praat script along with such factors as the place of articulation, the content versus function word information, the syllabic stress information, the location in a word, the location in an utterance, the speech rate of the three consecutive words, and the word frequency in the corpus. The result indicated that the formant patterns from the corpus were very different from those of earlier works although the overall pattern was similar and that the factors were strongly responsible for the realization of the two formants.

Reduction and Frequency Analyses of Vowels and Consonants in the Buckeye Speech Corpus

  • Yang, Byung-Gon
    • Phonetics and Speech Sciences
    • /
    • v.4 no.3
    • /
    • pp.75-83
    • /
    • 2012
  • The aims of this study were three. First, to examine the degree of deviation from dictionary prescribed symbols and actual speech made by American English speakers. Second, to measure the frequency of vowel and consonant production of American English speakers. And third, to investigate gender differences in the segmental sounds in a speech corpus. The Buckeye Speech Corpus was recorded by forty American male and female subjects for one hour per subject. The vowels and consonants in both the phonemic and phonetic transcriptions were extracted from the original files of the corpus and their frequencies were obtained using codes of a free software R. Results were as follows: Firstly, the American English speakers produced a reduced number of vowels and consonants in daily conversation. The reduction rate from the dictionary transcriptions to the actual transcriptions was around 38.2%. Secondly, the American English speakers used more front high and back low vowels while three-fourths of the consonants accounted for stops, fricatives, and nasals. This indicates that the segmental inventory has nonlinear frequency distribution in the speech corpus. Thirdly, the two gender groups produced vowels and consonants similarly even though there were a few noticeable differences in their speech. From these results we propose that English teachers consider pronunciation education reflecting the actual speech sounds and that linguists find a way to establish unmarked segmentals from speech corpora.