• Proceedings of the Korea Information Processing Society Conference
• /
• 2012.11a
• /
• pp.441-444
• /
• 2012

오늘날 스마트폰의 보급으로 SNS는 급속도로 성장하였고, 매일 엄청난 분량의 텍스트 데이터가 생성되고 있다. 본 연구에서는 다른 매체에 비해 개인의 의견이 좀 더 거침없이 올라오는 SNS의 특징에 주목해 SNS의 텍스트 데이터를 대상으로 하는 평판 분석 기법을 제안한다. 제안 방법은 분석하고자 하는 대상에 대한 SNS 데이터를 수집하여 DB에 저장한 다음, 광고 제거 과정과 자동 띄어쓰기 과정 및 형태소 분석을 거친 후 감성 포함 여부 확인 과정과 극성 분류 과정으로 구성된다. 평판 분석을 위해 본 연구에서는 감성 단어 사전의 쾌-불쾌 수치와 활성화 수치를 사용한다. 분석 결과 모든 문서에 대한 극성 분류 정확도는 55%였고, 감성 포함 여부 확인 과정이 올바르게 수행된 문서에 대한 극성 분류 정확도는 82%였다.

• Journal of KIISE:Software and Applications
• /
• v.35 no.6
• /
• pp.386-391
• /
• 2008

Some spoken word errors that violate grammatical or writing rules occurs frequently in communication environments like mobile phone and messenger. These unexpected errors cause a problem in a language processing system for many applications like speech recognition, text-to-speech translation, and so on. In this paper, we proposed and implemented an automatic correction system of ill-formed words and word spacing errors in SMS sentences that has been the major errors of poor accuracy. We experimented three methods of constructing the word correction dictionary and evaluated the results of those methods. They are (1) manual construction of error words from the vocabulary list of ill-formed communication languages, (2) automatic construction of error dictionary from the manually constructed corpus, and (3) context-dependent method of automatic construction of error dictionary.

• Journal of the Korean Institute of Telematics and Electronics
• /
• v.25 no.10
• /
• pp.1183-1192
• /
• 1988

In this paper, we describe heuristic information-added morphological dictionary and connection table, and automatic MUNJEUL separation process on the basis of least cost method for efficient morphological analysis. It is simplified the composition of connection and inflective word information by mutually interconnect conjugation table with connection tables. As a result, the applicability of system is increased. Translation dictionary consists of analysis and generation part and, increase the applicability by describing frequently using termination phrase which is extracted statistically as idiom and the procedure directly on the dictionary for the efficiency of analysis process and more natural generation of translation sentence.

• Journal of the Korean Institute of Intelligent Systems
• /
• v.20 no.2
• /
• pp.202-207
• /
• 2010

In this paper, we present an automatic sentiment classification method for on-line movie reviews that do not contain explicit sentiment rating scores. For the sentiment polarity classification, positive or negative, we use a Support Vector Machine classifier based on syllable kernel that is an extended model of string kernel. We give some experimental results which show that proposed syllable kernel model can be effectively used in sentiment classification tasks for on-line movie reviews that usually contain a lot of grammatical errors such as spacing or spelling errors.

• Journal of the Korean Society for information Management
• /
• v.13 no.1
• /
• pp.47-63
• /
• 1996

Conventional automatic indexing methods for Hangul texts can be classified into two groups as follows: One is to extract index terms by removing non-indexable segments from word-phrases, and the other is to generate index terms from the morphemes of word-phrases. The former suffers from the problem of word boundaries when documents contain many compound nouns. The latter can overcome the word boundary problem by extracting simple nouns, but has many overheads to develop a lot of linguistic knowledges needed in the indexing procedure. In this paper we propose a new indexing method based on n-grams. This method alleviates the problems of previous indexing methods related with word boundaries and linguistic knowledges. We also compare the effectiveness of the n-gram based indexing method with that of the previous ones.

• Smart Media Journal
• /
• v.7 no.4
• /
• pp.24-29
• /
• 2018

Text analysis technique for natural language processing in deep learning represents words in vector form through word embedding. In this paper, we propose a method of constructing a document vector and classifying it into spam and normal text message, using word embedding and deep learning method. Automatic spacing applied in the preprocessing process ensures that words with similar context are adjacently represented in vector space. Additionally, the intentional word formation errors with non-alphabetic or extraordinary characters are designed to avoid being blocked by spam message filter. Two embedding algorithms, CBOW and skip grams, are used to produce the sentence vector and the performance and the accuracy of deep learning based spam filter model are measured by comparing to those of SVM Light.

• Journal of Internet Computing and Services
• /
• v.21 no.1
• /
• pp.79-85
• /
• 2020

Today, to block spam texts on smartphone, a simple string comparison between text messages and spam keywords or a blocking spam phone numbers is used. As results, spam text is sent in a gradually hanged way to prevent if from being automatically blocked. In particular, for words included in spam keywords, spam texts are sent to abnormal words using special characters, Chinese characters, and whitespace to prevent them from being detected by simple string match. There is a limit that traditional spam filtering methods can't block these spam texts well. Therefore, new technologies are needed to respond to changing spam text messages. In this paper, we propose a knowledge graph-based new words detection mechanism that can detect new words frequently used in spam texts and respond to changing spam texts. Also, we show experimental results of the performance when detected Korean new words are applied to the Naive Bayes algorithm.

• Journal of KIISE:Software and Applications
• /
• v.37 no.12
• /
• pp.946-950
• /
• 2010

Many sentiment categorization systems based on machine learning methods use morphological analyzers in order to extract linguistic features from sentences. However, the morphological analyzers do not generally perform well in a customer review domain because online customer reviews include many spacing errors and spelling errors. These low performances of the underlying systems lead to performance decreases of the sentiment categorization systems. To resolve this problem, we propose a feature extraction method based on simple longest matching of Eojeol (a Korean spacing unit) and phoneme patterns. The two kinds of patterns are automatically constructed from a large amount of POS (part-of-speech) tagged corpus. Eojeol patterns consist of Eojeols including content words such as nouns and verbs. Phoneme patterns consist of leading consonant and vowel pairs of predicate words such as verbs and adjectives because spelling errors seldom occur in leading consonants and vowels. To evaluate the proposed method, we implemented a sentiment categorization system using a SVM (Support Vector Machine) as a machine learner. In the experiment with Korean customer reviews, the sentiment categorization system using the proposed method outperformed that using a morphological analyzer as a feature extractor.

• Journal of the Korea Convergence Society
• /
• v.12 no.12
• /
• pp.65-74
• /
• 2021

Recently, studies on Korean spelling correction have been actively conducted based on machine translation and automatic noise generation. These methods generate noise and use as train and data set. This has limitation in that it is difficult to accurately measure performance because it is unlikely that noise other than the noise used for learning is included in the test set In addition, there is no practical error type standard, so the type of error used in each study is different, making qualitative analysis difficult. This paper proposes new 'error type classification' for deep learning-based Korean spelling correction research, and error analysis perform on existing commercialized Korean spelling correctors (System A, B, C). As a result of analysis, it was found the three correction systems did not perform well in correcting other error types presented in this paper other than spacing, and hardly recognized errors in word order or tense.

• Journal of the HCI Society of Korea
• /
• v.7 no.2
• /
• pp.35-42
• /
• 2012

The purpose of this study was to evaluate the usability of the tablet PC-based Korean high-tech AAC(Augmentative Alternative Communication System) software. In order to develop an AAC software which is appropriate to Korean cultural/linguistic contexts and communication needs of the users, we examined the necessity and ease of use for the communication functions that are required in native Korean communication, such as polite expressions, tense expressions, negative expressions, subject-verb auto-matching, and automatic sentence generation functions, using a scenario-based user testing. We also investigated the users' needs, preferences, and satisfaction for the tablet PC-based Korean high tech AAC using a semi-structured and open questionnaires. The participants of this study were 9 special education teachers, 6 speech therapists, and 6 parents whose children had communication disabilities. The results of the usability testing of the tablet PC-based Korean high-tech AAC software presented positive responses in general, by indicating overall scores of above 4 out of 5 except in tense and negative expressions. The necessity and ease of use in the tense and negative expressions were evaluated relatively low, and it might be related to the inconsistent interface with the polite expressions. In terms of the user interface(UI), there were users' needs for clear visual feedback in the symbol selection and display, consistent interface for all functions, more natural subject-verb auto-matching, and spacing in the text within symbols. The results of the usability testing and users' feedback might serve as a guideline to compensate and improve the function and UI of the existing AAC software.