• The Journal of the Korea Contents Association
• /
• v.9 no.12
• /
• pp.113-120
• /
• 2009

Currently the verbal abuse in text message over on-line game is so serious. However we do not have any effective policy or technical tools yet. Till now in order to cope with this problem, the online game service providers have accumulated a set of forbidden words and applied this list on the textual word used in on-line game, which is called 'Swear filter'. But young on-line game players easily avoid this filtering method by coining another words which is not kept in the list. Especially Korean is very easy to make new variations of a vulgar word. In this paper, we propose one smart filtering algorithm to identify newly coined profanities. Important features of our method include the canonical form transformation of coined profanities, semi-global alignment between in the level of consonant and vowel units. For experiment, we have collected more than 1000 newly coined vulgar words in on-line gaming sites and tested these word against our methods. where our system have successfully filtered more than 90% of those newly coined vulgar words.

• Journal of Korea Multimedia Society
• /
• v.22 no.6
• /
• pp.695-705
• /
• 2019

The level of psychological damage caused by verbal abuse among cyberbully victims is very serious. It is going to introduce a system that determines the level of sanctions against chatting in real time using the automatic prohibited words filtering based on artificial neural network. In this paper, we propose a keyword filtering method that detects the modified prohibited words and determines whether the corresponding chat should be sanctioned in real time, and a real-time chatting screening system using it. The accuracy of filtering through machine learning was improved by processing data in advance through coding techniques that express consonants and vowels of similar pronunciation at close distances. After comparing and analyzing Mahalanobis-based clustering algorithms and artificial neural network-based algorithms, algorithms that utilize artificial neural networks showed high performance. If it is applied to Internet chatting, comments or online games, it is expected that it will be able to filter more effectively than the existing filtering method and that this will ease communication inconvenience due to existing indiscriminate filtering methods.

• International Journal of Fuzzy Logic and Intelligent Systems
• /
• v.6 no.2
• /
• pp.105-109
• /
• 2006

In this paper, we constructed a two-phase spam-mail filtering system based on the lexical and conceptual information. There are two kinds of information that can distinguish the spam mail from the ham (non-spam) mail. The definite information is the mail sender's information, URL, a certain spam keyword list, and the less definite information is the word list and concept codes extracted from the mail body. We first classified the spam mail by using the definite information, and then used the less definite information. We used the lexical information and concept codes contained in the email body for SVM learning in the 2nd phase. According to our results the ham misclassification rate was reduced if more lexical information was used as features, and the spam misclassification rate was reduced when the concept codes were included in features as well.

• Proceedings of the Korea Society of Information Technology Applications Conference
• /
• 2005.11a
• /
• pp.21-25
• /
• 2005

In this paper, we constructed a two-phase spam-mail filtering system based on the lexical and conceptual information. There are two kinds of information that can distinguish the spam mail from the legitimate mail. The definite information is the mail sender's information, URL, a certain spam list, and the less definite information is the word list and concept codes extracted from the mail body. We first classified the spam mail by using the definite information, and then used the less definite information. We used the lexical information and concept codes contained in the email body for SVM learning in the $2^{nd}$ phase. According to our results the spam precision was increased if more lexical information was used as features, and the spam recall was increased when the concept codes were included in features as well.

• Journal of KIISE:Computing Practices and Letters
• /
• v.16 no.12
• /
• pp.1254-1258
• /
• 2010

In information retrieval, different surface forms of the same object can cause poor performance of systems. In this paper, we propose the method extracting alternative words using translation words as features of each word extracted from parallel corpus, korean/english title pair of patent information. Also, we propose an association word filtering method to remove association words from an alternative word list. Evaluation results show that the proposed method outperforms other alternative word extraction methods.

• Journal of KIISE
• /
• v.42 no.1
• /
• pp.68-75
• /
• 2015

This paper proposes a method for the automatic word spacing of unsegmented Korean sentences. In our method, eojeol monograms are used for word spacing as opposed to the syllable n-grams that have been used in previous studies. The use of a Korean morphological analyzer is limited to the correction of typical word spacing errors. Our method gives a 98.06% syllable accuracy and a 94.15% eojeol recall, when 10-fold cross-validated with the Sejong corpus, after filtering out non-hangul eojeols. The processing rate is 250K eojeols or 1.8 MB per second on a typical personal computer. Syllable accuracy and eojeol recall are related to the size of the eojeol dictionary, better performance is expected with a bigger corpus.

• Smart Media Journal
• /
• v.7 no.4
• /
• pp.24-29
• /
• 2018

Text analysis technique for natural language processing in deep learning represents words in vector form through word embedding. In this paper, we propose a method of constructing a document vector and classifying it into spam and normal text message, using word embedding and deep learning method. Automatic spacing applied in the preprocessing process ensures that words with similar context are adjacently represented in vector space. Additionally, the intentional word formation errors with non-alphabetic or extraordinary characters are designed to avoid being blocked by spam message filter. Two embedding algorithms, CBOW and skip grams, are used to produce the sentence vector and the performance and the accuracy of deep learning based spam filter model are measured by comparing to those of SVM Light.

• Journal of the Korea Society of Computer and Information
• /
• v.21 no.7
• /
• pp.9-16
• /
• 2016

Collaborative filtering(CF) uses the purchase or item rating history of other users, but does not need additional properties or attributes of users and items. Hence CF is known th be the most successful recommendation technology. But conventional CF approach has some significant weakness, such as the new user problem. In this paper, we propose a approach using word embedding with skip-gram for learning distributed item representations. In particular, we show that this approach can be used to capture precise item for solving the "new user problem." The proposed approach has been tested on the Movielens databases. We compare the performance of the user based CF, item based CF and our approach by observing the change of recommendation results according to the different number of item rating information. The experimental results shows the improvement in our approach in measuring the precision applied to new user problem situations.

• Journal of KIISE:Software and Applications
• /
• v.32 no.9
• /
• pp.901-911
• /
• 2005

With the development of e-commerce and the proliferation of easily accessible information, information filtering has become a popular technique to prune large information spaces so that users are directed toward those items that best meet their needs and preferences. While many collaborative filtering systems have succeeded in capturing the similarities among users or items based on ratings to provide good recommendations, there are still some challenges for them to be more efficient, especially the user bias problem, non-transitive association problem and cold start problem. Those three problems impede us to capture more accurate similarities among users or items. In this paper, we provide probabilistic model approaches for UCHM and ICHM which are suggested to solve the addressed problems in hopes of achieving better performance. In this probabilistic model, objects (users or items) are classified into groups and predictions are made for users considering the Gaussian distribution of user ratings. Experiments on a real-word data set illustrate that our proposed approach is comparable with others.

• The Journal of the Korea Contents Association
• /
• v.8 no.1
• /
• pp.282-289
• /
• 2008

Although there were some technological developments in improving the collaborative filtering, they have yet to fully reflect the actual relation of the items. In this paper, we propose the recommendation system using associative web document classification by word frequency and ${\alpha}$-cut to address the short comings of the collaborative filtering. The proposed method extracts words from web documents through the morpheme analysis and accumulates the weight of term frequency. It makes associative rules and applies the weight of term frequency to its confidence by using Apriori algorithm. And it calculates the similarity among the words using the hypergraph partition. Lastly, it classifies related web document by using ${\alpha}$-cut and calculates similarity by using adjusted cosine similarity. The results show that the proposed method significantly outperforms the existing methods.