Browse > Article

Spam-Mail Filtering System Using Weighted Bayesian Classifier  

김현준 (인하대학교 컴퓨터정보공학부)
정재은 (인하대학교 컴퓨터정보공학)
조근식 (인하대학교 컴퓨터정보공학부)
Publication Information
Journal of KIISE:Software and Applications / v.31, no.8, 2004 , pp. 1092-1100 More about this Journal
Abstract
An E-mails have regarded as one of the most popular methods for exchanging information because of easy usage and low cost. Meanwhile, exponentially growing unwanted mails in user's mailbox have been raised as main problem. Recognizing this issue, Korean government established a law in order to prevent e-mail abuse. In this paper we suggest hybrid spam mail filtering system using weighted Bayesian classifier which is extended from naive Bayesian classifier by adding the concept of preprocessing and intelligent agents. This system can classify spam mails automatically by using training data without manual definition of message rules. Particularly, we improved filtering efficiency by imposing weight on some character by feature extraction from spam mails. Finally, we show efficiency comparison among four cases - naive Bayesian, weighting on e-mail header, weighting on HTML tags, weighting on hyperlinks and combining all of four cases. As compared with naive Bayesian classifier, the proposed system obtained 5.7% decreased precision, while the recall and F-measure of this system increased by 33.3% and 31.2%, respectively.
Keywords
Bayesian Classifier; Mail filtering; Intelligent Agent;
Citations & Related Records
연도 인용수 순위
  • Reference
1 한국전산원, '국가정보화백서(National Informatization White Paper)', pp. 23, 2002
2 Internet E-mail Corporate Usage Report, www.securitymanagenment.com/library/worldtalk0200.html
3 Ricardo, B.-Y. and Berthier, R.-N., Modern Information Retrieval, pp.27, Addison-Wesley, 1999
4 Provost, J., 'Naive-Bayes vs. Rule-Learning in Classification of Email,' Technical report, Dept. of Computer Sciences at the U. of Texas ay Austin, 1999
5 정보통신부, 정보통신망 이용촉진 및 정보보호 등에 관한 법률 시행령 제11조 (영리목적의 광고성 전자우편의 명시방법), 2002
6 Diao, Y., Lu, H. and Wu, D., 'A Comparative Study of Classification Based Personal E-mail Filtering,' Proc. of PAKDD-00, 4th Pacific-Asia Conference on Knowledge Discovery and Data Mining, 2000   DOI   ScienceOn
7 Cohen, W.W., 'Learning Rules that Classify E-Mail ,' Proc. of the AAAI Spring Symposium on Machine Learning in Information Access, 1996
8 Rev. T. B., 'An essay toward solving a problem in the doctrine of chances,' Philosopical Transactions of London, vol. 53, pp.370-418, 1763   DOI
9 Sahami, M., Dumais, S., Heckerman, D. and Horvitz, E., 'A Bayesian Approach to Filtering Junk E-Mail. In Learning for Text Categorization,' Proc. of the AAAI Workshop;AAAI Technical Report WS-98-05, 1998
10 Androutsopoulos, I., Koutsias, J., Chandrinos, K. V., Spyropoulos, C. D., 'An Experimental Comparison of Naive Bayesian and Keyword-Based Anti-Spam Filtering with Personal E-Mail Messages,' Proc of the 23rd Annual International ACM SIGIR Conference on Reach and Development in Information Retrieval, 2000
11 Thomas, G. and Peter, A. F., 'Weighted Bayesian Classification based on Support Vector Machine,' Proc. of the 18th International Conference on Machine Learning, pp. 207-209, 2001
12 Androutsopoulos, I., Koutsias, J., Chandrinos, K. V., Paliouras, G. and Spyropoulos, C. D., 'An Evaluation of Naive Bayesian Anti-Spam Filtering,' Proc fo the 11th European Conference on Machine Learning, pp.9-17, 2000
13 Mitchell, T. M., Machine Learning, Chapter 6: Bayesian Learning, McGraw-Hill, 1997
14 http://popfile.sourceforge.net/
15 Denzinger, J. and Ennis, S., 'Being the new guy in an experienced team - enhancing training on the job,' Proc. of the 1st International Joint Conference on Autonomous Agents and Multiagent Systems, Pt3, 2002
16 Ferreira, J. T. A. S., Denison, D. G. T., Hand, D. J., 'Weighted Naive Bayes modeling for data mining,' Technical report, Dept. of Mathematics at Imperial College. 2001
17 Russell, S. I. and Norving, P., Arificial Intelligence - A Modern Approach, Prentice Hall, pp.525-529, 1995
18 Han, J., 'Data Mining: Concepts and Techniques,' Morgan Kaufmann, 2001
19 Graham, P., 'Better Bayesian Filtering,' Article of Spam Conference, 2003
20 고수정, 이정현, 'Apriori 알고리즘에 의한 연관단어 지식 베이스에 의한 가중치가 부여된 베이지안 자동 문서 분류', 멀티미디어학회 논문지 제4권 제2호, 2001
21 조한철, 조근식, '나이브 베이지안 분류자와 메세지 규칙을 이용한 스팸메일 필터링 시스템', 한국정보과학회, 제29회 춘계학술대회, 2002   과학기술학회마을
22 Fox, C., 'Lexical analysis and stop lists. In Information Retrieval: Data Structures and Algorithms,' Prentice-Hall, 1992
23 http://email.about.com/cs/bayesianspamsw/
24 Joachims, T., 'Text Categorization with Support Vector Machines: Learning with Many Relevant Features,' European Conference on Machine Learning, 1998   DOI   ScienceOn