Browse > Article

An Information-theoretic Approach for Value-Based Weighting in Naive Bayesian Learning  

Lee, Chang-Hwan (동국대학교 정보통신학과)
Publication Information
Journal of KIISE:Databases / v.37, no.6, 2010 , pp. 285-291 More about this Journal
Abstract
In this paper, we propose a new paradigm of weighting methods for naive Bayesian learning. We propose more fine-grained weighting methods, called value weighting method, in the context of naive Bayesian learning. While the current weighting methods assign a weight to an attribute, we assign a weight to an attribute value. We develop new methods, using Kullback-Leibler function, for both value weighting and feature weighting in the context of naive Bayesian. The performance of the proposed methods has been compared with the attribute weighting method and general naive bayesian. The proposed method shows better performance in most of the cases.
Keywords
machine learning; data mining; naive Bayesian;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Dietrich Wettschereck, David W. Aha, and Takao Mohri. A review and empirical evaluation of feature weighting methods for a class of lazy learning algorithms. Articial Intelligence Review, vol.11, pp.273-314, 1997.   DOI   ScienceOn
2 J. Ross Quinlan. C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA. 1993.
3 Peter Clark and Robin Boswell. Rule induction with cn2: some recent improvements. In EWSL-91: Proceedings of the European working session on learning on Machine learning, pp.151-163, 1991.
4 S. Kullback and R. A. Leibler. On information and suciency. The Annals of Mathematical Statistics, vol.22, no.1, pp.79-86, 1951.   DOI   ScienceOn
5 C. Merz, P. Murphy, and D. Aha. UCI repository of machine learning databases. 1997.
6 Ron Kohavi and George H. John. Wrappers for feature subset selection. Articial Intelligence, vol.97, no.1-2, pp.273-324, 1997.   DOI   ScienceOn
7 C. A. Ratanamahatana and D. Gunopulos. Feature selection for the naive bayesian classier using decision trees. Applied Articial Intelligence, vol.17, no.5,6, pp.475-487, 2003.   DOI
8 Thomas Gartner and Peter A. Flach. Wbcsvm: Weighted bayesian classification based on support vector machines. In ICML '01: Proceedings of the Eighteenth International Conference on Machine Learning. 200l.
9 Mark Hall. A decision tree-based attribute weighting filter for naive bayes. Knowledge-Based Systems, vol.20, no.2, 2007. 13.
10 Harry Zhang and Shengli Sheng. Learning weighted naive bayes with accurate ranking. In ICDM '04: Proceedings of the Fourth IEEE International Conference on Data Mining, 2004.
11 U. Fayyad and K. Irani. Multi-interval discretization of continuous-valued attributes for classification learning. Proceedings of Thirteenth International Joint Conference on Artificial Intelligence. Morgan Kaufmann, 1993.
12 Pedro Domingos and Michael Pazzani. On the optimality of the simple bayesian classifier under zero-one loss. Machine Learning, 29(2-3), 1997.
13 Pat Langley and Stephanie Sage. Induction of selective bayesian classifiers. In Proceedings of the Tenth Conference on Uncertainty in Articial Intelligence, pp.399-406, 1994.
14 Claire Cardie and Nicholas Howe. Improving minority class prediction using case-specific feature weights. In Proceedings of the Fourteenth International Conference on Machine Learning, pp.57-65, 1997.