Journal of Computing Science and Engineering

Korean Institute of Information Scientists and Engineers (한국정보과학회)
권호 표지
DOI QR코드

DOI QR Code

Relation Based Bayesian Network for NBNN

  • Sun, Mingyang (School of Computing, Korea Advanced Institute of Science and Technology (KAIST)) ;
  • Lee, YoonSeok (School of Computing, Korea Advanced Institute of Science and Technology (KAIST)) ;
  • Yoon, Sung-eui (School of Computing, Korea Advanced Institute of Science and Technology (KAIST))
  • Received : 2015.11.26
  • Accepted : 2015.12.01
  • Published : 2015.12.30
Download PDF
⟨ Previous Next ⟩

Abstract

Under the conditional independence assumption among local features, the Naive Bayes Nearest Neighbor (NBNN) classifier has been recently proposed and performs classification without any training or quantization phases. While the original NBNN shows high classification accuracy without adopting an explicit training phase, the conditional independence among local features is against the compositionality of objects indicating that different, but related parts of an object appear together. As a result, the assumption of the conditional independence weakens the accuracy of classification techniques based on NBNN. In this work, we look into this issue, and propose a novel Bayesian network for an NBNN based classification to consider the conditional dependence among features. To achieve our goal, we extract a high-level feature and its corresponding, multiple low-level features for each image patch. We then represent them based on a simple, two-level layered Bayesian network, and design its classification function considering our Bayesian network. To achieve low memory requirement and fast query-time performance, we further optimize our representation and classification function, named relation-based Bayesian network, by considering and representing the relationship between a high-level feature and its low-level features into a compact relation vector, whose dimensionality is the same as the number of low-level features, e.g., four elements in our tests. We have demonstrated the benefits of our method over the original NBNN and its recent improvement, and local NBNN in two different benchmarks. Our method shows improved accuracy, up to 27% against the tested methods. This high accuracy is mainly due to consideration of the conditional dependences between high-level and its corresponding low-level features.

Keywords

References

  1. H. Zhang, A. Berg, M. Maire, and J. Malik, "SVM- KNN: discriminative nearest neighbor classification for visual category recognition," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), New York, NY, 2006, pp. 2126-2136.
  2. E. Alpaydin, Introduction to Machine Learning. Cambridge, MA: MIT Press, 2004.
  3. O. Boiman and E. Shechtman, and M. Irani, "In defense of nearest neighbor based image classification," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Anchorage, AK, 2008, pp. 1-8.
  4. R. Behmo, P. Marcombes, A. Dalalyan, and V. Prinet, "Towards optimal naive Bayes nearest neighbor," in Computer Vision-ECCV 2010, Heidelberg: Springer, pp. 171-184, 2010.
  5. Z. Wang, S. Gao, and L. T. Chia, "Learning class-to-image distance via large margin and l1-norm regularization," in Computer Vision-ECCV 2012, Heidelberg: Springer, pp. 230-244, 2012.
  6. T. Tuytelaars, M. Fritz, K. Saenko, and T. Darrell, "The NBNN kernel," in Proceedings of IEEE International Conference on Computer Vision (ICCV), Barcelona, Spain, 2011, pp. 1824-1831.
  7. S. McCann and D. G. Lowe, "Local naive Bayes nearest neighbor for image classification," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, 2012, pp. 3650-3656.
  8. A. Bosch, A. Zisserman, and X. Munoz, "Image classification using random forests and ferns," in Proceedings of 11th International Conference on Computer Vision (ICCV), Rio de Janeiro, Brazil, 2007, pp. 1-8.
  9. E. Bienenstock, S. Geman, and D. Potter, "Compositionality, MDL priors, and object recognition," Advances in Neural Information Processing Systems, vol. 9, pp. 838-844, 1997.
  10. M. Sahami, "Learning limited dependence Bayesian classifiers," in Proceedings of International Conference on Knowledge Discovery & Data Mining (KDD), Menlo Park, CA, 1996, pp. 335-338.
  11. G. F. Cooper, "The computational complexity of probabilistic inference using Bayesian belief networks," Artificial Intelligence, vol. 42, no. 2, pp. 393-405, 1990. https://doi.org/10.1016/0004-3702(90)90060-D
  12. S. N. Vitaladevuni, P. Natarajan, S. Wu, X. Zhuang, & R. Prasad, "Scene image categorization and video event detection using naive Bayes nearest neighbor," in Proceedings of IEEE Workshop on Applications of Computer-Vision (WACV), Tampa, FL, 2013, pp. 140-147.
  13. D. G. Lowe, "Distinctive image features from scale- invariant keypoints," International Journal of Computer Vision, vol. 60, no. 2, pp. 91-110, 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94
  14. R. E. Neapolitan, Learning Bayesian Networks. Upper Saddle River, NJ: Prentice Hall, 2003.
  15. D. D. Lewis, "Naive (Bayes) at forty: the independence assumption in information retrieval," in Proceedings of 10th European Conference on Machine Learning (ECML-98), Chemnitz, Germany, 1998, pp. 4-15.
  16. D. Heckerman, "A tutorial on learning with Bayesian networks," Microsoft Research, Redmond, WA, Tech. Rep. MSR-TR-95-06, 1995.
  17. D. Geiger, T. Vermas, and J. Pearl, "Identifying independence in Bayesian networks," Networks, vol. 20, no. 5, pp. 507-534, 1990. https://doi.org/10.1002/net.3230200504
  18. M. Muja and D. G. Lowe, "Scalable nearest neighbor algorithms for high dimensional data," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 11, pp. 2227-2240, 2014. https://doi.org/10.1109/TPAMI.2014.2321376
  19. J. P. Heo, Y. Lee, J. He, S. F. Chang, and S. E. Yoon, "Spherical hashing," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Providence, RI, 2012, pp. 2957-2964.
  20. J. P. Heo, Z. Lin, and S. E. Yoon, "Distance encoded product quantization," in Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Columbus, OH, 2014, pp. 2139-2146.
  21. D. Kim, J. Lee, J. Lee, I. Shin, J. H. Kim, and S. E. Yoon, "Scheduling in heterogeneous computing environments for proximity queries," IEEE Transactions on Visualization and Computer Graphics, vol. 19, no. 9, pp. 1513-1525, 2013. https://doi.org/10.1109/TVCG.2013.71