KSII Transactions on Internet and Information Systems (TIIS)

Korean Society for Internet Information (한국인터넷정보학회)
권호 표지
DOI QR코드

DOI QR Code

Learning Discriminative Fisher Kernel for Image Retrieval

  • Wang, Bin (Department of Automation, Shanghai Jiao Tong University) ;
  • Li, Xiong (Department of Automation, Shanghai Jiao Tong University) ;
  • Liu, Yuncai (Department of Automation, Shanghai Jiao Tong University)
  • Received : 2012.12.10
  • Accepted : 2013.03.04
  • Published : 2013.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

Content based image retrieval has become an increasingly important research topic for its wide application. It is highly challenging when facing to large-scale database with large variance. The retrieval systems rely on a key component, the predefined or learned similarity measures over images. We note that, the similarity measures can be potential improved if the data distribution information is exploited using a more sophisticated way. In this paper, we propose a similarity measure learning approach for image retrieval. The similarity measure, so called Fisher kernel, is derived from the probabilistic distribution of images and is the function over observed data, hidden variable and model parameters, where the hidden variables encode high level information which are powerful in discrimination and are failed to be exploited in previous methods. We further propose a discriminative learning method for the similarity measure, i.e., encouraging the learned similarity to take a large value for a pair of images with the same label and to take a small value for a pair of images with distinct labels. The learned similarity measure, fully exploiting the data distribution, is well adapted to dataset and would improve the retrieval system. We evaluate the proposed method on Corel-1000, Corel5k, Caltech101 and MIRFlickr 25,000 databases. The results show the competitive performance of the proposed method.

Keywords

References

  1. F. Faria, A. Veloso, H. Almeida, E. Valle, R. Torres, M. Goncalves and W. Meira Jr, "Learning to rank for content-based image retrieval," in Proc. of the international conference on Multimedia information retrieval, ACM, pp. 285-294, 2010.
  2. M. Arevalillo-Herraez, F. Ferri and J. Domingo, "A naive relevance feedback model for content-based image retrieval using multiple similarity measures," Pattern Recognition, vol. 43, no. 3, pp. 619-629, 2010. https://doi.org/10.1016/j.patcog.2009.08.010
  3. L. Yang, R. Jin, L. Mummert, R. Sukthankar, A. Goode, B. Zheng, S. Hoi and M. Satya-narayanan, "A boosting framework for visuality-preserving distance metric learning and its application to medical image retrieval," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 1, pp. 30-44, 2010. https://doi.org/10.1109/TPAMI.2008.273
  4. S. Hoi, W. Liu and S. Chang, "Semi-supervised distance metric learning for collaborative image retrieval and clustering," ACM Transactions on Multimedia Computing, Communications and Applications (TOMCCAP), vol. 6, no. 3, 18, 2010.
  5. A. Smeulders, M. Worring, S. Santini, A. Gupta and R. Jain, "Content-based image retrieval at the end of the early years," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 12, pp. 1349-1380, 2000. https://doi.org/10.1109/34.895972
  6. J. Wang, J. Yang, K. Yu, F. Lv, T. Huang and Y. Gong, "Locality-constrained linear coding for image classification," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3360-3367, 2010.
  7. S. Roweis and L. Saul, "Nonlinear dimensionality reduction by locally linear embedding," Science , vol. 290, no. 5500, pp. 2323-2326, 2000. https://doi.org/10.1126/science.290.5500.2323
  8. J. Goldberger, S. Roweis, G. Hinton and R. Salakhutdinov, "Neighborhood components analysis."
  9. L. Yang, R. Jin, R. Sukthankar and Y. Liu, "An efficient algorithm for local distance metric learning," in Proc. of the National Conference on Artificial Intelligence, Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press. vol. 21, pp. 543, 2006.
  10. A. Webb, "Statistical pattern recognition," Wiley, 2003.
  11. K. Weinberger and L. Saul, "Distance metric learning for large margin nearest neighbor classification," The Journal of Machine Learning Research, vol. 10, pp. 207-244, 2009.
  12. E. Xing, A. Ng, M. Jordan and S. Russell, "Distance metric learning, with application to clustering with side-information," Advances in neural information processing systems, vol. 15, pp.505-512, 2002.
  13. A. Bar-Hillel, T. Hertz, N. Shental and D. Weinshall, "Learning distance functions using equivalence relations," in Proc. of Machine Learning-International Work Shop Then Conference-, vol. 20, pp. 11,2003.
  14. S. Hoi, W. Liu, M. Lyu and W. Ma, "Learning distance metrics with contextual constraints for image retrieval," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 2, pp. 2072-2078, 2006.
  15. T. Kim, S. Wong, B. Stenger, J. Kittler and R. Cipolla, "Incremental linear discriminant analysis using sufficient spanning set approximations," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2007.
  16. A. Frome, Y. Singer and J. Malik, "Image retrieval and classification using local distance functions," in 19: in Proc. of the 2006 Conference Advances in Neural Information Processing Systems, vol. 19, MIT Press, pp. 417, 2007.
  17. J. Su, W. Huang, P. Yu and V. Tseng, "Efficient relevance feedback for content-based image retrieval by mining user navigation patterns," IEEE Transactions on Knowledge and Data Engineering, vol. 23, no. 3, pp. 360-372, 2011. https://doi.org/10.1109/TKDE.2010.124
  18. C. Ferreira, J. Santos, R. da S Torres, M. Goncalves, R. Rezende and W. Fan, "Relevance feedback based on genetic programming for image retrieval," Pattern Recognition Letters, vol. 32, no. 1, pp. 27-37, 2011. https://doi.org/10.1016/j.patrec.2010.05.015
  19. H. Cai, K. Mikolajczyk and J. Matas, "Learning linear discriminant projections for dimensionality reduction of image descriptors," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 2, pp. 338-352, 2011. https://doi.org/10.1109/TPAMI.2010.89
  20. J. Dillon, Y. Mao, G. Lebanon and J. Zhang, "Statistical translation, heat kernels and expected distances," arXiv preprint arXiv:1206.5248, 2012.
  21. L. Yang, R. Jin and R. Sukthankar, "Bayesian active distance metric learning," arXiv preprint arXiv:1206.5283, 2012.
  22. H. Chang and D. Yeung, "Kernel-based distance metric learning for content-based image retrieval," Image and Vision Computing 25 (5) (2007), pp. 695-703. https://doi.org/10.1016/j.imavis.2006.05.013
  23. J. Caicedo, J. BenAbdallah, F. González and O. Nasraoui, "Multimodal representation, indexing, automated annotation and retrieval of image collections via non-negative matrix factorization," Neurocomputing , vol. 76, no. 1, pp. 50-60, 2012. https://doi.org/10.1016/j.neucom.2011.04.037
  24. R. Vieux, J. Benois-Pineau and J. Domenger, "Content based image retrieval using bag of-regions," Advances in Multimedia Modeling, pp. 507-517, 2012.
  25. K. Van De Sande, T. Gevers, C. Snoek, "Evaluating color descriptors for object and scene recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 9, pp. 1582-1596, 2010. https://doi.org/10.1109/TPAMI.2009.154
  26. A. Bar-Hillel, T. Hertz, N. Shental and D. Weinshall, "Learning a Mahalanobis metric from equivalence constraints," Journal of Machine Learning Research, vol. 6, no. 1, pp. 937, 2006.
  27. S. Hoi, M. Lyu, R. Jin, "A unified log-based relevance feedback scheme for image retrieval," IEEE Transactions on Knowledge and Data Engineering, vol. 18, no. 4, pp. 509-524, 2006. https://doi.org/10.1109/TKDE.2006.1599389
  28. K. Chatfield, V. Lempitsky, A. Vedaldi and A. Zisserman, "The devil is in the details: an evaluation of recent feature encoding methods," in Proc. of British Machine Vision Conference, 2011.
  29. T. Jaakkola and D. Haussler, "Exploiting generative models in discriminative classifiers," In NIPS, pp. 487-493, 1999.
  30. M. Jordan, Z. Ghahramani, T. Jaakkola and S. Lawrence, "Introduction to variational methods for graphical models," Machine Learning, 37, pp. 183-233, 1999. https://doi.org/10.1023/A:1007665907178
  31. der Maaten and Laurens Van, "Learning discriminative fisher kernels," In ICML, pp. 217-224, 2011.
  32. J. Friedman, T. Hastie, and R. Tibshirani, "The Elements of Statistical Learning," Springer, 2008.
  33. P. Jain, B. Kulis, J. Davis and I. Dhillon, "Metric and kernel learning using a linear transformation," The Journal of Machine Learning Research, vol. 13, pp. 519-547, 2012.
  34. S. Xiang, F. Nie, C. Zhang, "Learning a mahalanobis distance metric for data clustering and classification," Pattern Recognition, vol. 41, no. 12, pp. 3600-3612, 2008. https://doi.org/10.1016/j.patcog.2008.05.018
  35. H. Becker, M. Naaman and L. Gravano, "Learning similarity metrics for event identification in social media," in Proceedings of the third ACM international conference on Web search and data mining, ACM, pp. 291-300, 2010.
  36. S. Cao and N. Snavely, "Learning to match images in large-scale collections," in Computer Vision-ECCV 2012. Workshops and Demonstrations, Springer, pp. 259-270, 2012.
  37. J. Kim, C. Shen and L. Wang, "A scalable algorithm for learning a Mahalanobis Distance Metric," in ACCV, 2010.
  38. Y. Ying and P. Li, "Distance metric learning with eigenvalue optimization," The Journal of Machine Learning Research , vol. 13, pp. 1-26, 2012.
  39. T. Jebara, R. Kondor, A. Howard, "Probability product kernels," Journal of Machine Learning Research, vol. 5, pp. 819-844, 2004.
  40. N. Vasconcelos, "On the efficient evaluation of probabilistic similarity functions for image retrieval," IEEE Trans. on Information Theory, vol. 50, no. 7, pp. 1482-1496, 2004. https://doi.org/10.1109/TIT.2004.830760
  41. C. Schmid, "Constructing models for content-based image retrieval," In CVPR 2001.
  42. A Perina, M. Cristani, U. Castellani, V. Murino, N. Jojic, "Free energy score spaces: using generative information in discriminative classifiers," IEEE Trans. on PAMI, 2011.
  43. X. Li, T.S. Lee, Y. Liu, "Hybrid generative-discriminative classification using posterior divergence," In CVPR 2011.
  44. J.C. Caicedo, J. BenAbdallah, F.A. Gonzalez, O. Nasraoui, "Multimodal representation, indexing, automated annotation and retrieval of image collections via non-negative matrix factorization," Neurocomputing, vol. 76, no.1, pp. 50-60, 2012. https://doi.org/10.1016/j.neucom.2011.04.037