Browse > Article
http://dx.doi.org/10.3837/tiis.2014.01.015

Medical Image Retrieval with Relevance Feedback via Pairwise Constraint Propagation  

Wu, Menglin (School of Computer Science and Engineering, Nanjing University of Science and Technology)
Chen, Qiang (School of Computer Science and Engineering, Nanjing University of Science and Technology)
Sun, Quansen (School of Computer Science and Engineering, Nanjing University of Science and Technology)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.8, no.1, 2014 , pp. 249-268 More about this Journal
Abstract
Relevance feedback is an effective tool to bridge the gap between superficial image contents and medically-relevant sense in content-based medical image retrieval. In this paper, we propose an interactive medical image search framework based on pairwise constraint propagation. The basic idea is to obtain pairwise constraints from user feedback and propagate them to the entire image set to reconstruct the similarity matrix, and then rank medical images on this new manifold. In contrast to most of the algorithms that only concern manifold structure, the proposed method integrates pairwise constraint information in a feedback procedure and resolves the small sample size and the asymmetrical training typically in relevance feedback. We also introduce a long-term feedback strategy for our retrieval tasks. Experiments on two medical image datasets indicate the proposed approach can significantly improve the performance of medical image retrieval. The experiments also indicate that the proposed approach outperforms previous relevance feedback models.
Keywords
Content-based medical image retrieval; manifold ranking; relevance feedback; pairwise constraint propagation; long-term feedback;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Z. Fu, H. Lp, H. Lu and Z. Lu, "Multi-modal constraint propagation for heterogeneous image clustering," in Proc. of 19th ACM Int. Conf. on Multimedia, pp. 143-152, November 28 - December 1, 2011.
2 B. Thomee and M.S. Lew, "Interactive search in image retrieval: a survey," International Journal of Multimedia Information Retrieval, vol. 1, no. 2, pp. 71-86, July, 2012.   DOI
3 H. Muller, W. Muller, S. Marchand-Maillet, T. Pun and D.M. Squire, "Strategies for positive and negative relevance feedback in image retrieval," in Proc. of 15th Int. Conf. on Pattern Recognition, pp. 1043-1046, September 3-7, 2000.
4 G. Das, S. Ray and C. Wilson, "Feature re-weighting in content-based image retrieval," in Proc. of 5th Int. Conf. on Image and Video Retrieval, pp. 193-200, July 13-15, 2006.
5 S. Tong and E. Chang, "Support vector machine active learning for image retrieval," in Proc. of 9th ACM Int. Conf. on Multimedia, pp. 107-118, September 30 - October 5, 2001.
6 C.H. Hoi, C.H. Chan, K.Z. Huang, M.R. Lyu and I. King. "Biased support vector machine for relevance feedback in image retrieval," in Proc. of IEEE Int. Conf. on Neural Networks, pp. 3189-3194, July 25-29, 2004.
7 D. Tao, X. Tang, X. Li and X. Wu, "Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval," IEEE Pattern Analysis and Machine Intelligence, vol. 28, no. 7, pp. 1088-1099, July, 2006.   DOI   ScienceOn
8 J. Ye, R. Janardan and Q. Li, "Two-dimensional linear discriminant analysis," in Proc. of the Advances in Neural Information Processing Systems, pp. 1569-1576, December 13-18, 2004.
9 H. Muller, N. Michoux, D. Bandon, and A. Geissbuhler, "A review of content-based image retrieval systems in medical applications-clinical benefits and future directions," International Journal of Medical Informatics, vol. 73, pp. 1-23, February, 2004.   DOI   ScienceOn
10 L. Yang, R. Jin, L. Mummert, R. Sukthankar, A. Goode, B. Zheng S. Hoi and M. Satya-narayanan, "A boosting framework for visuality-preserving distance metric learning and its application to medical image retrieval," IEEE Pattern Analysis and Machine Intelligence, vol. 32, no. 1, pp. 30-44, January, 2010.   DOI   ScienceOn
11 U. Avni, H. Greenspan, E. Konen, M. Sharon and J. Goldberger, "X-ray categorization and retrieval on the organ and pathology level, using patch-based visual words," IEEE Medical Imaging, vol. 30, no. 3, pp. 733-746, March, 2011.   DOI   ScienceOn
12 G. Quellec, M. Lamard, G. Cazuguel, "Case retrieval in medical databases by fusing heterogeneous information," IEEE Medical Imaging, vol. 30, no. 1, pp. 108-118, January, 2011.   DOI   ScienceOn
13 M.M. Rahman, S.K. Antani, and G.R. Thoma, "A learning-based similarity fusion and filtering approach for biomedical Image retrieval using SVM classification and relevance feedback," IEEE Information Technology in Biomedicine, vol. 15, no. 4, pp. 640-646, July, 2011.   DOI   ScienceOn
14 H.W. Chia, Y Li, C. Li. "Effective extraction of Gabor features for adaptive mammogram retrieval," in Proc. of IEEE Conf. on Multimedia & Expo 2007, pp. 1053-1056, July 2-5, 2007.
15 Y. Rui , T.S. Huang, M. Orgega and S. Mehrotra, "Relevance feedback: a power tool for interactive content-based image retrieval," IEEE Circuits and Systems for Video Technology, vol. 8, no. 5, pp. 644-655, September, 1998.   DOI   ScienceOn
16 D. Zhou, O. Bousquet, T.N. Lal, J. Weston and B. Scholkopf. "Learning with local and global consistency," in Proc. of the Advances in Neural Information Processing Systems, pp. 321-328, December 8-13, 2003.
17 X. Zhu, Z. Ghahramani and J. Lafferty. "Semi-supervised learning using gaussian fields and harmonic functions," in Proc. of 20th Int. Conf. on Machine Learning, pp. 912-919, August 21-24, 2003.
18 T. Tommasi, B. Caputo, P. Welter, M.O. Guld and T.M. Deserno, "Overview of the CLEF 2009 medical image annotation track," in Proc. of 10th Int. Conf. on Cross-language evaluation forum, pp. 85-93, September 30- October 2, 2009.
19 J. Oliveira, A. Machado, G. Chavez, A. Lopes, T.M. Deserno and A. Araujo, "MammoSys: a content-based image retrieval system using breast density patterns," Computer Methods and Programs in Biomedicine, vol. 99, no. 3, pp. 289-297, September, 2010.   DOI   ScienceOn
20 D. Xu, S. Yan, D. Tao, S. Lin and H.J. Zhang, "Marginal Fisher analysis and its variants for human gait recognition and content- based image retrieval," IEEE Image Processing, vol. 16, no. 11, pp. 2811-2821, November, 2007.   DOI   ScienceOn
21 X. He, D. Cai and J. Han. "Learning a maximum margin subspace for image retrieval," IEEE Knowledge and Data Engineering, vol. 20, no. 2, pp. 189-201, February, 2008.   DOI   ScienceOn
22 E.D. Vesa, J. Domingob, G. Ayalac and P. Zuccarello, "A novel Bayesian framework for relevance feedback in image content-based retrieval systems," Pattern Recognition, vol. 39, no. 9, pp. 1622-1632, September, 2006.   DOI
23 G. Giacinto and F. Roli. "Instance-based relevance feedback in image retrieval using dissimilarity spaces," Case-Based Reasoning on Images and Signals, vol. 73, no. 1, pp. 419-436, January, 2008.   DOI
24 M.A. Herraez and F.J. Ferri, "An improved distance-based relevance feedback strategy for image retrieval," Image and Vision Computing, vol. 31, no. 10, pp. 704-713, October, 2013.   DOI   ScienceOn
25 Z.H. Zhou, K.J Chen and H.B. Dai. "Enhancing relevance feedback in image retrieval using unlabeled data," ACM Information Systems, vol. 24, no. 2, pp. 219-244, April, 2006.   DOI   ScienceOn
26 T. Amin, M. Zeytinoglu and L. Guan, "Application of Laplacian mixture model to image and video retrieval," IEEE Multimedia, vol. 9, no. 7, pp. 1416-1429, November, 2007.   DOI   ScienceOn
27 M. Arevalillo-Herraeza, M. Zacaresb, X. Benaventc and E.D. Vesa, "A relevance feedback CBIR algorithm based on fuzzy sets," Signal Processing: Image Communication, vol. 23, no. 7, pp. 490-504, August, 2008.   DOI   ScienceOn
28 H. Chang, D.Y. Yeung, "Kernel-based distance metric learning for content-based image retrieval," Image and Vision Computing, vol. 25, no. 5, pp. 695-703, May, 2007.   DOI   ScienceOn
29 L. Zhang, L. Wang and W. Lin, "Semi-supervised biased maximum margin analysis for interactive image retrieval," IEEE Image Processing, vol. 21, no. 4, pp. 2294-2308, April, 2012.   DOI   ScienceOn
30 J. He, M. Li, H. Zhang and H. Tong, "Generalized manifold-ranking-based image retrieval," IEEE Image Processing, vol. 15, no. 10, pp. 3170-3177, October, 2006.   DOI   ScienceOn
31 S.R. Bulo, M. Rabbi and M. Pelillo, "Content-based image retrieval with relevance feedback using random walks," Pattern Recognition, vol. 44, no. 9, pp. 2109-2122, September, 2011.   DOI   ScienceOn
32 P. Yin, B. Bhanu, K. Chang and A. Dong, "Long-term cross-session relevance feedback using virtual features," IEEE Knowledge and Data Engineering, vol. 20, no. 3, pp. 352-368, March, 2008.   DOI   ScienceOn
33 Z. Li, J. Liu and X. Tang, "Pairwise constraint propagation by semidefinite programming for semi-supervised classification," in Proc. of 25th Int. Conf. on Machine learning, pp. 576-583, July 5-9, 2008.
34 Z. Lu and Y. Peng, "Exhaustive and Efficient Constraint Propagation: A Semi-Supervised Learning Perspective and Its Applications," International Journal of Computer Vision, vol. 103, no. 3, pp. 306-325, July, 2013.   DOI   ScienceOn
35 S.C.H Hoi, M.R. Lyu and R Jin, "A unified log-based relevance feedback scheme for image retrieval," IEEE Knowledge and Data Engineering, vol. 18, no. 4, pp. 509-524, April, 2006.   DOI   ScienceOn
36 J. Laaksonen, M. Koskela and E. Oja, "PicSOM: self-organizing image retrieval with MPEG-7 content descriptors," IEEE Neural Networks, vol. 13, no. 4, pp. 841-853, July, 2002.   DOI   ScienceOn