Browse > Article
http://dx.doi.org/10.3837/tiis.2021.03.010

A New Three-dimensional Integrated Multi-index Method for CBIR System  

Zhang, Mingzhu (School of Technology and Engineering, Xi'an Fanyi University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.15, no.3, 2021 , pp. 993-1014 More about this Journal
Abstract
This paper proposes a new image retrieval method called the 3D integrated multi-index to fuse SIFT (Scale Invariant Feature Transform) visual words with other features at the indexing level. The advantage of the 3D integrated multi-index is that it can produce finer subdivisions in the search space. Compared with the inverted indices of medium-sized codebook, the proposed method increases time slightly in preprocessing and querying. Particularly, the SIFT, contour and colour features are fused into the integrated multi-index, and the joint cooperation of complementary features significantly reduces the impact of false positive matches, so that effective image retrieval can be achieved. Extensive experiments on five benchmark datasets show that the 3D integrated multi-index significantly improves the retrieval accuracy. While compared with other methods, it requires an acceptable memory usage and query time. Importantly, we show that the 3D integrated multi-index is well complementary to many prior techniques, which make our method compared favorably with the state-of-the-arts.
Keywords
Multi-index; Content-based Image Retrieval; Feature Fusion; Indexing Strategy;
Citations & Related Records
연도 인용수 순위
  • Reference
1 K. Liao, F. Zhao, Y. Zheng, C. Cao, and M. Zhang, "Parallel N-Path Quantification Hierarchical K-Means Clustering Algorithm for Video Retrieval," International Journal of Pattern Recognition and Artificial Intelligence, vol. 31, no. 9, Sep, 2017.
2 Y. Rao, and W. Liu, "Region Division for Large-scale Image Retrieval," KSII Transactions on Internet and Information Systems, vol. 13, no. 10, pp. 5197-5218, Oct. 2019.   DOI
3 K. Liao and G. Liu, "An efficient content based video copy detection using the sample based hierarchical adaptive k-means clustering," Journal of Intelligent Information Systems, vol. 44, pp. 133-158, Feb. 2015.   DOI
4 J. Lu, V. E. Liong, and J. Zhou, "Simultaneous Local Binary Feature Learning and Encoding for Homogeneous and Heterogeneous Face Recognition," IEEE Transactions on Pattern Analysis And Machine Intelligence, vol. 40, no. 8, pp. 1979-1993, Aug. 2018.   DOI
5 Q. Thuy, Q. Huu, C. P. Van, and T. N. Quoc, "An efficient semantic - Related image retrieval method," Expert Systems with Applications, vol. 72, pp. 30-41, 2017.   DOI
6 C. Wu, H. Zhang, J. Hua, S. Hua, Y. Zhang, X. Lu, and Y. Tang, "A Novel Least Square and Image Rotation based Method for Solving the Inclination Problem of License Plate in Its Camera Captured Image," KSII Transactions on Internet and Information Systems, vol. 13, no. 12, pp. 5990-6008, Dec. 31, 2019.   DOI
7 X. Qian, H. Wang, Y. Zhao, X. Hou, R. Hong, M. Wang, and Y. Y. Tnag, "Image Location Inference by Multi-Saliency Enhancement," IEEE Transactions on Multimedia, vol. 19, no. 4, pp. 813-821, 2017.   DOI
8 K. T. Ahmed, S. Ummesafi, and A. Iqbal, "Content based image retrieval using image features information fusion," Information Fusion, vol. 51, pp. 76-99, Nov. 2019.   DOI
9 J. Liu, Z. Huang, H. Cai, H. T. Shen, C. W. Ngo, and W. Wang, "Near-duplicate video retrieval: Current research and future trends," ACM Computing Surveys(CSUR), vol. 45, no. 4, 2013.
10 H. Jegou, M. Douze, and C. Schmid, "Improving Bag-of-Features for Large Scale Image Search," International Journal of Computer Vision, vol. 87, no. 3, pp. 316-336, 2010.   DOI
11 C. Bin, A. thung, X. Zhang, and Z. Zhao, "Multiple feature fusion for social media applications," in Proc. of International Conference on Management of Data, pp. 435-446, 2010.
12 C. Wengert, M. Douze, and H. Jegou, "Bag-of-colors for improved image search," in Proc. of the 19th ACM International Conference on Multimedia ACM Multimedia, pp. 1437-1440, 2011.
13 P. Jain, B. Kulis, and K. Grauman, "Fast image search for learned metrics," in Proc. of Computer Vision and Pattern Recognition, pp. 1-8, 2008.
14 F. S. Khan, R. M. Anwer, J. Weijer, A. D. Bagdanov, M. Vanrell, and A. M. Lopez, "Color attributes for object detection," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3306-3313, 2012.
15 P. Li, M. Wang, J. Cheng, C. Xu, and H. Lu, "Spectral Hashing With Semantically Consistent Graph for Image Indexing," IEEE Transactions on Multimedia, vol. 15, no. 1, pp. 141-152, 2013.   DOI
16 P. Gehler and S. Nowozin, "On Feature Combination for Multiclass Object Classification," in Proc. of IEEE 12th International Conference on Computer Vision, pp. 221-228, 2009.
17 A. Oliva and A. Torralba, "Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope," International Journal of Computer Vision, vol. 42, no. 3, pp. 145-175, 2001.   DOI
18 Y. Gao, M. Shi, D. Tao, and C. Xu, "Database Saliency for Fast Image Retrieval," IEEE Transactions on Multimedia, vol. 17, no. 3, pp. 359-369, 2015.   DOI
19 D. Qin, S. Gammeter, L. Bossard, T. Quack, and L. Gool, "Hello neighbor: accurate object retrieval with k-reciprocal nearest neighbors," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 777-784. 2011.
20 Z. Gao, J. Xue, W. Zhou, S. Pang, and Q. Tian, "Democratic Diffusion Aggregation for Image Retrieval," IEEE Transactions on Multimedia, vol. 18, no. 8, pp.1661-1674, 2016.   DOI
21 D. Qin, C. Wengert, and L.Gool, "Query Adaptive Similarity for Large Scale Object Retrieval," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1610-1617, 2013.
22 C. Deng, R. Ji, W. Liu, D. Tao, and X. Gao, "Visual Reranking through Weakly Supervised Multi-Graph Learning," in Proc of IEEE International Conference on Computer Vision, pp. 2600-2607, 2013.
23 J. Song, T. He, L. Gao, X. Xu, A. Hanjalic, and H. T. Shen, "Unified Binary Generative Adversarial Network for Image Retrieval and Compression," International Journal of Computer Vision, vol. 128, no. 8, pp. 2243-2264, Sep. 2020.   DOI
24 J. Weijer, C. Schmid, J. Verbeek, and D. Larlus, "Learning Color Names for Real-World Applications," IEEE Transactions on Image Processing, vol. 18, no. 7, pp. 1512-1523, 2009.   DOI
25 R. Arandjelovic and A. Zisserman, "Three things everyone should know to improve object retrieval," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2911-2918, 2012.
26 B. K. Iwana, and S. Uchida, "Time series classification using local distance-based features in multi-modal fusion networks," Pattern Recognition, vol. 97, Jan. 2020.
27 S. Y. Jeong, and W. H. Kim, "Thermal Imaging Fire Detection Algorithm with Minimal False Detection," KSII Transactions on Internet and Information Systems, vol. 14, no. 5, pp. 2156-2170, May 31, 2020.   DOI
28 K. Liao, H. Lei, Y. Zheng, G. Lin, C. Cao, M. Zhang, and J. Ding, "IR Feature Embedded BOF Indexing Method for Near-Duplicate Video Retrieval," IEEE Transactions on Circuits and Systems for Video Technology, vol. 29, no. 12, pp. 3743-3753, Dec, 2019.   DOI
29 J. Sivic and A. Zisserman, "Efficient Visual Search of Videos Cast as Text Retrieval," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 4, pp. 591-606, 2009.   DOI
30 J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Object retrieval with large vocabularies and fast spatial matching," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2007.
31 D. Nister and H. Stewenius, "Scalable recognition with a vocabulary tree," in Proc. of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 2161-2168, 2006.
32 A. Babenko and V. Lempitsky, "The Inverted Multi-Index," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 6, pp. 1247-1260, 2015.   DOI
33 Z. Wu, S. Jiang, and Q. Huang, "Near-duplicate video matching with transformation recognition," in Proc. of the 17th ACM International Conference on Multimedia, pp. 549-552, 2009.
34 X. Yang, X. Qian, and Y. Xue, "Scalable Mobile Image Retrieval by Exploring Contextual Saliency," IEEE Transactions on Image Processing, vol. 24, no. 6, pp. 1709-1721, 2015.   DOI
35 L. Zheng, S. Wang, and Q. Tian, "Coupled Binary Embedding for Large-Scale Image Retrieval," IEEE Transactions on Image Processing, vol. 23, no. 8, pp. 3368-3380, 2014.   DOI
36 Z. A. Abduljabbar, A. Ibrahim, M. A. Hussain, Z. A. Hussien, M. A. Sibahee, and S. Lu, "EEIRI: Efficient Encrypted Image Retrieval in IoT-Cloud," KSII Transactions on Internet And Information Systems, vol. 13, no. 11, pp. 5692-5716, Nov. 2019.   DOI
37 L. Gao, X. Zhu, J. Song, Z. Zhao, and H. T. Shen, "Beyond product quantization: Deep progressive quantization for image retrieval," in Proc. of IJCAI International Joint Conference on Artificial Intelligence, pp. 723-729.
38 L. Zheng, S. J. Wang, Z. Q. Liu, and Q. Tian, "Packing and Padding: Coupled Multi-index for Accurate Image Retrieval," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1947-1954, 2014.
39 L. Zheng, S. Wang, Z. Liu, and Q. Tian, "L-p-norm IDF for Large Scale Image Search," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1626-1633, 2013.
40 L. Zheng, S. Wang, L. Tian, H. Fei, Z. Liu, and Q. Tian, "Query-adaptive late fusion for image search and person re-identification," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1741-1750, 2015.
41 J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Lost in quantization: Improving particular object retrieval in large scale image databases," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 1-8, 2008.
42 X. Wang, M. Yang, T. Cour, S. Zhu, K. Yu, and T. X. Han, "Contextual Weighting for Vocabulary Tree based Image Retrieval," in Proc. of IEEE International Conference on Computer Vision, pp. 209-216, 2011.
43 X. Shen, Z. Lin, J. Brandt, S. Avidan, and Y. Wu, "Object retrieval and localization with spatially-constrained similarity measure and k-NN re-ranking," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition, pp. 3013-3020, 2012.
44 H. Jegou, M. Douze, and C. Schmid, "Hamming embedding and weak geometric consistency for large scale image search," in Proc. of European Conference on Computer Vision, vol. 5302, pp. 304-317, 2018.
45 S. Zhang, M. Yang, T. Cour, K. Yu, and D. N. Metaxas, "Query Specific Rank Fusion for Image Retrieval," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 4, pp. 803-815, 2015.   DOI
46 C. G. M. Snoek and M. Worring, "Multimedia event-based video indexing using time intervals," IEEE Transactions on Multimedia, vol. 7, no. 4, pp. 638-647, 2005.   DOI
47 S. Zhang, M. Yang, X. Wang, Y. Lin, and Q. Tian, "Semantic-Aware Co-Indexing for Image Retrieval," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 37, no. 12, pp. 2573-2587, 2015.   DOI
48 S. Park, W. Jeong, and Y. S. Moon, "X-ray Image Segmentation using Multi-task Learning," KSII Transactions on Internet and Information Systems, vol. 14, no. 3, pp. 1104-1120, Mar. 2020.   DOI