Browse > Article
http://dx.doi.org/10.3837/tiis.2022.03.008

Image Retrieval Based on the Weighted and Regional Integration of CNN Features  

Liao, Kaiyang (Xi'an University of Technology, School of Printing, Packaging and Digital Media)
Fan, Bing (Xi'an University of Technology, School of Printing, Packaging and Digital Media)
Zheng, Yuanlin (Xi'an University of Technology, School of Printing, Packaging and Digital Media)
Lin, Guangfeng (Xi'an University of Technology, School of Printing, Packaging and Digital Media)
Cao, Congjun (Xi'an University of Technology, School of Printing, Packaging and Digital Media)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.16, no.3, 2022 , pp. 894-907 More about this Journal
Abstract
The features extracted by convolutional neural networks are more descriptive of images than traditional features, and their convolutional layers are more suitable for retrieving images than are fully connected layers. The convolutional layer features will consume considerable time and memory if used directly to match an image. Therefore, this paper proposes a feature weighting and region integration method for convolutional layer features to form global feature vectors and subsequently use them for image matching. First, the 3D feature of the last convolutional layer is extracted, and the convolutional feature is subsequently weighted again to highlight the edge information and position information of the image. Next, we integrate several regional eigenvectors that are processed by sliding windows into a global eigenvector. Finally, the initial ranking of the retrieval is obtained by measuring the similarity of the query image and the test image using the cosine distance, and the final mean Average Precision (mAP) is obtained by using the extended query method for rearrangement. We conduct experiments using the Oxford5k and Paris6k datasets and their extended datasets, Paris106k and Oxford105k. These experimental results indicate that the global feature extracted by the new method can better describe an image.
Keywords
Image retrieval; Weighting feature; Global feature; Convolutional neural network; Regional integration;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 K. Ahmad, M. Sahu, M. Shrivastava, M. A. Rizvi, and V. Jain, "An efficient image retrieval tool: query based image management system," International Journal of Information Technology, vol. 12, no. 1, pp. 103-111, May. 2020.   DOI
2 B. Khaldi, O. Aiadi, and K. M. Lamine, "Image representation using complete multi-texton histogram," Multimedia Tools and Applications, vol. 79, no. 11, pp. 8267-8285, Jan. 2020.   DOI
3 P. Jitesh, A. K. Pal, H. Banka, and P. Dansena, "Fusion of region based extracted features for instance-and class-based CBIR applications," Applied Soft Computing, vol. 102, pp. 1-24, Apr. 2021.
4 A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet classification with deep convolutional neural networks," Communications of the ACM, vol. 60, no. 6, pp. 84-90, Jun. 2017.   DOI
5 J. Kim, H. Park, and J. I. Park, "CNN-based image steganalysis using additional data embedding," Multimedia Tools and Applications, vol. 79, no. 1, pp. 1355-1372, Oct. 2020.   DOI
6 X. D. Wang, Z. D. Zheng, Y. He, F. an, Z. Z. Zeng, and Y. Yang, "Progressive local filter pruning for image retrieval acceleration," arXiv preprint arXiv:2001.08878, Jan. 2020.
7 J. Philbin, O. Chum, M. Isard, S J. ivic, and A. Zisserman, "Lost in quantization: Improving particular object retrieval in large scale image databases," in Proc. of 2008 IEEE conference on computer vision and pattern recognition, pp. 1-8, Jun. 2008.
8 F. Wei, W. N. Yi, P. Lin, and S. N. Hou, "A Method of License Plate Location and Character Recognition based on CNN," KSII Transactions on Internet and Information Systems, vol. 14, no. 8, pp. 3488-3500, 2020.   DOI
9 C. Y. Kao, and S. A. Mohammadi, "Extremal rearrangement problems involving Poisson's equation with Robin boundary conditions," Journal of Scientific Computing, vol. 86, no. 3, pp. 1-28, 2021.   DOI
10 Q. Yan, L. Xu, J. Shi, and J. Jia, "Hierarchical saliency detection," in Proc. of the IEEE conference on computer vision and pattern recognition, pp. 1155-1162, 2013.
11 A. Raza, T. Nawaz, H. Dawood, and H. Dawood, "Square texton histogram features for image retrieval," Multimedia Tools and Applications, vol. 78, no. 3, pp. 2719-2746, Feb. 2019.   DOI
12 Y. Gong, L. Wang, R. Guo, and S. Lazebnik, "Multi-scale Orderless Pooling of Deep Convolutional Activation Features," in Proc. of European Conference on Computer Vision, Springer International Publishing, vol. 8695, pp. 392-407, 2014.
13 K. Simonyan, and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, Apr. 2014.
14 H. Jegou, M. Douze, and C. Schmid, "Improving bag-of-features for large scale image search," International journal of computer vision, vol. 87, no. 3, pp. 316-336, 2010.   DOI
15 A. S. Razavian, J. Sullivan, S. Carlsson, and A. Maki, "Visual instance retrieval with deep convolutional networks," ITE Transactions on Media Technology and Applications, vol. 4, no. 3, pp. 251-258, 2016.   DOI
16 Z. Yang, X. Gao, Z. Xie, and K. W. Wu, "Scene categorization of local Gist feature match kernel," Chinese Journal of Graphics, vol. 18, no. 3, pp. 264-270, Mar. 2013.
17 J. Chaki, N. Dey, Texture feature extraction techniques for image recognition, Springer, Singapore, 2020.
18 J. Sivic, and A. Zisserman, "Video Google: A text retrieval approach to object matching in videos," in Proc. of Ninth IEEE International Conference on Computer Vision, vol. 2, pp. 1470-1477, Oct. 2003.
19 Q. Liu, X.Y. Xiang, J. H. Qin, Y. Tan, and Q. Zhang, "Reversible Sub-Feature Retrieval: Toward Robust Coverless Image Steganography for Geometric Attacks Resistance," KSII Transactions on Internet and Information Systems, vol. 15, no. 3, pp. 1078-1099, Feb. 2021.
20 A. Vedaldi, and K. Lenc, "Matconvnet: Convolutional neural networks for matlab," in Proc. of the 23rd ACM international conference on Multimedia, pp. 689-692, Oct. 2015.
21 J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Object retrieval with large vocabularies and fast spatial matching," in Proc. of 2007 IEEE conference on computer vision and pattern recognition, pp. 1-8, Jun. 2007.
22 Y. Kalantidis, C. Mellina, and S. Osindero, "Cross-dimensional weighting for aggregated deep convolutional features," in Proc. of European conference on computer vision, pp. 685-701, Oct. 2016.
23 G. Tolias, R. Sicre, and H. Jegou, "Particular object retrieval with integral max-pooling of CNN activations," arXiv preprint arXiv:1511.05879, Nov. 2015.
24 D. G. Lowe, "Local feature view clustering for 3D object recognition," in Proc. of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. 291-301, Dec. 2001.
25 F. Perronnin, Y. Liu, J. Sanchez, and H. Poirier, "Large-scale image retrieval with compressed fisher vectors," in Proc. of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 3384-3391, Jun. 2010.
26 H. Jegou, and A. Zisserman, "Triangulation embedding and democratic aggregation for image search," in Proc. of the IEEE conference on computer vision and pattern recognition, pp. 3310-3317, 2014.
27 A. Babenko, A. Slesarev, A. Chigorin, and V. Lempitsky, "Neural codes for image retrieval," in Proc. of European conference on computer vision, Springer, Cham, pp. 584-599, Sep. 2014.
28 F. Chen, Z. H. Lv, J. Li, X. D. Wang, and Y. Dou, "Multi-Label Image Retrieval by Hashing with Object Proposal," Chinese Journal of Image Graphics, vol. 22, no. 2, pp. 232-240, 2017.
29 A. Babenko, and V. Lempitsky, "Aggregating deep convolutional features for image retrieval," in Proc. of the IEEE conference on computer vision and pattern recognition, arXiv preprint arXiv:1510.07493, 2015.
30 H. Jegou, M. Douze, C. Schmid, and P. Perez, "Aggregating local descriptors into a compact image representation," in Proc. of 2010 IEEE computer society conference on computer vision and pattern recognition, pp. 3304-3311, Jun. 2010.