ETRI Journal

Electronics and Telecommunications Research Institute (한국전자통신연구원)
권호 표지
DOI QR코드

DOI QR Code

Fast and Accurate Visual Place Recognition Using Street-View Images

  • Received : 2016.04.15
  • Accepted : 2016.11.29
  • Published : 2017.02.01
Download PDF
⟨ Previous Next ⟩

Abstract

A fast and accurate building-level visual place recognition method built on an image-retrieval scheme using street-view images is proposed. Reference images generated from street-view images usually depict multiple buildings and confusing regions, such as roads, sky, and vehicles, which degrades retrieval accuracy and causes matching ambiguity. The proposed practical database refinement method uses informative reference image and keypoint selection. For database refinement, the method uses a spatial layout of the buildings in the reference image, specifically a building-identification mask image, which is obtained from a prebuilt three-dimensional model of the site. A global-positioning-system-aware retrieval structure is incorporated in it. To evaluate the method, we constructed a dataset over an area of $0.26km^2$. It was comprised of 38,700 reference images and corresponding building-identification mask images. The proposed method removed 25% of the database images using informative reference image selection. It achieved 85.6% recall of the top five candidates in 1.25 s of full processing. The method thus achieved high accuracy at a low computational complexity.

Keywords

References

  1. D.M. Chen et al., "City-Scale Landmark Identification on Mobile Devices," IEEE Conf. Comput. Vis. Pattern Recog., Colorado Springs, CO, USA, June 20-25, 2011, pp. 737-744.
  2. A. Torii et al., "Visual Place Recognition with Repetitive Structures," IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 11, Nov. 2015, pp. 2346-2359. https://doi.org/10.1109/TPAMI.2015.2409868
  3. J. Knopp, J. Sivic, and T. Pajdla, "Avoiding Confusing Features in Place Recognition," European Conf. Comput. Vis., Crete, Greece, Sept. 5-11, 2010, pp. 748-761.
  4. R. Arandjelovic and A. Zisserman, "DisLocation: Scalable Descriptor Distinctiveness for Location Recognition," Asian Conf. Comput. Vis., Singapore, Nov. 1-2, 2014, pp. 188-204.
  5. G. Schindler, M. Brown, and R. Szeliski, "City-Scale Location Recognition," IEEE Conf. Comput. Vis. Pattern Recog., Los Angeles, CA, USA, June 17-22, 2007, pp. 1-7.
  6. P. Turcot and D.G. Lowe, "Better Matching with Fewer Features: the Selection of Useful Features in Large Database Recognition Problems," IEEE Int. Conf. Comput. Vis. Workshop, Kyoto, Japan, Sept. 27-Oct. 4, 2009, pp. 2109-2116.
  7. A.R. Zamir and M. Shah, "Accurate Image Localization Based on Google Maps Street View," European Conf. Comput. Vis., Crete, Greece, Sept. 5-11, 2010, pp. 255-269.
  8. A. Torii et al., "24/7 Place Recognition by View Synthesis," IEEE Conf. Comput. Vis. Pattern Recogn., Boston, MA, USA, June 7-12, 2015, pp. 1808-1817.
  9. R. Arandjelovic et al., "NetVLAD: CNN Architecture for Weakly Supervised Place Recognition," IEEE Conf. Comput. Vis. Pattern Recogn., Las Vegas, NV, USA, June 2016, pp. 5297-5307.
  10. J. Philbin et al., "Object Retrieval with Large Vocabularies and Fast Spatial Matching," Comput. Vis. Pattern Recog., Los Angeles, CA, USA, June 17-22, 2007, pp. 1-8.
  11. J. Sivic and A. Zisserman, "Video Google: a Text Retrieval Approach to Object Matching in Videos," IEEE Int. Conf. Comput. Vis., Nice, France, Oct. 13-16, 2003, pp. 1470-1477.
  12. H. Jegou et al., "Aggregating Local Descriptors into a Compact Image Representation," IEEE Conf. Comput. Vis. Pattern Recog., San Francisco, CA, USA, June 13-18, 2010, pp. 3304-3311.
  13. H. Jegou et al., "Aggregating Local Image Descriptors into Compact Codes," IEEE Trans. Pattern Mach. Intell., vol. 34, no. 9, Sept. 2012, pp. 1704-1716. https://doi.org/10.1109/TPAMI.2011.235
  14. D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int. J. Comput. Vis., vol. 60, no. 2, Nov. 2004, pp. 91-110. https://doi.org/10.1023/B:VISI.0000029664.99615.94
  15. Kakao Co., Daum map. http://map.daum.net
  16. G. Francini, S. Lepsoy, and M. Balestri, "Selection of Local Features for Visual Search," Signal Process.: Image Commun., vol. 28, no. 4, Apr. 2013, pp. 311-322. https://doi.org/10.1016/j.image.2012.11.002
  17. L.Y. Duan et al., "Overview of the MPEG-CDVS Standard," IEEE Trans. Image Proc., vol. 25, no. 1, Jan. 2016, pp. 179-194. https://doi.org/10.1109/TIP.2015.2500034
  18. K. Lee et al., "Extensive Analysis of Feature Selection for Compact Descriptor," Korea-Japan Joint Workshop Frontiers Comput. Vis., Incheon, Rep. of Korea, Jan. 2013, pp. 53-57.
  19. California Institute of Technology, Pasadena Buildings 2010 dataset. http://www.vision.caltech.edu/archive.html
  20. H. Jegou, D. Matthijs, and C. Schmid, "Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search," European Conf. Comput. Vis., Marseille, France, Oct. 12-18, 2008, pp. 304-317.
  21. Telecom Italia, 201 Books, InternetArchive and DistractorPairs Dataset. http://pacific.tilab.com/
  22. University of Oxford, The Oxford Buildings Dataset. http://www.robots.ox.ac.uk/-vgg/data/oxbuildings/
  23. K. Lee, S. Lee, and W.G. Oh, "Accelerating Local Feature Extraction Using Two Stage Feature Selection and Partial Gradient Computation," Asian Conf. Comput. Vis. Workshops, Singapore, Nov. 1-2, 2014, pp. 366-380.
  24. ISO/IEC JTC1/SC29/WG11, "Test Model 8: Compact Descriptors for Visual Search," 106th MPEG Meeting, N13926, Nov. 2013.
  25. ISO/IEC JTC1/SC29/WG11, "Test Model 14: Compact Descriptors for Visual Search," 112th MPEG Meeting, N15372, June 2015.
  26. S. Lepsoy et al., "Statistical Modeling of Outliers for Fast Visual Search," IEEE Conf. Multimedia Expo, Barcelona, Spain, July 11-15, 2011, pp. 1-6.
  27. J. Lie et al., "Rate-Adaptive Compact Fisher Codes for Mobile Visual Search," IEEE Signal Process. Lett., vol. 20, no. 2, Feb. 2014, pp. 195-198.
  28. R. Arandjelovic and A. Zisserman, "Three Things Everyone Should Know to Improve Object Retrieval," IEEE Conf. Comput. Vis. Pattern Recogn., Providence, RI, USA, June 16-21, 2012, pp. 2911-2918.
  29. H. Jegou and O. Chum, "Negative Evidences and Co-occurences in Image Tetrieval: the Benefit of PCA and Whitening," Computer Vision - ECCV 2012, Hidelderg, Berlin: Springer, 2012, pp. 774-787.

Cited by

  1. Spatializing an Artist-Resident Community Area at a Building-Level: A Case Study of Garosu-Gil, South Korea vol.12, pp.15, 2020, https://doi.org/10.3390/su12156116