Browse > Article
http://dx.doi.org/10.5909/JBE.2021.26.1.26

Deep Learning-based Keypoint Filtering for Remote Sensing Image Registration  

Sung, Jun-Young (Department of Electronic Engineering, Kwangwoon University)
Lee, Woo-Ju (Kwangwoon University)
Oh, Seoung-Jun (Department of Electronic Engineering, Kwangwoon University)
Publication Information
Journal of Broadcast Engineering / v.26, no.1, 2021 , pp. 26-38 More about this Journal
Abstract
In this paper, DLKF (Deep Learning Keypoint Filtering), the deep learning-based keypoint filtering method for the rapidization of the image registration method for remote sensing images is proposed. The complexity of the conventional feature-based image registration method arises during the feature matching step. To reduce this complexity, this paper proposes to filter only the keypoints detected in the artificial structure among the keypoints detected in the keypoint detector by ensuring that the feature matching is matched with the keypoints detected in the artificial structure of the image. For reducing the number of keypoints points as preserving essential keypoints, we preserve keypoints adjacent to the boundaries of the artificial structure, and use reduced images, and crop image patches overlapping to eliminate noise from the patch boundary as a result of the image segmentation method. the proposed method improves the speed and accuracy of registration. To verify the performance of DLKF, the speed and accuracy of the conventional keypoints extraction method were compared using the remote sensing image of KOMPSAT-3 satellite. Based on the SIFT-based registration method, which is commonly used in households, the SURF-based registration method, which improved the speed of the SIFT method, improved the speed by 2.6 times while reducing the number of keypoints by about 18%, but the accuracy decreased from 3.42 to 5.43. Became. However, when the proposed method, DLKF, was used, the number of keypoints was reduced by about 82%, improving the speed by about 20.5 times, while reducing the accuracy to 4.51.
Keywords
Computer Vision; Deep Learning; Image Registration; Remote Sensing Image; Image Segmentation;
Citations & Related Records
연도 인용수 순위
  • Reference
1 B. Zitova, J. Flusser, "Image registration methods: a survey" Image and Vision Computing, Vol.21, No.11, pp.977-1000, June 2003.   DOI
2 S. Suri, P. Reinartz, "Mutual-information-based registration of terrasar-x and ikonos imagery in urban areas" IEEE Transactions on Geoscience and Remote Sensing, Vol. 48, No.2, pp.939-949, November 2009.   DOI
3 J.P. Kern, M.S. Pattichis, "Robust multispectral image registration using mutual-information models" IEEE Transactions on Geoscience and Remote Sensing, Vol.45, No.5, pp.1494-1505, April 2007.   DOI
4 H. Bay, A. Ess, T. Tuytelaars, L.V. Gool, "Speeded-up robust features (surf)" Computer Vision and Image Understanding, Vol.110, No3, pp.346-359, June 2008.   DOI
5 D. Lowe, "Distinctive image features from scale-invariant keypoints" International Journal of Computer Vision, Vol.60, No.2, pp.91-110, January 2004.   DOI
6 M.I. Patel, V.K. Thakar, "Speed Improvement in Image Registration using Maximum Likelihood based Mutual Information" International Conference on Advanced Computing and Communication Systems, Coimbatore, India, pp.1-3, 2015.
7 L.C. Chiu, T.S. Chang, J.Y. Chen, N.Y.C. Chang, "Fast SIFT Design for Real-Time Visual Feature Extraction" IEEE Transactions on Image Processing, Vol. 22, No. 8, pp.3158-3167, August 2013.   DOI
8 Y. LeCun, Y. Bengion and G. Hinton, "Deep learning," Nature, Vol. 521, pp.436-444, May 2015.   DOI
9 O. Ronneberger, P. Fischer, and T. Brox, "U-Net: Convolutional networks for biomedical image segmentation" Medical Image Computing and Computer-Assisted Intervention, Vol.9351. pp.234-241, November 2015.
10 C. Harris, M. Stephens, "A combined corner and edge detector" Alvey Vision Conference, Manchester, UK, pp. 147-151, 1988.
11 T. Lindeberg, "Feature detection with automatic scale selection", International Journal of Computer Vision, Vol. 30, No.2, pp.79-116, Novemver 1998.   DOI
12 K. Mikolajczyk, C. Schmid, "Indexing based on scale invariant interest points" International Conference on Computer Vision, Vancouver, Canada, pp.525-531, 2001.
13 L.M.J. Florack, B.M. ter Haar Romeny, J.J. Koenderink, M.A. Viergever, "General intensity transformations and differential invariants" Journal of Mathematical Imaging and Vision, Vol.4, No.2, pp.171-187, May 1994.   DOI
14 D. Lowe, "Object recognition from local scale-invariant features" International Conference on Computer Vision, Kerkyra, Greece, pp.1150-1157, 1999.
15 T. Kadir, M. Brady, "Scale, saliency and image description" International Journal of Computer Vision, Vol. 45, No.2, pp.83-105, Novemver 2001.   DOI
16 F. Jurie, C. Schmid, "Scale-invariant shape features for recognition of object categories" IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, pp.90-96, 2004.
17 F. Mindru, T. Tuytelaars, L. Van Gool, T. Moons, "Moment invariants for recognition under changing viewpoint and illumination" Computer Vision and Image Understanding, Vol.94 No.1-3, pp.3-27, April 2004.   DOI
18 A. Baumberg, "Reliable feature matching across widely separated views" IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, SC, USA, pp.774-781, 2000.
19 W.T. Freeman, E.H. Adelson, "The design and use of steerable filters" IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.13, No.9, pp.891-906, September 1991.   DOI
20 G. Carneiro, A.D. Jepson, "Multi-scale phase-based local features" IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Madison, WI, USA, pp.736-743, 2003.
21 Y. Ke, R. Sukthankar, "PCA-SIFT: a more distinctive representation for local image descriptors" IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, USA, pp.506-513, 2004.
22 H. Goncalves, J.A. Goncalves, L. Corte-Real, "Measures for an objective evaluation of the geometric correction process quality" IEEE Geoscience and Remote Sensing Letters, Vol.6, No.2, pp.292-296, April 2009.   DOI
23 K. Mikolajczyk, C. Schmid, "A performance evaluation of local descriptors" IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol.27, No.10, pp.1615-1630, October 2003.   DOI