International Journal of Computer Science & Network Security

International Journal of Computer Science & Network Security (국제컴퓨터통신보호논문지학회)
권호 표지
DOI QR코드

DOI QR Code

A Remote Sensing Scene Classification Model Based on EfficientNetV2L Deep Neural Networks

  • Aljabri, Atif A. (Faculty of Computer and Information Systems, Islamic University of Madinah) ;
  • Alshanqiti, Abdullah (Faculty of Computer and Information Systems, Islamic University of Madinah) ;
  • Alkhodre, Ahmad B. (Faculty of Computer and Information Systems, Islamic University of Madinah) ;
  • Alzahem, Ayyub (Robotics and Internet-of-Things Lab, Prince Sultan University) ;
  • Hagag, Ahmed (Faculty of Computers and Artificial Intelligence, Benha University)
  • Received : 2022.10.05
  • Published : 2022.10.30
Download PDF
⟨ Previous Next ⟩

Abstract

Scene classification of very high-resolution (VHR) imagery can attribute semantics to land cover in a variety of domains. Real-world application requirements have not been addressed by conventional techniques for remote sensing image classification. Recent research has demonstrated that deep convolutional neural networks (CNNs) are effective at extracting features due to their strong feature extraction capabilities. In order to improve classification performance, these approaches rely primarily on semantic information. Since the abstract and global semantic information makes it difficult for the network to correctly classify scene images with similar structures and high interclass similarity, it achieves a low classification accuracy. We propose a VHR remote sensing image classification model that uses extracts the global feature from the original VHR image using an EfficientNet-V2L CNN pre-trained to detect similar classes. The image is then classified using a multilayer perceptron (MLP). This method was evaluated using two benchmark remote sensing datasets: the 21-class UC Merced, and the 38-class PatternNet. As compared to other state-of-the-art models, the proposed model significantly improves performance.

Keywords

References

  1. F. Hu, G.-S. Xia, J. Hu, and L. Zhang, "Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery," Remote Sensing, vol. 7, pp. 14680-14707, 2015. https://doi.org/10.3390/rs71114680
  2. Y. Yang and S. Newsam, "Bag-of-visual-words and spatial extensions for land-use classification," in Proceedings of the 18th SIGSPATIAL international conference on advances in geographic information systems, 2010, pp. 270-279.
  3. W. Zhou, S. Newsam, C. Li, and Z. Shao, "PatternNet: A benchmark dataset for performance evaluation of remote sensing image retrieval," ISPRS journal of photogrammetry and remote sensing, vol. 145, pp. 197-209, 2018. https://doi.org/10.1016/j.isprsjprs.2018.01.004
  4. G.-S. Xia, W. Yang, J. Delon, Y. Gousseau, H. Sun, and H. Maitre, "Structural high-resolution satellite image indexing," in ISPRS TC VII Symposium-100 Years ISPRS, 2010, pp. 298-303.
  5. D. Dai and W. Yang, "Satellite image classification via two-layer sparse coding with biased image representation," IEEE Geoscience and remote sensing letters, vol. 8, pp. 173-176, 2010. https://doi.org/10.1109/LGRS.2010.2055033
  6. G.-S. Xia, J. Hu, F. Hu, B. Shi, X. Bai, Y. Zhong, et al., "AID: A benchmark data set for performance evaluation of aerial scene classification," IEEE Transactions on Geoscience and Remote Sensing, vol. 55, pp. 3965-3981, 2017. https://doi.org/10.1109/TGRS.2017.2685945
  7. D. G. Lowe, "Distinctive image features from scale-invariant keypoints," International journal of computer vision, vol. 60, pp. 91-110, 2004. https://doi.org/10.1023/B:VISI.0000029664.99615.94
  8. B. Luo, S. Jiang, and L. Zhang, "Indexing of remote sensing images with different resolutions by multiple features," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 6, pp. 1899-1912, 2013. https://doi.org/10.1109/JSTARS.2012.2228254
  9. B. Luo, J.-F. Aujol, Y. Gousseau, and S. Ladjal, "Indexing of satellite images with different resolutions by wavelet features," IEEE Transactions on Image Processing, vol. 17, pp. 1465-1472, 2008. https://doi.org/10.1109/TIP.2008.925367
  10. B. Luo, J.-F. Aujol, and Y. Gousseau, "Local scale measure from the topographic map and application to remote sensing images," Multiscale modeling & simulation, vol. 8, pp. 1-29, 2009. https://doi.org/10.1137/080730627
  11. C. Chen, B. Zhang, H. Su, W. Li, and L. Wang, "Land-use scene classification using multi-scale completed local binary patterns," Signal, image and video processing, vol. 10, pp. 745-752, 2016. https://doi.org/10.1007/s11760-015-0804-2
  12. L. Chen, W. Yang, K. Xu, and T. Xu, "Evaluation of local features for scene classification using VHR satellite images," in 2011 Joint Urban Remote Sensing Event, 2011, pp. 385-388.
  13. W. J. Scheirer, N. Kumar, P. N. Belhumeur, and T. E. Boult, "Multi-attribute spaces: Calibration for attribute fusion and similarity search," in 2012 IEEE Conference on Computer Vision and Pattern Recognition, 2012, pp. 2933-2940.
  14. F. Hu, G.-S. Xia, Z. Wang, X. Huang, L. Zhang, and H. Sun, "Unsupervised feature learning via spectral clustering of multidimensional patches for remotely sensed scene classification," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 8, 2015.
  15. F. Zhang, B. Du, and L. Zhang, "Saliency-guided unsupervised feature learning for scene classification," IEEE Transactions on Geoscience and Remote Sensing, vol. 53, pp. 2175-2184, 2014. https://doi.org/10.1109/TGRS.2014.2357078
  16. F. P. Luus, B. P. Salmon, F. Van den Bergh, and B. T. J. Maharaj, "Multiview deep learning for land-use classification," IEEE Geoscience and Remote Sensing Letters, vol. 12, pp. 2448-2452, 2015. https://doi.org/10.1109/LGRS.2015.2483680
  17. F. Zhang, B. Du, and L. Zhang, "Scene classification via a gradient boosting random convolutional network framework," IEEE Transactions on Geoscience and Remote Sensing, vol. 54, pp. 1793-1802, 2015. https://doi.org/10.1109/TGRS.2015.2488681
  18. Q. Zou, L. Ni, T. Zhang, and Q. Wang, "Deep learning based feature selection for remote sensing scene classification," IEEE Geoscience and Remote Sensing Letters, vol. 12, pp. 2321-2325, 2015. https://doi.org/10.1109/LGRS.2015.2475299
  19. S. Chaib, H. Liu, Y. Gu, and H. Yao, "Deep feature fusion for VHR remote sensing scene classification," IEEE Transactions on Geoscience and Remote Sensing, vol. 55, pp. 4775-4784, 2017. https://doi.org/10.1109/TGRS.2017.2700322
  20. G. Cheng, Z. Li, X. Yao, L. Guo, and Z. Wei, "Remote sensing image scene classification using bag of convolutional features," IEEE Geoscience and Remote Sensing Letters, vol. 14, pp. 1735-1739, 2017. https://doi.org/10.1109/LGRS.2017.2731997
  21. M. Shahriari and R. Bergevin, "Land-use scene classification: a comparative study on bag of visual word framework," Multimedia Tools and Applications, vol. 76, pp. 23059-23075, 2017. https://doi.org/10.1007/s11042-016-4316-z
  22. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in neural information processing systems, vol. 25, 2012.
  23. K. Simonyan and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
  24. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, et al., "Going deeper with convolutions," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, pp. 1-9.
  25. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770-778.
  26. K. Qi, C. Yang, C. Hu, H. Zhai, Q. Guan, and S. Shen, "A multi-level improved circle pooling for scene classification of high-resolution remote sensing imagery," Neurocomputing, vol. 462, pp. 506-522, 2021. https://doi.org/10.1016/j.neucom.2021.08.022
  27. O. A. Shawky, A. Hagag, E.-S. A. El-Dahshan, and M. A. Ismail, "Remote sensing image scene classification using CNN-MLP with data augmentation," Optik, vol. 221, p. 165356, 2020. https://doi.org/10.1016/j.ijleo.2020.165356
  28. X. Zhang, W. An, J. Sun, H. Wu, W. Zhang, and Y. Du, "Best Representation Branch Model for Remote Sensing Image Scene Classification," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 9768-9780, 2021. https://doi.org/10.1109/JSTARS.2021.3114404
  29. Y. Liu, Y. Zhong, F. Fei, Q. Zhu, and Q. Qin, "Scene classification based on a deep random-scale stretched convolutional neural network," Remote Sensing, vol. 10, p. 444, 2018. https://doi.org/10.3390/rs10030444
  30. Q. Zhu, Y. Zhong, Y. Liu, L. Zhang, and D. Li, "A deep-local-global feature fusion framework for high spatial resolution imagery scene classification," Remote Sensing, vol. 10, p. 568, 2018. https://doi.org/10.3390/rs10040568
  31. D. Zeng, S. Chen, B. Chen, and S. Li, "Improving remote sensing scene classification by integrating global-context and local-object features," Remote Sensing, vol. 10, p. 734, 2018. https://doi.org/10.3390/rs10050734
  32. Y. Chen, L. Feng, X. Zhang, Z. Shen, and X. Zhou, "Supervised and adaptive feature weighting for object-based classification on satellite images," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 11, pp. 3224-3234, 2018. https://doi.org/10.1109/JSTARS.2018.2851753
  33. O. A. Shawky, A. Hagag, E.-S. A. El-Dahshan, and M. A. Ismail, "A very high-resolution scene classification model using transfer deep CNNs based on saliency features," Signal, Image and Video Processing, vol. 15, pp. 817-825, 2021. https://doi.org/10.1007/s11760-020-01801-5
  34. H. Alhichri, A. S. Alswayed, Y. Bazi, N. Ammour, and N. A. Alajlan, "Classification of Remote Sensing Images Using EfficientNet-B3 CNN Model With Attention," IEEE Access, vol. 9, pp. 14078-14094, 2021. https://doi.org/10.1109/ACCESS.2021.3051085
  35. C. Peng, Y. Li, L. Jiao, and R. Shang, "Efficient Convolutional Neural Architecture Search for Remote Sensing Image Scene Classification," IEEE Transactions on Geoscience and Remote Sensing, vol. 59, pp. 6092-6105, 2021. https://doi.org/10.1109/TGRS.2020.3020424
  36. X. Xu, Y. Chen, J. Zhang, Y. Chen, P. Anandhan, and A. Manickam, "A novel approach for scene classification from remote sensing images using deep learning methods," European Journal of Remote Sensing, vol. 54, pp. 383-395, 2021. https://doi.org/10.1080/22797254.2020.1790995
  37. X. Tang, Q. Ma, X. Zhang, F. Liu, J. Ma, and L. Jiao, "Attention consistent network for remote sensing scene classification," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 14, pp. 2030-2045, 2021. https://doi.org/10.1109/JSTARS.2021.3051569
  38. M. Tan and Q. Le, "Efficientnet: Rethinking model scaling for convolutional neural networks," in International conference on machine learning, 2019, pp. 6105-6114.
  39. M. Tan and Q. Le, "Efficientnetv2: Smaller models and faster training," in International Conference on Machine Learning, 2021, pp. 10096-10106.
  40. J. Duchi, E. Hazan, and Y. Singer, "Adaptive subgradient methods for online learning and stochastic optimization," Journal of machine learning research, vol. 12, 2011.
  41. X. Gu, C. Zhang, Q. Shen, J. Han, P. P. Angelov, and P. M. Atkinson, "A Self-Training Hierarchical Prototype-based Ensemble Framework for Remote Sensing Scene Classification," Information Fusion, vol. 80, pp. 179-204, 2022. https://doi.org/10.1016/j.inffus.2021.11.014
  42. X. Bian, C. Chen, L. Tian, and Q. Du, "Fusing local and global features for high-resolution scene classification," IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, vol. 10, pp. 2889-2901, 2017. https://doi.org/10.1109/JSTARS.2017.2683799
  43. B. Yuan, L. Han, X. Gu, and H. Yan, "Multi-deep features fusion for high-resolution remote sensing image scene classification," Neural Computing and Applications, vol. 33, pp. 2047-2063, 2021. https://doi.org/10.1007/s00521-020-05071-7
  44. R. Cao, L. Fang, T. Lu, and N. He, "Self-attention-based deep feature fusion for remote sensing scene classification," IEEE Geoscience and Remote Sensing Letters, vol. 18, pp. 43-47, 2020.
  45. X. Sun, Q. Zhu, and Q. Qin, "A multi-level convolution pyramid semantic fusion framework for high-resolution remote sensing image scene classification and annotation," IEEE Access, vol. 9, pp. 18195-18208, 2021. https://doi.org/10.1109/ACCESS.2021.3052977
  46. S. Ansith and A. Bini, "Land use classification of high resolution remote sensing images using an encoder based modified GAN architecture," Displays, vol. 74, p. 102229, 2022. https://doi.org/10.1016/j.displa.2022.102229