International Journal of Computer Science & Network Security

International Journal of Computer Science & Network Security (국제컴퓨터통신보호논문지학회)
권호 표지
DOI QR코드

DOI QR Code

A Novel RGB Channel Assimilation for Hyperspectral Image Classification using 3D-Convolutional Neural Network with Bi-Long Short-Term Memory

  • M. Preethi (Department of Computer Science, S.T. Hindu College, Nagercoil, Affiliated to Manonmaniam Sundaranar University) ;
  • C. Velayutham (Department of Computer Science, Aditanar College) ;
  • S. Arumugaperumal (Department of Computer Science, S.T. Hindu College)
  • Received : 2023.03.05
  • Published : 2023.03.30
Download PDF
⟨ Previous Next ⟩

Abstract

Hyperspectral imaging technology is one of the most efficient and fast-growing technologies in recent years. Hyperspectral image (HSI) comprises contiguous spectral bands for every pixel that is used to detect the object with significant accuracy and details. HSI contains high dimensionality of spectral information which is not easy to classify every pixel. To confront the problem, we propose a novel RGB channel Assimilation for classification methods. The color features are extracted by using chromaticity computation. Additionally, this work discusses the classification of hyperspectral image based on Domain Transform Interpolated Convolution Filter (DTICF) and 3D-CNN with Bi-directional-Long Short Term Memory (Bi-LSTM). There are three steps for the proposed techniques: First, HSI data is converted to RGB images with spatial features. Before using the DTICF, the RGB images of HSI and patch of the input image from raw HSI are integrated. Afterward, the pair features of spectral and spatial are excerpted using DTICF from integrated HSI. Those obtained spatial and spectral features are finally given into the designed 3D-CNN with Bi-LSTM framework. In the second step, the excerpted color features are classified by 2D-CNN. The probabilistic classification map of 3D-CNN-Bi-LSTM, and 2D-CNN are fused. In the last step, additionally, Markov Random Field (MRF) is utilized for improving the fused probabilistic classification map efficiently. Based on the experimental results, two different hyperspectral images prove that novel RGB channel assimilation of DTICF-3D-CNN-Bi-LSTM approach is more important and provides good classification results compared to other classification approaches.

Keywords

References

  1. Bandos, T., Bruzzone, L., and Camps-Valls, G., "Classification of hyperspectral images with regularized linear discriminant analysis," IEEE Transactions on Geoscience and Remote Sensing, vol. 47, no. 3, pp. 862-873, 2009. https://doi.org/10.1109/TGRS.2008.2005729
  2. Chen, Y., Lin, Z., Zhao, X., Wang, G., and Gu, Y., "Deep learning-based classification of hyperspectral data," IEEE Journal Of Topics In Applied Earth Observations and Remote Sensing,vol. 7, no. 6, pp. 2094-2107,
  3. Chen, Y., Nasrabadi, N. M., and Tran, T. D., "Hyperspectral image classification via kernel sparse representation," IEEE Transactions on Geoscience and Remote Sensing, vol. 51, no. 1, pp. 217-231, 2013. https://doi.org/10.1109/TGRS.2012.2201730
  4. Dalla Mura, M, Villa, A., Benediktsson, J. A., Chanussot, J., and Bruzzone, L., "Classification of hyperspectral images by using extended morphological attribute profiles and independent component analysis," IEEE Geoscience and Remote Sensing Letters, vol. 8, no. 3, pp. 542-546, 2011. https://doi.org/10.1109/LGRS.2010.2091253
  5. Ellis, D.M., Draper, N.P., and Smith, H.S., "Applied regression analysis." Applied Statistics, Vol. 17, no. 1, pp. 83-90,
  6. Fauvel, M., Benediktsson, J., Chanussot, J.,and Sveinsson, J., "Spectral and spatial classification of hyperspectral data using SVMs and morphological profiles," IEEE Transactions on Geoscience and Remote Sensing., vol. 46, no. 11, pp. 3804-3814, 2008. https://doi.org/10.1109/TGRS.2008.922034
  7. Guo, Y., Cao, H., Han, S., Sun, Y., and Bai, Y., "Spectral-spatial hyperspectral image classification with K-Nearest neighbor and guided filter." IEEE Access. Vol. 6, pp. 18582-18591, 2018.
  8. Haokui Zhang , Ying Li , Yenan Jiang, Peng Wang, Qiang Shen , and Chunhua Shen, "Hyperspectral Classification Based on Lightweight 3-D-CNN With Transfer Learning," IEEE Transactions On Geoscience And Remote Sensing, VOL. 57, NO. 8, pp. 5813-5830, 2019. https://doi.org/10.1109/TGRS.2019.2902568
  9. He, K.M., Sun, J., and Tang, X.O. Guided image filtering. IEEE Trans. Pattern Anal. Mach. Intell. Vol. 35, no.6, pp.1397-1409, 2013. https://doi.org/10.1109/TPAMI.2012.213
  10. Hochreiter, S. and Schmidhuber, J., "Long short-term memory. Neural computation, vol. 9, no. 8, pp.1735-1780,1997. https://doi.org/10.1162/neco.1997.9.8.1735
  11. Hughes, G. F., "On the mean accuracy of statistical pattern recognizers," IEEE Transaction Information Theory, vol. 14, no. 1, pp 55-63, 1968. https://doi.org/10.1109/TIT.1968.1054102
  12. Hu, W.., Huang, Y., Li, W., Zhang, F., an Li, d H.., "Deep convolutional neural networks for hyperspectral image classification," Journal of Sensors, vol. 501, pp. 258619, 2015.
  13. Jia, S., Shen, L., Zhu, J., and Li, Q., "A 3-D Gabor phase-based coding and matching framework for hyperspectral imagery classification." IEEE Transactions on Cybernetics, Vol. 48, No. 4, pp. 1176-1188, 2018.
  14. Jun li., and jose M. Bioucas, "Spectral- spatial hyperspectral image segmentation using subspace multinomial logistic regression and markov random fields," IEEE vol. 50, no.3, 2012.
  15. Kang, X., Li, S., and Benediktsson, J. A, "Feature extraction of hyperspectral images with image fusion and recursive filtering." IEEE Transactions on Geoscience and Remote Sensing, vol. 52, no. 6, pp. 3742-3752,
  16. Konstantinos Makantasis, Konstantinos Karantzalos, and Anastasios Doulamis, "Deep supervised learning for hyperspectral data classification through convolutional neural networks," IEEE. pp.4959-4962, 2015.
  17. Landgrede, D, A., "Hyperspectral image data analysis, IEEE signal process," Mag. 1053-5888, pp.17-28, 2002.
  18. Licciardi, G., Marpu, P. R., Chanussot, J., and Benediktsson, J. A., "Linear versus nonlinear pca for the classification of hyperspectral data based on the extended morphological profiles," IEEE Geoscience and Remote Sensing Letters, vol. 9, no. 3, pp. 447-451, 2012. https://doi.org/10.1109/LGRS.2011.2172185
  19. Li, J., Bioucas-Dias, J. M ., and Plaza, A., "Semisupervised hyperspectral image segmentation using multinomial logistic regression with active learning," IEEE Transactions on Geoscience and Remote Sensing, vol. 48, no. 11, pp. 4085-4098, 2010.
  20. Li, J., Bioucas-Dias, J. M., and Plaza, A., "Spectral-spatial hyperspectral image segmentation using subspace multinomial logistic regression and markov random fields," IEEE Transactions on Geoscience and Remote Sensing, vol. 50, no. 3, pp. 809-823, 2012. https://doi.org/10.1109/TGRS.2011.2162649
  21. Li, J., Bioucas-Dias, J. M., and Plaza, A., "Hyperspectral image segmentation using new Bayesian approach with active learning," IEEE Transactions on Geoscience and Remote Sensing., vol. 49, no. 10, pp3947-3960, 2011. https://doi.org/10.1109/TGRS.2011.2128330
  22. Li, Y., Zhang, H., and Shen, Q., "Spectral-spatial classification of hyperspectral imagery with 3d convolutional neural network," Remote Sens, vol.9, pp. 67-74, 2017. https://doi.org/10.3390/rs9010067
  23. Lin Zhu, Yushi Chen and Pedram Ghamisi , "Generative Adversarial Networks for Hyperspectral Image Classification," IEEE Transactions On Geoscience And Remote Sensing, Volume: 56, no. 6,pp. 5046 - 5063, 2018. https://doi.org/10.1109/TGRS.2018.2805286
  24. Liu, Q.,Feng, Z., Hang, R., and Yuan, X., "Bidirectional-Convolutional LSTM Based Spectral-Spatial Feature Learning for Hyperspectral Image Classification," Remote Sens, vol. 9, pp. 1330-1340, 2017. https://doi.org/10.3390/rs9121330
  25. Mandic, D.P., and Chambers, J., "Recurrent neural networks for prediction: learning algorithms, architectures and stability," John Wiley & Sons, Inc. pp. 1-14, 2001.
  26. Melgani, F., and Bruzzone, L., "Classification of hyperspectral remote sensing images with support vector machines," IEEE Transactions On Geoscience And Remote Sensing, vol. 42, no. 8, pp. 1778-1790, 2004. https://doi.org/10.1109/TGRS.2004.831865
  27. Oliveira, M.M., and Gastal, E.S.,"Domain transform for edge-aware image and video processing." ACM Transactions on Graphics (ToG). ACM, Vol. 30,no. 4, pp. 69. 2011.
  28. Qin Xu , Yong Xiao, Dongyue Wang and Bin Luo, "CSA-MSO3DCNN: Multiscale Octave 3D-CNN with Channel and Spatial Attention for Hyperspectral Image Classification," Remote Sens. 2020.
  29. Radhesyam Vaddi, and Prabukumar Manoharan, "Hyperspectral image classification using CNN with spectral and spatial features integration," Infrared Physics and Technology, vol. 107, Elsevier, 2020.
  30. Shaohui Mei , Jingyu Ji , Qianqian Bi , Junhui Hou , and Qian Du, "Integrating spectral and spatial information into deep convolutional neural networks for hyperspectral classification," IEEE . 5067-5070 2016.
  31. Shen, L., and Jia, S., "Three-dimensional gabor wavelets for pixel-based hyperspectral imagery classification," IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 12, pp. 5039-5046, 2011. https://doi.org/10.1109/TGRS.2011.2157166
  32. Shi, X., Chen, Z., Wang, H., Yeung, D.Y., Wong, W.K., and Woo, W.C., " Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting," Advances in Neural Information Processing Systems, Volume 28, pp. 1049-5258, 2015. Available online: https://papers.nips.cc/paper/5955-convolutional-lstmnetworka-machine-learning-approach-for-precipitationnowcasting (accessed on 10 October 2019).
  33. Sun, X., Qu, Q., Nasrabadi, N. M. and Tran, T. D. "Structured priors for sparse-representation-based hyperspectral image classification," IEEE Geoscience and Remote Sensing Letters, vol. 11, no. 7, pp. 1235-1239,
  34. Szegedy, C., Toshev, A., and Erhan, D., "Deep neural networks for object detection," in Advances in Neural Information Processing Systems, pp. 2553-2561, 2013.
  35. Yushi Chen , Zhouhan Lin and Xing Zhao, "Deep Learning-Based Classification of Hyperspectral Data," IEEE Journal Of Selected Topics In Applied Earth Observations And Remote Sensing, IEEE, vol.7 no.6, pp.2094-2107, 2014.  https://doi.org/10.1109/JSTARS.2014.2329330