Journal of Information Processing Systems

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

A Noisy Infrared and Visible Light Image Fusion Algorithm

  • Shen, Yu (School of Electronic and Information Engineering, Lanzhou Jiaotong University) ;
  • Xiang, Keyun (School of Electronic and Information Engineering, Lanzhou Jiaotong University) ;
  • Chen, Xiaopeng (School of Electronic and Information Engineering, Lanzhou Jiaotong University) ;
  • Liu, Cheng (School of Electronic and Information Engineering, Lanzhou Jiaotong University)
  • Received : 2019.07.04
  • Accepted : 2020.07.11
  • Published : 2021.10.31
Download PDF
⟨ Previous Next ⟩

Abstract

To solve the problems of the low image contrast, fuzzy edge details and edge details missing in noisy image fusion, this study proposes a noisy infrared and visible light image fusion algorithm based on non-subsample contourlet transform (NSCT) and an improved bilateral filter, which uses NSCT to decompose an image into a low-frequency component and high-frequency component. High-frequency noise and edge information are mainly distributed in the high-frequency component, and the improved bilateral filtering method is used to process the high-frequency component of two images, filtering the noise of the images and calculating the image detail of the infrared image's high-frequency component. It can extract the edge details of the infrared image and visible image as much as possible by superimposing the high-frequency component of infrared image and visible image. At the same time, edge information is enhanced and the visual effect is clearer. For the fusion rule of low-frequency coefficient, the local area standard variance coefficient method is adopted. At last, we decompose the high- and low-frequency coefficient to obtain the fusion image according to the inverse transformation of NSCT. The fusion results show that the edge, contour, texture and other details are maintained and enhanced while the noise is filtered, and the fusion image with a clear edge is obtained. The algorithm could better filter noise and obtain clear fused images in noisy infrared and visible light image fusion.

Keywords

Acknowledgement

The work has been supported by National Natural Science Foundation of China (Nos. 61861025, 61562057, 51969011); Ministry of Education (No. KFKT2018-9); Longyuan Youth Innovative and Entrepreneurial Talents (Team) Project in 2021; Gansu Provincial Department of Education: Young Doctor Fund Project (No. 2021QB-049); College Students Employment and Entrepreneurship Ability Improvement Project of Gansu Province (No. 2021-C-123); Intelligent Tunnel Supervision Robot Research Project (China Railway Research Institute [Research Institute], No. 2020-KJ016-Z016-A2); Youth Foundation of Lanzhou Jiaotong University (No. 2015005); Scientific Research Project of Higher Education Institutions of Gansu Province (No. 2016A-018); and Open project of Gansu Provincial Research Center for Conservation of Dunhuang Cultural Heritage (No. GDW2021YB15).

References

  1. M. N. Do and M. Vetterli, "Contourlets: a new directional multiresolution image representation," in Conference Record of the 36th Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, 2002, pp. 497-501.
  2. A. L. Da Cunha, J. Zhou, and M. N. Do, "The nonsubsampled contourlet transform: theory, design, and applications," IEEE Transactions on Image Processing, vol. 15, no. 10, pp. 3089-3101, 2006. https://doi.org/10.1109/TIP.2006.877507
  3. X. L. Wang and C. X. Chen, "Image fusion for synthetic aperture radar and multispectral images based on sub-band-modulated non-subsampled contourlet transform and pulse coupled neural network methods," The Imaging Science Journal, vol. 64, no. 2, pp. 87-93, 2016. https://doi.org/10.1080/13682199.2015.1136101
  4. P. S. Gomathi and B. Kalaavathi, "Multimodal medical image fusion in non-subsampled contourlet transform domain," Circuits and Systems, vol. 7, no. 8, pp. 1598-1610, 2016. https://doi.org/10.4236/cs.2016.78139
  5. X. Feng, "Fusion of infrared and visible images based on Tetrolet framework," Acta Photonica Sinica, vol. 48, no. 2, pp. 76-84, 2019.
  6. H. Deng, C. Wang, Y. Hu, and Y. Zhang, "Fusion of infrared and visible images based on non-subsampled dualtree complex contourlet and adaptive block," Acta Photonica Sinica, vol. 48, no. 7, pp. 136-146, 2019.
  7. L. Yan and T. Z. Xiang, "Fusion of infrared and visible images based on edge feature and adaptive PCNN in NSCT domain," Acta Electonica Sinica, vol. 44, no. 4, pp. 761-766, 2019.
  8. W. Z. Dai, X. L. Jiang, J. F. Li, "Adaptive medical image fusion based on human visual features," Acta Electonica Sinica, vol. 44, no. 8, pp. 1932-1939, 2016.
  9. Z. Chen, X. Yang, and C. Zhang, "Infrared and visible image fusion based on the compensation mechanism in NSCT domain," Chinese Journal of Scientific Instrument, vol. 37, no. 4, pp. 860-870, 2016.
  10. H. Y. Cai, L. R. Zhuo, P. Zhu, Z. H. Huang, and X. Y. Wu, "Fusion of infrared and visible images based on non-subsampled contourlet transform and intuitionistic fuzzy set," Acta Photonica Sinica, vol. 47, no. 6, pp. 225-234, 2018.
  11. C. H. Zha and Y. T. Guo, "Fast image fusion algorithm based on sparse representation and non-subsampled contourlet transform," Journal of Electronics & Information Technology, vol. 38, no. 7, pp. 1773-1780, 2016.
  12. F. Wang and Y. M. Cheng, "Visible and infrared image enhanced fusion based on MSSTO and NSCT transform," Control and Decision, vol. 32, no. 2, pp. 269-274, 2017.
  13. Q. Zhang and X. Maldague, "An adaptive fusion approach for infrared and visible images based on NSCT and compressed sensing," Infrared Physics & Technology, vol. 74, pp. 11-20, 2016. https://doi.org/10.1016/j.infrared.2015.11.003
  14. Z. Fu, X. Wang, X. Li, and J. Xu, "Infrared and visible image fusion based on visual saliency and NSCT," Journal of University of Electronic Science and Technology of China, vol. 46, no. 2, pp. 357-362, 2017.
  15. G. Wen, J. Pengchong, and Z. Tianchen, "Infrared image and visual image fusion algorithm based on NSCT and improved weight average," in Proceedings of 2015 6th International Conference on Intelligent Systems Design and Engineering Applications (ISDEA), Guiyang, China, 2015, pp. 456-459.
  16. J. Chen, X. Li, L. Luo, X. Mei, and J. Ma, "Infrared and visible image fusion based on target-enhanced multiscale transform decomposition," Information Sciences, vol. 508, pp. 64-78, 2020. https://doi.org/10.1016/j.ins.2019.08.066
  17. C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images," in Proceedings of the 6th International Conference on Computer Vision (IEEE Cat. No. 98CH36271), Bombay, India, 1998, pp. 839-846.
  18. E. Lallier and M. Farooq, "A real time pixel-level based image fusion via adaptive weight averaging," in Proceedings of the 3rd International Conference on Information Fusion, Paris, France, 2000.
  19. C. Q. Ye, "Research on multi-sensor image fusion algorithm based on multiscale decomposition," Xidian University, Xi'an, China, 2009.
  20. H. G. Jia, Z. P. Wu, M. C. Zhu, M. Xuan, and H. Liu, "Infrared image enhancement based on generalized linear operation and bilateral filter," Optics and Precision Engineering, vol. 21, no. 12, pp. 3272-3282, 2013. https://doi.org/10.3788/OPE.20132112.3272
  21. Y. Shen, J. W. Dang, Y. P. Wang, X. Feng, and W. W. Luo, "A novel medical image fusion method based on the multi-scale geometric analysis tool," Journal of Optoelectronics.Laser, vol. 24, no. 12, pp. 2446-2451, 2013.