Browse > Article
http://dx.doi.org/10.3837/tiis.2021.08.003

Reflectance estimation for infrared and visible image fusion  

Gu, Yan (North Night Vision Technology Corp., Ltd.)
Yang, Feng (North Night Vision Technology Corp., Ltd.)
Zhao, Weijun (North Night Vision Technology Corp., Ltd.)
Guo, Yiliang (North Night Vision Technology Corp., Ltd.)
Min, Chaobo (The College of Internet of Things Engineering, HoHai University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.15, no.8, 2021 , pp. 2749-2763 More about this Journal
Abstract
The desirable result of infrared (IR) and visible (VIS) image fusion should have textural details from VIS images and salient targets from IR images. However, detail information in the dark regions of VIS image has low contrast and blurry edges, resulting in performance degradation in image fusion. To resolve the troubles of fuzzy details in dark regions of VIS image fusion, we have proposed a method of reflectance estimation for IR and VIS image fusion. In order to maintain and enhance details in these dark regions, dark region approximation (DRA) is proposed to optimize the Retinex model. With the improved Retinex model based on DRA, quasi-Newton method is adopted to estimate the reflectance of a VIS image. The final fusion outcome is obtained by fusing the DRA-based reflectance of VIS image with IR image. Our method could simultaneously retain the low visibility details in VIS images and the high contrast targets in IR images. Experiment statistic shows that compared to some advanced approaches, the proposed method has superiority on detail preservation and visual quality.
Keywords
Image fusion; reflectance estimation; Retinex; infrared; detail preservation;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Y. Liu, S. Liu and Z.Wang, "A general framework for image fusion based on multi-scale transform and sparse representation," Inf. Fusion, vol. 24, pp. 147-164, Jul. 2015.   DOI
2 Y. Liu, X. Chen, H. Peng, Z. Wang, "Multi-focus image fusion with a deep convolutional neural network," Inf. Fusion, vol. 36, pp. 191-207, Jul. 2017.   DOI
3 J. Ma, C. Chen, C. Li and J. Huang, "Infrared and visible image fusion via gradient transfer and total variation minimization," Inf. Fusion, vol. 31, pp. 100-109, Sep. 2016.   DOI
4 J. Qu, Y. Li, Q. Du and H. Xia, "Hyperspectral and panchromatic image fusion via adaptive tensor and multi-scale retinex algorithm," IEEE Access, vol. 8, pp. 30522-30532, Feb. 2020.   DOI
5 D. J. Jobson, Z. Rahman, and G. A. Woodell, "A multiscale Retinex for bridging the gap between color images and the human observation of scenes," IEEE Trans. Image Process., vol. 6, no. 7, pp. 965-976, Jul. 1997.   DOI
6 Y. Liu, X. Chen, R. K. Ward and Z. J. Wang, "Image fusion with convolutional sparse representation," IEEE Signal Process. Lett., vol. 23, no. 12, pp. 1882-1886, Dec. 2016.   DOI
7 J. Wang, J. Peng, X. Feng, G. He and J. Fan, "Fusion method for infrared and visible images by using non-negative sparse representation," Infrared Phys. Technol., vol. 67, pp. 477-489, Nov. 2014.   DOI
8 W. Kong, Y. Lei and H. Zhao, "Adaptive fusion method of visible light and infrared images based on non-subsampled shearlet transform and fast non-negative matrix factorization," Infrared Phys. Technol., vol. 67, pp. 161-172, Nov. 2014.   DOI
9 P. Hill, M. E. Al-Mualla and D. Bull, "Perceptual image fusion using wavelets," IEEE Trans. Image Process., vol. 26, no. 3, pp. 1076-1088, Mar. 2017.   DOI
10 M. Choi, R. Kim, M. Nam and H. Kim, "Fusion of multispectral and panchromatic satellite images using the curvelet transform," IEEE Geosci.Remote Sens. Lett., vol. 2, no. 2, pp. 136-140, Apr. 2005.   DOI
11 T. Xiang, L. Yan and R. Gao, "A fusion algorithm for infrared and visible images based on adaptive dual-channel unit-linking PCNN in NSCT domain," Infrared Phys. Technol., vol. 69, pp. 53-61, Mar. 2015.   DOI
12 N. Otsu, "A threshold selection method from gray-level histograms," IEEE Trans. Syst. Man Cy., vol. 9, no. 1, pp. 62-66, Jan. 1979.   DOI
13 F. Meng, M. Song, B. Guo, R. Shi and D. Shan, "Image fusion based on object region detection and non-subsampled contourlet transform," Comput. Electr. Eng., vol. 62, pp. 375-383, Aug. 2017.   DOI
14 X. Zhang, Y. Ma, F. Fan, Y. Zhang and J. Huang, "Infrared and visible image fusion via saliency analysis and local edge-preserving multi-scale decomposition," JOSA A., vol. 34, no. 8, pp. 1400-1410, Aug. 2017.   DOI
15 J. Zhao, Y. Chen, H. Feng, Z. Xu and Q. Li, "Infrared image enhancement through saliency feature analysis based on multi-scale decomposition," Infrared Phys. Technol., vol. 62, pp. 86-93, Jan. 2014.   DOI
16 J. W. Roberts, J. Van Aardt and F. Ahmed, "Assessment of image fusion procedures using entropy, image quality, and multispectral classification," J. Appl. Remote Sens., vol. 2, no. 1, pp. 023522, Jan. 2008.   DOI
17 A. M. Eskicioglu and P. S. Fisher, "Image quality measures and their performance," IEEE Trans. Commun., vol. 43, no. 12, pp. 2959-2965, Dec. 1995.   DOI
18 A. Toet, "Image fusion by a ratio of low-pass pyramid," Pattern Recognit. Lett., vol. 9, no. 4, pp. 245-253, May. 1989.   DOI
19 H. Li, X.-J. Wu, J. Kittler, "Infrared and visible image fusion using a deep learning framework," in Proc. of International Conference on Pattern Recognition, pp. 2705-2710, Nov. 2018.
20 X. Mao, Q. Li, H. Xie, R.Y. Lau, Z. Wang, S.P. Smolley, "Least squares generative adversarial networks," in Proc. of IEEE International Conference on Computer Vision, pp. 2813-2821, Dec. 2017.
21 E. H. Land, "The Retinex theory of color vision," Sci. Amer., vol. 237, no. 6, pp. 108-128, Dec. 1977.   DOI
22 X. Fu, D. Zeng, Y. Huang, X. Zhang, and X. Ding, "A weighted variational model for simultaneous reflectance and illumination estimation," in Proc. of IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), pp. 2782-2790, Jun. 2016.
23 D. J. Jobson, Z. Rahman, and G. A. Woodell, "Properties and performance of a center/surround Retinex," IEEE Trans. Image Process., vol. 6, no. 3, pp. 451-462, Mar. 1997.   DOI
24 X. Ren, W. Yang, W. Cheng and J. Liu, "LR3M: robust low-light enhancement via low-rank regularized Retinex model," IEEE Trans. Image Process., vol. 29, pp. 5862-5876, Apr. 2020.   DOI
25 D. P. Bavirisetti and R. Dhuli, "Two-scale image fusion of visible and infrared images using saliency detection," Infrared Phys. Technol., vol. 76, pp. 52-64, May. 2016.   DOI
26 R. Yu, W. Chen and D. Zhou, "Infrared and visible image fusion based on gradient transfer optimization model," IEEE Access, vol. 8, pp. 50091-50106, Mar. 2020.   DOI
27 M. Deshmukh and U. Bhosale, "Image fusion and image quality assessment of fused images," Int. J. Image Process., vol. 4, no. 5, pp. 484-508, Dec. 2010.
28 J. Ma, P. Liang, W. Yu, C. Chen, X. Guo, J. Wu and J. Jiang, "Infrared and visible image fusion via detail preserving adversarial learning," Inf. Fusion, vol. 54, pp. 85-98, Jul. 2019.   DOI
29 C. Deng, Z. Wang, X. Li, H. Li and C. C. Cavalcante, "An improved remote sensing image fusion algorithm based on IHS transformation," KSII Trans. Int. Inf. Syst., vol. 11, no. 3, pp. 1633-1649, Mar. 2017.   DOI
30 D. P. Bavirisetti, G. Xiao and G. Liu, "Multi-sensor image fusion based on fourth order partial differential equations," in Proc. of Int. Conf. on Inf. Fusion, pp. 701-709, July 10-13, 2017.
31 Z. Zhou, B.Wang, S. Li and M. Dong, "Perceptual fusion of infrared and visible images through a hybrid multi-scale decomposition with Gaussian and bilateral filters," Inf. Fusion, vol. 30, pp. 15-26, Jul. 2016.   DOI
32 W. Zhao, H. Lu and D. Wang, "Multisensor image fusion and enhancement in spectral total variation domain," IEEE Trans. Multimedia, vol. 20, no. 4, pp. 866-879, Apr. 2018.   DOI
33 Y. Rao, "In-fibre bragg grating sensors," Meas. Sci. Technol., vol. 8, no. 4, pp. 355, 1997.   DOI
34 Y. Han, Y. Cai, Y. Cao and X. Xu, "A new image fusion performance metric based on visual information fidelity," Inf. fusion, vol. 14, no. 2, pp. 127-135, Apr. 2013.   DOI
35 Z. Wang and A. C. Bovik, "A universal image quality index," IEEE Signal Process. Lett., vol. 9, no 3, pp. 81-84, Mar. 2002.   DOI
36 S. Li, X. Kang, J. Hu, "Image fusion with guided filtering," IEEE Trans. Image Process., vol. 22, no. 7, pp. 2864-2875, Jul. 2013.   DOI
37 M. Yin, P. Duan,W. Liu and X. Liang, "A novel infrared and visible image fusion algorithm based on shift-invariant dual-tree complex shearlet transform and sparse representation," Neurocomputing, vol. 226, pp. 182-191, Feb. 2017.   DOI
38 W. Kong, L. Zhang and Y. Lei, "Novel fusion method for visible light and infrared images based on NSST-SF-PCNN," Infrared Phys. Technol., vol. 65, pp. 103-112, Jul. 2014.   DOI
39 M. H. Loke and R. D. Barker, "Rapid least-squares inversion of apparent resistivity pseudosections by a quasi-Newton method," Geophysical Prospecting, vol. 44, no. 1, pp. 131-152, Jan. 1996.   DOI
40 J. Ma, W. Yu, P. Liang, C. Li and J. Jiang, "FusionGAN: a generative adversarial network for infrared and visible image fusion," Inf. Fusion, vol. 48, pp. 11-26, Aug. 2019.   DOI
41 F. An, X. Zhou and D. Lin, "Multiscale self-coordination of bidimensional empirical mode decomposition in image fusion," KSII Trans. Int. Inf. Syst., vol. 9, no. 4, pp. 1441-1456, Apr. 2015.   DOI
42 W. Jeong, B. Han, H. Yang and Y. Moon, "Real-time visible-infrared image fusion using multi-guided filter," KSII Trans. Int. Inf. Syst., vol. 13, no. 6, pp. 3092-3107, Jun. 2019.   DOI