Browse > Article
http://dx.doi.org/10.9766/KIMST.2022.25.5.476

Synthetic Infra-Red Image Dataset Generation by CycleGAN based on SSIM Loss Function  

Lee, Sky (Department of Aerospace Engineering, Chosun University)
Leeghim, Henzeh (Department of Aerospace Engineering, Chosun University)
Publication Information
Journal of the Korea Institute of Military Science and Technology / v.25, no.5, 2022 , pp. 476-486 More about this Journal
Abstract
Synthetic dynamic infrared image generation from the given virtual environment is being the primary goal to simulate the output of the infra-red(IR) camera installed on a vehicle to evaluate the control algorithm for various search & reconnaissance missions. Due to the difficulty to obtain actual IR data in complex environments, Artificial intelligence(AI) has been used recently in the field of image data generation. In this paper, CycleGAN technique is applied to obtain a more realistic synthetic IR image. We added the Structural Similarity Index Measure(SSIM) loss function to the L1 loss function to generate a more realistic synthetic IR image when the CycleGAN image is generated. From the simulation, it is applicable to the guided-missile flight simulation tests by using the synthetic infrared image generated by the proposed technique.
Keywords
Generative Adversarial Networks; Synthetic Image; Dataset; Structural Similarity Index Measure;
Citations & Related Records
Times Cited By KSCI : 2  (Citation Analysis)
연도 인용수 순위
1 Ronneberger, Olaf and Fischer, Philipp and Brox, Thomas, "U-Net: Convolutional Networks for Biomedical Image Segmentation," International Conference on Medical Image Computing and Computer-assisted Intervention, pp. 234-241, 2015.
2 Koch, Michael and Denzler, Joachim and Redies, Christoph, "1/f2 Characteristics and Isotropy in the Fourier Power Spectra of Visual Art, Cartoons, Comics, Mangas, and Different Categories of Photographs," PLoS One, Vol. 5, No. 8, p. e12268, 2010.   DOI
3 Kaiming He, Xiangyu Zhang, and Jian Sun, "Deep Residual Learning for Image Recognition," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770-778, 2016
4 Hang Zhao, Orazio Gallo, and Jan Kautz, "Loss Functions for Image Restoration With Neural Networks," IEEE Transactions on Computational Imaging, pp. 47-57, 2017.
5 Karras, Tero and Aila, Timo and Laine, Samuli and Lehtinen, Jaakko, "Progressive Growing of Gans for Improved Quality, Stability, and Variation," arXiv preprint arXiv:1710.10196, 2017.
6 Li, Wenda and Wang, Jian, "Residual Learning of Cycle-GAN for Seismic Data Denoising," IEEE Access, Vol. 9, pp. 11585-11597, 2021.   DOI
7 Teng, Long and Fu, Zhongliang and Yao, Yu, "Interactive Translation in Echocardiography Training System with Enhanced Cycle-GAN," IEEE Access, Vol. 8, pp. 106147-106156, 2020.   DOI
8 Tao, Li and Zhu, Chuang and Xiang, Guoqing and Li, Yuan and Jia, Huizhu and Xie, Xiaodong, "LLCNN: A Convolutional Neural Network for Low-Light Image Enhancement," 2017 IEEE Visual Communications and Image Processing(VCIP), pp. 1-4, 2017.
9 Shi, Haoyue and Wang, Le and Zheng, Nanning and Hua, Gang and Tang, Wei, "Loss Functions for Pose Guided Person Image Generation," Pattern Recognition, Vol. 122, p. 108351, 2022.   DOI
10 L. J. P. van der Maaten and G. E. Hinton, "Visualizing High-Dimensional Data Using t-SNE," Journal of Machine Learning Research, Vol. 9, pp. 2579-2695, 2008.
11 A. van der SCHAAF, J. H. van HATEREN "Modelling the Power Spectra of Natural Images: Statistics and Information," Vision Research, Volume 36, Issue 17, pp. 2759-2770, 1996.   DOI
12 Obukhov, Artem and Krasnyanskiy, Mikhail, "Quality Assessment Method for GAN based on Modified Metrics Inception Score and Frechet Inception Distance," Proceedings of the Computational Methods in Systems and Software, pp. 102-114, 2020.
13 Karras, Tero and Laine, Samuli and Aila, Timo, "A Style-based Generator Architecture for Generative Adversarial Networks," Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4401-4410, 2019.
14 Venkataramanan, Abhinau K and Wu, Chengyang and Bovik, Alan C and Katsavounidis "A Hitchhiker's Guide to Structural Similarity," IEEE Access, Vol. 9, pp. 28872-28896, 2021.   DOI
15 Moorthy, Anush K and Bovik, Alan C, "Perceptually Significant Spatial Pooling Techniques for Image Quality Assessment," Human Vision and Electronic Imaging XIV, Vol. 7240, pp. 339-349, 2009.
16 FLIR Systems, 2019. FREE FLIR Thermal Dataset for Algorithm Training. https://flir.com/oem/adas/adas-dataset-form
17 Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A. Efros, "Image-to-Image Translation with Conditional Adversarial Networks," CVPR, 2017.
18 Zhang, Ruiheng and Mu, Chengpo and Xu, Min and Xu, Lixin and Shi, Qiaolin and Wang, Junbo, "Synthetic IR Image Refinement Using Adversarial Learning with Bidirectional Mappings," IEEE Access, Vol. 7, pp. 153734-153750, 2019.   DOI
19 Kniaz, VV and Gorbatsevich, VS and Mizginov, VA, "Thermalnet: a Deep Convolutional Network for Synthetic Thermal Image Generation," The International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, Vol. 42, p. 41, 2017,
20 Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza and Bing Xu, "Generative Adversarial Networks," Neural Information Processing Systems(NIPS), Motreal, 2014.
21 Jun-Yan Zhu, Taesung Park, and Alexei A. Efros, "Unpaired Image-to-Image Translation using CycleConsistent Adversarial Networks," Proceedings of the IEEE International Conference on Computer Vision, pp. 2223-2232, 2017.
22 Maniyath, Shima Ramesh and Vijayakumar, K and Singh, Laxman and Sharma, Sudhir Kumar and Olabiyisi, Tunde, "Learningbased Approach to Underwater Image Dehazing Using CycleGAN," IEEE Access, Vol. 14, No. 18, pp. 1-11, 2021.
23 Hwang, Jieon and Yu, Chushi and Shin, Yoan, "SAR-to-Optical Image Translation Using SSIM and Perceptual Loss based Cycleconsistent GAN," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 191-194. 2020.
24 Heusel, Martin and Ramsauer, Hubert and Unterthiner, Thomas and Nessler, Bernhard and Hochreiter, Sepp, "Gans Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium," Advances in Neural Information Processing Systems, Vol. 30, 2017.
25 G. E. Hinton and S. T. Roweis, "Stochastic Neighbor Embedding," Proceedings of Advances in Neural Information Processing Systems(NIPS), pp. 833-840, 2002.