Browse > Article
http://dx.doi.org/10.3837/tiis.2021.12.004

Generative Adversarial Networks for single image with high quality image  

Zhao, Liquan (Key Laboratory of Modern Power System Simulation and Control and Renewable Energy Technology, Ministry of Education, Northeast Electric Power University)
Zhang, Yupeng (Key Laboratory of Modern Power System Simulation and Control and Renewable Energy Technology, Ministry of Education, Northeast Electric Power University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.15, no.12, 2021 , pp. 4326-4344 More about this Journal
Abstract
The SinGAN is one of generative adversarial networks that can be trained on a single nature image. It has poor ability to learn more global features from nature image, and losses much local detail information when it generates arbitrary size image sample. To solve the problem, a non-linear function is firstly proposed to control downsampling ratio that is ratio between the size of current image and the size of next downsampled image, to increase the ratio with increase of the number of downsampling. This makes the low-resolution images obtained by downsampling have higher proportion in all downsampled images. The low-resolution images usually contain much global information. Therefore, it can help the model to learn more global feature information from downsampled images. Secondly, the attention mechanism is introduced to the generative network to increase the weight of effective image information. This can make the network learn more local details. Besides, in order to make the output image more natural, the TVLoss function is introduced to the loss function of SinGAN, to reduce the difference between adjacent pixels and smear phenomenon for the output image. A large number of experimental results show that our proposed model has better performance than other methods in generating random samples with fixed size and arbitrary size, image harmonization and editing.
Keywords
Artificial Intelligence; Generative adversarial networks; Attention mechanism; Random samples;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Mao X, Li Q, Xie H, et al., "Least Squares Generative Adversarial Networks," in Proc. of IEEE International Conference on Computer Vision (ICCV), pp. 2813-2821, 2017.
2 Arjovsky M, Chintala S, Bottou L, "Wasserstein generative adversarial networks," in Proc. of the 34th International Conference on Machine Learning, PMLR 70, pp. 214-223, 2017.
3 Radford A, Metz L, Chintala S, "Unsupervised representation learning with deep convolutional generative adversarial networks," arXiv preprint arXiv:1511.06434, 2015.
4 Mirza M, Osindero S, "Conditional generative adversarial nets," arXiv preprint arXiv:1411.1784, 2014.
5 Xin R, Zhang J, Shao Y, "Complex network classification with convolutional neural network," Tsinghua Science and Technology, vol. 25, no. 4, pp. 447-457, Aug. 2020.   DOI
6 Dong Y, Liu Y, Zhang H, et al., "FD-GAN: Generative Adversarial Networks with FusionDiscriminator for Single Image Dehazing," in Proc. of the AAAI Conference on Artificial Intelligence, pp. 10729-10736, 2020.
7 Jetchev N, Bergmann U, Vollgraf R, "Texture synthesis with spatial generative adversarial networks," arXiv preprint arXiv:1611.08207, 2016.
8 C. G. Turhan and H. S. Bilge, "Recent Trends in Deep Generative Models: a Review," in Proc. of International Conference on Computer Science and Engineering (UBMK), pp. 574-579, 2018.
9 G. Bombuwala and G. Poravi, "A Review of Generative Image Modeling Techniques," in Proc. of IEEE 5th International Conference for Convergence in Technology (I2CT), pp. 1-5, 2019.
10 Li Y, Wang H, and Dong X, "The Denoising of Desert Seismic Data Based on Cycle-GAN With Unpaired Data Training," IEEE Geoscience and Remote Sensing Letters, vol. 18, pp. 2016-2020, 2020.
11 Shaham T R, Dekel T, and Michaeli T, "Singan: Learning a generative model from a single natural image," in Proc. of IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4569- 4579, 2019.
12 Gulrajani I, Ahmed F, Arjovsky M, et al., "Improved training of wasserstein GANs," in Proc. of the 31st International Conference on Neural Information Processing Systems (NIPS'17), pp. 5769- 5779, Dec. 2017.
13 Denton E, Chintala S, Szlam A, et al., "Deep generative image models using a Laplacian pyramid of adversarial networks," in Proc. of the 28th International Conference on Neural Information Processing Systems (NIPS'15), vol.1, pp. 1486-1494, 2015.
14 Li C, Wand M, "Precomputed Real-Time Texture Synthesis with Markovian Generative Adversarial Networks," Lecture Notes in Computer Science, vol. 9907, pp. 702-716, 2016.
15 Zhou Y, Zhu Z, Bai X, et al., "Non-stationary texture synthesis by adversarial expansion," ACM Transactions on Graphics, vol. 37, no. 4, pp. 1-13, 2018.
16 Luan F, Paris S, Shechtman E, et al., "Deep painterly harmonization," Computer graphics forum, vol. 37, no. 4, pp. 95-106, 2018.   DOI
17 Shocher A, Bagon S, Isola P, et al., "InGAN: Capturing and Remapping the "DNA" of a Natural Image," arXiv preprint arXiv:1812.00231, 2018.
18 Tilon S, Nex F, Kerle N, et al., "Post-Disaster Building Damage Detection from Earth Observation Imagery Using Unsupervised and Transferable Anomaly Detecting Generative Adversarial Networks," Remote Sensing, vol. 12, p. 4193, 2020.   DOI
19 Ledig C, Theis L, Huszar F, et al., "Photo-realistic single image super-resolution using a generative adversarial network," in Proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 105-114, 2017.
20 Bergmann U, Jetchev N, Vollgraf R, "Learning texture manifolds with the Periodic Spatial GAN," in Proc. of the 34th International Conference on Machine Learning, PMLR, vol. 70, pp. 469-477, 2017.
21 Hinz T, Fisher M, Wang O, et al., "Improved Techniques for Training Single-Image GANs," in Proc. of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pp. 1300- 1309, 2021.
22 Yi X, Walia E, and Babyn P, "Generative adversarial network in medical imaging: A review," Medical image analysis, vol. 58, 2019.
23 Wu B, Liu L, Dai Z, et al, "Detecting Malicious Social Robots with Generative Adversarial Networks," KSII Transactions on Internet and Information Systems, vol. 13, no. 11, pp. 5594-5515, 2019.   DOI
24 Karras T, Laine S, Aila T, "A Style-Based Generator Architecture for Generative Adversarial Networks," in Proc. of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4396-4405, 2019.
25 Goodfellow I J, Pouget-Abadie J, Mirza M, et al., "Generative adversarial nets," Advances in Neural Information Processing Systems, vol. 27, pp. 2672-2680, 2014.