Browse > Article
http://dx.doi.org/10.9708/jksci.2022.27.07.027

A Multi-domain Style Transfer by Modified Generator of GAN  

Lee, Geum-Boon (SW Convergence Education Institute, Chosun University)
Publication Information
Journal of the Korea Society of Computer and Information / v.27, no.7, 2022 , pp. 27-33 More about this Journal
Abstract
In this paper, we propose a novel generator architecture for multi-domain style transfer method not an image to image translation, as a method of generating a styled image by transfering a style to the content image. A latent vector and Gaussian noises are added to the generator of GAN so that a high quality image is generated while considering the characteristics of various data distributions for each domain and preserving the features of the content data. With the generator architecture of the proposed GAN, networks are configured and presented so that the content image can learn the styles for each domain well, and it is applied to the domain composed of images of the four seasons to show the high resolution style transfer results.
Keywords
Multi domain; Latent vector; Generator architecture; Gaussian noise; Style transfer;
Citations & Related Records
연도 인용수 순위
  • Reference
1 X. Huang, M.-Y. Liu, S. Belongie, and J. Kautz, "Multimodal unsupervised image-to-image translation," 2018. arXiv preprint arXiv: https://arxiv.org/abs/1812.04948,
2 Y. Choi, Y. Uh, J. Yoo, and J-W. Ha, "StarGAN v2: Diverse Image Synthesis for Multiple Domains," 2020. arXiv preprint arXiv: https://arxiv.org/abs/1912.01865
3 T. Karras, S. Laine, and T. Aila, "A Style-Based Generator architecture for Generative Adeversarial Networks," 2019. https://arxiv.org/abs/1812.04948
4 S. Na, S. Yoo, and J. Choo, "MISO: Mutual Information Loss with Stochastic Style Representaions for Multimodal Image-to-Image Translation," 2019. arXiv preprint arXiv: https://arxiv.org/abs/1902.03938
5 J-Y Zhu, R. Z, D, Pathak, T. Darrell, A. A. Efros, O. Wang and E. Shechtman, "Toward Multimodal Image-to-Image Translation," 2017. arXiv preprint arXiv: https://arxiv.org/abs/1711.11586
6 P. Isola, J-Y. Zhu, T. Zhou, and A. A. Efros, "Image-to-Image Translation with Conditional Adversarial Networks," 2017. arXiv preprint arXiv: https://arxiv.org/abs/1611.07004
7 J-Y. Zhu, T. Park, P. Isola, and A. A. Efros. "Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks," 2018. arXiv preprint arXiv: https://arxiv.org/abs/1703.10593
8 Q. Mao, H-Y. Lee, H-Y. Tseng, S. Ma, and M-H. Yang, "Mode Seeking Generative Adversarial Networks for Divese Image Synthesis," 2019. arXiv preprint arXiv: https://arxiv.org/abs/1903.05628
9 F. Luan, S. Paris, E. Shechtman, and K. Bala. "Deep Photo Style Transfer," 2017. arXiv preprint arXiv: https://arxiv.org/abs/1703.07511
10 Y. Choi, M. Choi, M. Kim, J.-W. Ha, S. Kim, and J. Choo, "StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation," 2018. arXiv preprint arXiv: https://arxiv.org/abs/1711.09020
11 D. P. Kingma and M. Welling, "Auto-Encoding Variaitonal Bayes," 2014. arXiv preprint arXiv: https://arxiv.org/abs/1312.6114
12 A. Odena, C. Olah, and J. Shlens, "Conditional Image Synthesis With Auxiliary Classifier GANs," 2017. arXiv preprint arXiv: https://arxiv.org/abs/1610.09585
13 I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative Adversarial Nets," 2014. arXiv preprint arXiv: https://arxiv.org/abs/1406.2661
14 L. A. Gatys, A. S. Ecker, and Matthias Bethge, "Image Style Transfer Using Convolutional Neural Networks," In Proceeding of IEEE Conference on Computer Vision and Pattern Recognition. CVPR. Las Vegas, NV, pp. 2414-2423, 2016.
15 E. Risser, P. Wilmot, and C. Barnes, "Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses," 2017. arXiv preprint arXiv: https://arxiv.org/abs/1701.08893
16 D. Ulyanov, A. Vedaldi, and V. Lempitsky, "Improved texture networks: Maximizing quality and diversity in feed-forward stylization and texture synthsesis," 2017. arXiv preprint arXiv: https://arxiv.org/pdf/1701.02096
17 A. Almahairi, S. Rajeshwar, A. Sordoni, P. Bachman, and A. Courville, "Augmented cyclegan: Learning many-to-many mappings from unpaired data," In Proceeding of the 35th International Conference on Machine. PMLR. Stockholmsmassan, pp. 195-204, 2018.
18 S. Loffe and C. Szegedy, "Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift," 2015. arXiv preprint arXiv: https://arxiv.org/abs/1502.03167
19 J. Chang, Q. Mao, Z. Zhao, S. Wang, S. Wang, H. Zhu and S. Ma, "Layered Conceptual Image Compression Via Deep Semantic Synthesis," 2019. In Proceeding of the IEEE Conference on Image Processing. ICIP. Taipei, Taiwan, 694-698. DOI: https://doi.org/10.1109/ICIP.2019.8803805.   DOI
20 A. Radford, L. Mets, and S. Chintala, "Unsupervised representation learning with deep convolutional generative adversarial network," 2016. arXiv preprint arXiv: https://arxiv.org/abs/1511.06434
21 Y. Shen, C. Yang, X. Tang, and B. Zhou, "InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs," 2020. arXiv preprint arXiv: https://arxiv.org/pdf/2005.09635
22 D. Ylyanov, V.Lebedev, A. Vedaldi, and V. Lempitsky, "Texture networks: Feed-forward synthesis of textures and stylized images," 2016. arXiv preprint arXiv: https://arxiv.org/pdf/2005.09635
23 M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler and S. Hochreiter, "GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium," 2018. arXiv preprint arXiv: https://arxiv.org/abs/1706.08500
24 X. Huang and S. Belongie, "Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization," 2017. arXiv preprint arXiv: https://arxiv.org/abs/1703.06868