The Journal of the Korea institute of electronic communication sciences (한국전자통신학회논문지)

Korea Institute of Electronic Communication Science (한국전자통신학회)
권호 표지
DOI QR코드

DOI QR Code

A Study on Image Creation and Modification Techniques Using Generative Adversarial Neural Networks

생성적 적대 신경망을 활용한 부분 위변조 이미지 생성에 관한 연구

  • 송성헌 (동서대학교 소프트웨어학과) ;
  • 최봉준 (동서대학교 소프트웨어융합대학) ;
  • 문미경 (동서대학교 소프트웨어학과)
  • Received : 2022.03.07
  • Accepted : 2022.04.17
  • Published : 2022.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

A generative adversarial network (GAN) is a network in which two internal neural networks (generative network and discriminant network) learn while competing with each other. The generator creates an image close to reality, and the delimiter is programmed to better discriminate the image of the constructor. This technology is being used in various ways to create, transform, and restore the entire image X into another image Y. This paper describes a method that can be forged into another object naturally, after extracting only a partial image from the original image. First, a new image is created through the previously trained DCGAN model, after extracting only a partial image from the original image. The original image goes through a process of naturally combining with, after re-styling it to match the texture and size of the original image using the overall style transfer technique. Through this study, the user can naturally add/transform the desired object image to a specific part of the original image, so it can be used as another field of application for creating fake images.

생성적 적대 신경망(Generative Adversarial Networks, GAN)은 내부의 두 신경망(생성망, 판별망)이 상호 경쟁하면서 학습하는 네트워크이다. 생성자는 현실과 가까운 이미지를 만들고, 구분자는 생성자의 이미지를 더 잘 감별하도록 프로그래밍 되어있다. 이 기술은 전체 이미지 X를 다른 이미지 Y로 생성, 변환 및 복원하기 위해 다양하게 활용되고 있다. 본 논문에서는 원본 이미지에서 부분 이미지만 추출한 후, 이를 자연스럽게 다른 객체로 위변조할 수 있는 방법에 관해 기술한다. 먼저 원본 이미지에서 부분 이미지만 추출한 후, 기존에 학습시켜놓은 DCGAN 모델을 통해 새로운 이미지를 생성하고, 이를 전체적 스타일 전이(overall style transfer) 기술을 사용하여 원본 이미지의 질감과 크기에 어울리도록 리스타일링(re-styling) 한 후, 원본 이미지에 자연스럽게 결합하는 과정을 거친다. 본 연구를 통해 원본 이미지의 특정 부분에 사용자가 원하는 객체 이미지를 자연스럽게 추가/변형할 수 있음으로써 가짜 이미지 생성의 또 다른 활용 분야로 사용될 수 있을 것이다.

Keywords

Acknowledgement

이 논문은 2021년도 동서대학교 "Dongseo Cluster Project"지원에 의하여 이루어진 것임 (DSU-20210007)

References

  1. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, A. Ozair, S., Courville, and Y. Bengio, "Generative adversarial nets," Report, 2014.
  2. Y. Jeong and G. Choi, "Efficient iris recognition using deep-learning convolution neural network(CNN)," J. of the Korea Institute of Electronic Communication Sciences, vol. 15, no. 3, 2020, pp. 521-526. https://doi.org/10.13067/JKIECS.2020.15.3.521
  3. J. Yoo, "A Design of Small Scale Deep CNN Model for Facial Expression Recognition using the Low Resolution Image Datasets," J. of the Korea Institute of Electronic Communication Sciences, vol. 16, no. 1, 2021, pp. 75-80. https://doi.org/10.13067/JKIECS.2021.16.1.75
  4. C. Donahue, J. Macauley, and M. Puckette. "Adversarial audio synthesis," Report, 2018.
  5. S. Huang, Q. Li, C. Anil, X. Bao, S. Oore, and R. Grosse, "Timbretron: A wavenet (cyclegan (cqt (audio))) pipeline for musical timbre transfer," Report, 2018.
  6. R. Tolosana, R. Vera-Rodriguez, J. Fierrez, A. Morales, and J. Ortega-Garcia, "Deepfakes and beyond: A survey of face manipulation and fake detection," Information Fusion, vol. 64, 2020, pp. 131-148. https://doi.org/10.1016/j.inffus.2020.06.014
  7. A. Radford, L. Metz, and S. Chintala. "Unsupervised representation learning with deep convolutional generative adversarial networks," Report, 2015.
  8. P. Isola, J. Zhu, T. Zhou, and A. A. Efros, "Image-to-Image Translation with Conditional Adversarial Networks," In Proc. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, June 2017, pp. 5967-5976.
  9. T. Park, M. Liu, T. Wang, and J. Zhu, "Semantic image synthesis with spatially-adaptive normalization," In Proc. of the IEEE/CVF conference on computer vision and pattern recognition, Long Beach USA. June 2019, pp. 2337-2346.
  10. W. Sun and T. Wu. "Image synthesis from reconfigurable layout and style," In Proc. of the IEEE/CVF International Conference on Computer Vision, Seoul, Korea, Oct. 2019.
  11. J. Y. Zhu, T. Park, P. Isola, and A. Efros, "Unpaired image-to-image translation using cycle-consistent adversarial networks," In Proc. of the IEEE international conference on computer vision, Venice Italy, Oct. 2017, pp. 2223-2232.
  12. Y. Choi, M. Choi, M. Kim, J. Ha, S. Kim, and J. Choo, "Stargan: Unified generative adversarial networks for multi-domain image-to-image translation," In Proc. of the IEEE conference on computer vision and pattern recognition, Salt Lake City, USA, June 2018, pp. 8789-8797.
  13. T. Karras, S. Laine, and T. Aila. "A style-based generator architecture for generative adversarial networks," In Proc. of the IEEE/CVF conference on computer vision and pattern recognition, Long Beach USA. June 2019, pp. 4401-4410.
  14. L. A. Gatys, A. S. Ecker, and M. Bethge. "Image style transfer using convolutional neural networks," In Proc. of the IEEE conference on computer vision and pattern recognition, Las Vegas, USA, June 2016, pp.2414-2423.
  15. Bau, D., J. Zhu, H. Strobelt, B. Zhou, J. Tenenbaum, W. T. Freeman, and A. Torralba, "Gan dissection: Visualizing and understanding generative adversarial networks," Report, 2018.
  16. K. Nazeri, E. Ng, T. Joseph, F. Z. Qureshi, and M. Ebrahimi, "Edgeconnect: Generative image inpainting with adversarial edge learning," Report, 2019.
  17. A. Telea. "An image inpainting technique based on the fast marching method," Journal of graphics tools, vol. 9, no. 1, 2004, pp. 23-34. https://doi.org/10.1080/10867651.2004.10487596
  18. M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, "Image inpainting," In Proc. of the 27th annual conference on Computer graphics and interactive techniques, New Orleans, USA, July 2000, pp.417-424.