Browse > Article
http://dx.doi.org/10.7471/ikeee.2022.26.2.176

A Scheme for Preventing Data Augmentation Leaks in GAN-based Models Using Auxiliary Classifier  

Shim, Jong-Hwa (Dept. of Electrical Engineering, Korea University)
Lee, Ji-Eun (Dept. of Electrical Engineering, Korea University)
Hwang, Een-Jun (Dept. of Electrical Engineering, Korea University)
Publication Information
Journal of IKEEE / v.26, no.2, 2022 , pp. 176-185 More about this Journal
Abstract
Data augmentation is general approach to solve overfitting of machine learning models by applying various data transformations and distortions to dataset. However, when data augmentation is applied in GAN-based model, which is deep learning image generation model, data transformation and distortion are reflected in the generated image, then the generated image quality decrease. To prevent this problem called augmentation leak, we propose a scheme that can prevent augmentation leak regardless of the type and number of augmentations. Specifically, we analyze the conditions of augmentation leak occurrence by type and implement auxiliary augmentation task classifier that can prevent augmentation leak. Through experiments, we show that the proposed technique prevents augmentation leak in the GAN model, and as a result improves the quality of the generated image. We also demonstrate the superiority of the proposed scheme through ablation study and comparison with other representative augmentation leak prevention technique.
Keywords
Data augmentation; GAN; Deep Learning; Augmentation Leak; Auxiliary Classifier;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 M. Arjovsky, S. Chintala and L. Bottou, "Wasserstein generative adversarial networks," International conference on machine learning, pp.214-223, 2017.
2 T. Karras et al., "A style-based generator architecture for generative adversarial networks." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401-4410, 2019.
3 H. Thanh-Tung et al., "Improving generalization and stability of generative adversarial networks," arXiv preprint, 2019. DOI: 10.48550/arXiv.1902.03984   DOI
4 A. Radford, L. Metz and S. Chintala, "Unsupervised representation learning with deep convolutional generative adversarial networks," arXiv preprint, 2015.
5 A. Kolesnikov, X. Zhai and L. Beyer "Revisiting self-supervised visual representation learning," Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp.1920-1929, 2019. DOI: 10.48550/arXiv.1901.09005   DOI
6 A. Krizhevsky, I. Sutskever and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Advances in neural information processing systems 2012, 2012.
7 L. Perez. and J. Wang, "The effectiveness of data augmentation in image classification using deep learning," arXiv preprint, 2017. DOI: 10.48550/arXiv.1712.04621
8 I. Goodfellow et al., "Generative adversarial nets," Advances in neural information processing systems 2014, 2014.
9 Z. Zhao et al, "Image augmentations for GAN training," arXiv preprint, 2020.
10 M. Heusel et al., "Gans trained by a two time-scale update rule converge to a local nash equilibrium," Advances in neural information processing systems, 2017.
11 T. Salimans et al., "Improved techniques for training GANs," Advances in neural information processing systems, 2016. DOI: 10.48550/arXiv.1606.03498   DOI
12 Z. Lin et al., "PacGAN: The power of two samples in generative adversarial networks," Advances in neural information processing systems, 2018. DOI: 10.1109/JSAIT.2020.2983071   DOI
13 Z. Cai et al., "Generative adversarial networks: A survey toward private and secure applications," ACM Computing Surveys, Vol.54, No.6, pp.1-38, 2021. DOI: 10.48550/arXiv.2106.03785   DOI
14 X. Mao et al., "Least squares generative adversarial networks," Proceedings of the IEEE international conference on computer vision, pp.2794-2802, 2017.
15 T. Karras et al., "Progressive growing of GANs for improved quality, stability, and variation," arXiv preprint, 2017.
16 K. He et al., "Deep residual learning for image recognition," Proceedings of the IEEE conference on computer vision and pattern recognition, pp.770-778, 2016. DOI: 10.1109/CVPR.2016.90   DOI
17 L. Metz et al., "Unrolled generative adversarial networks," arXiv preprint, 2016.
18 T. Miyato et al,. "Spectral normalization for generative adversarial networks," arXiv preprint, 2018. DOI: 10.48550/arXiv.1802.05957   DOI
19 Y. Yazici et al., "Empirical analysis of overfitting and mode drop in GAN training," IEEE International Conference on Image Processing, pp.1651-1655, 2020. DOI: 10.48550/arXiv.2006.14265   DOI
20 S. Loffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," International conference on machine learning, pp.448-456, 2015. DOI: 10.48550/arXiv.1502.03167   DOI
21 B. Zhou et al., "Learning deep features for discriminative localization," Proceedings of the IEEE conference on computer vision and pattern recognition, pp.2921-2929, 2016. DOI: 10.48550/arXiv.1512.04150   DOI
22 Z. Zhao et al., "Improved consistency regularization for GANs," AAAI Conference on Artificial Intelligence, 2021.
23 P. Isola et al., "Image-to-image translation with conditional adversarial networks," Proceedings of the IEEE conference on computer vision and pattern recognition, pp.1125-1134, 2017. DOI: 10.48550/arXiv.1611.07004   DOI
24 D. P. Kingma, and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint, 2014.
25 C. Szegedy et al., "Rethinking the inception architecture for computer vision," Proceedings of the IEEE conference on computer vision and pattern recognition, pp.2818-2826, 2016. DOI: 10.48550/arXiv.1512.00567   DOI
26 S. Yang et al., "From facial parts responses to face detection: A deep learning approach," Proceedings of the IEEE international conference on computer vision, pp.3676-3684, 2015. DOI: 10.1109/ICCV.2015.419   DOI
27 Shorten., Conner, and Taghi M. Khoshgoftaar, "A survey on image data augmentation for deep learning," Journal of big data, Vol.6, No.1, pp. 1-48, 2019. DOI: 10.1186/s40537-019-0197-0   DOI
28 N. t. Tran et al., "On data augmentation for GAN training," IEEE Transactions on Image Processing, Vol.30, pp.1882-1897, 2021. DOI: 10.1109/TIP.2021.3049346   DOI
29 H. Zhang et al., "Consistency regularization for generative adversarial networks," International Conference of Leaning Representations, 2020.
30 A. Radford., L. Metz and S. Chintala, "Unsupervised representation learning with deep convolutional generative adversarial networks," arXiv preprint, 2015.
31 Gulrajani et al., "Improved training of wasserstein GANs," Advances in neural information processing systems 2017, 2017.
32 U. Ruby, V. Yendapalli, "Binary cross entropy with deep learning technique for image classification," Int. J. Adv. Trends Comput. Sci. Eng, Vol.9, No.10, 2020. DOI: 10.30534/ijatcse/2020/175942020   DOI
33 F. Quiroga et al., "Revisiting data augmentation for rotational invariance in convolutional neural networks," International Conference on Modelling and Simulation in Management Sciences 2018, pp.127-141, 2018. DOI: 10.1007/978-3-030-15413-4_10   DOI
34 T. Karras et al., "Training generative adversarial networks with limited data. Advances in Neural Information Processing Systems," Advances in Neural Information Processing Systems 2020, pp. 12104-12114, 2020. DOI: 10.5555/3495724.3496739   DOI