Browse > Article
http://dx.doi.org/10.3837/tiis.2019.08.025

Game Sprite Generator Using a Multi Discriminator GAN  

Hong, Seungjin (School of Games, Hongik University)
Kim, Sookyun (Department of Game Engineering, Paichai University)
Kang, Shinjin (School of Games, Hongik University)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.13, no.8, 2019 , pp. 4255-4269 More about this Journal
Abstract
This paper proposes an image generation method using a Multi Discriminator Generative Adversarial Net (MDGAN) as a next generation 2D game sprite creation technique. The proposed GAN is an Autoencoder-based model that receives three areas of information-color, shape, and animation, and combines them into new images. This model consists of two encoders that extract color and shape from each image, and a decoder that takes all the values of each encoder and generates an animated image. We also suggest an image processing technique during the learning process to remove the noise of the generated images. The resulting images show that 2D sprites in games can be generated by independently learning the three image attributes of shape, color, and animation. The proposed system can increase the productivity of massive 2D image modification work during the game development process. The experimental results demonstrate that our MDGAN can be used for 2D image sprite generation and modification work with little manual cost.
Keywords
Generative Adversarial Nets (GANs); 2D Game Sprite; Deep Learning; Game Development; Neural Network;
Citations & Related Records
연도 인용수 순위
  • Reference
1 M. Mathieu, C. Couprie, and Y. LeCun, "Deep multiscale video prediction beyond mean square error," in Proc. of international conference on learning representations, arXiv preprint arXiv:1511.05440, 2016.
2 C. Vondrick, H. Pirsiavash, and A. Torralba, "Generating videos with scene dynamics," In advances in neural information processing systems, pp. 613-621, 2016.
3 J. Wu, C. Zhang, T. Xue, B. Freeman, and J. Tenenbaum, "Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling," advances in neural information processing systems, pp. 82-90, 2016.
4 A. Radford, L. Metz, and S. Chintala, "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks," in Proc. of international conference on learning representations (ICLR), arXiv preprint arXiv:1511.06434, 2016.
5 A. Makhzani, J. Shlens, N. Jaitly, and I. Goodfellow, "Adversarial Autoencoders. In International Conference on Learning Representations," arXiv preprint arXiv:1511.05644, 2016.
6 P. Isola, J. Y. Zhu, T. Zhou, and A. A. Efros, "Image-to-image translation with conditional adversarial networks," in Proc. of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5967 - 5976, 2017.
7 O. Ronneberger, P. Fischer, and T. Brox, "U-net: Convolutional networks for biomedical image segmentation," in Proc. of MICCAI, pp. 234-241, 2015.
8 J. Donahue, P. Krahenbühl, and T. Darrell, "Adversarial feature learning," arXiv preprint arXiv:1605.09782, 2016.
9 J. Y. Zhu, T. Park, P. Isola, and A. A. Efros, "Unpaired image-to-image translation using cycle-consistent adversarial networks," in Proc. of the IEEE international conference on computer vision, pp. 2223-2232, 2017.
10 T. Kim, M. Cha, H. Kim, J. K. Lee, and J. Kim, "Learning to discover cross-domain relations with generative adversarial networks," arXiv:1703.05192, 2017.
11 Y. Choi, M. Choi, M. Kim, J. W. Ha, S. Kim, and J. Choo, "StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation," in Proc. of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8789 - 8797, 2018.
12 D. Joo, D. Kim, and J. Kim, "Generating a Fusion Image: One's Identity and Another's Shape," in Proc. of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1635 - 1643, 2018.
13 D. Berthelot, T. Schumm, and L. Metz, "Began: Boundary equilibrium generative adversarial networks," arXiv preprint arXiv:1703.10717, 2017.
14 R. Jain, A. Isaksen, C. Holmgard, and J. Togelius, "Autoencoders for level generation, repair, and recognition," in Proc. of the ICCC workshop on computational creativity and games, 2016.
15 T. Xue, J. Wu, K. Bouman, and B. Freeman, "Visual dynamics: Probabilistic future frame synthesis via cross convolutional networks," advances in neural information processing systems, pp. 91-99, 2016.
16 S. E. Reed, Y. Zhang, Y. Zhang, and H. Lee, "Deep visual analogy-making," advances in neural information processing systems, pp. 1252-1260, 2015.
17 A. Summerville, M. Guzdial, M. Mateas, and M. O. Riedl, "Learning player tailored content from observation: Platformer level generation from video traces using lstms," in Proc. of artificial intelligence and interactive digital entertainment conference, 2016.
18 S. Snodgrass, and S. Ontanon, “Learning to generate video game maps using markov models,” IEEE transactions on computational intelligence and AI in games, Vol. 9, No. 4, pp. 410-422, 2016.   DOI
19 L., Horsley and D. Perez-Liebana, "Building an automatic sprite generator with deep convolutional generative adversarial networks," in Proc. of 2017 IEEE conference on computational intelligence and games (CIG), pp. 134-141, 2017.
20 H. H. Kim, "Content generation using variational auto-encoder," in Proc. of Nexon developer conference (NDC), 2017.
21 C. Beckham, and C. Pal, "A step towards procedural terrain generation with GANs," arXiv preprint arXiv:1707.03383, 2017.
22 E. Giacomello, P. L. Lanzi, and D. Loiacono, "DOOM level generation using generative adversarial networks," in Proc. of 2018 IEEE games, entertainment, media conference (GEM), pp. 316-323, 2018.
23 E. L. Denton, S. Chintala, and R. Fergus, "Deep generative image models using a laplacian pyramid of adversarial networks," advances in neural information processing systems, pp. 1486-1494, 2015.
24 K. Simonyan, and A. Zisserman, "Very deep convolutional networks for large-scale image recognition," arXiv preprint arXiv:1409.1556, 2014.
25 C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, "Rethinking the inception architecture for computer vision," in Proc. of the IEEE conference on computer vision and pattern recognition, pp. 2818-2826, 2016.
26 K. He, X. Zhang, S. Ren and J. Sun, "Deep residual learning for image recognition," in Proc. of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
27 I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, and Y. Bengio, "Generative adversarial nets," advances in neural information processing systems, pp. 2672-2680, 2014.
28 Jie Li, Junjie Jia, Donglai Xu, "Unsupervised representation learning with deep convolutional generative adversarial networks," in Proc. of 2018 37th Chinese Control Conference (CCC), 2018.
29 A. Radford, L. Metz, and S. Chintala., "Unsupervised representation learning with deep convolutional generative adversarial networks," arXiv preprint arXiv:1511.06434, 2015.
30 J. Y. Zhu, P. Krähenbuhl, E. Shechtman, and A. A. Efros, "Generative visual manipulation on the natural image manifold," in Proc. of european conference on computer vision, pp. 597-613, 2016
31 T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and X. Chen, "Improved techniques for training gans," arXiv preprint arXiv:1606.03498, 2016.
32 D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros, "Context encoders: Feature learning by inpainting," in Proc. of the IEEE conference on computer vision and pattern recognition, 2016.
33 M. F. Mathieu, J. Zhao, A. Ramesh, P. Sprechmann, and Y. LeCun. "Disentangling factors of variation in deep representation using adversarial training," advances in neural information processing systems, pp. 5040-5048, 2016.
34 S. Reed, Z. Akata, X. Yan, L. Logeswaran, B. Schiele, and H. Lee, "Generative adversarial text to image synthesis," arXiv preprint arXiv:1605.05396, 2016.