Browse > Article
http://dx.doi.org/10.7472/jksii.2020.21.4.9

Image-to-Image Translation Based on U-Net with R2 and Attention  

Lim, So-hyun (Department of Computer Science, Kyonggi University)
Chun, Jun-chul (Department of Computer Science, Kyonggi University)
Publication Information
Journal of Internet Computing and Services / v.21, no.4, 2020 , pp. 9-16 More about this Journal
Abstract
In the Image processing and computer vision, the problem of reconstructing from one image to another or generating a new image has been steadily drawing attention as hardware advances. However, the problem of computer-generated images also continues to emerge when viewed with human eyes because it is not natural. Due to the recent active research in deep learning, image generating and improvement problem using it are also actively being studied, and among them, the network called Generative Adversarial Network(GAN) is doing well in the image generating. Various models of GAN have been presented since the proposed GAN, allowing for the generation of more natural images compared to the results of research in the image generating. Among them, pix2pix is a conditional GAN model, which is a general-purpose network that shows good performance in various datasets. pix2pix is based on U-Net, but there are many networks that show better performance among U-Net based networks. Therefore, in this study, images are generated by applying various networks to U-Net of pix2pix, and the results are compared and evaluated. The images generated through each network confirm that the pix2pix model with Attention, R2, and Attention-R2 networks shows better performance than the existing pix2pix model using U-Net, and check the limitations of the most powerful network. It is suggested as a future study.
Keywords
Image-to-Image Translation; conditional GAN; U-Net;
Citations & Related Records
Times Cited By KSCI : 6  (Citation Analysis)
연도 인용수 순위
1 D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros. "Context encoders: Feature learning by inpainting.", In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016. https://doi.org/10.1109/cvpr.2016.278
2 J. Long, E. Shelhamer, and T. Darrell. "Fully convolutional networks for semantic segmentation." In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2015. https://doi.org/10.1109/cvpr.2015.7298965
3 G. Larsson, M. Maire, and G. Shakhnarovich. "Learning representations for automatic colorization.", In Computer Vision - ECCV ,pp. 577-593, 2016. https://doi.org/10.1007/978-3-319-46493-0_35
4 Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, Alexei A. Efros, "Image-to-Image Translation with Conditional Adversarial Networks", In IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE., 2017.
5 Olaf Ronneberger, Philipp Fischer, and Thomas Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation", In Lecture Notes in Computer Science, pp.234-241, 2015. https://doi.org/10.1007/978-3-319-24574-4_28
6 Ozan Oktay, Jo Schlemper, Loic Le Folgoc, Matthew Lee, Mattias Heinrich, Kazunari Misawa, Kensaku Mori, Steven McDonagh, Nils Y Hammerla, Bernhard Kainz, Ben Glocker, Daniel Rueckert, "Attention U-Net: Learning Where to Look for the Pancreas", In MIDL, arXiv preprint arXiv:1804.03999, 2018. https://arxiv.org/abs/1804.03999
7 Md Zahangir Alom, Chris Yakopcic, Tarek M. Taha, and Vijayan K. Asari, "Nuclei Segmentation with Recurrent Residual Convolutional Neural Networks based U-Net (R2U-Net)", In NAECON - IEEE National Aerospace and Electronics Conference. IEEE, 2018. https://doi.org/10.1109/naecon.2018.8556686
8 Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, Yoshua Bengio, "Generative Adversarial Networks", In NIPS, arXiv preprint arXiv:1406.2661, 2014. https://arxiv.org/pdf/1406.2661.pdf
9 Alec Radford & Luke Metz, "Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks", In CoRR, arXiv preprint arXiv:1511.06434, 2016. https://arxiv.org/pdf/1511.06434.pdf
10 R. Mu, X. Zeng, "A Review of Deep Learning Research", In TIIS, Vol. 13, No.4, 2019. https://doi.org/10.3837/tiis.2019.04.001
11 Mehdi Mirza, Simon Osindero, "Conditional Generative Adversarial Nets", In CoRR, arXiv preprint arXiv:1411.1784, 2014. https://arxiv.org/pdf/1411.1784.pdf
12 Hee-jin Yoon, Kang-jik Kim, Jun-chul Chun, "GAN-based shadow removal using context information", Journal of Internet Computing and Services, Vol. 20, No. 6, pp. 29-36, 2019. https://doi.org/10.7472/jksii.2019.20.6.29   DOI
13 Zhang, H., Sindagi, V., & Patel, V. M. "Image De-raining Using a Conditional Generative Adversarial Network". IEEE Transactions on Circuits and Systems for Video Technology, 1-1. 2019. https://doi.org/10.1109/tcsvt.2019.2920407
14 Yao Wang, Woojin Jeong, Young Shik Moon, "Single Image Dehazing Based on Depth Map Estimation via Generative Adversarial Networks", Journal of Internet Computing and Services, Vol. 19, No. 5, pp. 43-54, 2018. https://doi.org/10.7472/jksii.2018.19.5.43   DOI
15 Eirikur Agustsson, Michael Tschannen, Fabian Mentzer, Radu Timofte, Luc Van Gool, "Generative Adversarial Networks for Extreme Learned Image Compression", The IEEE International Conference on Computer Vision (ICCV), pp. 221-231, 2019 https://arxiv.org/pdf/1804.02958.pdf
16 Liqun Chen, Yizhe Zhang, Ruiyi Zhang, Chenyang Tao, Zhe Gan, Haichao Zhang, Bai Li, Dinghan Shen, Changyou Chen, Lawrence Carin, "Improving Sequence-to-Sequence Learning via Optimal Transport", In ICLR, 2019. https://arxiv.org/pdf/1901.06283.pdf
17 Han Zhang, Ian Goodfellow, Dimitris Metaxas, Augustus Odena, "Self-Attention Generative Adversarial Networks", In CoRR, 2018. https://arxiv.org/pdf/1805.08318.pdf
18 Heusel, M., Ramsauer, H., Unterthiner, T., Nessler, B., & Hochreiter, S, "Gans trained by a two time-scale update rule converge to a local nash equilibrium", In Advances in Neural Information Processing Systems, pp. 6626-6637, 2017. https://arxiv.org/pdf/1706.08500.pdf