Browse > Article
http://dx.doi.org/10.15701/kcgs.2018.24.2.1

Deep Learning based Color Restoration of Corrupted Black and White Facial Photos  

Woo, Shin Jae (Dept. of Convergence Software, Hallym University)
Kim, Jong-Hyun (School of Software Application, Kangnam University)
Lee, Jung (Dept. of Convergence Software, Hallym University)
Song, Chang-Germ (Dept. of Convergence Software, Hallym University)
Kim, Sun-Jeong (Dept. of Convergence Software, Hallym University)
Publication Information
Journal of the Korea Computer Graphics Society / v.24, no.2, 2018 , pp. 1-9 More about this Journal
Abstract
In this paper, we propose a method to restore corrupted black and white facial images to color. Previous studies have shown that when coloring damaged black and white photographs, such as old ID photographs, the area around the damaged area is often incorrectly colored. To solve this problem, this paper proposes a method of restoring the damaged area of input photo first and then performing colorization based on the result. The proposed method consists of two steps: BEGAN (Boundary Equivalent Generative Adversarial Networks) model based restoration and CNN (Convolutional Neural Network) based coloring. Our method uses the BEGAN model, which enables a clearer and higher resolution image restoration than the existing methods using the DCGAN (Deep Convolutional Generative Adversarial Networks) model for image restoration, and performs colorization based on the restored black and white image. Finally, we confirmed that the experimental results of various types of facial images and masks can show realistic color restoration results in many cases compared with the previous studies.
Keywords
Inpainting; Colorization; Deep Learning; BEGAN; CNN;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Radford, A., Metz, L., & Chintala, S., "Unsupervised representation learning with deep convolutional generative adversarial networks," arXiv preprint, arXiv:1511.06434, 2015.
2 Yeh, R. A., Chen, C., Lim, T. Y., Schwing, A. G., Hasegawa-Johnson, M., & Do, M. N., "Semantic image inpainting with deep generative models," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5485-5493, 2017.
3 Berthelot, D., Schumm, T., & Metz, L., "Began: Boundary equilibrium generative adversarial networks," arXiv preprint, arXiv: 1703.10717, 2017.
4 LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., & Jackel, L. D., "Backpropagation applied to handwritten zip code recognition," Neural computation, Vol. 1, No. 4, pp. 541-551, 1989.   DOI
5 Bertalmio, M., Sapiro, G., Caselles, V., & Ballester, G., "Image inpainting," Proceedings of the 27th annual conference on Computer graphics and interactive techniques, pp. 417-424, 2000.
6 Efros, A. A., and Leung T. K. ''Texture synthesis by non-parametric sampling," Proceedings of the Seventh IEEE International Conference on Computer Vision, Vol. 2, pp. 1033-1038, 1999.
7 Hays, J., & Efros, A. A., "Scene completion using millions of photographs," ACM Transactions on Graphics, Vol. 26, No. 3, p. 4, 2007.   DOI
8 Pathak, D., krahenbuhl, P., Donahue, J. Darrell, T., & Efros, A. A., "Context encoders: Feature learning by inpainting," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2536-2544, 2016.
9 Levin, A., Lischinski, D., & Weiss, Y., "Colorization using optimization," ACM transactions on graphics, Vol. 23, No. 3, pp. 689-694, 2004.   DOI
10 Welsh, T., Ashikhmin, M., & Mueller, K., "Transferring color to greyscale image," ACM Transactions on Graphics, Vol. 21, No. 3, pp. 277-280, 2002.
11 Iizuka, S., Simo-Serra, E., & Ishikawa, H., "Let there be color!: joint end-to-end learning of global and local image priors for automatic linage colorization with simultaneous classification," ACM Transactions on Graphics, Vol. 35, No. 4, p. 110, 2016.
12 Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., & Bengio, Y., "Generative adversarial nets," Advances in neural information processing systems, pp. 2672-2680, 2014.
13 Noh, H., Hong, S., & Han, B., "Learning deconvolution network for semantic segmentation," Proceedings of the IEEE International Conference on Computer Vision, pp. 1520-1528, 2015.
14 Bengio, Y., "Learning deep architectures for AI," Foundations and trends(R) in Machine Learning, Vol. 2, No. 1, pp. 1-127, 2009.   DOI
15 Arjovsky, M., Chintala, S., & Bottou, L., "Wasserstein gan," arXiv preprint, arXiv: 1701.07875, 2017.
16 Perez, P., Gangnet, M., & Blake, A., "Poisson linage editing," ACM Transactions on graphics, Vol. 22, No. 3, pp. 313-318, 2003.
17 Sun, T., Chen, Y., Wang, X., & Tang, X., "Deep learning face representation by joint identification-verification," Advances in neural information proceesing systems, pp. 1988-1996, 2014.
18 Viola, P ., & Jones, M., "Rapid object detection using a boosted cascade of simple features," Proceedings of the 200I IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol. 1, pp. 1-1, 2001.
19 Huang, G. B., Ramesh, M., Berg, T., & Learned-Miller, E., "Labeled faces in the wild: A database for studying face recognition in unconstrained environments," Technical Report 07-49, University of Massachusetts, Amherst, Vol. 1, No. 2, p. 3, 2007.