Journal of Semiconductor Engineering

The Institute of Semiconductor Engineers (반도체공학회)
권호 표지
DOI QR코드

DOI QR Code

GAN-based Color Palette Extraction System by Chroma Fine-tuning with Reinforcement Learning

  • Kim, Sanghyuk (QCT WT RF SW Applications Engineering, Qualcomm CDMA Technologies) ;
  • Kang, Suk-Ju (Department of Electronic Engineering, Sogang University)
  • Received : 2020.11.24
  • Accepted : 2021.01.22
  • Published : 2021.03.31
Download PDF
⟨ Previous Next ⟩

Abstract

As the interest of deep learning, techniques to control the color of images in image processing field are evolving together. However, there is no clear standard for color, and it is not easy to find a way to represent only the color itself like the color-palette. In this paper, we propose a novel color palette extraction system by chroma fine-tuning with reinforcement learning. It helps to recognize the color combination to represent an input image. First, we use RGBY images to create feature maps by transferring the backbone network with well-trained model-weight which is verified at super resolution convolutional neural networks. Second, feature maps are trained to 3 fully connected layers for the color-palette generation with a generative adversarial network (GAN). Third, we use the reinforcement learning method which only changes chroma information of the GAN-output by slightly moving each Y component of YCbCr color gamut of pixel values up and down. The proposed method outperforms existing color palette extraction methods as given the accuracy of 0.9140.

Keywords

References

  1. W. K. Leow, and R. Li, " The analysis and applications of adaptive-binning color histograms," Computer Vision and Image Understanding, vol. 94. No. 1-3, pp.67-91, 2004. https://doi.org/10.1016/j.cviu.2003.10.010
  2. A. Ahmad, and L. Dey, "A k-mean clustering algorithm for mixed numeric and categorical data," Data & Knowledge Engineering, vol. 63, no. 2, pp. 503-527, 2007. https://doi.org/10.1016/j.datak.2007.03.016
  3. I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative Adversarial Nets," Proceedings of Advances in Neural Information Processing Systems 27 (NIPS 2014), pp. 2672-2680, 2014.
  4. V. Mnih, K. Kavukcuoglu, D. Silver, A. Graves, I. Antono glou, D. Wierstra, and M. Riedmiller, "Playing Atari with Deep Reinforcement Learning," arXiv, 2013.
  5. C. Dong, C. C. Loy, K. He and X. Tang, "Image Super-Resolu tion Using Deep Convolutional Networks," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 38, no. 2, pp. 295-307, 2015. https://doi.org/10.1109/TPAMI.2015.2439281
  6. C. Tan, F. Sun, T. Kong, W. Zhang, C. Yang, and C. Liu, "A Survey on Deep Transfer Learning," arXiv, 2018.
  7. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "ImageNet Classification with Deep Convolutional Neural Networks," Proceedings of Advances in Neural Information Processing Systems 25 (NIPS 2012), pp. 1097-1105, 2012.
  8. W. Liu, Y. Wen, Z. Yu, and M. Yang, "Large-Margin Soft max Loss for Convolutional Neural Networks," Proceedings of International Conference on Machine Learning (ICML), vol. 2, no. 3, p. 7, 2016.
  9. Colormind, http://colormind.io/
  10. D. Bloomberg, "Color Quantization Using Modified Median Cut," http://leptonica.com/papers/mediancut.pdf, 2008.
  11. B. Lim, S. Son, H. Kim, S. Nah, and K. M. Lee, "En hanced Deep Residual Networks for Single Image Super-Resolution," arXiv, 2017.
  12. Y. Zhang, Y. Tian, Y. Kong, B. Zhong, and Y. Fu, "Resid ual Dense Network for Image Super-Resolution," arXiv, 2018.