Browse > Article
http://dx.doi.org/10.3745/KTSDE.2021.10.11.457

Adversarial Learning-Based Image Correction Methodology for Deep Learning Analysis of Heterogeneous Images  

Kim, Junwoo (국민대학교 비즈니스IT)
Kim, Namgyu (국민대학교 비즈니스IT전문대학원)
Publication Information
KIPS Transactions on Software and Data Engineering / v.10, no.11, 2021 , pp. 457-464 More about this Journal
Abstract
The advent of the big data era has enabled the rapid development of deep learning that learns rules by itself from data. In particular, the performance of CNN algorithms has reached the level of self-adjusting the source data itself. However, the existing image processing method only deals with the image data itself, and does not sufficiently consider the heterogeneous environment in which the image is generated. Images generated in a heterogeneous environment may have the same information, but their features may be expressed differently depending on the photographing environment. This means that not only the different environmental information of each image but also the same information are represented by different features, which may degrade the performance of the image analysis model. Therefore, in this paper, we propose a method to improve the performance of the image color constancy model based on Adversarial Learning that uses image data generated in a heterogeneous environment simultaneously. Specifically, the proposed methodology operates with the interaction of the 'Domain Discriminator' that predicts the environment in which the image was taken and the 'Illumination Estimator' that predicts the lighting value. As a result of conducting an experiment on 7,022 images taken in heterogeneous environments to evaluate the performance of the proposed methodology, the proposed methodology showed superior performance in terms of Angular Error compared to the existing methods.
Keywords
Adversarial Learning; Color Constancy; Heterogeneous Images; Illumination Estimation; Image Correction;
Citations & Related Records
연도 인용수 순위
  • Reference
1 T. Kim and N. Kim, "Deep learning-based professional image interpretation using expertise transplant," Journal of Intelligence and Information Systems, Vol.26, No.2, pp.79-104, 2020.   DOI
2 H. Kim, H. S. Oh, and D. Kim, "CNN architecture predicting movie rating from audience's reviews written in Korean," KIPS Transactions on Computer and Communication Systems, Vol.9, No.1, pp.17-24, 2020.   DOI
3 F. Laakom, J. Raitoharju, J. Nikkanen, A. Iosifidis, and M. Gabbouj, "Intel-tau: A color constancy dataset," IEEE Access, Vol.9, pp.39560-39567, 2021.   DOI
4 K. F. Yang, S. B. Gao, and Y. J. Li, "Efficient illuminant estimation for color constancy using grey pixels," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.2254-2263, 2015.
5 J. Xiao, S. Gu, and L. Zhang, "Multi-domain learning for accurate and few-shot color constancy," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.3258-3267, 2020.
6 Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, and V. Lempitsky, "Domain-adversarial training of neural networks," Journal of Machine Learning Research, Vol.17, No.1, pp.2096-2030, 2016.
7 E. Land, "The retinex theory of color vision," Scientific American, Vol.237, No.6, pp.108-128, 1977.   DOI
8 A. Gijsenij, T. Gevers, and J. V. Weijer, "Computational color constancy: Survey and experiments," IEEE Transactions on Image Processing, Vol.20, No.9, pp.2475-2489, 2011.   DOI
9 G. Buchsbaum, "A spacial processor model for object color perception," Journal of the Franklin institute, Vol.310, No.1, pp.1-26, 1980.   DOI
10 D. Cheng, B. Price, S. Cohen, and M. S. Brown, "Effective learning-based illuminant estimation using simple features," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1000-1008, 2015.
11 J. T Barron and Y. T. Tsai, "Fast fourier color constancy," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp.886-894, 2017.
12 J. T. Barron, "Convolutional color constancy," in Proceedings of the IEEE International Conference on Computer Vision, pp.379-387, 2015.
13 I. Goodfellow, J. P. Abadie, M. Mirza, B. Xu, D. W. Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets," Advances in Neural Information Processing Systems, pp.2672-2680, 2014.
14 Z. Yang, J. Hu, R. Salakhutdinov, and W. Cohen, "Semi-supervised qa with generative domain-adaptive nets," arXiv preprint arXiv:1702.02206, 2017.
15 H. Yuzuguzel, "Learning colour constancy using convolutional neural networks," MS thesis, Tampere University of Technology, Finland, 2015.
16 J. V. Weijer, T. Gevers, and A. Gijsenji, "Edge-based color constancy," IEEE Transactions on Image Processing, Vol.16, No.9, pp.2207-2214, 2007.   DOI
17 K. Barnard and G. Finlayson, "Shadow identification using colour ratios," in Proceedings of the IS&T's Color and Imaging Conference, Vol.2000, No.1, pp.97-101, 2000.
18 Y. Ganin and V. Lempitsky, "Unsupervised domain adaptation by backpropagation," in International Conference on Machine Learning, pp.1180-1189, 2015.
19 B. Funt and W. Xiong, "Estimating illumination chromaticity via support vector regression," in Proceedings of the IS&T's Color and Imaging Conference, Vol.2004, No.1, pp.47-52, 2004.
20 S. Bianco, C. Cusano, and R. Schettini, "Color constancy using cnns," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp.81-89, 2015.
21 Y. Hu, B. Wang, and S. Lin, "Fc4: Fully convolutional color constancy with confidence weighted pooling," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.4085-4094, 2017.
22 M. Rizzo, C. Conati, D. Jang, and H. Hu, "Cascading convolutional temporal colour constancy," arXiv preprint arXiv: 2106.07955, 2021.
23 Y. Wang, F. Ma, Z. jin, Y. Yuan, G. Xun, K. Jha, L. Su, and J. Gao, "Eann: Event adversarial neural networks for multimodal fake news detection," in Proceedings of the 24th ACM SIGKDD International Conference On Knowledge Discovery & Data Mining, pp.849-857, 2018.
24 H. R. V. Joze, M. S. Drew, G. D. Finlayson, and P. A. T. Rey, "The role of bright pixels in illumination estimation," in Proceedings of the IS&T's Color and Imaging Conference, Vol.2012, No.1, pp.41-46, 2012.
25 D. Li, D. Chen, L. Shi, B. Jin, J. Goh, and S. K. Ng, "MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks," in International Conference on Artificial Neural Networks, Springer, pp.703-716, 2019.
26 B. Sun, J. Feng, and K. Saenko, "Return of frustratingly easy domain adaptation," in Proceedings of the AAAI Conference on Artificial Intelligence, Vol.30, No.1, 2016.