KIPS Transactions on Software and Data Engineering (정보처리학회논문지:소프트웨어 및 데이터공학)

Korea Information Processing Society (한국정보처리학회)
권호 표지
DOI QR코드

DOI QR Code

Adversarial Learning-Based Image Correction Methodology for Deep Learning Analysis of Heterogeneous Images

이질적 이미지의 딥러닝 분석을 위한 적대적 학습기반 이미지 보정 방법론

  • Received : 2021.09.28
  • Accepted : 2021.10.11
  • Published : 2021.11.30
Download PDF
⟨ Previous Next ⟩

Abstract

The advent of the big data era has enabled the rapid development of deep learning that learns rules by itself from data. In particular, the performance of CNN algorithms has reached the level of self-adjusting the source data itself. However, the existing image processing method only deals with the image data itself, and does not sufficiently consider the heterogeneous environment in which the image is generated. Images generated in a heterogeneous environment may have the same information, but their features may be expressed differently depending on the photographing environment. This means that not only the different environmental information of each image but also the same information are represented by different features, which may degrade the performance of the image analysis model. Therefore, in this paper, we propose a method to improve the performance of the image color constancy model based on Adversarial Learning that uses image data generated in a heterogeneous environment simultaneously. Specifically, the proposed methodology operates with the interaction of the 'Domain Discriminator' that predicts the environment in which the image was taken and the 'Illumination Estimator' that predicts the lighting value. As a result of conducting an experiment on 7,022 images taken in heterogeneous environments to evaluate the performance of the proposed methodology, the proposed methodology showed superior performance in terms of Angular Error compared to the existing methods.

빅데이터 시대의 도래는 데이터에서 스스로 규칙을 배우는 딥러닝의 비약적인 발전을 가능하게 하였으며, 특히 CNN 알고리즘이 거둔 성과는 모델의 구조를 넘어 소스 데이터 자체를 조정하는 수준에 이르렀다. 하지만 기존의 이미지 처리 방법은 이미지 데이터 자체를 다룰 뿐, 해당 이미지가 생성된 이질적 환경을 충분히 고려하지 않았다. 이질적 환경에서 촬영된 이미지는 동일한 정보임에도 촬영 환경에 따라 각 이미지의 특징(Feature)이 상이하게 표현될 수 있다. 이는 각 이미지가 갖는 상이한 환경 정보뿐 아니라 이미지 고유의 정보조차 서로 상이한 특징으로 표현되며, 이로 인해 이들 이미지 정보는 서로 잡음(Noise)으로 작용해 모델의 분석 성능을 저해할 수 있음을 의미한다. 따라서 본 논문은 이질적 환경에서 생성된 이미지 데이터들을 동시에 사용하는 앤드-투-앤드(End-To-End) 구조의 적대적 학습(Adversarial Learning) 기반의 이미지 색 항상성 모델 성능 향상 방안을 제안한다. 구체적으로 제안 방법론은 이미지가 촬영된 환경인 도메인을 예측하는 '도메인 분류기'와 조명 값을 예측하는 '조명 예측기'의 상호 작용으로 동작하며, 도메인 분류의 성능을 떨어뜨리는 방향의 학습을 통해 도메인 특성을 제거한다. 제안 방법론의 성능을 평가하기 위해 이질적 환경에서 촬영된 이미지 데이터 셋 7,022장에 대한 색 항상성 실험을 수행한 결과, 제안 방법론이 기존 방법론에 비해 Angular Error 측면에서 우수한 성능을 나타냄을 확인하였다.

Keywords

Acknowledgement

이 논문은 2021년 대한민국 교육부와 한국연구재단의 지원을 받아 수행된 연구임(NRF-2021S1A5A2A01061459). 이 논문은 과학기술정보통신부와 정보통신산업진흥원의 '고성능 컴퓨팅 지원' 사업의 지원을 받아 수행하였음.

References

  1. T. Kim and N. Kim, "Deep learning-based professional image interpretation using expertise transplant," Journal of Intelligence and Information Systems, Vol.26, No.2, pp.79-104, 2020. https://doi.org/10.13088/JIIS.2020.26.2.079
  2. H. Kim, H. S. Oh, and D. Kim, "CNN architecture predicting movie rating from audience's reviews written in Korean," KIPS Transactions on Computer and Communication Systems, Vol.9, No.1, pp.17-24, 2020. https://doi.org/10.3745/KTCCS.2020.9.1.17
  3. A. Gijsenij, T. Gevers, and J. V. Weijer, "Computational color constancy: Survey and experiments," IEEE Transactions on Image Processing, Vol.20, No.9, pp.2475-2489, 2011. https://doi.org/10.1109/TIP.2011.2118224
  4. J. Xiao, S. Gu, and L. Zhang, "Multi-domain learning for accurate and few-shot color constancy," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp.3258-3267, 2020.
  5. Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, and V. Lempitsky, "Domain-adversarial training of neural networks," Journal of Machine Learning Research, Vol.17, No.1, pp.2096-2030, 2016.
  6. G. Buchsbaum, "A spacial processor model for object color perception," Journal of the Franklin institute, Vol.310, No.1, pp.1-26, 1980. https://doi.org/10.1016/0016-0032(80)90058-7
  7. E. Land, "The retinex theory of color vision," Scientific American, Vol.237, No.6, pp.108-128, 1977. https://doi.org/10.1038/scientificamerican1277-108
  8. J. V. Weijer, T. Gevers, and A. Gijsenji, "Edge-based color constancy," IEEE Transactions on Image Processing, Vol.16, No.9, pp.2207-2214, 2007. https://doi.org/10.1109/TIP.2007.901808
  9. K. F. Yang, S. B. Gao, and Y. J. Li, "Efficient illuminant estimation for color constancy using grey pixels," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.2254-2263, 2015.
  10. K. Barnard and G. Finlayson, "Shadow identification using colour ratios," in Proceedings of the IS&T's Color and Imaging Conference, Vol.2000, No.1, pp.97-101, 2000.
  11. B. Funt and W. Xiong, "Estimating illumination chromaticity via support vector regression," in Proceedings of the IS&T's Color and Imaging Conference, Vol.2004, No.1, pp.47-52, 2004.
  12. D. Cheng, B. Price, S. Cohen, and M. S. Brown, "Effective learning-based illuminant estimation using simple features," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.1000-1008, 2015.
  13. S. Bianco, C. Cusano, and R. Schettini, "Color constancy using cnns," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp.81-89, 2015.
  14. J. T. Barron, "Convolutional color constancy," in Proceedings of the IEEE International Conference on Computer Vision, pp.379-387, 2015.
  15. J. T Barron and Y. T. Tsai, "Fast fourier color constancy," in Proceedings of the IEEE conference on computer vision and pattern recognition, pp.886-894, 2017.
  16. Y. Hu, B. Wang, and S. Lin, "Fc4: Fully convolutional color constancy with confidence weighted pooling," in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp.4085-4094, 2017.
  17. M. Rizzo, C. Conati, D. Jang, and H. Hu, "Cascading convolutional temporal colour constancy," arXiv preprint arXiv: 2106.07955, 2021.
  18. I. Goodfellow, J. P. Abadie, M. Mirza, B. Xu, D. W. Farley, S. Ozair, A. Courville, and Y. Bengio, "Generative adversarial nets," Advances in Neural Information Processing Systems, pp.2672-2680, 2014.
  19. Y. Wang, F. Ma, Z. jin, Y. Yuan, G. Xun, K. Jha, L. Su, and J. Gao, "Eann: Event adversarial neural networks for multimodal fake news detection," in Proceedings of the 24th ACM SIGKDD International Conference On Knowledge Discovery & Data Mining, pp.849-857, 2018.
  20. D. Li, D. Chen, L. Shi, B. Jin, J. Goh, and S. K. Ng, "MAD-GAN: Multivariate anomaly detection for time series data with generative adversarial networks," in International Conference on Artificial Neural Networks, Springer, pp.703-716, 2019.
  21. Z. Yang, J. Hu, R. Salakhutdinov, and W. Cohen, "Semi-supervised qa with generative domain-adaptive nets," arXiv preprint arXiv:1702.02206, 2017.
  22. Y. Ganin and V. Lempitsky, "Unsupervised domain adaptation by backpropagation," in International Conference on Machine Learning, pp.1180-1189, 2015.
  23. B. Sun, J. Feng, and K. Saenko, "Return of frustratingly easy domain adaptation," in Proceedings of the AAAI Conference on Artificial Intelligence, Vol.30, No.1, 2016.
  24. H. Yuzuguzel, "Learning colour constancy using convolutional neural networks," MS thesis, Tampere University of Technology, Finland, 2015.
  25. H. R. V. Joze, M. S. Drew, G. D. Finlayson, and P. A. T. Rey, "The role of bright pixels in illumination estimation," in Proceedings of the IS&T's Color and Imaging Conference, Vol.2012, No.1, pp.41-46, 2012.
  26. F. Laakom, J. Raitoharju, J. Nikkanen, A. Iosifidis, and M. Gabbouj, "Intel-tau: A color constancy dataset," IEEE Access, Vol.9, pp.39560-39567, 2021. https://doi.org/10.1109/ACCESS.2021.3064382