대한임베디드공학회논문지 (IEMEK Journal of Embedded Systems and Applications)

대한임베디드공학회 (Institute of Embedded Engineering of Korea)
권호 표지
DOI QR코드

DOI QR Code

웨이블릿 퓨전에 의한 딥러닝 색상화의 성능 향상

High-performance of Deep learning Colorization With Wavelet fusion

  • 투고 : 2018.09.11
  • 심사 : 2018.10.26
  • 발행 : 2018.12.31
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

We propose a post-processing algorithm to improve the quality of the RGB image generated by deep learning based colorization from the gray-scale image of an infrared camera. Wavelet fusion is used to generate a new luminance component of the RGB image luminance component from the deep learning model and the luminance component of the infrared camera. PSNR is increased for all experimental images by applying the proposed algorithm to RGB images generated by two deep learning models of SegNet and DCGAN. For the SegNet model, the average PSNR is improved by 1.3906dB at level 1 of the Haar wavelet method. For the DCGAN model, PSNR is improved 0.0759dB on the average at level 5 of the Daubechies wavelet method. It is also confirmed that the edge components are emphasized by the post-processing and the visibility is improved.

키워드

OBDDBE_2018_v13n6_313_f0001.png 이미지

그림 1. SegNet 구조 Fig. 1 SegNet architecture

OBDDBE_2018_v13n6_313_f0002.png 이미지

그림 2. SegNet 기반의 색상화 모델 Fig. 2 SegNet based colorization model

OBDDBE_2018_v13n6_313_f0003.png 이미지

그림 3. SegNet 기반의 색상화 결과 Fig. 3 SegNet based colorization results

OBDDBE_2018_v13n6_313_f0004.png 이미지

그림 4. 원본과 출력 이미지의 휘도 성분 비교 Fig. 4. Comparison of luminance components between ground truth and output images

OBDDBE_2018_v13n6_313_f0005.png 이미지

그림 5. DCGAN 기반 색상화 모델 학습 알고리즘 Fig. 5 DCGAN based colorization model training algorithm

OBDDBE_2018_v13n6_313_f0006.png 이미지

그림 6. DCGAN 기반의 색상화 모델 Fig. 6 DCGAN based colorization model

OBDDBE_2018_v13n6_313_f0007.png 이미지

그림 7. DCGAN 기반의 색상화 결과 Fig. 7 DCGAN based colorization results

OBDDBE_2018_v13n6_313_f0008.png 이미지

그림 8. 딥러닝과 웨이블릿 융합에 의한 색상화 Fig. 8 Colorization by deep-learning and wavelet fusion

OBDDBE_2018_v13n6_313_f0009.png 이미지

그림 9. 웨이블릿 변환 및 이미지 융합 Fig. 9 Wavelet transform and image fusion

OBDDBE_2018_v13n6_313_f0010.png 이미지

그림 10. SegNet 기반 색상화 이미지와 후처리에 따른 가시성 비교 Fig. 10 Comparison of visibility SegNet based colorization and post-processing

OBDDBE_2018_v13n6_313_f0011.png 이미지

그림 11. DCGAN 기반 색상화 이미지와 후처리에 따른 가시성 비교 Fig. 11 Comparison of visibility DCGAN based colorization and post-processing

표 1. 융합 파라미터에 따른 PSNR Table 1. PSNR according to fusion parameters

OBDDBE_2018_v13n6_313_t0001.png 이미지

표 2. 웨이블릿 기법과 분해단계에 따른 PSNR Table 2. PSNR according to wavelet method and each decomposition step

OBDDBE_2018_v13n6_313_t0002.png 이미지

참고문헌

  1. Mengchi He, Xiaojing Gu, Xingsheng Gu, "A Fast Colorization Algorithm for Infrared Video," Proceedings of International Conference on Life System Modeling and Simulation and International Conference on Intelligent Computing for Sustainable Energy and Environment, pp. 282-292, 2014.
  2. Patricia L. Suarez, Angel D. Sappa, Boris X. Vintimilla, "Infrared Image Colorization based on a Triplet DCGAN Architecture," Proceedings of IEEE Computer Vision and Pattern Recognition Workshop, pp. 212-217, 2017.
  3. Matthias Limmer, Hendrik P.A. Lensch, "Infrared Colorization Using Deep Convolutional Neural Networks," Proceedings of IEEE Machine Learning and Applications, pp. 61-68, 2016.
  4. Richard Zhang, Phillip Isola, Alexei A. Efros, "Colorful Image Colorization," Proceedings of European Conference on Computer Vision, pp. 649-666, 2016.
  5. Choi Hyun, "Colorization Based on SegNet Architecture and Post-processing Using Wavelet Fusion," M.D. Thesis, Incheon National University, 2018. (in Korean)
  6. Kamyar Nazeri, Eric Ng, "Image Colorization with Generative Adversarial Networks," Faculty of Science, University of Ontario Institute of Technology, arXiv:1803.05400, 2018.
  7. Vijay Badrinarayanan, Alex Kendall, Roberto Cipolla, Senior Member, "SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation," Proceedings of IEEE Pattern Analysis and Machine Intelligence, Vol. 39, No. 12, pp. 2481-2495, 2017.
  8. Olaf Ronneberger, Philipp Fischer, Thomas Brox, "U-Net: Convolutional Networks for Biomedical Image Segmentation," Proceedings of International Conference on Meidcal Image Computing and Computer-Assisted Intervention, pp. 234-241, 2015.