KSCE Journal of Civil and Environmental Engineering Research (대한토목학회논문집)

Korean Society of Civil Engeneers (대한토목학회)
권호 표지
DOI QR코드

DOI QR Code

A Simulation of Nighttime Thermal Infrared Image Colorization considering Temperature Change between Day and Night

주야간 온도변화를 고려한 야간 열적외영상 컬러화 모의

  • Jung, Ji Heon (Konkuk University) ;
  • Jo, Su Min (Konkuk University) ;
  • Eo, Yang Dam (Konkuk University) ;
  • Park, Jinhyeok (Lodics Co,. LTD) ;
  • Choi, Yeon Oh (Lodics Co,. LTD)
  • 정지헌 (건국대학교 대학원 기술융합공학과) ;
  • 조수민 (건국대학교 대학원 기술융합공학과) ;
  • 어양담 (건국대학교 사회환경공학부) ;
  • 박진혁 ((주)로딕스 신기술연구소) ;
  • 최연오 ((주)로딕스 신기술연구소)
  • Received : 2024.01.24
  • Accepted : 2024.02.25
  • Published : 2024.06.01
Download PDF
⟨ Previous Next ⟩

Abstract

In order to improve the visibility of nighttime thermal infrared images, a simulation method with daytime color images was proposed. As a simulation method consisting of two steps, the daytime thermal infrared image was simulated by learning the unpaired nighttime thermal infrared image and daytime thermal infrared image, then the result was translated into a daytime color image. A temperature change regression equation was constructed and applied to reflect the systematic characteristics of temperature changes in daytime and nighttime images, and day and night simulation and colorization were trained and modeled by CycleGAN. For the experimental area, 100 images were captured and used for training. As a result, the simulation showed an average SSIM of 0.2449 and a PSNR of 51.2254. It was confirmed that the method could simulate complex and detailed features such as vegetation.

야간에 촬영한 열적외영상의 가시성 향상을 위해 주간에 촬영한 가시영역 컬러영상으로의 모의방법을 제안하였다. 짝지어지지 않은 야간 열적외영상을 주간 열적외영상으로 모의하고 그 결과를 주간 컬러영상으로 변환하는 단계별 모의를 하였다. 주간영상과 야간영상의 온도 영상변화의 계통적 특성을 반영하기 위해 온도변화 회귀식을 구성하여 적용하였고, 주야간 모의와 컬러화는 CycleGAN으로 학습하여 모델링하였다. 실험지역에 대하여 100장의 영상을 촬영하여 학습한 결과 SSIM은 0.2449, PSNR은 51.2254를 평균값으로 모의되었으며, 식생과 같은 복잡하고 세부적인 모의도 가능한 것을 확인하였다.

Keywords

Acknowledgement

This work was supported by KOITA grant funded by MSIT(R&DCENTER Capability Enhancement Project, 1711199726).

References

  1. Bae, J., Park, S., Ho, H. and Sohn, H. (2020). "A study on the application of object detection technique after transforming night image into daytime image." KSCE 2020 Convention Conference, Jeju, Korea, pp. 1474-1475 (in Korean).
  2. Ciftci, O. and Akcayol, M. A. (2021). "Thermal infrared colorization using deep learning." 2021 8th International Conference on Electrical and Electronics Engineering (ICEEE), Antalya, Turkey, pp. 323-326, https://doi.org/10.1109/iceee52452.2021.9415929.
  3. Hamam, T., Dordek, Y. and Cohen, D. (2012). "Single-band infrared texture-based image colorization," 2012 IEEE 27th Convention of Electrical and Electronics Engineers in Israel, Eilat, Israel, pp. 1-5, https://doi.org/10.1109/EEEI.2012.6377111.
  4. Jensen, J. R. (2007). Remote sensing of the environment: An earth resource perspective. Pearson Prentice Hall.
  5. Liu, S., Gao, M., John, V., Liu, Z. and Blasch, E. (2020). "Deep learning thermal image translation for night vision perception." ACM Translations on Intelligent Systems and Technology, Vol. 12, No. 1, pp. 1-18, https://doi.org/10.1145/3426239. 
  6. Luo, F., Cao, C. and Li, Y. (2021). "Nighttime thermal infrared image colorization with dynamic label mining." In Image and Graphics: 11th International Conference, ICIG 2021, Haikou, China, 2021, Proceedings, Part III. Springer-Verlag, Berlin, Heidelberg, pp. 388-399, https://doi.org/10.1007/978-3-030-87361-5_32.
  7. Luo, F., Li, Y., Zeng, G., Peng, P., Wang, G. and Li, Y. (2022). "Thermal infrared image colorization for nighttime, driving scenes with top-down guided attention." Institute of Electrical and Electronics Engineers (IEEE) Transactions on Intelligent Transportation Systems, Vol. 23, No. 9, pp. 15808-15823, https://doi.org/10.1109/tits.2022.3145476. 
  8. Nyberg, A., Eldesokey, A., Bergstrom, D. and Gustafsson, D. (2019). "Unpaired thermal to visible spectrum transfer using adversarial training." Computer Vision-ECCV 2018 Workshops.
  9. Toet, A. (2005). "Colorizing single band intensified nightvision images." Displays, Vol. 26, No. 1, pp. 15-21, https://dx.doi.org/10.1016/j.displa.2004.09.007. 
  10. Wang, H., Cheng, C., Zhang, X. and Sun, H. (2022). "Towards high-quality thermal infrared image colorization via attention-based hierarchical network." Neurocomputing, Vol. 501, Issue C, pp. 318-327, https://doi.org/10.1016/j.neucom.2022.06.021. 
  11. Yang, S., Sun, M., Lou, X., Yang, H. and Zhou, H. (2023). "An unpaired thermal infrared image translation method using GMA-CycleGAN." Remote Sensing, Vol. 15, No. 3, https://doi.org/10.3390/rs15030663.