Journal of Korea Water Resources Association (한국수자원학회논문집)

Korea Water Resources Association (한국수자원학회)
권호 표지
DOI QR코드

DOI QR Code

Rainfall image DB construction for rainfall intensity estimation from CCTV videos: focusing on experimental data in a climatic environment chamber

CCTV 영상 기반 강우강도 산정을 위한 실환경 실험 자료 중심 적정 강우 이미지 DB 구축 방법론 개발

  • Byun, Jongyun (Department of Civil and Environmental Engineering, College of Engineering, Chung-Ang University) ;
  • Jun, Changhyun (Department of Civil and Environmental Engineering, College of Engineering, Chung-Ang University) ;
  • Kim, Hyeon-Joon (Department of Civil and Environmental Engineering, College of Engineering, Chung-Ang University) ;
  • Lee, Jae Joon (ICT Convergence Division Software Testing Center, Korea Conformity Laboratories) ;
  • Park, Hunil (New Business Development Team, Korea Conformity Laboratories) ;
  • Lee, Jinwook (Department of Civil and Environmental Engineering, College of Engineering, Chung-Ang University)
  • 변종윤 (중앙대학교 일반대학원 토목공학과) ;
  • 전창현 (중앙대학교 공과대학 사회기반시스템공학부) ;
  • 김현준 (중앙대학교 공과대학 사회기반시스템공학부) ;
  • 이재준 (한국건설생활환경시험연구원 ICT 융합본부 소프트웨어평가센터) ;
  • 박헌일 (한국건설생활환경시험연구원 대외사업본부 신사업기획팀) ;
  • 이진욱 (중앙대학교 공과대학 사회기반시스템공학부)
  • Received : 2023.04.12
  • Accepted : 2023.06.14
  • Published : 2023.06.30
Download PDF
⟨ Previous Next ⟩

Abstract

In this research, a methodology was developed for constructing an appropriate rainfall image database for estimating rainfall intensity based on CCTV video. The database was constructed in the Large-Scale Climate Environment Chamber of the Korea Conformity Laboratories, which can control variables with high irregularity and variability in real environments. 1,728 scenarios were designed under five different experimental conditions. 36 scenarios and a total of 97,200 frames were selected. Rain streaks were extracted using the k-nearest neighbor algorithm by calculating the difference between each image and the background. To prevent overfitting, data with pixel values greater than set threshold, compared to the average pixel value for each image, were selected. The area with maximum pixel variability was determined by shifting with every 10 pixels and set as a representative area (180×180) for the original image. After re-transforming to 120×120 size as an input data for convolutional neural networks model, image augmentation was progressed under unified shooting conditions. 92% of the data showed within the 10% absolute range of PBIAS. It is clear that the final results in this study have the potential to enhance the accuracy and efficacy of existing real-world CCTV systems with transfer learning.

본 연구에서는 CCTV 영상 기반 강우강도 산정 시 필수적으로 요구되는 적정 강우 이미지 DB를 구축하기 위한 방법론을 개발하였다. 먼저, 실환경에서 불규칙적이고 높은 변동성을 보일 수 있는 변수들(바람으로 인한 빗줄기의 변동성, 녹화 환경에서 포함되는 움직이는 객체, 렌즈 위의 흐림 현상 등)에 대한 통제가 가능한 한국건설생활환경시험연구원 내 기후환경시험실에서 CCTV 영상 DB를 구축하였다. 서로 다른 5개의 실험 조건을 고려하여 이상적 환경에서 총 1,728개의 시나리오를 구성하였다. 본 연구에서는 1,920×1,080 사이즈의 30 fps (frame per second) 영상 36개에 대하여 프레임 분할을 진행하였으며, 총 97,200개의 이미지를 사용하였다. 이후, k-최근접 이웃 알고리즘을 기반으로 산정된 최종 배경과 각 이미지와의 차이를 계산하여 빗줄기 이미지를 분리하였다. 과적합 방지를 위해 각 이미지에 대한 평균 픽셀 값을 계산하고, 설정한 픽셀 임계치보다 큰 자료를 선별하였다. 180×180 사이즈로의 재구성을 위해서 관심영역을 설정하고 10 Pixel 단위로 이동을 진행하여 픽셀 변동성이 최대가 되는 영역을 산정하였다. 합성곱 신경망 모델의 훈련을 위해서 120×120 사이즈로 재변환하고 과적합 방지를 위해 이미지 증강 과정을 거쳤다. 그 결과, 이미지 기반 강우 강도 합성곱 신경망 모델을 통해 산정된 결과값과 우량계에서 취득된 강우자료가 전반적으로 유사한 양상을 보였으며, 모든 강우강도 실험 조건에 대해서 약 92%의 데이터의 PBIAS (percent bias)가 절댓값 범위 10% 이내에 해당하였다. 본 연구의 결과물과 전이학습 등의 방법을 연계하여 기존 실환경 CCTV의 한계점을 개선할 수 있을 것으로 기대된다.

Keywords

Acknowledgement

이 연구는 기상청<｢스마트시티 기상기후 융합기술 개발｣사업>(KMI2022-01910)의 지원으로 수행되었음. 또한 이 성과 는 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임(No. NRF-2022R1A4A3032838). 본 연구는 과학기술정보통신부 한국건설기술연구원 연구운영비지 원(주요사업)사업으로 수행되었음(과제번호 20230115-001, 디지털뉴딜 기반 통합물관리 기술 융합 플랫폼(IWRM-K) 개발).

References

  1. Abdelraouf, A., Abdel-Aty, M., and Wu, Y. (2022). "Using vision transformers for spatial-context-aware rain and road surface condition detection on freeways." IEEE Transactions on Intelligent Transportation Systems, Vol. 23, No. 10, pp. 18546-18556. https://doi.org/10.1109/TITS.2022.3150715
  2. Abraham, M.T., Satyam, N., Pradhan, B., and Alamri, A.M. (2020). "IoT-based geotechnical monitoring of unstable slopes for landslide early warning in the Darjeeling Himalayas." Sensors, Vol. 20, No. 9, 2611.
  3. Abudi, I., Carmi, G., and Berliner, P. (2012). "Rainfall simulator for field runoff studies." Journal of Hydrology, Vol. 454, pp. 76-81. https://doi.org/10.1016/j.jhydrol.2012.05.056
  4. Ajith, J.B., Manimegalai, R., and Ilayaraja, V. (2020). "An IoT based smart water quality monitoring system using cloud." In 2020 International Conference on Emerging Trends in Information Technology and Engineering (ic-ETITE), IEEE, Vellore, India, pp. 1-7.
  5. Allamano, P., Croci, A., and Laio, F. (2015). "Toward the camera rain gauge." Water Resources Research, Vol. 51, No. 3, 1744-1757. https://doi.org/10.1002/2014WR016298
  6. Amale, O., and Patil, R. (2019). "IOT Based rainfall monitoring system using WSN enabled architecture." In 2019 3rd International Conference on Computing Methodologies and Communication (ICCMC), Erode, India, IEEE, pp. 789-791.
  7. Barbosa, J.H.S., Fernandes, A., Lima, A., and Assis, L. (2019). "The influence of spatial discretization on HEC-HMS modelling: A case study." International Journal of Hydrology, Vol. 3, pp. 442-449. https://doi.org/10.15406/ijh.2019.03.00209
  8. Byun, J., Jun, C., Kim, J., Cha, J., and Narimani, R. (2023). "Deep Learning-based Rainfall Prediction using Cloud Image Analysis." IEEE Transactions on Geoscience and Remote Sensing, Vol. 61, pp. 1-11. https://doi.org/10.1109/TGRS.2023.3263872
  9. Cardoso de Salis, H.H., Monteiro da Costa, A., Moreira Vianna, J. H., Azeneth Schuler, M., Kunne, A., Sanches Fernandes, L.F., and Leal Pacheco, F.A. (2019). "Hydrologic modeling for sustainable water resources management in urbanized karst areas." International Journal of Environmental Research and Public Health, Vol. 16, No. 14, 2542.
  10. Dawson, C.W., Abrahart, R.J., and See, L.M. (2007). "HydroTest: A web-based toolbox of evaluation metrics for the standardised assessment of hydrological forecasts." Environmental Modelling and Software, Vol. 22, No. 7, pp. 1034-1052. https://doi.org/10.1016/j.envsoft.2006.06.008
  11. Dong, R., Liao, J., Li, B., Zhou, H., and Crookes, D. (2017). "Measurements of rainfall rates from videos." 2017 10th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), IEEE, Shanghai, China, pp. 1-9.
  12. Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., and Paisley, J. (2017). "Removing rain from single images via a deep detail network." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, U.S., pp. 3855-3863.
  13. Garg, K., and Nayar, S.K. (2004). "Detection and removal of rain from videos." Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, IEEE, Washington, DC, U.S., Vol. 1, pp. I-I.
  14. Guo, B., Han, Q., Chen, H., Shangguan, L., Zhou, Z., and Yu, Z. (2017). "The emergence of visual crowdsensing: Challenges and opportunities." IEEE Communications Surveys and Tutorials, Vol. 19, No. 4, pp. 2526-2543. https://doi.org/10.1109/COMST.2017.2726686
  15. Haidar, A., and Verma, B. (2018). "Monthly rainfall forecasting using one-dimensional deep convolutional neural network." IEEE Access, Vol. 6, pp. 69053-69063. https://doi.org/10.1109/ACCESS.2018.2880044
  16. Hakim, A.L., and Dewi, R. (2021). "Automatic rain detection system based on digital images of CCTV cameras using convolutional neural network method." IOP Conference Series: Earth and Environmental Science, Vol. 893, No. 1, 012048.
  17. Han, K., and Xiang, X. (2020). "Decomposed cyclegan for single image deraining with unpaired data." ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), IEEE, Barcelona, Spain, pp. 1828-1832.
  18. Humphry, J.B., Daniel, T.C., Edwards, D.R., and Sharpley, A.N. (2002). "A portable rainfall simulator for plot-scale runoff studies." Applied Engineering in Agriculture, Vol. 18, No. 2, 199.
  19. Jiang, S., Babovic, V., Zheng, Y., and Xiong, J. (2019). "Advancing opportunistic sensing in hydrology: A novel approach to measuring rainfall with ordinary surveillance cameras." Water Resources Research, Vol. 55, No. 4, pp. 3004-3027. https://doi.org/10.1029/2018WR024480
  20. Lee, J., Byun, J., Baik, J., Jun, C., and Kim, H.-J. (2023). "Estimation of raindrop size distribution and rain rate with infrared surveillance camera in dark conditions." Atmospheric Measurement Techniques, Vol. 16, No. 3, pp. 707-725. https://doi.org/10.5194/amt-16-707-2023
  21. Lee, S.H., Jeon, J.H., Choi, D.Y., Park, J.M., and Song, B.C. (2022). "Image based rainfall amount estimation for auto-wiping of vehicles." Neural Computing and Applications, Vol. 34, pp. 15543-15554. https://doi.org/10.1007/s00521-022-07269-3
  22. Li, M., Xie, Q., Zhao, Q., Wei, W., Gu, S., Tao, J., and Meng, D. (2018a). "Video rain streak removal by multiscale convolutional sparse coding." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, U.S., pp. 6644-6653.
  23. Li, Z., Hyeon, J., and Choi, H. J. (2018b). "Rainfall recognition from road surveillance videos using TSN." Journal of Korean Society for Atmospheric Environment, Vol. 34, No. 5, pp. 735-747. https://doi.org/10.5572/KOSAE.2018.34.5.735
  24. Liu, T., Xu, M., and Wang, Z. (2019). "Removing rain in videos: A large-scale database and a two-stream ConvLSTM approach." In 2019 IEEE International Conference on Multimedia and Expo (ICME), IEEE, Shanghai, China, pp. 664-669.
  25. Notarangelo, N.M., Hirano, K., Albano, R., and Sole, A. (2021). "Transfer learning with convolutional neural networks for rainfall detection in single images." Water, Vol. 13, No. 5, 588.
  26. Rani, D.S., Jayalakshmi, G.N., and Baligar, V.P. (2020). "Low cost IoT based flood monitoring system using machine learning and neural networks: flood alerting and rainfall prediction." In 2020 2nd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA), IEEE, Bangalore, India, pp. 261-267.
  27. Regmi, T.P., and Thompson, A.L. (2000). "Rainfall simulator for laboratory studies." Applied Engineering in Agriculture, Vol. 16, No. 6, 641.
  28. Sadeghi, M., Asanjan, A.A., Faridzad, M., Nguyen, P., Hsu, K., Sorooshian, S., and Braithwaite, D. (2019). "PERSIANN-CNN: Precipitation estimation from remotely sensed information using artificial neural networks-convolutional neural networks." Journal of Hydrometeorology, Vol. 20, No. 12, pp. 2273-2289. https://doi.org/10.1175/JHM-D-19-0110.1
  29. Santos, S.C., Firmino, R.M., Mattos, D.M., and Medeiros, D.S. (2020). "An IoT rainfall monitoring application based on wireless communication technologies." In 2020 4th Conference on Cloud and Internet of Things (CIoT), IEEE, Niteroi, Brazil, pp. 53-56.
  30. Singh, V.K., Kumar, M.M., Yuvaraj, J., Rubeshkumar, T., Kumar, S., and Padmapriya, S. (2022). "IoT based rainfall surveillance system with sensor integrated infrastructure." 2022 7th International Conference on Communication and Electronics Systems (ICCES), IEEE, Coimbatore, India, pp. 468-472.
  31. Thompson, A.L., and James, L.G. (1985). "Water droplet impact and its effect on infiltration." Transactions of the ASAE, Vol. 28, No. 5, pp. 1506-1510. https://doi.org/10.13031/2013.32468
  32. Tu, Z., Talebi, H., Zhang, H., Yang, F., Milanfar, P., Bovik, A., and Li, Y. (2022). "Maxim: Multi-axis mlp for image processing." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, U.S., pp. 5769-5780.
  33. Wang, T., Yang, X., Xu, K., Chen, S., Zhang, Q., and Lau, R.W. (2019). "Spatial attentive single-image deraining with a high quality real rain dataset." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, U.S., pp. 12270-12279.
  34. Wei, Y., Zhang, Z., Wang, Y., Xu, M., Yang, Y., Yan, S., and Wang, M. (2021). "Deraincyclegan: Rain attentive cyclegan for single image deraining and rainmaking." IEEE Transactions on Image Processing, Vol. 30, pp. 4788-4801. https://doi.org/10.1109/TIP.2021.3074804
  35. Xue, M., Hang, R., Liu, Q., Yuan, X.T., and Lu, X. (2021). "CNN-based near-real-time precipitation estimation from Fengyun-2 satellite over Xinjiang, China." Atmospheric Research, Vol. 250, 105337.
  36. Yang, W., Tan, R.T., Feng, J., Liu, J., Guo, Z., and Yan, S. (2017). "Deep joint rain detection and removal from a single image." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, U.S., pp. 1357-1366.
  37. Zamir, S.W., Arora, A., Khan, S., Hayat, M., Khan, F.S., and Yang, M.H. (2022). "Restormer: Efficient transformer for highresolution image restoration." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, LA, U.S., pp. 5728-5739.
  38. Zen, R., Arsa, D.M.S., Zhang, R., Er, N.A.S., and Bressan, S. (2019). "Rainfall estimation from traffic cameras." Database and Expert Systems Applications: 30th International Conference, Linz, Austria, pp. 18-32.
  39. Zhang, H., and Patel, V.M. (2018). "Density-aware single image de-raining using a multi-stream dense network." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, U.S., pp. 695-704.
  40. Zhu, H., Peng, X., Zhou, J.T., Yang, S., Chanderasekh, V., Li, L., and Lim, J.H. (2019). "Singe image rain removal with unpaired information: A differentiable programming perspective." Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, No. 01, pp. 9332-9339.