DOI QR코드

DOI QR Code

A Development of Intelligent Pumping Station Operation System Using Deep Reinforcement Learning

심층 강화학습을 이용한 지능형 빗물펌프장 운영 시스템 개발

  • 강승호 (동신대학교/융합정보보안전공) ;
  • 박정현 (동신대학교/토목환경공학과) ;
  • 주진걸 (동신대학교/토목환경공학과)
  • Received : 2020.01.30
  • Accepted : 2020.03.20
  • Published : 2020.03.31

Abstract

The rainwater pumping station located near a river prevents river overflow and flood damages by operating several pumps according to the appropriate rules against the reservoir. At the present time, almost all of rainwater pumping stations employ pumping policies based on the simple rules depending only on the water level of reservoir. The ongoing climate change caused by global warming makes it increasingly difficult to predict the amount of rainfall. Therefore, it is difficult to cope with changes in the water level of reservoirs through the simple pumping policy. In this paper, we propose a pump operating method based on deep reinforcement learning which has the ability to select the appropriate number of operating pumps to keep the reservoir to the proper water level using the information of the amount of rainfall, the water volume and current water level of the reservoir. In order to evaluate the performance of the proposed method, the simulations are performed using Storm Water Management Model(SWMM), a dynamic rainfall-runoff-routing simulation model, and the performance of the method is compared with that of a pumping policy being in use in the field.

하천 인근에 위치한 빗물펌프장은 유수지를 대상으로 적절한 규칙에 따라 펌프를 가동함으로써 도심지 및 농경지 침수 피해를 예방하는 기능을 수행한다. 현재 대부분의 빗물펌프장은 유수지의 수위를 기준으로 단순한 규칙 기반의 펌프운영 정책을 사용하고 있다. 최근 지구온난화로 인한 기후 변화가 예측하기 어려운 강우량의 변화를 발생시키고 있다. 따라서 단순한 펌프정책으로는 지구온난화로 인한 갑작스러운 유수지 변화에 적절하게 대처하기 어렵다. 본 논문은 강우량과 저수량, 유수지 수위 등의 정보를 이용해 시스템이 적정 유수지 수위을 유지할 수 있도록 펌프 가동을 선택할 수 있는 심층 강화학습 기반의 자동 빗물펌프 운용 방법을 제시한다. 제안한 방법의 타당성을 검증하기 위해 강우-유출 모의 모델인 Storm Water Management Model(SWMM)을 이용해 모의실험을 수행하고 현장에서 사용되고 있는 기존 펌프 정책과 성능을 비교하였다.

Keywords

References

  1. S. U. Yun and J. T. Lee, "A study on the development of the operation models for storm water pumps in detention pond", J. KWRA, Vol. 28, No. 6, pp. 203-215, 1995
  2. 추연문, 유도근, 이승엽, 임오성, 주진걸, 김중훈, "도시 빗물펌프장 예측운영시스템의 스마트 운영기법 개발", 한국방재학회 논문집, 17(2), pp. 39-50, 2017.4 https://doi.org/10.9798/KOSHAM.2017.17.2.39
  3. J. H. Sim, W. C. Joo, and W. H. Lee, "An adaptive control of inland pumping station using self-tuning of fuzzy control technique", KWRA Hydro Research Paper, pp. 291-299, 1992.
  4. 전환돈, 이양재, 이정호, 김중훈, "도시 내배수시스템 실시간 운영모형의 개발", 한국물환경학회지, 23권 5호, pp. 748-755, 2007.
  5. X. Feng, B. Qiu, X. Yang, and B. Pei, "Optimal methods and its application of large pumping station operation", J. Drainage and Irrigation Machinery Engineering, Vol. 2, No. 10, 2011.
  6. X. Zhuan, and X. Xiaohua, "Optimal operation scheduling of a pumping station with multiple pumps", Applied Energy, Vol. 104, pp. 250-257, 2013. https://doi.org/10.1016/j.apenergy.2012.10.028
  7. T. Baumeister, S. L. Brunton, and J. N. Kutz, "Deep learning and model predictive control for self-tuning mode-locked lasers", J. Optical Society of America B, Vol. 35, No. 3, pp. 617-626, 2018. https://doi.org/10.1364/JOSAB.35.000617
  8. X. Y. Lee, et al., "A Case Study of Deep Reinforcement Learning for Engineering Design: Application to Microfluidic Devices for Flow Sculpting", J. Mechanical Design, Vol 141, No. 11, 111401(10 pages), 2019. https://doi.org/10.1115/1.4044397
  9. https://www.epa.gov/water-research/storm-watermanagement-model-swmm
  10. https://cfpub.epa.gov/si/si_public_record_report.cfm?Lab=NRMRL&direntryid=335780
  11. Richard S. Sutton and Andrew G. Barto, "Reinforcement Learning: An Introduction second edition", The MIT Press, 2018.
  12. K. Arulkumaran, M. P. Deisenroth, M. Brundage, and A. A. Bharath, "Deep Reinforcement Learning: A Brief Survey", IEEE Signal Processing Magazine, Vol. 34, pp. 26-38, 2017. https://doi.org/10.1109/MSP.2017.2743240
  13. T. D. Bruin, J. Kober, K. Tuyls, and R. Babuska, "Experience Selection in Deep Reinforcement Learning for Control", The Journal of Machine Learning Research, Vol. 19, No. 1, pp. 1-56, 2018.
  14. A. Ilyas, L. Engstrom, S. Santurkar, D. Tsipras, F. Janoos, L. Rudolph, A. Madry, "Are deep policy gradient algorithms truly policy gradient algorithms?", CoRR abs/1811.02553. arXiv:1811.02553.
  15. R. Williams, "Simple statistical gradient-following algorithms for connectionist reinforcement learning", 1992.