한국빅데이터학회지 (The Journal of Bigdata)

  • 제9권1호
  • /
  • Pages.153-168
  • /
  • 2024
  • /
  • 2508-1829(pISSN)
  • /
  • 2713-6361(eISSN)
한국빅데이터학회 (Korea Bigdata Society)
권호 표지
DOI QR코드

DOI QR Code

최소가동시간 제약을 고려한 심층 강화학습 기반의 다중 냉동기 운영 모델

A Multi-chiller Operation Model Based on Deep Reinforcement Learning Considering Minimum Up-time Constraint

  • 김종은 (인천대학교 산업경영공학과) ;
  • 김관호 (동국대학교 산업시스템공학과) ;
  • 김재곤 (인천대학교 산업경영공학과)
  • 투고 : 2024.05.30
  • 심사 : 2024.06.12
  • 발행 : 2024.06.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

여름철 냉동기가 건물의 주 에너지 소비자로 고려됨에 따라 효율적인 냉동기 운영은 매우 중요한 문제로 고려된다. 그러나, 건물의 냉방수요가 건물 내외부 환경, 건물 재실자의 행동 등의 여러 요인에 의해 변동하고 냉동기의 가동제약조건으로 인해 현재 시점의 운영이 미래 시점의 운영에 제약을 발생시킴에 따라 건물의 냉방수요를 정확하게 만족하도록 냉동기를 운영하는 것은 어렵다. 본 연구에서는 이러한 문제를 해결하기 위해 냉동기의 최소가동시간을 고려한 심층 강화학습 기반의 다중 냉동기 운영 모델을 제안한다. 제안한 모델은 외기 정보와 냉방시스템 내부 정보로 구성된 상태에 따른 냉동기 운영 조합이 갖는 가치를 학습하고 실현 가능한 냉동기 운영 중 건물의 냉방수요와 냉동기에 의한 공급 부하 간의 차이를 최소화할 수 있는 냉동기 운영 조합을 결정한다. 냉동기의 최소가동시간 제약을 고려한 훈련 알고리즘을 적용하여 제안한 모델의 현실 적용 가능성을 높였으며 실제 국내 A대학교의 데이터를 바탕으로 실험한 결과, 제안한 다중 냉동기 운영 모델이 최소가동시간을 준수함과 동시에 건물 냉방 부하와의 차이 측면에서 A대학교의 기존 냉동기 운영 로직보다 우수한 성능을 보임을 확인하였다.

In summer, as chillers are considered the main energy consumer of building, the efficient chiller operation is considered important. However, it is difficult to operate chillers to meet the cooling demand of the building as the demand fluctuates with various factors like the internal, external environment and behavior of the occupants and as chiller's constraint cause the current operation constrains operation in future. To address these problems, this study proposes a multi-chiller operation model based on deep reinforcement learning considering the minimum up-time of the chiller. The proposed model learns the value of the chiller operations according to the state composed of metrological and cooling system information and determines operation that minimizes the difference between the supply load and the cooling demand among feasible operations. The practical applicability was improved by applying the training algorithm considering the minimum up-time constraint and Experiments results using the actual data from a Korean university confirmed that the proposed model complies with the chiller constraints and outperforms the existing chiller operation logic of the university in terms of differences from the building cooling demand.

키워드

과제정보

본 연구는 과학기술정보통신부 및 정보통신기획평가원의 학석사연계ICT핵심인재양성사업의 연구결과로 수행되었음 (IITP-2024-RS-2023-00260175)

참고문헌

  1. Beghi, A., Cecchinato, L., Cosi, G., and Rampazzo, M. (2012). "A PSO-based algorithm for optimal multiple chiller systems operation" Applied Thermal Engineering, 32, pp. 31-40.
  2. Chang, Y. C. (2006). "An outstanding method for saving energy-optimal chiller operation" IEEE transactions on energy conversion, 21(2), pp. 527-532.
  3. dos Santos Coelho, L., Klein, C. E., Sabat, S. L., and Mariani, V. C. (2014). "Optimal chiller loading for energy conservation using a new differential cuckoo search approach" Energy, 75, pp. 237-243.
  4. Du, Y., Li, F., Kurte, K., Munk, J., and Zandi, H. (2022) "Demonstration of intelligent HVAC load management with deep reinforcement learning: real-world experience of machine learning in demand control" IEEE Power and Energy Magazine, 20(3), pp. 42-53.
  5. He, K., Fu, Q., Lu, Y., Wang, Y., Luo, J., Wu, H., and Chen, J. (2023). "Predictive control optimization of chiller plants based on deep reinforcement learning" Journal of Building Engineering, 76, pp. 107158.
  6. Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., and Meger, D. (2018). "Deep reinforcement learning that matters" In Proceedings of the AAAI conference on artificial intelligence, Louisiana, USA, 32(1).
  7. Huang, S., and Ontanon, S. (2020). "A closer look at invalid action masking in policy gradient algorithms" arXiv preprint arXiv:2006.14171.
  8. Hyun, S. H., Park, C. S., and Augenbroe, G. L. M. (2008). "Analysis of uncertainty in natural ventilation predictions of high-rise apartment buildings" Building Services Engineering Research and Technology, 29(4), pp. 311-326.
  9. Jia, W., Li, J., and Zhao, Y. (2021). "DQN Algorithm Based on Target Value Network Parameter Dynamic Update" IEEE 4th International Conference on Computer and Communication Engineering Technology (CCET), Beijing, China, pp. 285-289.
  10. Kwok, S. S., Yuen, R. K., and Lee, E. W. (2011). "An intelligent approach to assessing the effect of building occupancy on building cooling load prediction" Building and Environment, 46(8), pp. 1681-1690.
  11. Kwon, Y. D., Choo, J., Kim, B., Yoon, I., Gwon, Y., and Min, S. (2020). "Pomo: Policy optimization with multiple optima for reinforcement learning" Advances in Neural Information Processing Systems (NeurIPS), Vancouver, Canada, 33, pp. 21188-21198.
  12. Lewis, F. L., and Vrabie, D. (2009). "Reinforcement learning and adaptive dynamic programming for feedback control" IEEE circuits and systems magazine, 9(3), pp. 32-50.
  13. Liao, Y., and Huang, G. (2019). "A hybrid predictive sequencing control for multi-chiller plant with considerations of indoor environment control, energy conservation and economical operation cost" Sustainable Cities and Society, 49, pp. 101616.
  14. Liu, R., and Zou, J. (2018). "The effects of memory replay in reinforcement learning" In 2018 56th annual allerton conference on communication, control, and computing (Allerton), IEEE, Monticello, USA, pp. 478-485.
  15. Liu, Z., Tan, H., Luo, D., Yu, G., Li, J., and Li, Z. (2017). "Optimal chiller sequencing control in an office building considering the variation of chiller maximum cooling capacity" Energy and Buildings, 140, pp. 430-442.
  16. Mavrovouniotis, M., Li, C., and Yang, S. (2017). "A survey of swarm intelligence for dynamic optimization: Algorithms and applications" Swarm and Evolutionary Computation, 33, pp, 1-17.
  17. Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., and Riedmiller, M. (2013). "Playing atari with deep reinforcement learning" arXiv preprint arXiv:1312.5602.
  18. Nair, V., and Hinton, G. E. (2010). "Rectified linear units improve restricted boltzmann machines" In Proceedings of the 27th international conference on machine learning (ICML-10), Haifa, Israel, pp. 807-814.
  19. Sala-Cardoso, E., Delgado-Prieto, M., Kampouropoulos, K., and Romeral, L. (2020). "Predictive chiller operation: A data-driven loading and scheduling approach" Energy and Buildings, 208, pp. 109639.
  20. Wong, W., Dutta, P., Voicu, O., Chervonyi, Y., Paduraru, C., and Luo, J. (2022). "Optimizing industrial hvac systems with hierarchical reinforcement learning" arXiv preprint arXiv:2209.08112.
  21. Wu, Z., Li, Y., Zhan, W., Liu, C., Liu, Y. H., and Tomizuka, M. (2024). "Efficient Reinforcement Learning of Task Planners for Robotic Palletization through Iterative Action Masking Learning" arXiv preprint arXiv:2404.04772.
  22. Ye, Y., Sharma, R., and Guo, F. (2016). "A multi-layer optimal chiller operation management framework" IEEE Industry Applications Society Annual Meeting (IAS), Portland, USA, pp. 1-6.
  23. Zhuang, C., Wang, S., and Shan, K. (2020). "A risk-based robust optimal chiller sequencing control strategy for energy-efficient operation considering measurement uncertainties" Applied Energy, 280, pp. 115983.