Journal of Korean Association for Spatial Structures (한국공간구조학회논문집)

Korean Association for Spatial Structures (한국공간구조학회)
권호 표지
DOI QR코드

DOI QR Code

Development of Optimal Design Technique of RC Beam using Multi-Agent Reinforcement Learning

다중 에이전트 강화학습을 이용한 RC보 최적설계 기술개발

  • Received : 2023.05.08
  • Accepted : 2023.05.27
  • Published : 2023.06.15
Download PDF
⟨ Previous Next ⟩

Abstract

Reinforcement learning (RL) is widely applied to various engineering fields. Especially, RL has shown successful performance for control problems, such as vehicles, robotics, and active structural control system. However, little research on application of RL to optimal structural design has conducted to date. In this study, the possibility of application of RL to structural design of reinforced concrete (RC) beam was investigated. The example of RC beam structural design problem introduced in previous study was used for comparative study. Deep q-network (DQN) is a famous RL algorithm presenting good performance in the discrete action space and thus it was used in this study. The action of DQN agent is required to represent design variables of RC beam. However, the number of design variables of RC beam is too many to represent by the action of conventional DQN. To solve this problem, multi-agent DQN was used in this study. For more effective reinforcement learning process, DDQN (Double Q-Learning) that is an advanced version of a conventional DQN was employed. The multi-agent of DDQN was trained for optimal structural design of RC beam to satisfy American Concrete Institute (318) without any hand-labeled dataset. Five agents of DDQN provides actions for beam with, beam depth, main rebar size, number of main rebar, and shear stirrup size, respectively. Five agents of DDQN were trained for 10,000 episodes and the performance of the multi-agent of DDQN was evaluated with 100 test design cases. This study shows that the multi-agent DDQN algorithm can provide successfully structural design results of RC beam.

Keywords

Acknowledgement

본 논문은 2019년도 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임. (No. NRF-2019R1A2C1002385)

References

  1. Thai, H.T., "Machine learning for structural engineering: A state-of-the-art review", Structures, Vol.38, pp.448-491, 2022, doi: 10.1016/j.istruc.2022.02.003 
  2. Degtyarev, V.V. & Naser, M.Z., "Boosting machines for predicting shear strength of CFS channels with staggered web perforations", Structures, Vol.34, pp.3391-3403, 2021, doi: 10.1016/j.istruc.2021.09.060 
  3. Feng, D.C., Wang, W.J., Mangalathu, S., Hu, G. & Wu, T., "Implementing ensemble learning methods to predict the shear strength of RC deep beams with/without web reinforcements", Engineering Structures, Vol.235, 2021, doi: 10.1016/j.engstruct.2021.111979 
  4. Lee, C. & Ahn, J., "Flexural design of reinforced concrete frames by genetic algorithm", Journal of Structural Engineering, Vol.129, pp.762-774, 2003, doi: 10.1061/(ASCE)0733-9445(2003)129:6(762) 
  5. Lee, K.S. & Geem, Z.W., "A new structural optimization method based on the harmony search algorithm", Computers and Structures, Vol.82, pp.781-798, 2004, doi: 10.1016/j.compstruc.2004.01.002 
  6. Jeong, J.H. & Jo, H., "Deep reinforcement learning for automated design of reinforced concrete structures", Computer-Aided Civil and Infrastructure Engineering, Vol.36, pp.1508-1529, 2021, doi: 10.1111/mice.12773 
  7. Volodymyr, M., Koray, K., David, S., Andrei, A.R., Joel, V., Marc, G.B., Alex, G., Martin, R., Andreas, K.F., Georg, O., Stig, P., Charles, B., Amir, S., Ioannis, A., Helen, K., Dharshan, K., Daan, W., Shane, L. & Demis, H., "Human-level control through deep reinforcement learning", Nature, Vol. 518, pp. 529-533, 2015, doi: 10.1038/nature14236 
  8. Hafiz, A.M. & Bhat, G.M., "Deep q-network based multi-agent reinforcement learning with binary action agents", Computer Science, 2020, doi: 10.48550/arXiv.2008.04109 
  9. Leps, M., & Sejnoha, M., "New approach to optimization of reinforced concrete beams", Computers and Structures, Vol.81, pp.1957-1966, 2003, doi: 10.1016/S0045-7949(03)00215-3 
  10. Zhang, K., Yang, Z. & Basar, T., "Multi-agent reinforcement learning: a selective overview of theories and algorithms", Studies in Systems, Decision and Control, Vol. 325, pp. 321-384, 2021, doi: 10.1007/978-3-030-60990-0_12 
  11. Hasselt, H.V., Guez, A. & Silver, D., "Deep reinforcement learning with double q-learning", Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp.2094-2100, 2016, doi: 10.1609/aaai.v30i1.10295