Journal of Korean Association for Spatial Structures (한국공간구조학회논문집)

Korean Association for Spatial Structures (한국공간구조학회)
권호 표지
DOI QR코드

DOI QR Code

Performance Evaluation of Reinforcement Learning Algorithm for Control of Smart TMD

스마트 TMD 제어를 위한 강화학습 알고리즘 성능 검토

  • Received : 2021.04.19
  • Accepted : 2021.04.26
  • Published : 2021.06.15
Download PDF
⟨ Previous Next ⟩

Abstract

A smart tuned mass damper (TMD) is widely studied for seismic response reduction of various structures. Control algorithm is the most important factor for control performance of a smart TMD. This study used a Deep Deterministic Policy Gradient (DDPG) among reinforcement learning techniques to develop a control algorithm for a smart TMD. A magnetorheological (MR) damper was used to make the smart TMD. A single mass model with the smart TMD was employed to make a reinforcement learning environment. Time history analysis simulations of the example structure subject to artificial seismic load were performed in the reinforcement learning process. Critic of policy network and actor of value network for DDPG agent were constructed. The action of DDPG agent was selected as the command voltage sent to the MR damper. Reward for the DDPG action was calculated by using displacement and velocity responses of the main mass. Groundhook control algorithm was used as a comparative control algorithm. After 10,000 episode training of the DDPG agent model with proper hyper-parameters, the semi-active control algorithm for control of seismic responses of the example structure with the smart TMD was developed. The simulation results presented that the developed DDPG model can provide effective control algorithms for smart TMD for reduction of seismic responses.

Keywords

Acknowledgement

본 논문은 2019년도 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임. (No. NRF-2019R1A2C1002385)

References

  1. Spencer Jr., B. F., & Nagarajaiah, S., "State of the art of structural control", Journal of Structural Engineering, Vol.129, No.7, pp.845~856, 2003, doi: 10.1061/(ASCE)0733-9445(2003)129:7(845)
  2. Kim, H. S., & Kang, J. W., "Vibration Control Performance Evaluation of Hybrid Mid-Story Isolation System for a Tall Building", Journal of Korean Association for Spatial Structures, Vol.18, No.3 pp.37~44, 2018, doi: 10.9712/KASS.2018.18.3.37
  3. Korkmaz, S., "A review of active structural control: challenges for engineering informatics", Computers & Structures, Vol.89, No.23-24, pp.2113~2132, 2011, doi: 10.1016/j.compstruc.2011.07.010
  4. Ok, S. Y., Park, K. S., Song, J. H., & Koh, H. M., "Multi-Objective Integrated Optimal Design of Hybrid Structure-Damper System Satisfying Target Reliability", Journal of the Earthquake Engineering Society of Korea, Vol.12, No.2, pp.9~22, 2008, doi: 10.5000/EESK.2008.12.2.009
  5. Pastia, C., & Luca, S. G., "Vibration Control of a Frame Structure Using Semi-Active Tuned Mass Damper", Bulletin of the Polytechnic Institute of Jassy, CONSTRUCTIONS. ARCHITECTURE Section, Vol.59, No.4, pp.31~40, 2013, Retrieved from http://www.bipcons.ce.tuiasi.ro/Archive/392.pdf
  6. Nagarajaiah, S., & Narasimhan, S., "Smart base-isolated benchmark building. Part II: phase I sample controllers for linear isolation systems", Structural Control Health Monitoring, Vol.13, No.2-3, pp.589~604, 2006, doi: 10.1002/stc.100
  7. Kim, H. S., & Kang, J. W., "Seismic Response Control of Retractable-roof Spatial Structure Using Smart TMD", Journal of Korean Association for Spacial Structures, Vol.16, No.4, pp.91~100, 2016, doi: 10.9712/KASS.2016.16.4.091
  8. Kim, H. S., & Kang, J. W., "Optimal Design of Smart Outrigger Damper for Multiple Control of Wind and Seismic Responses", Journal of Korean Association for Spacial Structures, Vol.16, No.3, pp.79~88, 2016, doi: 10.9712/KASS.2016.16.3.079
  9. Bitaraf, M., Ozbulut, O. E., Hurlebaus, S., & Barroso, L., "Application of semi-active control strategies for seismic protection of buildings with MR dampers", Engineering Structures, Vol.32, No.10, pp.3040~3047, 2010, doi: 10.1016/j.engstruct.2010.05.023
  10. Koo, J. H. (2003). Using Magneto-Rheological Dampers in Semiactive Tuned Vibration Absorbers to Control Structural Vibrations (Doctoral dissertation). Virginia Polytechnic Institute and State University, USA.
  11. Dyke, S. J., Spencer Jr., B. F., Sain, M. K., & Carlson, J. D., "Modeling and control of magnetorheological dampers for seismic response reduction", Smart Materials and Structures, Vol.5, No.5, pp.565~575, 1996 https://doi.org/10.1088/0964-1726/5/5/006
  12. Bathaei, A., Zahrai, S. M., & Ramezani, M., "Semi-active seismic control of an 11-DOF building model with TMD+MR damper using type-1 and -2 fuzzy algorithms", Journal of Vibration and Control, Vol.24, No.13, pp.2938~2953, 2018 https://doi.org/10.1177/1077546317696369
  13. Kim, H. S., & Kang, J. W., "Multi-objective Integrated Optimization of Diagrid Structure-smart Control Device", Journal of the Computational Structural Engineering Institute of Korea, Vol.26, No.1, pp.69~77, 2013 https://doi.org/10.7734/COSEIK.2013.26.1.69
  14. Cioffi, R., Travaglioni, M., Piscitelli, G., Petrillo, A., & De Felice, F., "Artificial Intelligence and Machine Learning Applications in Smart Production: Progress, Trends, and Directions", Sustainability, Vol.12, No.2, 2020, doi: 10.3390/su12020492
  15. Busoniu, L., de Bruin, T., Tolic, D., Kober, J., & Palunko, I., "Reinforcement learning for control: Performance, stability, and deep approximators", Annual Reviews in Control, Vol.46, pp.8~28, 2018, doi: 10.1016/j.arcontrol.2018.09.005
  16. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., ... Hassabis, D., "Human-level control through deep reinforcement learning", Nature, Vol.518, pp.529~533, 2015, doi: 10.1038/nature14236
  17. Warburton, G. B., "Optimum absorber parameters for various combinations of response and excitation parameters", Earthquake Engineering and Structural Dynamics, Vol.10, No.3, pp.381~401, 1982, doi: 10.1002/eqe.4290100304
  18. Sueoka, T., Torii, S., & Tsuneki, Y. (2004). The Application of Response Control Design Using Middle-Story Isolation System to High-Rise Building. Proceedings of the 13th World Conference on Earthquake Engineering, Canada, Retrieved from https://www.iitk.ac.in/nicee/wcee/article/13_3457.pdf
  19. Lillicrap, T. P., Hunt, J. J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., ... Wierstra, D. (2016). Continuous control with deep reinforcement learning. Proceedings of the 4th International Conference on Learning Representations, Puerto Rico, Retrieved from https://arxiv.org/pdf/1509.02971.pdf
  20. Uhlenbeck, G. E., & Ornstein, L. S., "On the Theory of the Brownian Motion", Physical Review, Vol.36, No.5, pp.823, 1930, doi: 10.1103/PhysRev.36.823