Journal of Korean Association for Spatial Structures (한국공간구조학회논문집)

Korean Association for Spatial Structures (한국공간구조학회)
권호 표지
DOI QR코드

DOI QR Code

Reward Design of Reinforcement Learning for Development of Smart Control Algorithm

스마트 제어알고리즘 개발을 위한 강화학습 리워드 설계

  • Kim, Hyun-Su (Division of Architecture, Sunmoon University) ;
  • Yoon, Ki-Yong (Department of Civil Infrastructure Systems and Safety Engineering, Sunmoon University)
  • 김현수 (선문대학교 건축학부) ;
  • 윤기용 (선문대학교 건설시스템안전공학과)
  • Received : 2022.05.12
  • Accepted : 2022.06.10
  • Published : 2022.06.15
Download PDF
⟨ Previous Next ⟩

Abstract

Recently, machine learning is widely used to solve optimization problems in various engineering fields. In this study, machine learning is applied to development of a control algorithm for a smart control device for reduction of seismic responses. For this purpose, Deep Q-network (DQN) out of reinforcement learning algorithms was employed to develop control algorithm. A single degree of freedom (SDOF) structure with a smart tuned mass damper (TMD) was used as an example structure. A smart TMD system was composed of MR (magnetorheological) damper instead of passive damper. Reward design of reinforcement learning mainly affects the control performance of the smart TMD. Various hyper-parameters were investigated to optimize the control performance of DQN-based control algorithm. Usually, decrease of the time step for numerical simulation is desirable to increase the accuracy of simulation results. However, the numerical simulation results presented that decrease of the time step for reward calculation might decrease the control performance of DQN-based control algorithm. Therefore, a proper time step for reward calculation should be selected in a DQN training process.

Keywords

Acknowledgement

본 논문은 2019년도 정부(과학기술정보통신부)의 재원으로 한국연구재단의 지원을 받아 수행된 연구임.(No. NRF-2019R1A2C1002385).

References

  1. Matta, E., "Performance of tuned mass dampers against near-field earthquakes", Structural Engineering and Mechanics, Vol.39, No.5, pp. 621-642, 2011, doi: https://doi.org/10.12989/sem.2011.39.5.621
  2. Warburton, G.B., "Optimum absorber parameters for various combinations of response and excitation parameters", Earthquake Engrg. and Struct. Dyn., Vol.10, pp.381-401, 1982., doi: 10.1002/eqe.4290100304
  3. Huu, T.P., Miura, N. and Iba, D., "Multi active tuned mass dampers for earthquake- induced vibration response control of high rise building", Journal of Mechanical Science and Technology, Vol.18, 2022, doi: 10.1007/s12206-022-0304-6
  4. Koo, J.H., Using magneto-rheological dampers in semiactive tuned vibration absorbers to control structural vibrations, Ph.D. Dissertation, Virginia Polytechnic Institute and State University, USA, 2003.
  5. Kim, H.S. and Kang, J.W., "Seismic response control of retractable-roof spatial structure using smart TMD", Journal of the Korean Association for Spacial Structures, Vol.16, No.4, pp.91-100, 2016, doi: 10.9712/KASS.2016.16.4.091
  6. Bathaei, A., Zahrai, S.M. and Ramezani, M., "Semi-active seismic control of an 11-DOF building model with TMD+MR damper using type-1 and -2 fuzzy algorithms", Journal of Vibration and Control, Vol. 24, No. 13, pp. 2938-2953, 2018, doi: 10.1177/1077546317696369
  7. Yi, F., Dyke, S.J., Caicedo, J.M., and Carlson, J.D., "Experimental verification of multi-input seismic control strategies for smart dampers," Journal of Engineering Mechanics, ASCE, Vol. 127, No. 11, pp. 1152-1164, 2001, doi: 10.1061/(ASCE)0733-9399(2001)127:11(1152)
  8. Dyke, S.J., Spencer, B.F., Sain, M.K. and Carlson, J.D., "Modeling and control of magnetorheological dampers for seismic response reduction", Smart Materials and Structures, Vol. 5, pp. 565-575, 1996. https://doi.org/10.1088/0964-1726/5/5/006
  9. Kim, H.S. and Kang, J.W., "Vibration control performance evaluation of hybrid mid-story isolation system for a tall building", Journal of the Korean Association for Spacial Structures, Vol.18, No.3, pp.37-44, 2018, doi:10.9712/KASS.2018.18.3.37
  10. Kim, H.S. and Kang, J.W., "Seismic response control of retractable-roof spatial structure using smart TMD", Journal of the Korean Association for Spacial Structures, Vol.16, No.4, pp.91-100, 2016, doi: 10.9712/KASS.2016.16.4.091
  11. Vorman, M.C., Maximum likelihood inverse reinforcement learning, Ph.D. Dissertation, , The State University of New Jersey, USA, 2014, doi: 10.7282/T3GQ70C8
  12. Kim, H.S. and Kang, J.W., "Development of semi-active control algorithm using deep q-network", Journal of the Korean Association for Spacial Structures, Vol.21, No.1, pp.79-86, 2021, doi: 10.9712/KASS.2021.21.1.79
  13. Kim, H.S. and Kang, J.W., "Seismic Response Control of Spacial Arch Structures using Multiple Smart TMD", Journal of the Korean Association for Spacial Structures, Vol.16, No.1, pp.43-51, 2016, doi: 10.9712/KASS.2016.16.1.043
  14. Warburton, G.B., "Optimum absorber parameters for various combinations of response and excitation parameters", Earthquake Engineering and Structural Dynamics, Vol.10, pp.381-401, 1982, doi:10.1002/eqe.4290100304
  15. Sues, R. H., Mau, S. T. and Wen, Y. K., "System identifcation of degrading hysteretic restoring forces", Journal of Engineering Mechanics, ASCE, Vol.114, No.5, pp.833-846, 1988, doi: 10.1061/(ASCE)0733-9399(1988)114:5(833)
  16. Volodymyr, M., Koray, K., David, S., Andrei, A.R., Joel, V., Marc, G.B., Alex, G., Martin, R., Andreas, K.F., Georg, O., Stig, P., Charles, B., Amir, S., Ioannis, A., Helen, K., Dharshan, K., Daan, W., Shane, L. and Demis, H., "Human-level control through deep reinforcement learning", Nature, Vol.518, pp.529-533, 2015, doi: 10.1038/nature14236