Browse > Article
http://dx.doi.org/10.9712/KASS.2022.22.2.39

Reward Design of Reinforcement Learning for Development of Smart Control Algorithm  

Kim, Hyun-Su (Division of Architecture, Sunmoon University)
Yoon, Ki-Yong (Department of Civil Infrastructure Systems and Safety Engineering, Sunmoon University)
Publication Information
Journal of Korean Association for Spatial Structures / v.22, no.2, 2022 , pp. 39-46 More about this Journal
Abstract
Recently, machine learning is widely used to solve optimization problems in various engineering fields. In this study, machine learning is applied to development of a control algorithm for a smart control device for reduction of seismic responses. For this purpose, Deep Q-network (DQN) out of reinforcement learning algorithms was employed to develop control algorithm. A single degree of freedom (SDOF) structure with a smart tuned mass damper (TMD) was used as an example structure. A smart TMD system was composed of MR (magnetorheological) damper instead of passive damper. Reward design of reinforcement learning mainly affects the control performance of the smart TMD. Various hyper-parameters were investigated to optimize the control performance of DQN-based control algorithm. Usually, decrease of the time step for numerical simulation is desirable to increase the accuracy of simulation results. However, the numerical simulation results presented that decrease of the time step for reward calculation might decrease the control performance of DQN-based control algorithm. Therefore, a proper time step for reward calculation should be selected in a DQN training process.
Keywords
Reinforcement learning; Smart TMD; Deep Q-network; Reward calculation; Seismic response reduction;
Citations & Related Records
Times Cited By KSCI : 6  (Citation Analysis)
연도 인용수 순위
1 Matta, E., "Performance of tuned mass dampers against near-field earthquakes", Structural Engineering and Mechanics, Vol.39, No.5, pp. 621-642, 2011, doi: https://doi.org/10.12989/sem.2011.39.5.621   DOI
2 Huu, T.P., Miura, N. and Iba, D., "Multi active tuned mass dampers for earthquake- induced vibration response control of high rise building", Journal of Mechanical Science and Technology, Vol.18, 2022, doi: 10.1007/s12206-022-0304-6   DOI
3 Koo, J.H., Using magneto-rheological dampers in semiactive tuned vibration absorbers to control structural vibrations, Ph.D. Dissertation, Virginia Polytechnic Institute and State University, USA, 2003.
4 Yi, F., Dyke, S.J., Caicedo, J.M., and Carlson, J.D., "Experimental verification of multi-input seismic control strategies for smart dampers," Journal of Engineering Mechanics, ASCE, Vol. 127, No. 11, pp. 1152-1164, 2001, doi: 10.1061/(ASCE)0733-9399(2001)127:11(1152)   DOI
5 Kim, H.S. and Kang, J.W., "Seismic response control of retractable-roof spatial structure using smart TMD", Journal of the Korean Association for Spacial Structures, Vol.16, No.4, pp.91-100, 2016, doi: 10.9712/KASS.2016.16.4.091   DOI
6 Kim, H.S. and Kang, J.W., "Development of semi-active control algorithm using deep q-network", Journal of the Korean Association for Spacial Structures, Vol.21, No.1, pp.79-86, 2021, doi: 10.9712/KASS.2021.21.1.79   DOI
7 Kim, H.S. and Kang, J.W., "Seismic Response Control of Spacial Arch Structures using Multiple Smart TMD", Journal of the Korean Association for Spacial Structures, Vol.16, No.1, pp.43-51, 2016, doi: 10.9712/KASS.2016.16.1.043   DOI
8 Sues, R. H., Mau, S. T. and Wen, Y. K., "System identifcation of degrading hysteretic restoring forces", Journal of Engineering Mechanics, ASCE, Vol.114, No.5, pp.833-846, 1988, doi: 10.1061/(ASCE)0733-9399(1988)114:5(833)   DOI
9 Bathaei, A., Zahrai, S.M. and Ramezani, M., "Semi-active seismic control of an 11-DOF building model with TMD+MR damper using type-1 and -2 fuzzy algorithms", Journal of Vibration and Control, Vol. 24, No. 13, pp. 2938-2953, 2018, doi: 10.1177/1077546317696369   DOI
10 Warburton, G.B., "Optimum absorber parameters for various combinations of response and excitation parameters", Earthquake Engineering and Structural Dynamics, Vol.10, pp.381-401, 1982, doi:10.1002/eqe.4290100304   DOI
11 Warburton, G.B., "Optimum absorber parameters for various combinations of response and excitation parameters", Earthquake Engrg. and Struct. Dyn., Vol.10, pp.381-401, 1982., doi: 10.1002/eqe.4290100304   DOI
12 Kim, H.S. and Kang, J.W., "Seismic response control of retractable-roof spatial structure using smart TMD", Journal of the Korean Association for Spacial Structures, Vol.16, No.4, pp.91-100, 2016, doi: 10.9712/KASS.2016.16.4.091   DOI
13 Dyke, S.J., Spencer, B.F., Sain, M.K. and Carlson, J.D., "Modeling and control of magnetorheological dampers for seismic response reduction", Smart Materials and Structures, Vol. 5, pp. 565-575, 1996.   DOI
14 Vorman, M.C., Maximum likelihood inverse reinforcement learning, Ph.D. Dissertation, , The State University of New Jersey, USA, 2014, doi: 10.7282/T3GQ70C8   DOI
15 Kim, H.S. and Kang, J.W., "Vibration control performance evaluation of hybrid mid-story isolation system for a tall building", Journal of the Korean Association for Spacial Structures, Vol.18, No.3, pp.37-44, 2018, doi:10.9712/KASS.2018.18.3.37   DOI
16 Volodymyr, M., Koray, K., David, S., Andrei, A.R., Joel, V., Marc, G.B., Alex, G., Martin, R., Andreas, K.F., Georg, O., Stig, P., Charles, B., Amir, S., Ioannis, A., Helen, K., Dharshan, K., Daan, W., Shane, L. and Demis, H., "Human-level control through deep reinforcement learning", Nature, Vol.518, pp.529-533, 2015, doi: 10.1038/nature14236   DOI