Wind and Structures

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

Active flutter control of long-span bridges via deep reinforcement learning: A proof of concept

  • Teng Wu (University at Buffalo) ;
  • Jiachen He (China Railway Siyuan Survey and Design Group Co., Ltd.) ;
  • Shaopeng Li (University of Florida)
  • Received : 2022.08.07
  • Accepted : 2023.12.19
  • Published : 2023.05.25
⟨ Previous Next ⟩

Abstract

Aeroelastic instability (i.e., flutter) is a critical issue that threatens the safety of flexible bridges with increasing span length. As a promising technique for flutter prevention, active aerodynamic control using auxiliary surfaces attached to the bridge deck (e.g., winglets and flaps) can be utilized to extract the stabilizing forces from the surrounding wind flow. Conventional controllers for the active aerodynamic control are usually designed using linear model-based schemes [e.g., linear quadratic regulator (LQR) and H-infinity control]. In addition to suffering from model inaccuracies, the obtained linear controller may not work well considering the high complexity of the inherently nonlinear wind-bridge-control system. To this end, this study proposes a nonlinear model-free controller based on deep reinforcement learning for active flutter control of long-span bridges. Specifically, a deep neural network (DNN), with the powerful ability to approximate nonlinear functions, is introduced to map from the system state (e.g., the motion of bridge deck) to the control command (e.g., reference position of the actively controlled surface). The DNN weights are obtained by interacting with the wind-bridge-control environment in a trial-and-error fashion (hence the explicit model of system dynamics is not required) using reinforcement learning algorithms of deep deterministic policy gradient (DDPG) due to its ability to tackle continuous actions with high training efficiency. As a proof of concept, numerical examples on active flutter control of a flat plate and a bridge deck are conducted to demonstrate the good performance of the proposed scheme.

Keywords

Acknowledgement

The support from Institute of Bridge Engineering at University at Buffalo is gratefully acknowledged.

References

  1. Bani-Hani, K.A., (2007), "Vibration control of wind-induced response of tall buildings with an active tuned mass damper using neural networks", Struct. Control Health Moni., 14(1), 83-108. https://doi.org/10.1002/stc.85.
  2. Battaini, M., Casciati, F. and Faravelli, L. (1998), "Fuzzy control of structural vibration. An active mass system driven by a fuzzy controller", Earthq. Eng. Struct. D., 27(11), 1267-1276. https://doi.org/10.1002/(SICI)1096-9845(1998110)27:11%3C1267::AID-EQE782%3E3.0.CO;2-D.
  3. Bera, K.K. and Chandiramani, N.K. (2019), "Flutter control of bridge deck using experimental aeroderivatives and LQR-driven winglets", J. Bridge Eng., 24(11), 04019100. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001467.
  4. Chen, X. and Kareem, A. (2003), "Efficacy of tuned mass dampers for bridge flutter control", J. Struct. Eng., 129(10), 1291-1300. https://doi.org/10.1061/(ASCE)0733-9445(2003)129:10(1291).
  5. Chen, X., Kareem, A. and Matsumoto, M. (2001), "Multimode coupled flutter and buffeting analysis of long span bridges", J. Wind Eng. Ind. Aerod., 89(7-8), 649-664. https://doi.org/10.1016/S0167-6105(01)00064-2.
  6. Ghaboussi, J. and Joghataie, A. (1995), "Active control of structures using neural networks", J. Eng. Mech., 121(4), 555-567. https://doi.org/10.1061/(ASCE)0733-9399(1995)121:4(555).
  7. Gu, M., Chen, S.R. and Chang, C.C. (2002), "Control of wind-induced vibrations of long-span bridges by semi-active lever-type TMD", J. Wind Eng. Ind. Aerod., 90(2),111-126. https://doi.org/10.1016/S0167-6105(01)00165-9.
  8. Guo, Z. (2013), "Principle and method of active control for vortex-induced vibration and flutter of long-span suspension bridge", Ph.D. Dissertation, Tongji University, Shanghai.
  9. Hinton, G., Deng, L., Yu, D., Dahl, G.E., Mohamed, A.R., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T.N. and Kingsbury, B. (2012), "Deep neural networks for acoustic modeling in speech recognition: the shared views of four research groups", IEEE Signal Process. Mag., 29(6), 82-97. https://doi.org/ 10.1109/MSP.2012.2205597.
  10. Hornik, K., Stinchcombe, M. and White, H. (1990), "Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks", Neural Networks, 3(5), 551-560. https://doi.org/10.1016/0893-6080(90)90005-6.
  11. Huynh, T. and Thoft-Christensen, P. (2001), "Suspension bridge flutter for girders with separate control flaps", J. Bridge Eng., 6(3), 168-175. https://doi.org/10.1061/(ASCE)1084-0702(2001)6:3(168).
  12. Kobayashi, H. and Nagaoka, H. (1992), "Active control of flutter of a suspension bridge", J. Wind Eng. Ind. Aerod., 41(1-3), 143-151. https://doi.org/10.1016/0167-6105(92)90402-V.
  13. Korlin, R. and Starossek, U. (2007), "Wind tunnel test of an active mass damper for bridge decks", J. Wind Eng. Ind. Aerod., 95(4), 267-277. https://doi.org/10.1016/j.jweia.2006.06.015.
  14. Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012), "ImageNet classification with deep convolutional neural networks", Adv. Neural Inf. Process. Syst., 25, 1097-1105.
  15. Li, K., Ge, Y.J., Guo, Z.W. and Zhao, L. (2015), "Theoretical framework of feedback aerodynamic control of flutter oscillation for long-span suspension bridges by the twin-winglet system", J. Wind Eng. Ind. Aerod., 145, 166-177. https://doi.org/10.1016/j.jweia.2015.06.012.
  16. Li, K., Zhao, L., Ge, Y.J. and Guo, Z.W. (2017), "Flutter suppression of a suspension bridge sectional model by the feedback controlled twin-winglet system", J. Wind Eng. Ind. Aerod., 168, 101-109. https://doi.org/10.1016/j.jweia.2017.05.007.
  17. Li, S. and Wu, T. (2022), "Deep reinforcement learning-based decision support system for transportation infrastructure management under hurricane events", Struct. Safety, 99, 102254. https://doi.org/10.1016/j.strusafe.2022.102254.
  18. Li, S., Snaiki, R. and Wu, T. (2021a), "A knowledge-enhanced deep reinforcement learning-based shape optimizer for aerodynamic mitigation of wind-sensitive structures", Comp.-Aided Civil Infrastruct. Eng., 36(6), 733-746. https://doi.org/10.1111/mice.12655.
  19. Li, S., Snaiki, R. and Wu, T. (2021b), "Active simulation of transient wind field in a multiple-fan wind tunnel via deep reinforcement learning", J. Eng. Mech., 147(9), 04021056. https://doi.org/10.1061/(ASCE)EM.1943-7889.0001967.
  20. Li, Y.L., Chen, X.Y., Yu, C.J., Togbenou, K., Wang, B. and Zhu, L.D. (2018), "Effects of wind fairing angle on aerodynamic characteristics and dynamic responses of a streamlined trapezoidal box girder", J. Wind Eng. Ind. Aerod., 177, 69-78. https://doi.org/10.1016/j.jweia.2018.04.006.
  21. Lillicrap, T.P., Hunt, J.J., Pritzel, A., Heess, N., Erez, T., Tassa, Y., Silver, D., and Wierstra, D. (2016), "Continuous control with deep reinforcement learning", Proceeding 4th Int. Conf. Learning Represent., ICLR, San Juan, Sept. https://doi.org/10.48550/arXiv.1509.02971.
  22. Lin, Y.K. and Yang, J.N. (1983), "Multimode bridge response to wind excitations", J. Eng. Mech., 109(2), 586-603. https://doi.org/10.1061/(ASCE)0733-9399(1983)109:2(586).
  23. Lin, Y.Y., Cheng, C.M. and Lee, C.H. (2000), "A tuned mass damper for suppressing the coupled flexural and torsional buffeting response of long-span bridges", Eng. Struct., 22(9), 1195-1204. https://doi.org/10.1016/S0141-0296(99)00049-8.
  24. Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G. and Petersen, S. (2015), "Human-level control through deep reinforcement learning", Nature, 518(7540), 529-533. https://doi.org/10.1038/nature14236.
  25. Montoya, M.C., Hernandez, S. and Nieto, F. (2018b), "Shape optimization of streamlined decks of cable-stayed bridges considering aeroelastic and structural constraints", J. Wind Eng. Ind. Aerod., 177, 429-455. https://doi.org/10.1016/j.jweia.2017.12.018.
  26. Montoya, M.C., Nieto, F., Hernandez, S., Kusano, I., Alvarez, A.J. and Jurado, J.A. (2018a), "CFD-based aeroelastic characterization of streamlined bridge deck cross-sections subject to shape modifications using surrogate models", J. Wind Eng. Ind. Aerod., 177, 405-428. https://doi.org/10.1016/j.jweia.2018.01.014.
  27. Omenzetter, P., Wilde, K. and Fujino, Y. (2002a), "Study of passive deck-flaps flutter control system on full bridge model. I: theory", J. Eng. Mech., 128(3), 264-279. https://doi.org/10.1061/(ASCE)0733-9399(2002)128:3(264).
  28. Omenzetter, P., Wilde, K. and Fujino, Y. (2002b), "Study of passive deck-flaps flutter control system on full bridge model. II: results", J. Eng. Mech., 128(3), 280-286. https://doi.org/10.1061/(ASCE)0733-9399(2002)128:3(280).
  29. Rahmani, H.R., Chase, G., Wiering, M. and Konke, C. (2019), "A framework for brain learning-based control of smart structures", Adv. Eng. Inf., 42, 100986. https://doi.org/10.1016/j.aei.2019.100986.
  30. Roger, K.L. (1977), "Airplane math modeling methods for active control design", AGARD-CP-228, 4-1-4-11.
  31. Sallab, A.E., Abdou, M., Perot, E. and Yogamani, S. (2017), "Deep reinforcement learning framework for autonomous driving", Electro. Imag., 2017(19), 70-76. https://doi.org/10.48550/arXiv.1704.02532.
  32. Scanlan, R.H. and Tomko, J. (1971), "Air foil and bridge deck flutter derivatives", J. Eng. Mech. Division., 97(6), 1717-1737. https://doi.org/10.1061/JMCEA3.0001526.
  33. Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A. and Chen, Y. (2017), "Mastering the game of go without human knowledge", Nature, 550(7676), 354-359. https://doi.org/10.1038/nature24270.
  34. Sutton, R. S. and Barto, A. G. (2018), Reinforcement Learning: An Introduction, MIT press, Cambridge, MA, United States of America.
  35. Wang, Q., Yan, L., Hu, G., Li, C., Xiao, Y., Xiong, H., Rabault, J. and Noack, B.R. (2022), "DRLinFluids: an open-source python platform of coupling deep reinforcement learning and OpenFOAM", Phys. Fluids, 34(8), 081801. https://doi.org/10.1063/5.0103113.
  36. Wang, Y., Sun, J., He, H. and Sun, C. (2019), "Deterministic policy gradient with integral compensator for robust quadrotor control", IEEE Transactions Syst., Man, Cybernet.: Syst., 50(10), 3713-3725. https://doi.org/10.1109/TSMC.2018.2884725.
  37. Wilde, K. and Fujino, Y. (1998). "Aerodynamic control of bridge deck flutter by active surfaces", J. Eng. Mech., 124(7),718-727. https://doi.org/10.1061/(ASCE)0733-9399(1998)124:7(718).
  38. Wilde, K., Omenzetter, P. and Fujino, Y. (2001), "Suppression of bridge flutter by active deck-flaps control system", J. Eng. Mech., 127(1), 80-89. https://doi.org/10.1061/(ASCE)0733-9399(2001)127:1(80).
  39. Wu, T. and Snaiki, R. (2022), "Applications of machine learning to wind engineering", Front. Built Environ., 8, 811460. https://doi.org/10.3389/fbuil.2022.811460.
  40. Wu, T., Kareem, A. and Ge, Y. (2013), "Linear and nonlinear aeroelastic analysis frameworks for cable-supported bridges", Nonlinear Dyn., 74, 487-516. https://doi.org/10.1007/s11071-013-0984-7.
  41. Xu, F., Wu, T., Ying, X. and Kareem, A. (2016), "Higher-order self-excited drag forces on bridge decks", J. Eng. Mech., 142(3), 06015007. https://doi.org/10.1061/(ASCE)EM.1943-7889.0001036.
  42. Yan, G. and Zhou, L.L. (2006), "Integrated fuzzy logic and genetic algorithms for multi-objective control of structures using MR dampers", J. Sound Vib., 296(1-2), 368-382. https://doi.org/10.1016/j.jsv.2006.03.011.
  43. Yang, Y., Wu, T., Ge, Y. and Kareem, A. (2015), "Aerodynamic stabilization mechanism of a twin box girder with various slot widths", J. Bridge Eng., 20(3), 04014067. https://doi.org/10.1061/(ASCE)BE.1943-5592.0000645.