Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

An image-based deep learning network technique for structural health monitoring

  • Lee, Dong-Han (Department of Mechanical, Robotics, and Energy Engineering, Dongguk University-Seoul) ;
  • Koh, Bong-Hwan (Department of Mechanical, Robotics, and Energy Engineering, Dongguk University-Seoul)
  • Received : 2021.04.15
  • Accepted : 2021.09.27
  • Published : 2021.12.25
⟨ Previous Next ⟩

Abstract

When monitoring the structural integrity of a bridge using data collected through accelerometers, identifying the profile of the load exerted on the bridge from the vehicles passing over it becomes a crucial task. In this study, the speed and location of vehicles on the deck of a bridge is reconfigured using real-time video to implicitly associate the load applied to the bridge with the response from the bridge sensors to develop an image-based deep learning network model. Instead of directly measuring the load that a moving vehicle exerts on the bridge, the intention in the proposed method is to replace the correlation between the movement of vehicles from CCTV images and the corresponding response by the bridge with a neural network model. Given the framework of an input-output-based system identification, CCTV images secured from the bridge and the acceleration measurements from a cantilevered beam are combined during the process of training the neural network model. Since in reality, structural damage cannot be induced in a bridge, the focus of the study is on identifying local changes in parameters by adding mass to a cantilevered beam in the laboratory. The study successfully identified the change in the material parameters in the beam by using the deep-learning neural network model. Also, the method correctly predicted the acceleration response of the beam. The proposed approach can be extended to the structural health monitoring of actual bridges, and its sensitivity to damage can also be improved through optimization of the network training.

Keywords

Acknowledgement

This work was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2018R1D1A1B07043813).

References

  1. Byun, Y.S., Jeong, R.G. and Kang, S.W. (2015), "Vehicle position estimation based on magnetic markers: Enhanced accuracy by compensation of time delays", Sensors, 15(11), 28807-28825. https://doi.org/10.3390/s151128807
  2. Cantero, D., Arvidsson, T., OBrien, E. and Karoumi, R. (2016), "Train-track-bridge modelling and review of parameters", Struct. Infrastruct. Eng., 12(9), 1051-1064. https://doi.org/10.1080/15732479.2015.1076854
  3. Cantero, D., Hester, D. and Brownjohn, J. (2017), "Evolution of bridge frequencies and modes of vibration during truck passage", Eng. Struct., 152, 452-464. https://doi.org/10.1016/j.engstruct.2017.09.039
  4. Chang, M. and Pakzad, S.N. (2013), "Modified natural excitation technique for stochastic modal identification", Journal of Structural Engineering, 139(10), 1753-1762. https://doi.org/10.1061/(asce)st.1943-541x.0000559
  5. Chang, M. and Pakzad, S.N. (2014), "Observer Kalman filter identification for output-only systems using interactive structural modal identification toolsuite", J. Bridge Eng., 19(5), 04014002. https://doi.org/10.1061/(ASCE)BE.1943-5592.0000530
  6. Cho, S., Jo, H., Jang, S., Park, J., Jung, H.J., Yun, C.B., Spencer Jr, B.F. and Seo, J.W. (2010), "Structural health monitoring of a cable-stayed bridge using wireless smart sensor technology: data analyses", Smart Struct. Syst., Int. J., 6(5-6), 461-480. https://doi.org/10.12989/sss.2010.6.5_6.461
  7. Comanducci, G., Ubertini, F. and Materazzi, A.L. (2015), "Structural health monitoring of suspension bridges with features affected by changing wind speed", J. Wind Eng. Indust. Aerodyn., 141, 12-26. https://doi.org/10.1016/j.jweia.2015.02.007
  8. Deng, L. and Cai, C.S. (2009), "Identification of parameters of vehicles moving on bridges", Eng. Struct., 31(10), 2474-2485. https://doi.org/10.1016/j.engstruct.2009.06.005
  9. Deng, L., He, W., Yu, Y. and Cai, C.S. (2018), "Equivalent shear force method for detecting the speed and axles of moving vehicles on bridges", J. Bridge Eng., 23(8), 04018057. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001278
  10. Dong, H., Wen, M. and Yang, Z. (2019), "Vehicle speed estimation based on 3d convnets and non-local blocks", Future Internet, 11(6), 123. https://doi.org/10.3390/fi11060123
  11. Enckell, M., Glisic, B., Myrvoll, F. and Bergstrand, B. (2011), "Evaluation of a large-scale bridge strain, temperature and crack monitoring with distributed fibre optic sensors", J. Civil Struct. Health Monitor., 1(1-2), 37-46. https://doi.org/10.1007/s13349-011-0004-x
  12. Goodfellow, I., Bengio, Y. and Courville, A. (2016), Deep Learning, Vol. 1, MIT press Cambridge.
  13. Guo, A., Jiang, A., Lin, J. and Li, X. (2020), "Data mining algorithms for bridge health monitoring: Kohonen clustering and LSTM prediction approaches", J. Supercomput., 76(2), 932-947. https://doi.org/10.1007/s11227-019-03045-8
  14. Hata, A.Y. and Wolf, D.F. (2016), "Feature detection for vehicle localization in urban environments using a multilayer LIDAR", IEEE Transact. Intell. Transport. Syst., 17(2), 420-429. https://doi.org/10.1109/TITS.2015.2477817
  15. Hou, R., Zhang, Y., O'Connor, S., Hong, Y. and Lynch, J.A. (2015), "Monitoring and identification of vehicle-bridge interaction using mobile truck-based wireless sensors", Proceedings of 11th International Workshop on Advanced Smart Materials and Smart Structures Technology, 1-2.
  16. Imai, H., Yun, C.B., Maruyama, O. and Shinozuka, M. (1989), "Fundamentals of system identification in structural dynamics", Probabil. Eng. Mech., 4(4), 162-173. https://doi.org/10.1016/0266-8920(89)90022-2
  17. Jahanshahi, M.R. and Masri, S.F. (2012), "Adaptive vision-based crack detection using 3D scene reconstruction for condition assessment of structures", Automat. Constr., 22, 567-576. https://doi.org/10.1016/j.autcon.2011.11.018
  18. James, G.H., Carne, T.G. and Lauffer, J.P. (1995), "The Natural Excitation Technique (NExT) for Modal Parameter Extraction from Operating Structures", Modal Anal. - Int. J. Anal. Experim. Modal Anal., 10(4), 260.
  19. Jang, S., Jo, H., Cho, S., Mechitov, K., Rice, J.A., Sim, S.H., Jung, H.J., Yun, C.B., Spencer Jr, B.F. and Agha, G. (2010), "Structural health monitoring of a cable-stayed bridge using smart sensor technology: deployment and evaluation", Smart Struct. Syst., Int. J., 6(5-6), 439-459. https://doi.org/10.12989/sss.2010.6.5_6.461
  20. Javadi, S., Dahl, M. and Pettersson, M.I. (2019), "Vehicle speed measurement model for video-based systems", Comput. Electr. Eng., 76, 238-248. https://doi.org/10.1016/j.compeleceng.2019.04.001
  21. Juang, J.N. (1994), Applied System Identification, Prentice-Hall, Inc.
  22. Khuc, T. and Catbas, F.N. (2018), "Structural identification using computer vision-based bridge health monitoring", J. Struct. Eng., 144(2), 04017202. https://doi.org/10.1061/(ASCE)ST.1943-541X.0001925
  23. Kim, T. and Park, T.H. (2020), "Extended Kalman filter (EKF) design for vehicle position tracking using reliability function of radar and lidar", Sensors, 20(15), 4126. https://doi.org/10.3390/s20154126
  24. Kim, C.W., Kawatani, M. and Kim, K.B. (2005), "Three-dimensional dynamic analysis for bridge-vehicle interaction with roadway roughness", Comput. Struct., 83(19), 1627-1645. https://doi.org/10.1016/j.compstruc.2004.12.004
  25. Li, J., Chen, S., Zhang, F., Li, E., Yang, T. and Lu, Z. (2019), "An adaptive framework for multi-vehicle ground speed estimation in airborne videos", Remote Sensing, 11(10), 1241. https://doi.org/10.3390/rs11101241
  26. Ljung, L. (2010), "Perspectives on system identification", Annual Rev. Control, 34(1), 1-12. https://doi.org/10.1016/j.arcontrol.2009.12.001
  27. Luvizon, D.C., Nassu, B.T. and Minetto, R. (2016), "A video-based system for vehicle speed measurement in urban roadways", IEEE Transact. Intell. Transport. Syst., 18(6), 1393-1404. https://doi.org/10.1109/TITS.2016.2606369
  28. Lynch, J.P., Law, K.H., Kiremidjian, A.S., Carryer, E.D., Farrar, C.R., Sohn, H., Allen, D.W., Nadler, B. and Wait, J.R. (2004), "Design and performance validation of a wireless sensing unit for structural monitoring applications", Struct. Eng. Mech., Int. J., 17(3-4), 393-408. https://doi.org/10.12989/sem.2004.17.3_4.393
  29. Malekjafarian, A., McGetrick, P.J. and OBrien, E.J. (2015), "A review of indirect bridge monitoring using passing vehicles", Shock Vib., 2015. https://doi.org/10.1155/2015/286139
  30. Mei, Q. and Gul, M. (2019), "A crowdsourcing-based methodology using smartphones for bridge health monitoring", Struct. Health Monitor., 18(5-6), 1602-1619. https://doi.org/10.1177/1475921718815457
  31. Meng, X., Dodson, A.H. and Roberts, G.W. (2007), "Detecting bridge dynamics with GPS and triaxial accelerometers", Eng. Struct., 29(11), 3178-3184. https://doi.org/10.1016/j.engstruct.2007.03.012
  32. Miyamoto, A., Yabe, A. and Lucio, V.J. (2017), "Damage detection sensitivity of a vehicle-based bridge health monitoring system", J. Phys.: Conference Series, 842, 012032. https://doi.org/10.1088/1742-6596/842/1/012032
  33. Moreu, F., Kim, R.E. and Spencer Jr, B.F. (2017), "Railroad bridge monitoring using wireless smart sensors", Struct. Control Health Monitor., 24(2), e1863. https://doi.org/10.1002/stc.1863
  34. Mufti, A.A., Tadros, G. and Jones, P.R. (1997), "Field assessment of fibre-optic Bragg grating strain sensors in the confederation bridge", Can. J. Civil Eng., 24(6), 963-966. https://doi.org/10.1139/l97-080
  35. Nagarajaiah, S., Dyke, S., Lynch, J.P., Smyth, A., Agrawal, A., Symans, M. and Johnson, E. (2008), "Current directions of structural health monitoring and control in USA", In: Advances in Science and Technology, Volume 56, pp. 277-286, Trans Tech Publ.
  36. Nayek, R. and Narasimhan, S. (2020), "Extraction of contact-point response in indirect bridge health monitoring using an input estimation approach", J. Civil Struct. Health Monitor., 10(5), 815-831. https://doi.org/10.1007/s13349-020-00418-z
  37. Pan, Y., Dong, Y., Wang, D., Chen, A. and Ye, Z. (2019), "Three-dimensional reconstruction of structural surface model of heritage bridges using UAV-based photogrammetric point clouds", Remote Sensing, 11(10), 1204. https://doi.org/10.3390/rs11101204
  38. Papageorgiou, C. and Poggio, T. (2000), "A trainable system for object detection", Int. J. Comput. Vision, 38(1), 15-33. https://doi.org/10.1023/A:1008162616689
  39. Redmon, J. and Farhadi, A. (2018), "Yolov3: An incremental improvement", ArXiv Preprint ArXiv:1804.02767.
  40. Smyth, A.W., Pei, J.S. and Masri, S.F. (2003), "System identification of the Vincent Thomas suspension bridge using earthquake records", Earthq. Eng. Struct. Dyn., 32(3), 339-367. https://doi.org/10.1002/eqe.226
  41. Sochor, J., Juranek, R. and Herout, A. (2017), "Traffic surveillance camera calibration by 3d model bounding box alignment for accurate vehicle speed measurement", Comput. Vision Image Understand., 161, 87-98. https://doi.org/10.1016/j.cviu.2017.05.015
  42. Tan, G.H., Brameld, G.H. and Thambiratnam, D.P. (1998), "Development of an analytical model for treating bridge-vehicle interaction", Eng. Struct., 20(1), 54-61. https://doi.org/10.1016/S0141-0296(97)00051-5
  43. Tang, Z., Wang, G., Xiao, H., Zheng, A. and Hwang, J.N. (2018), "Single-camera and inter-camera vehicle tracking and 3D speed estimation based on fusion of visual and semantic features", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 108-115.
  44. Tong, X., Yang, H., Wang, L. and Miao, Y. (2019), "The development and field evaluation of an IoT system of low-power vibration for bridge health monitoring", Sensors, 19(5), 1222. https://doi.org/10.3390/s19051222
  45. Vagnoli, M., Remenyte-Prescott, R. and Andrews, J. (2018), "Railway bridge structural health monitoring and fault detection: State-of-the-art methods and future challenges", Struct. Health Monitor., 17(4), 971-1007. https://doi.org/10.1177/1475921717721137
  46. Xiao, F., Fan, J., Chen, G.S. and Hulsey, J.L. (2019), "Bridge health monitoring and damage identification of truss bridge using strain measurements", Adv. Mech. Eng., 11(3), 1-7. https://doi.org/10.1177/1687814019832216
  47. Yang, Y., Li, S., Nagarajaiah, S., Li, H. and Zhou, P. (2016), "Real-time output-only identification of time-varying cable tension from accelerations via complexity pursuit", J. Struct. Eng., 142(1), 04015083. https://doi.org/10.1061/(ASCE)ST.1943-541X.0001337
  48. Yang, L., Li, M., Song, X., Xiong, Z., Hou, C. and Qu, B. (2019), "Vehicle speed measurement based on binocular stereovision system", IEEE Access, 7, 106628-106641. https://doi.org/10.1109/ACCESS.2019.2932120
  49. Yi, T.H., Li, H.N. and Gu, M. (2013), "Wavelet based multi-step filtering method for bridge health monitoring using GPS and accelerometer", Smart Struct. Syst., Int. J., 11(4), 331-348. https://doi.org/10.12989/sss.2013.11.4.331
  50. Yu, H., Wang, B., Li, Y., Zhang, Y. and Zhang, W. (2018), "Road vehicle-bridge interaction considering varied vehicle speed based on convenient combination of simulink and ANSYS", Shock Vib., 2018. https://doi.org/10.1155/2018/1389628
  51. Yu, Z., Shen, Y. and Shen, C. (2021), "A real-time detection approach for bridge cracks based on YOLOv4-FPM", Automat. Constr., 122, 103514. https://doi.org/10.1016/j.autcon.2020.103514
  52. Zhang, X., Story, B. and Rajan, D. (2020), "Night Time Vehicle Detection and Tracking by Fusing Sensor Cues from Autonomous Vehicles", Proceedings of 2020 IEEE 91st Vehicular Technology Conference (VTC2020-Spring), pp. 1-7.
  53. Zhu, D., Guo, J., Cho, C., Wang, Y. and Lee, K.M. (2012), "Wireless mobile sensor network for the system identification of a space frame bridge", Ieee/Asme Transact. Mechatron., 17(3), 499-507. https://doi.org/10.1109/TMECH.2012.2187915