Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

A novel DNN tracking algorithm for structural system identification

  • Peng, Sheng-Yun (College of Civil Engineering, Tongji University) ;
  • Yan, Ling-Feng (State Key Laboratory of Disaster Reduction in Civil Engineering, Tongji University) ;
  • He, Bin (College of Electronic and Information Engineering, Tongji University) ;
  • Zhou, Ying (State Key Laboratory of Disaster Reduction in Civil Engineering, Tongji University)
  • Received : 2020.07.11
  • Accepted : 2021.01.20
  • Published : 2021.05.25
⟨ Previous Next ⟩

Abstract

In the field of structural health monitoring (SHM), cameras record videos and tracking methods can be applied to calculate the structural displacement. Commercial and unmanned aerial vehicle (UAV) cameras are promising non-contact sensors owning to their high availability and easy installation. However, effective tracking methods need to be developed. In this study, we firstly propose an end-to-end vision measuring framework with a novel deep neural network (DNN) tracker, named Siamese Single Decoder Network (SiamSDN). The system requires no target installation and uses cellphone cameras. For SiamSDN, the position and scale of bounding box are formulated through statistical parameter estimation. Unlike generative trackers, SiamSDN does not require manually extracted features or pre-defined motion areas. The tracking object is solely identified in the first frame. A shaking table test of a five-storey structure is carried out to demonstrate the efficiency. Besides, a UAV is used to simulate the field test. To minimize the error caused by the vibrations of UAV, digital video stabilization (DVS) is proposed to eliminate the drifts. Videos taken by both the commercial and UAV cameras are analyzed to calculate the displacements. Comparing our DNN tracker with feature point matching approach, SiamSDN improves the displacement measuring accuracy by 66.16% and 57.54%, respectively, and the frequency characteristics are obtained precisely.

Keywords

Acknowledgement

This research work is supported by the National Nature Science Foundation of China (Grant No. 52025083, 51878449) and the Fundamental Research Funds for the Central Universities.

References

  1. Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A. and Torr, P.H.S. (2016), "Fully-convolutional siamese networks for object tracking", Lecture Notes in Computer Science: Computer Vision - ECCV 2016 Workshops, Vol. 9914, pp. 850-865.
  2. Bierling, M. (1988), "Displacement estimation by hierarchical blockmatching", Proceedings of Visual Communications and Image Processing '88: Third in a Series, November, Cambridge, MA, USA, Vol. 1001, pp. 942-953. https://doi.org/10.1117/12.969046
  3. Busca, G., Cigada, A., Mazzoleni, P. and Zappa, E. (2014), "Vibration monitoring of multiple bridge points by means of a unique vision-based measuring system", Experim. Mech., 54(2), 255-271. https://doi.org/10.1007/s11340-013-9784-8
  4. Dong, C.Z., Bas, S. and Catbas, F.N. (2019), "A completely noncontact recognition system for bridge influence line using portable cameras and computer vision", Smart Struct. Syst., Int. J., 24(5), 617-630. https://doi.org/10.12989/sss.2019.24.5.617
  5. Feng, D.M. and Feng, M.Q. (2016), "Vision-based multipoint displacement measurement for structural health monitoring", Struct. Control Health Monitor., 23(5), 876-890. https://doi.org/10.1002/stc.1819
  6. Feng, M.Q., Fukuda, Y., Feng, D.M. and Mizuta, M. (2015), "Nontarget vision sensor for remote measurement of bridge dynamic response", J. Bridge Eng., 20(12), 04015023. https://doi.org/10.1061/(ASCE)BE.1943-5592.0000747
  7. Forsyth, D.A. and Ponce, J. (2002), Computer Vision: A Modern Approach, Prentice Hall Professional Technical Reference.
  8. Fukuda, Y., Feng, M.Q., Narita, Y., Kaneko, S. and Tanaka, T. (2013), "Vision-based displacement sensor for monitoring dynamic response using robust object search algorithm", IEEE Sensors J., 13(12), 4725-4732. https://doi.org/10.1109/JSEN.2013.2273309
  9. Harris, C. and Stephens, M. (1988), "A combined corner and edge detector", Proceedings of 4th Alvey Vision Conference.
  10. He, K., Zhang, X., Ren, S. and Sun, J. (2015), "Delving deep into rectifiers: surpassing human-level performance on imagenet classification", Proceedings of the IEEE International Conference on Computer Vision (ICCV), December, pp. 1026-1034.
  11. Huang, N.E., Shen, Z., Long, S.R. and Wu, M.C. (1998), "The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis", Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Science, 454(1971), 903-995. https://doi.org/10.1098/rspa.1998.0193
  12. Huang, L., Zhao, X. and Huang, K. (2019), "Got-10k: A large high-diversity benchmark for generic object tracking in the wild", IEEE Transact. Pattern Anal. Mach. Intell., 43(5), 1562-1577. https://doi.org/10.1109/TPAMI.2019.2957464
  13. Ji, Y.F. and Chang, C.C. (2008), "Nontarget image-based technique for small cable vibration measurement", J. Bridge Eng., 13(1), 34-42. https://doi.org/10.1061/(ASCE)1084-0702(2008)13:1(34)
  14. Jo, H., Sim, S.H., Tatkowski, A., Spencer, B.F. and Nelson, M.E. (2013), "Feasibility of displacement monitoring using low-cost GPS receivers", Struct. Control Health Monitor., 20(9), 1240-1254. https://doi.org/10.1002/stc.1532
  15. Jung, H.J., Lee, J.H., Yoon, S. and Kim, I.H. (2019), "Bridge Inspection and condition assessment using Unmanned Aerial Vehicles (UAVs): Major challenges and solutions from a practical perspective", Smart Struct. Syst., Int. J., 24(5), 669-681. https://doi.org/10.12989/sss.2019.24.5.669
  16. Kohut, P., Holak, K., Uhl, T., Ortyl, L., Owerko, T., Kuras, P. and Kocierz, R. (2013), "Monitoring of a civil structure's state based on noncontact measurements", Struct. Health Monitor., 12(5-6), 411-429. https://doi.org/10.1177/1475921713487397
  17. Lee, J.H., Jung, C.Y., Choi, E. and Cheung, J.H. (2017), "Vision-based multipoint measurement systems for structural in-plane and out-of-plane movements including twisting rotation", Smart Struct. Syst., Int. J., 20(5), 563-572. https://doi.org/10.12989/sss.2017.20.5.563
  18. Li, B., Yan, J.J., Wu, W., Zhu, Z. and Hu, X.L. (2018), "High performance visual tracking with siamese region proposal network", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June, pp. 8971-8980.
  19. Liu, C.X., Chen, L.C., Schroff, F., Adam, H., Hua, W., Yuille, A.L. and Li, F.F. (2019a), "Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June, pp. 82-92.
  20. Liu, J.T., Yang, X.X. and Li, L. (2019b), "VibroNet: Recurrent neural networks with multi-target learning for image-based vibration frequency measurement", J. Sound Vib., 457, 51-66. https://doi.org/10.1016/j.jsv.2019.05.027
  21. Lu, L. and Du, W.T. (2011), "The vehicle-borne electronic image stabilization system based on Gray Projection Algorithm", Proceedings of International Conference on Electric Information and Control Engineering, April, Wuhan, China, pp. 4687-4690.
  22. Luo, L.X. and Feng, M.Q. (2018), "Edge-enhanced matching for gradient-based computer vision displacement measurement", Comput.-Aided Civil Infrastruct. Eng., 33(12), 1019-1040. https://doi.org/10.1111/mice.12415
  23. Lydon, D., Taylor, S.E., Lydon, M., Rincon, J.M. and Hester, D. (2019), "Development and testing of a composite system for bridge health monitoring utilizing computer vision and deep learning", Smart Struct. Syst., Int. J., 24(6), 723-732. https://doi.org/10.12989/sss.2019.24.6.723
  24. Mas, D., Ferrer, B., Espinosa, J., Perez Rodriguez, J., Roig Hernandez, A.B. and Illueca Contri, C. (2011), "High speed imaging and algorithms for non invasive vibrations measurement", EVACES 2011 - Proceedings of the 4th International Conference on Experimental Vibration Analysis for Civil Engineering Structures, Varenna, Italy, October.
  25. Narazaki, Y., Hoskere, V., Eick, B.A., Smith, M.D. and Spencer, B.F. (2019), "Vision-based dense displacement and strain estimation of miter gates with the performance evaluation using physics-based graphiscs models", Smart Struct. Syst., Int. J., 24(6), 709-721. https://doi.org/10.12989/sss.2019.24.6.709
  26. Nassif, H.H., Gindy M. and Davis, J. (2005), "Comparison of laser Doppler vibrometer with contact sensors for monitoring bridge deflection and vibration", NDT & E International, 38(3), 213-218. https://doi.org/10.1016/j.ndteint.2004.06.012
  27. Oh, B.K., Glisic, B., Kim, Y. and Park, H.S. (2019), "Convolutional neural network-based wind-induced response estimation model for tall buildings", Comput.-Aided Civil Infrastruct. Eng., 34(10), 843-858. https://doi.org/10.1111/mice.12476
  28. Pan, B., Tian, L. and Song, X.L. (2016), "Real-time, non-contact and targetless measurement of vertical de- flection of bridges using off-axis digital image correlation", NDT & E Int., 79, 73-80. https://doi.org/10.1016/j.ndteint.2015.12.006
  29. Peng, S.Y., Yu, Y.X., Wang, K. and He, L. (2020), "Accurate Anchor Free Tracking", arXiv, 2006.07560.
  30. Pukelsheim, F. (1994), "The three sigma rule", The American Statistician, 48(2), 88-91. https://doi.org/10.2307/2684253
  31. Ribeiro, D., Calcada, R., Ferreira, J. and Martins, T. (2014), "Noncontact measurement of the dynamic dis- placement of railway bridges using an advanced video-based system", Eng. Struct., 75, 164-180. https://doi.org/10.1016/j.engstruct.2014.04.051
  32. Rosten, E. and Drummond, T. (2005), "Fusing points and lines for high performance tracking", Proceedings of the 10th IEEE International Conference on Computer Vision, ICCV 2005, pp. 1508-1515.
  33. Shi, J.B. (1994), "Good features to track", Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, June, pp. 593-600. https://doi.org/10.1109/CVPR.1994.323794
  34. Spencer, B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Engineering, 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030
  35. Wahbeh, A.M., Caffrey, J.P. and Masri, S.F. (2003), "A vision-based approach for the direct measurement of displacements in vibrating systems", Smart Mater. Struct., 12(5), 785-794. https://doi.org/10.1088/0964-1726/12/5/016
  36. Wu, Y., Lim, J. and Yang, M. (2015), "Object tracking benchmark", IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9), 1834-1848. https://doi.org/10.1109/TPAMI.2014.2388226
  37. Ye, X.W., Dong, C.Z. and Liu, T. (2016), "A review of machine vision-based structural health monitoring: methodologies and applications", J. Sensors, 7103039. https://doi.org/10.1155/2016/7103039
  38. Ye, X.W., Jin, T. and Yun, C.B. (2019), "A review on deep learning-based structural health monitoring of civil infrastructures", Smart Struct. Syst., Int. J., 24(5), 567-585. https://doi.org/10.12989/sss.2019.24.5.567
  39. Yoon, H., Elanwar, H., Choi, H., Golparvar-Fard, M. and Spencer, B.F. (2016), "Target-free approach for vision-based structural system identification using consumer-grade cameras", Struct. Control Health Monitor., 23(12), 1405-1416. https://doi.org/10.1002/stc.1850
  40. Yoon, H., Shin, J. and Spencer, B.F. (2018), "Structural displacement measurement using an unmanned aerial system", Comput.-Aided Civil Infrastruct. Eng., 33(3), 183-192. https://doi.org/10.1111/mice.12338
  41. Yu, S.S. and Zhang, J. (2020), "Fast bridge deflection monitoring through an improved feature tracing algorithm", Comput.-Aided Civil Infrastruct. Eng., 35(3), 292-302. https://doi.org/10.1111/mice.12499
  42. Zhang, X.C., Ye, P., Peng, S.Y., Liu, J., Gong, K. and Xiao, G. (2019a), "SiamFT: An RGB-infrared fusion tracking method via fully convolutional siamese networks", IEEE Access, 7, 122122-122133. https://doi.org/10.1109/ACCESS.2019.2936914
  43. Zhang, Y.Q., Miyamori, Y., Mikami, S. and Saito, T. (2019b), "Vibration-based structural state identification by a 1-dimensional convolutional neural network", Comput.-Aided Civil Infrastruct. Eng., 34(9), 822-839. https://doi.org/10.1111/mice.12447
  44. Zhang, X.C., Ye, P., Peng, S.Y., Liu, J. and Xiao, G. (2020), "DSiamMFT: An RGB-T fusion tracking method via dynamic Siamese networks using multi-layer feature fusion", Signal Process.: Image Commun., 84, 115756. https://doi.org/10.1016/j.image.2019.115756