Browse > Article
http://dx.doi.org/10.12989/sss.2021.27.5.803

A novel DNN tracking algorithm for structural system identification  

Peng, Sheng-Yun (College of Civil Engineering, Tongji University)
Yan, Ling-Feng (State Key Laboratory of Disaster Reduction in Civil Engineering, Tongji University)
He, Bin (College of Electronic and Information Engineering, Tongji University)
Zhou, Ying (State Key Laboratory of Disaster Reduction in Civil Engineering, Tongji University)
Publication Information
Smart Structures and Systems / v.27, no.5, 2021 , pp. 803-818 More about this Journal
Abstract
In the field of structural health monitoring (SHM), cameras record videos and tracking methods can be applied to calculate the structural displacement. Commercial and unmanned aerial vehicle (UAV) cameras are promising non-contact sensors owning to their high availability and easy installation. However, effective tracking methods need to be developed. In this study, we firstly propose an end-to-end vision measuring framework with a novel deep neural network (DNN) tracker, named Siamese Single Decoder Network (SiamSDN). The system requires no target installation and uses cellphone cameras. For SiamSDN, the position and scale of bounding box are formulated through statistical parameter estimation. Unlike generative trackers, SiamSDN does not require manually extracted features or pre-defined motion areas. The tracking object is solely identified in the first frame. A shaking table test of a five-storey structure is carried out to demonstrate the efficiency. Besides, a UAV is used to simulate the field test. To minimize the error caused by the vibrations of UAV, digital video stabilization (DVS) is proposed to eliminate the drifts. Videos taken by both the commercial and UAV cameras are analyzed to calculate the displacements. Comparing our DNN tracker with feature point matching approach, SiamSDN improves the displacement measuring accuracy by 66.16% and 57.54%, respectively, and the frequency characteristics are obtained precisely.
Keywords
structural health monitoring; commercial camera; unmanned aerial vehicle; siamese network; frequency characteristics;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Jung, H.J., Lee, J.H., Yoon, S. and Kim, I.H. (2019), "Bridge Inspection and condition assessment using Unmanned Aerial Vehicles (UAVs): Major challenges and solutions from a practical perspective", Smart Struct. Syst., Int. J., 24(5), 669-681. https://doi.org/10.12989/sss.2019.24.5.669   DOI
2 Zhang, X.C., Ye, P., Peng, S.Y., Liu, J. and Xiao, G. (2020), "DSiamMFT: An RGB-T fusion tracking method via dynamic Siamese networks using multi-layer feature fusion", Signal Process.: Image Commun., 84, 115756. https://doi.org/10.1016/j.image.2019.115756   DOI
3 Li, B., Yan, J.J., Wu, W., Zhu, Z. and Hu, X.L. (2018), "High performance visual tracking with siamese region proposal network", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June, pp. 8971-8980.
4 Liu, J.T., Yang, X.X. and Li, L. (2019b), "VibroNet: Recurrent neural networks with multi-target learning for image-based vibration frequency measurement", J. Sound Vib., 457, 51-66. https://doi.org/10.1016/j.jsv.2019.05.027   DOI
5 Lu, L. and Du, W.T. (2011), "The vehicle-borne electronic image stabilization system based on Gray Projection Algorithm", Proceedings of International Conference on Electric Information and Control Engineering, April, Wuhan, China, pp. 4687-4690.
6 Luo, L.X. and Feng, M.Q. (2018), "Edge-enhanced matching for gradient-based computer vision displacement measurement", Comput.-Aided Civil Infrastruct. Eng., 33(12), 1019-1040. https://doi.org/10.1111/mice.12415   DOI
7 Forsyth, D.A. and Ponce, J. (2002), Computer Vision: A Modern Approach, Prentice Hall Professional Technical Reference.
8 Huang, N.E., Shen, Z., Long, S.R. and Wu, M.C. (1998), "The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis", Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Science, 454(1971), 903-995. https://doi.org/10.1098/rspa.1998.0193   DOI
9 Lydon, D., Taylor, S.E., Lydon, M., Rincon, J.M. and Hester, D. (2019), "Development and testing of a composite system for bridge health monitoring utilizing computer vision and deep learning", Smart Struct. Syst., Int. J., 24(6), 723-732. https://doi.org/10.12989/sss.2019.24.6.723   DOI
10 Narazaki, Y., Hoskere, V., Eick, B.A., Smith, M.D. and Spencer, B.F. (2019), "Vision-based dense displacement and strain estimation of miter gates with the performance evaluation using physics-based graphiscs models", Smart Struct. Syst., Int. J., 24(6), 709-721. https://doi.org/10.12989/sss.2019.24.6.709   DOI
11 Pukelsheim, F. (1994), "The three sigma rule", The American Statistician, 48(2), 88-91.   DOI
12 Feng, M.Q., Fukuda, Y., Feng, D.M. and Mizuta, M. (2015), "Nontarget vision sensor for remote measurement of bridge dynamic response", J. Bridge Eng., 20(12), 04015023. https://doi.org/10.1061/(ASCE)BE.1943-5592.0000747   DOI
13 Nassif, H.H., Gindy M. and Davis, J. (2005), "Comparison of laser Doppler vibrometer with contact sensors for monitoring bridge deflection and vibration", NDT & E International, 38(3), 213-218. https://doi.org/10.1016/j.ndteint.2004.06.012   DOI
14 Oh, B.K., Glisic, B., Kim, Y. and Park, H.S. (2019), "Convolutional neural network-based wind-induced response estimation model for tall buildings", Comput.-Aided Civil Infrastruct. Eng., 34(10), 843-858. https://doi.org/10.1111/mice.12476   DOI
15 Pan, B., Tian, L. and Song, X.L. (2016), "Real-time, non-contact and targetless measurement of vertical de- flection of bridges using off-axis digital image correlation", NDT & E Int., 79, 73-80. https://doi.org/10.1016/j.ndteint.2015.12.006   DOI
16 Peng, S.Y., Yu, Y.X., Wang, K. and He, L. (2020), "Accurate Anchor Free Tracking", arXiv, 2006.07560.
17 Ribeiro, D., Calcada, R., Ferreira, J. and Martins, T. (2014), "Noncontact measurement of the dynamic dis- placement of railway bridges using an advanced video-based system", Eng. Struct., 75, 164-180. https://doi.org/10.1016/j.engstruct.2014.04.051   DOI
18 Rosten, E. and Drummond, T. (2005), "Fusing points and lines for high performance tracking", Proceedings of the 10th IEEE International Conference on Computer Vision, ICCV 2005, pp. 1508-1515.
19 Harris, C. and Stephens, M. (1988), "A combined corner and edge detector", Proceedings of 4th Alvey Vision Conference.
20 Fukuda, Y., Feng, M.Q., Narita, Y., Kaneko, S. and Tanaka, T. (2013), "Vision-based displacement sensor for monitoring dynamic response using robust object search algorithm", IEEE Sensors J., 13(12), 4725-4732. https://doi.org/10.1109/JSEN.2013.2273309   DOI
21 He, K., Zhang, X., Ren, S. and Sun, J. (2015), "Delving deep into rectifiers: surpassing human-level performance on imagenet classification", Proceedings of the IEEE International Conference on Computer Vision (ICCV), December, pp. 1026-1034.
22 Huang, L., Zhao, X. and Huang, K. (2019), "Got-10k: A large high-diversity benchmark for generic object tracking in the wild", IEEE Transact. Pattern Anal. Mach. Intell., 43(5), 1562-1577. https://doi.org/10.1109/TPAMI.2019.2957464   DOI
23 Ji, Y.F. and Chang, C.C. (2008), "Nontarget image-based technique for small cable vibration measurement", J. Bridge Eng., 13(1), 34-42. https://doi.org/10.1061/(ASCE)1084-0702(2008)13:1(34)   DOI
24 Jo, H., Sim, S.H., Tatkowski, A., Spencer, B.F. and Nelson, M.E. (2013), "Feasibility of displacement monitoring using low-cost GPS receivers", Struct. Control Health Monitor., 20(9), 1240-1254. https://doi.org/10.1002/stc.1532   DOI
25 Kohut, P., Holak, K., Uhl, T., Ortyl, L., Owerko, T., Kuras, P. and Kocierz, R. (2013), "Monitoring of a civil structure's state based on noncontact measurements", Struct. Health Monitor., 12(5-6), 411-429. https://doi.org/10.1177/1475921713487397   DOI
26 Lee, J.H., Jung, C.Y., Choi, E. and Cheung, J.H. (2017), "Vision-based multipoint measurement systems for structural in-plane and out-of-plane movements including twisting rotation", Smart Struct. Syst., Int. J., 20(5), 563-572. https://doi.org/10.12989/sss.2017.20.5.563   DOI
27 Mas, D., Ferrer, B., Espinosa, J., Perez Rodriguez, J., Roig Hernandez, A.B. and Illueca Contri, C. (2011), "High speed imaging and algorithms for non invasive vibrations measurement", EVACES 2011 - Proceedings of the 4th International Conference on Experimental Vibration Analysis for Civil Engineering Structures, Varenna, Italy, October.
28 Spencer, B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Engineering, 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030   DOI
29 Dong, C.Z., Bas, S. and Catbas, F.N. (2019), "A completely noncontact recognition system for bridge influence line using portable cameras and computer vision", Smart Struct. Syst., Int. J., 24(5), 617-630. https://doi.org/10.12989/sss.2019.24.5.617   DOI
30 Shi, J.B. (1994), "Good features to track", Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, June, pp. 593-600. https://doi.org/10.1109/CVPR.1994.323794   DOI
31 Wahbeh, A.M., Caffrey, J.P. and Masri, S.F. (2003), "A vision-based approach for the direct measurement of displacements in vibrating systems", Smart Mater. Struct., 12(5), 785-794. https://doi.org/10.1088/0964-1726/12/5/016   DOI
32 Wu, Y., Lim, J. and Yang, M. (2015), "Object tracking benchmark", IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(9), 1834-1848. https://doi.org/10.1109/TPAMI.2014.2388226   DOI
33 Ye, X.W., Dong, C.Z. and Liu, T. (2016), "A review of machine vision-based structural health monitoring: methodologies and applications", J. Sensors, 7103039. https://doi.org/10.1155/2016/7103039   DOI
34 Ye, X.W., Jin, T. and Yun, C.B. (2019), "A review on deep learning-based structural health monitoring of civil infrastructures", Smart Struct. Syst., Int. J., 24(5), 567-585. https://doi.org/10.12989/sss.2019.24.5.567   DOI
35 Yoon, H., Elanwar, H., Choi, H., Golparvar-Fard, M. and Spencer, B.F. (2016), "Target-free approach for vision-based structural system identification using consumer-grade cameras", Struct. Control Health Monitor., 23(12), 1405-1416. https://doi.org/10.1002/stc.1850   DOI
36 Yoon, H., Shin, J. and Spencer, B.F. (2018), "Structural displacement measurement using an unmanned aerial system", Comput.-Aided Civil Infrastruct. Eng., 33(3), 183-192. https://doi.org/10.1111/mice.12338   DOI
37 Busca, G., Cigada, A., Mazzoleni, P. and Zappa, E. (2014), "Vibration monitoring of multiple bridge points by means of a unique vision-based measuring system", Experim. Mech., 54(2), 255-271. https://doi.org/10.1007/s11340-013-9784-8   DOI
38 Liu, C.X., Chen, L.C., Schroff, F., Adam, H., Hua, W., Yuille, A.L. and Li, F.F. (2019a), "Auto-deeplab: Hierarchical neural architecture search for semantic image segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June, pp. 82-92.
39 Yu, S.S. and Zhang, J. (2020), "Fast bridge deflection monitoring through an improved feature tracing algorithm", Comput.-Aided Civil Infrastruct. Eng., 35(3), 292-302. https://doi.org/10.1111/mice.12499   DOI
40 Zhang, X.C., Ye, P., Peng, S.Y., Liu, J., Gong, K. and Xiao, G. (2019a), "SiamFT: An RGB-infrared fusion tracking method via fully convolutional siamese networks", IEEE Access, 7, 122122-122133. https://doi.org/10.1109/ACCESS.2019.2936914   DOI
41 Zhang, Y.Q., Miyamori, Y., Mikami, S. and Saito, T. (2019b), "Vibration-based structural state identification by a 1-dimensional convolutional neural network", Comput.-Aided Civil Infrastruct. Eng., 34(9), 822-839. https://doi.org/10.1111/mice.12447   DOI
42 Feng, D.M. and Feng, M.Q. (2016), "Vision-based multipoint displacement measurement for structural health monitoring", Struct. Control Health Monitor., 23(5), 876-890. https://doi.org/10.1002/stc.1819   DOI
43 Bertinetto, L., Valmadre, J., Henriques, J.F., Vedaldi, A. and Torr, P.H.S. (2016), "Fully-convolutional siamese networks for object tracking", Lecture Notes in Computer Science: Computer Vision - ECCV 2016 Workshops, Vol. 9914, pp. 850-865.
44 Bierling, M. (1988), "Displacement estimation by hierarchical blockmatching", Proceedings of Visual Communications and Image Processing '88: Third in a Series, November, Cambridge, MA, USA, Vol. 1001, pp. 942-953. https://doi.org/10.1117/12.969046   DOI