Smart Structures and Systems

Techno-Press (테크노프레스)
권호 표지
DOI QR코드

DOI QR Code

Dynamic characteristics monitoring of wind turbine blades based on improved YOLOv5 deep learning model

  • W.H. Zhao (Institution of Earthquake Protection and Disaster Mitigation, Lanzhou University of Technology) ;
  • W.R. Li (Institution of Earthquake Protection and Disaster Mitigation, Lanzhou University of Technology) ;
  • M.H. Yang (Institution of Earthquake Protection and Disaster Mitigation, Lanzhou University of Technology) ;
  • N. Hong (Institution of Earthquake Protection and Disaster Mitigation, Lanzhou University of Technology) ;
  • Y.F. Du (Institution of Earthquake Protection and Disaster Mitigation, Lanzhou University of Technology)
  • Received : 2022.08.03
  • Accepted : 2023.03.19
  • Published : 2023.05.25
⟨ Previous Next ⟩

Abstract

The dynamic characteristics of wind turbine blades are usually monitored by contact sensors with the disadvantages of high cost, difficult installation, easy damage to the structure, and difficult signal transmission. In view of the above problems, based on computer vision technology and the improved YOLOv5 (You Only Look Once v5) deep learning model, a non-contact dynamic characteristic monitoring method for wind turbine blade is proposed. First, the original YOLOv5l model of the CSP (Cross Stage Partial) structure is improved by introducing the CSP2_2 structure, which reduce the number of residual components to better the network training speed. On this basis, combined with the Deep sort algorithm, the accuracy of structural displacement monitoring is mended. Secondly, for the disadvantage that the deep learning sample dataset is difficult to collect, the blender software is used to model the wind turbine structure with conditions, illuminations and other practical engineering similar environments changed. In addition, incorporated with the image expansion technology, a modeling-based dataset augmentation method is proposed. Finally, the feasibility of the proposed algorithm is verified by experiments followed by the analytical procedure about the influence of YOLOv5 models, lighting conditions and angles on the recognition results. The results show that the improved YOLOv5 deep learning model not only perform well compared with many other YOLOv5 models, but also has high accuracy in vibration monitoring in different environments. The method can accurately identify the dynamic characteristics of wind turbine blades, and therefore can provide a reference for evaluating the condition of wind turbine blades.

Keywords

Acknowledgement

The work described in this paper was jointly supported by the National Science Foundation of China (Grant Nos. 52068049 and 51908266), the Science Fund for Distinguished Young Scholars of Gansu Province (No. 21JR7RA267), and Hongliu Outstanding Young Talents Program of Lanzhou University of Technology.

References

  1. Amenabar, I., Mendikute, A., Lopez-Arraiza, A., Lizaranzu, M. and Aurrekoetxea, J. (2011), "Comparison and analysis of nondestructive testing techniques suitable for delamination inspection in wind turbine blades", Compos. B. Eng., 42(5), 1298-1305. https://doi.org/10.1016/j.compositesb.2011.01.025
  2. Beganovic, N. and Soffker, D. (2016), "Structural health management utilization for lifetime prognosis and advanced control strategy deployment of wind turbines: An overview and outlook concerning actual methods, tools, and obtained results", Renew. Sustain. Energy Rev., 64, 68-83. https://doi.org/10.1016/j.rser.2016.05.083
  3. Dong, C.Z., Celik, O. and Catbas, F.N. (2019), "Marker-free monitoring of the grandstand structures and modal identification using computer vision methods", Struct. Health Monit., 18(5-6), 1491-1509. https://doi.org/10.1177/1475921718806895
  4. Dong, C.Z., Celik, O., Catbas, F.N., O'Brien, E.J. and Taylor, S. (2020), "Structural displacement monitoring using deep learning-based full field optical flow methods", Struct. Infrastruct. Eng., 16(1), 51-71. https://doi.org/10.1080/15732479.2019.1650078
  5. Du, Y., Zhou, S., Jing, X., Peng, Y., Wu, H. and Kwok, N. (2020), "Damage detection techniques for wind turbine blades: A review", Mech. Syst. Signal Process., 141, 106445. https://doi.org/10.1016/j.ymssp.2019.106445
  6. Ehrhardt, D.A., Allen, M.S., Yang, S. and Beberniss, T.J. (2017), "Full-field linear and nonlinear measurements using continuous-scan laser doppler vibrometry and high speed three-dimensional digital image correlation", Mech. Syst. Signal Process., 86, 82-97. https://doi.org/10.1016/j.ymssp.2015.12.003
  7. Felipe-Sese, L. and Diaz, F.A. (2018), "Damage methodology approach on a composite panel based on a combination of Fringe Projection and 2D Digital Image Correlation", Mech. Syst. Signal Process., 101, 467-479. https://doi.org/10.1016/j.ymssp.2017.09.002
  8. Feng, D. and Feng, M.Q. (2017), "Experimental validation of cost-effective vision-based structural health monitoring", Mech. Syst. Signal Process., 88, 199-211. https://doi.org/10.1016/j.ymssp.2016.11.021
  9. Feng, D., Feng, M.Q., Ozer, E. and Fukuda, Y. (2015), "A vision-based sensor for noncontact structural displacement measurement", Sensors, 15(7), 16557-16575. https://doi.org/10.3390/s150716557
  10. Hernandez-Estrada, E., Lastres-Danguillecourt, O., Robles-Ocampo, J.B., Lopez-Lopez, A., Sevilla-Camacho, P.Y., Perez-Sarinana, B.Y. and Dorrego-Portela, J.R. (2021), "Considerations for the structural analysis and design of wind turbine towers: A review", Renew. Sust. Energ. Rev., 137, 110447. https://doi.org/10.1016/j.rser.2020.110447
  11. Hoskere, V., Park, J.W., Yoon, H. and Spencer Jr, B.F. (2019), "Vision-based modal survey of civil infrastructure using unmanned aerial vehicles", J. Struct. Eng., 145(7), 04019062. https://doi.org/10.1061/(ASCE)ST.1943-541X.0002321
  12. Hoskere, V., Narazaki, Y. and Spencer Jr, B.F. (2022), "Physics-based graphics models in 3D synthetic environments as autonomous vision-based inspection testbeds", Sensors, 22(2), 532. https://doi.org/10.3390/s22020532
  13. Jasinien, E., Raiutis, R., Voleiis, A., Vladiauskas, A., Mitchard, D. and Amos, M. (2009), "NDT of wind turbine blades using adapted ultrasonic and radiographic techniques", INSIGHT, 51(9), 477-483. https://doi.org/10.1016/j.ymssp.2019.106445
  14. Jia, W., Xu, S., Liang, Z., Zhao, Y., Min, H., Li, S. and Yu, Y. (2021), "Real-time automatic helmet detection of motorcyclists in urban traffic using improved YOLOv5 detector", IET Image Process, 15(14), 3623-3637. https://doi.org/10.1049/ipr2.12295
  15. Kalaitzakis, M., Kattil, S.R., Vitzilaios, N., Rizos, D. and Sutton, M. (2019), "Dynamic structural health monitoring using a DIC-enabled drone", Proceedings of 2019 International Conference on Unmanned Aircraft Systems (ICUAS), Atlanta, GA, USA, June.
  16. Kaewniam, P., Cao, M., Alkayem, N.F., Li, D. and Manoach, E. (2022), "Recent advances in damage detection of wind turbine blades: A state-of-the-art review", Renew. Sust. Energ. Rev., 167, 112723. https://doi.org/10.1016/j.rser.2022.112723
  17. Khadka, A., Fick, B., Afshar, A., Tavakoli, M. and Baqersad, J. (2020), "Non-contact vibration monitoring of rotating wind turbines using a semi-autonomous UAV", Mech. Syst. Signal Process., 138, 106446. https://doi.org/10.1016/j.ymssp.2019.106446
  18. Kuddus, M.A., Li, J., Hao, H., Li, C. and Bi, K. (2019), "Target-free vision-based technique for vibration measurements of structures subjected to out-of-plane movements", Eng. Struct., 190, 210-222. https://doi.org/10.1016/j.engstruct.2019.04.019
  19. Li, S., Guo, Y., Xu, Y. and Li, Z. (2019), "Real-time geometry identification of moving ships by computer vision techniques in bridge area", Smart Struct. Syst., Int. J., 23(4), 359-371. https://doi.org/10.12989/sss.2019.23.4.359
  20. Li, W., Zhao, W., Gu, J., Fan, B. and Du, Y. (2022), "Dynamic characteristics monitoring of large wind turbine blades based on target-free DSST vision algorithm and UAV", Remote Sens., 14(13), 3113. https://doi.org/10.3390/rs14133113
  21. Liang, X. (2019), "Image-based post-disaster inspection of reinforced concrete bridge systems using deep learning with Bayesian optimization", Comput.-Aided Civil Inf., 34(5), 415-430. https://doi.org/10.1111/mice.12425
  22. Lin, Z., Cevasco, D. and Collu, M. (2020), "A methodology to develop reduced-order models to support the operation and maintenance of offshore wind turbines", Appl. Energy, 259, 114228. https://doi.org/10.1016/j.apenergy.2019.114228
  23. Lydon, D., Taylor, S.E., Lydon, M., del Rincon, J.M. and Hester, D. (2019), "Development and testing of a composite system for bridge health monitoring utilising computer vision and deep learning", Smart Struct. Syst., Int. J., 24(6), 723-732. https://doi.org/10.12989/sss.2019.24.6.723
  24. Mao, Q.C., Sun, H.M., Liu, Y.B. and Jia, R.S. (2019), "Mini-YOLOv3: real-time object detector for embedded applications", IEEE Access, 7, 133529-133538. 10.1109/ACCESS.2019.2941547
  25. Narazaki, Y., Hoskere, V., Eick, B.A., Smith, M.D. and Spencer Jr., B.F. (2019), "Vision-based dense displacement and strain estimation of miter gates with the performance evaluation using physics-based graphics models", Smart Struct. Syst., Int. J., 24(6), 709-721. https://doi.org/10.12989/sss.2019.24.6.709
  26. Narazaki, Y., Hoskere, V., Yoshida, K., Spencer Jr., B.F. and Fujino, Y. (2021), "Synthetic environments for vision-based structural condition assessment of Japanese high-speed railway viaducts", Mech. Syst. Signal Process., 160, 107850. https://doi.org/10.1016/j.ymssp.2021.107850
  27. Onat, O. and Gul, M. (2018), "Application of artificial neural networks to the prediction of out-of-plane response of infill walls subjected to shake table", Smart Struct. Syst., Int. J., 21(4), 521-535. https://doi.org/10.12989/sss.2018.21.4.521
  28. Perry, B.J. and Guo, Y. (2021), "A portable three-component displacement measurement technique using an unmanned aerial vehicle (UAV) and computer vision: a proof of concept", Measurement, 176, 109222. https://doi.org/10.1016/j.measurement.2021.109222
  29. Shao, Y., Li, L., Li, J., An, S. and Hao, H. (2021), "Computer vision based target-free 3D vibration displacement measurement of structures", Eng. Struct., 246, 113040. https://doi.org/10.1016/j.engstruct.2021.113040
  30. Song, Q., Wu, J., Wang, H., An, Y. and Tang, G. (2022), "Computer vision-based illumination-robust and multi-point simultaneous structural displacement measuring method", Mech. Syst. Signal Process., 170, 108822. https://doi.org/10.1016/j.ymssp.2022.108822
  31. Sony, S., Laventure, S. and Sadhu, A. (2019), "A literature review of next-generation smart sensing technology in structural health monitoring", Struct. Control Health Monit., 26(3), e2321. https://doi.org/10.1002/stc.2321
  32. Spencer Jr, B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Eng., 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030
  33. Sun, S., Wang, T., Yang, H. and Chu, F. (2022), "Damage identification of wind turbine blades using an adaptive method for compressive beamforming based on the generalized minimax-concave penalty function", Renew. Energ., 181, 59-70. https://doi.org/10.1016/j.renene.2021.09.024
  34. Tian, Y., Zhang, J. and Yu, S. (2019a), "Vision-based structural scaling factor and flexibility identification through mobile impact testing", Mech. Syst. Signal Process., 122, 387-402. https://doi.org/10.1016/j.ymssp.2018.12.029
  35. Tian, Y., Zhang, J. and Yu, S. (2019b), "Rapid impact testing and system identification of footbridges using particle image velocimetry", Comput.-Aided Civil Inf., 34(2), 130-145. https://doi.org/10.1111/mice.12390
  36. Wu, D., Lv, S., Jiang, M. and Song, H. (2020), "Using channel pruning-based YOLO v4 deep learning algorithm for the real-time and accurate detection of apple flowers in natural environments", Comput. Electron. Agric., 178, 105742. https://doi.org/10.1016/j.compag.2020.105742
  37. Xu, D., Wen, C. and Liu, J. (2019), "Wind turbine blade surface inspection based on deep learning and UAV-taken images", J. Renew. Sustain. Energy, 11(5), 053305. https://doi.org/10.1063/1.5113532
  38. Xu, D., Liu, P.F. and Chen, Z.P. (2021), "Damage mode identification and singular signal detection of composite wind turbine blade using acoustic emission", Compos. Struct., 255, 112954. https://doi.org/10.1016/j.compstruct.2020.112954
  39. Ye, X.W., Dong, C.Z. and Liu, T. (2016), "Image-based structural dynamic displacement measurement using different multi-object tracking algorithms", Smart Struct. Syst., Int. J., 17(6), 935-956. https://doi.org/10.12989/sss.2016.17.6.935
  40. Ye, X.W., Jin, T. and Yun, C.B. (2019), "A review on deep learning-based structural health monitoring of civil infrastructures", Smart Struct. Syst., Int. J., 24(5), 567-585. 10.12989/sss.2019.24.5.567
  41. Yu, Y., Cao, H., Yan, X., Wang, T. and Ge, S.S. (2020), "Defect identification of wind turbine blades based on defect semantic features with transfer feature extractor", Neurocomputing, 376, 1-9. https://doi.org/10.1016/j.neucom.2019.09.071
  42. Zhang, Z. (2000), "A flexible new technique for camera calibration", IEEE Trans. Pattern Anal. Machine Intell., 22(11), 1330-1334. DOI: 10.1109/34.888718
  43. Zhao, X., Liu, H., Yu, Y., Zhu, Q., Hu, W., Li, M. and Ou, J. (2016), "Displacement monitoring technique using a smartphone based on the laser projection-sensing method", Sens. Actuator A Phys., 246, 35-47. https://doi.org/10.1016/j.sna.2016.05.012
  44. Zhou, Y., Zhou, S., Hao, G. and Zhang, J. (2021), "Bridge influence line identification based on big data and interval analysis with affine arithmetic", Measurement, 183, 109807. https://doi.org/10.1016/j.measurement.2021.109807
  45. Zhu, L., Geng, X., Li, Z. and Liu, C. (2021), "Improving yolov5 with attention mechanism for detecting boulders from planetary images", Remote Sens., 13(18), 3776. https://doi.org/10.3390/rs13183776