Browse > Article
http://dx.doi.org/10.12989/sss.2022.29.1.141

Smartphone-based structural crack detection using pruned fully convolutional networks and edge computing  

Ye, X.W. (Department of Civil Engineering, Zhejiang University)
Li, Z.X. (Department of Civil Engineering, Zhejiang University)
Jin, T. (Department of Civil Engineering, Zhejiang University)
Publication Information
Smart Structures and Systems / v.29, no.1, 2022 , pp. 141-151 More about this Journal
Abstract
In recent years, the industry and research communities have focused on developing autonomous crack inspection approaches, which mainly include image acquisition and crack detection. In these approaches, mobile devices such as cameras, drones or smartphones are utilized as sensing platforms to acquire structural images, and the deep learning (DL)-based methods are being developed as important crack detection approaches. However, the process of image acquisition and collection is time-consuming, which delays the inspection. Also, the present mobile devices such as smartphones can be not only a sensing platform but also a computing platform that can be embedded with deep neural networks (DNNs) to conduct on-site crack detection. Due to the limited computing resources of mobile devices, the size of the DNNs should be reduced to improve the computational efficiency. In this study, an architecture called pruned crack recognition network (PCR-Net) was developed for the detection of structural cracks. A dataset containing 11000 images was established based on the raw images from bridge inspections. A pruning method was introduced to reduce the size of the base architecture for the optimization of the model size. Comparative studies were conducted with image processing techniques (IPTs) and other DNNs for the evaluation of the performance of the proposed PCR-Net. Furthermore, a modularly designed framework that integrated the PCR-Net was developed to realize a DL-based crack detection application for smartphones. Finally, on-site crack detection experiments were carried out to validate the performance of the developed system of smartphone-based detection of structural cracks.
Keywords
deep learning; edge computing; fully convolutional networks; structural crack detection; structural health monitoring;
Citations & Related Records
Times Cited By KSCI : 5  (Citation Analysis)
연도 인용수 순위
1 Li, G., Ma, B., He, S.H., Ren, X.L. and Liu, Q.W. (2020), "Automatic tunnel crack detection based on u-net and a convolutional neural network with alternately updated clique", Sensors, 20(3), 717. https://doi.org/10.3390/s20030717   DOI
2 Liu, Q. (2019), U-Net Implementation in PyTorch. Retrieved from https://github.com/Qiuyan918/Unet_Implementation_PyTorch/b lob/master/Unet_Implementation_PyTorch.ipynb
3 Liu, Z., Li, J., Shen, Z., Huang, G., Yan, S. and Zhang, C. (2017), "Learning efficient convolutional networks through network slimming", Proceedings of 2017 IEEE International Conference on Computer Vision, Venice, Italy. (CD-ROM) https://doi.org/10.1109/ICCV.2017.298   DOI
4 Ni, Y.Q., Ye, X.W. and Ko, J.M. (2012), "Modeling of stress spectrum using long-term monitoring data and finite mixture distributions", J. Eng. Mech., 138(2), 175-183. https://doi.org/10.1061/(ASCE)EM.1943-7889.0000313   DOI
5 Ryu, E., Kang, J., Lee, J., Shin, Y. and Kim, H. (2020), "Automated detection of surface cracks and numerical correlation with thermal-structural behaviors of fire damaged concrete beams", Int. J. Concrete Struct. Mater., 14(1), 12. https://doi.org/10.1186/s40069-019-0387-3   DOI
6 Spencer Jr., B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Engineering, 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030   DOI
7 Mondal, T.G. and Jahanshahi, M.R. (2020), "Autonomous vision-based damage chronology for spatiotemporal condition assessment of civil infrastructure using unmanned aerial vehicle", Smart Struct. Syst., Int. J., 25(6), 733-749. https://doi.org/10.12989/sss.2020.25.6.733   DOI
8 Ni, Y.Q., Ye, X.W. and Ko, J.M. (2010), "Monitoring-based fatigue reliability assessment of steel bridges: analytical model and application", J. Struct. Eng., 136(12), 1563-1573. https://doi.org/10.1061/(ASCE)ST.1943-541X.0000250   DOI
9 Li, S.Y. and Zhao, X.F. (2019), "Image-based concrete crack detection using convolutional neural network and exhaustive search technique", Adv. Civil Eng., 2019. https://doi.org/10.1155/2019/6520620   DOI
10 Tang, Z.Y., Chen, Z.C., Bao, Y.Q. and Li, H. (2019), "Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring", Struct. Control. Health Monitor., 26(1), e2296. https://doi.org/10.1002/stc.2296   DOI
11 Xu, Y., Wei, S.Y., Bao, Y.Q. and Li, H. (2019), "Automatic seismic damage identification of reinforced concrete columns from images by a region-based deep convolutional neural network", Struct. Control. Health Monitor., 26(3), e2313. https://doi.org/10.1002/stc.2313   DOI
12 Ye, X.W., Ni, Y.Q., Wong, K.Y. and Ko, J.M. (2012), "Statistical analysis of stress spectra for fatigue life assessment of steel bridges with structural health monitoring data", Eng. Struct., 45, 166-176. https://doi.org/10.1016/j.engstruct.2012.06.016   DOI
13 Ye, X.W., Ni, Y.Q., Wai, T.T., Wong, K.Y., Zhang, X.M. and Xu, F. (2013), "A vision-based system for dynamic displacement measurement of long-span bridges: algorithm and verification", Smart Struct. Syst., Int. J., 12(3-4), 363-379. https://doi.org/10.12989/sss.2013.12.3_4.363   DOI
14 Ronneberger, O., Fischer, P. and Brox, T. (2015), "U-net: convolutional networks for biomedical image segmentation", Proceedings of the 18th International Conference on Medical Image Computing and Computer Assisted Intervention, Munich, Germany. (CD-ROM) https://doi.org/10.1007/978-3-319-24574-4_28   DOI
15 Ye, X.W., Jin, T. and Yun, C.B. (2019a), "A review on deep learning-based structural health monitoring of civil infrastructures", Smart Struct. Syst., Int. J., 24(5), 567-585. https://doi.org/10.12989/sss.2019.24.5.567   DOI
16 Ye, X.W., Jin, T. and Chen, P.Y. (2019b), "Structural crack detection using deep learning-based fully convolutional networks", Adv. Struct. Eng., 22(16), 3412-3419. https://doi.org/10.1177/1369433219836292   DOI
17 Ye, X.W., Jin, T., Ang, P.P., Bian, X.C. and Chen, Y.M. (2021), "Computer vision-based monitoring of the 3-D structural deformation of an ancient structure induced by shield tunneling construction", Struct. Control. Health Monitor., 28(4), e2702. https://doi.org/10.1002/stc.2702   DOI
18 Bao, Y.Q., Tang, Z.Y. and Li, H. (2019), "Compressive-sensing data reconstruction for structural health monitoring: a machine-learning approach", Struct. Health Monitor., 19(1), 293-304. https://doi.org/10.1177/1475921719844039   DOI
19 Abdel-Qader, I., Abudayyeh, O. and Kelly, M.E. (2003), "Analysis of edge-detection techniques for crack identification in bridges", J. Comput. Civil Eng., 17(4), 255-263. https://doi.org/10.1061/(ASCE)0887-3801(2003)17:4(255)   DOI
20 Alipour, M., Harris, D.K. and Miller, G.R. (2019), "Robust pixel-level crack detection using deep fully convolutional neural networks", J. Comput. Civil Eng., 33(6), 04019040. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000854   DOI
21 Fujita, Y. and Hamamoto, Y. (2011), "A robust automatic crack detection method from noisy concrete surfaces", Mach. Vision Appl., 22(2), 245-254. https://doi.org/10.1007/s00138-009-0244-5   DOI
22 Gresil, M., Yu, L., Shen, Y. and Giurgiutiu, V. (2013), "Predictive model of fatigue crack detection in thick bridge steel structures with piezoelectric wafer active sensors", Smart Struct. Syst., Int. J., 12(2), 97-119. https://doi.org/10.12989/sss.2013.12.2.097   DOI
23 Li, S.Y. and Zhao, X.F. (2020), "Automatic crack detection and measurement of concrete structure using convolutional encoder-decoder network", IEEE Access, 8, 134602-134618. https://doi.org/10.1109/ACCESS.2020.3011106   DOI
24 Hakim, S.J.S. and Razak, H.A. (2014), "Modal parameters based structural damage detection using artificial neural networks - a review", Smart Struct. Syst., Int. J., 14(2), 159-189. https://doi.org/10.12989/sss.2014.14.2.159   DOI
25 Jang, J., Shin, M., Lim, S., Park, J. and Paik, J. (2019), "Intelligent image-based railway inspection system using deep learning-based object detection and weber contrast-based image comparison", Sensors, 19(21), 4738. https://doi.org/10.3390/s19214738   DOI
26 Kingma, D.P. and Ba, J.L. (2015), "Adam: a method for stochastic optimization", Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA. (CD-ROM)