Browse > Article
http://dx.doi.org/10.12989/sss.2021.29.1.029

A three-stage deep-learning-based method for crack detection of high-resolution steel box girder image  

Meng, Shiqiao (State Key Laboratory of Disaster Reduction in Civil Engineering, Tongji University)
Gao, Zhiyuan (College of Civil Engineering, Tongji University)
Zhou, Ying (State Key Laboratory of Disaster Reduction in Civil Engineering, Tongji University)
He, Bin (College of Electronic and Information Engineering, Tongji University)
Kong, Qingzhao (State Key Laboratory of Disaster Reduction in Civil Engineering, Tongji University)
Publication Information
Smart Structures and Systems / v.29, no.1, 2022 , pp. 29-39 More about this Journal
Abstract
Crack detection plays an important role in the maintenance and protection of steel box girder of bridges. However, since the cracks only occupy an extremely small region of the high-resolution images captured from actual conditions, the existing methods cannot deal with this kind of image effectively. To solve this problem, this paper proposed a novel three-stage method based on deep learning technology and morphology operations. The training set and test set used in this paper are composed of 360 images (4928 × 3264 pixels) in steel girder box. The first stage of the proposed model converted high-resolution images into sub-images by using patch-based method and located the region of cracks by CBAM ResNet-50 model. The Recall reaches 0.95 on the test set. The second stage of our method uses the Attention U-Net model to get the accurate geometric edges of cracks based on results in the first stage. The IoU of the segmentation model implemented in this stage attains 0.48. In the third stage of the model, we remove the wrong-predicted isolated points in the predicted results through dilate operation and outlier elimination algorithm. The IoU of test set ascends to 0.70 after this stage. Ablation experiments are conducted to optimize the parameters and further promote the accuracy of the proposed method. The result shows that: (1) the best patch size of sub-images is 1024 × 1024. (2) the CBAM ResNet-50 and the Attention U-Net achieved the best results in the first and the second stage, respectively. (3) Pre-training the model of the first two stages can improve the IoU by 2.9%. In general, our method is of great significance for crack detection.
Keywords
crack detection; high-resolution image; steel box girder; three-stage method;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Bao, Y., Li, J., Nagayama, T., Xu, Y., Spencer, B.F. and Li, H. (2021), "The 1st International Project Competition for Structural Health Monitoring (IPC-SHM, 2020): A summary and benchmark problem", Struct. Health Monitor., 20(4), 2229-2239. https://doi.org/10.1177/14759217211006485   DOI
2 Cha, Y.-J., Choi, W., Suh, G., Mahmoudkhani, S. and Buyukozturk, O. (2018), "Autonomous Structural Visual Inspection Using Region-Based Deep Learning for Detecting Multiple Damage Types", Comput.-Aided Civil Infrastr. Eng., 33(9), 731-747. https://doi.org/10.1111/mice.12334   DOI
3 de Freitas, S.T., Kolstein, H. and Bijlaard, F. (2012), "Parametric study on the interface layer of renovation solutions for orthotropic steel bridge decks", Comput.-Aided Civil Infrastr. Eng., 27(2), 143-153. https://doi.org/10.1111/j.1467-8667.2010.00693.x   DOI
4 Qiao, W., Liu, Q., Wu, X., Ma, B. and Li, G. (2021), "Automatic Pixel-Level Pavement Crack Recognition Using a Deep Feature Aggregation Segmentation Network with a scSE Attention Mechanism Module", Sensors, 21(9), 2902. https://doi.org/10.3390/s21092902   DOI
5 Bao, Y. and Li, H. (2021), "Machine learning paradigm for structural health monitoring", Struct. Health Monitor., 20(4), 1353-1372. https://doi.org/10.1177/1475921720972416   DOI
6 Chen, F.-C. and Jahanshahi, M.R. (2017), "NB-CNN: Deep learning-based crack detection using convolutional neural network and Naive Bayes data fusion", IEEE Transact. Indust. Electro., 65(5), 4392-4400. https://doi.org/10.1109/TIE.2017.2764844   DOI
7 Canny, J. (1986), "A computational approach to edge detection", IEEE Transact. Pattern Anal. Mach. Intell., (6), 679-698.
8 Li, S., Zhao, X. and Zhou, G. (2019), "Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network", Comput.-Aided Civil Infrastr. Eng., 34(7), 616-634. https://doi.org/10.1111/mice.12433   DOI
9 Bao, Y., Chen, Z., Wei, S., Xu, Y., Tang, Z. and Li, H. (2019), "The state of the art of data science and engineering in structural health monitoring", Engineering, 5(2), 234-242. https://doi.org/10.1016/j.eng.2018.11.027   DOI
10 Pan, Y., Zhang, G. and Zhang, L. (2020). "A spatial-channel hierarchical deep learning network for pixel-level automated crack detection", Automat. Constr., 119, 103357. https://doi.org/10.1016/j.autcon.2020.103357   DOI
11 Ren, S., He, K., Girshick, R. and Sun, J. (2016), "Faster R-CNN: Towards real-time object detection with region proposal networks", arXiv:1506.01497 [cs]. http://arxiv.org/abs/1506.01497
12 Saouma, V.E., Barton, C.C. and Gamaleldin, N.A. (1990), "Fractal characterization of fracture surfaces in concrete", Eng. Fract. Mech., 35(1-3), 47-53. https://doi.org/10.1016/0013-7944(90)90182-G   DOI
13 Dung, C.V. and Anh, L.D. (2019), "Autonomous concrete crack detection using deep fully convolutional neural network", Automat. Constr., 99, 52-58. https://doi.org/10.1016/j.autcon.2018.11.028   DOI
14 Hou, L., Samaras, D., Kurc, T.M., Gao, Y., Davis, J.E. and Saltz, J.H. (2016), "Patch-based con- volutional neural network for whole slide tissue image classification", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, VA, USA, June.
15 Sofia, T.D.F., Henk, K. and Frans, B. (2012), "Parametric study on the interface layer of renovation solutions for orthotropic steel bridge decks", Comput.-Aided Civil Infrastr. Eng., 27(2), 143-153. https://doi.org/10.1111/j.1467-8667.2010.00693.x   DOI
16 Yang, X., Li, H., Yu, Y., Luo, X., Huang, T. and Yang, X. (2018), "Automatic pixel-level crack detection and measurement using fully convolutional network", Comput.-Aided Civil Infrastr. Eng., 33(12), 1090-1109. https://doi.org/10.1111/mice.12412   DOI
17 Ye, X.-W., Jin, T. and Chen, P.-Y. (2019), "Structural crack detection using deep learning-based fully convolutional networks", Adv. Struct. Eng., 22(16), 3412-3419. https://doi.org/10.1177/1369433219836292   DOI
18 Zhang, A., Wang, K.C.P., Fei, Y., Liu, Y., Tao, S., Chen, C., Li, J.Q. and Li, B. (2018), "Deep learning-based fully automated pavement crack detection on 3D asphalt surfaces with an improved CrackNet", J. Comput. Civil Eng., 32(5), 04018041. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000775   DOI
19 Oh, J.-K., Jang, G., Oh, S., Lee, J.H., Yi, B.-J., Moon, Y.S., Lee, J.S. and Choi, Y. (2009). "Bridge inspection robot system with machine vision", Automat. Constr., 18(7), 929-941. https://doi.org/10.1016/j.autcon.2009.04.003   DOI
20 Kingma, D.P. and Ba, J. (2014), "Adam: A method for stochastic optimization", Proceedings of the 3rd International Conference on Learning Representations, San Diego, CA, USA, May.
21 Ronneberger, O., Fischer, P. and Brox, T. (2015), "U-net: Convolutional networks for biomedical image segmentation", Proceedings of International Conference on Medical Image Computing and Computer- Assisted Intervention, Munich, Germany, October.
22 Sun, B.-C. and Qiu, Y. (2007). "Automatic identification of pavement cracks using mathematic morphology", Proceedings of International Conference on Transportation Engineering 2007, pp. 1783-1788.
23 Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. and Polosukhin, I. (2017), "Attention is all you need", arXiv preprint arXiv:1706.03762.
24 Spencer Jr, B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Eng., 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030   DOI
25 Wan, H., Gao, L., Su, M., Sun, Q. and Huang, L. (2021), "Attention-Based Convolutional Neural Network for Pavement Crack Detection", Adv. Mater. Sci. Eng. https://doi.org/10.1155/2021/5520515   DOI
26 Deng, J., Dong, W., Socher, R., Li, L.J., Li, K. and Fei-Fei, L. (2009), "Imagenet: A large-scale hierarchical image database", Proceedings of the IEEE conference on computer vision and pattern recognition, Miami Beach, FL, USA, June.
27 Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2017), "ImageNet classification with deep convolutional neural networks", Commun. ACM, 60(6), 84-90. https://doi.org/10.1145/3065386   DOI
28 Bang, S., Park, S., Kim, H. and Kim, H. (2019), "Encoder-decoder network for pixel-level road crack detection in black-box images", Comput.-Aided Civil Infrastr. Eng., 34(8), 713-727. https://doi.org/10.1111/mice.12440   DOI
29 Girshick, R., Donahue, J., Darrell, T. and Malik, J. (2014), "Rich feature hierarchies for accurate object detection and semantic segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH, USA, June.
30 Han, Q., Xu, J., Carpinteri, A. and Lacidogna, G. (2015), "Localization of acoustic emission sources in structural health monitoring of masonry bridge", Struct. Control Health Monitor., 22(2), 314-329. https://doi.org/10.1002/stc.1675   DOI
31 Song, W., Jia, G., Jia, D. and Zhu, H. (2019), "Automatic Pavement Crack Detection and Classification Using Multiscale Feature Attention Network", IEEE Access, 7, 171001-171012. https://doi.org/10.1109/ACCESS.2019.2956191   DOI
32 Woo, S., Park, J., Lee, J.Y. and Kweon, I.S. (2018), "Cbam: Convolutional block attention module", Proceedings of the European Conference on Computer Vision, Munich, Germany, September.
33 Xu, Y., Bao, Y., Chen, J., Zuo, W. and Li, H. (2019), "Surface fatigue crack identification in steel box girder of bridges by a deep fusion convolutional neural network based on consumer-grade camera images", Struct. Health Monitor., 18(3), 653-674. https://doi.org/10.1177/1475921718764873   DOI
34 Yang, Y. and Nagarajaiah, S. (2014), "Blind identification of damage in time-varying systems using independent component analysis with wavelet transform", Mech. Syst. Signal Process., 47(1-2), 3-20. https://doi.org/10.1016/j.ymssp.2012.08.029   DOI
35 Yang, Y. and Nagarajaiah, S. (2016), "Harnessing data structure for recovery of randomly missing structural vibration responses time history: Sparse representation versus low-rank structure", Mech. Syst. Signal Process., 74, 165-182. https://doi.org/10.1016/j.ymssp.2015.11.009   DOI
36 Mutlib, N.K., Baharom, S.B., El-Shafie, A. and Nuawi, M.Z. (2016), "Ultrasonic health monitoring in structural engineering: buildings and bridges", Struct. Control Health Monitor., 23(3), 409-422. https://doi.org/10.1002/stc.1800   DOI
37 Lim, R.S., La, H.M. and Sheng, W. (2014), "A robotic crack inspection and mapping system for bridge deck maintenance", IEEE Transact. Automat. Sci. Eng., 11(2), 367-378. https://doi.org/10.1109/TASE.2013.2294687   DOI
38 He, K., Zhang, X., Ren, S. and Sun, J. (2016), "Deep residual learning for image recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, VA, USA, June.
39 Hu, J., Shen, L. and Sun, G. (2018), "Squeeze-and-excitation networks", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA, June.
40 Huang, G., Liu, Z., Van Der Maaten, L. and Weinberger, K.Q. (2017), "Densely connected convolutional networks", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, July.
41 Ong, E.P., Lee, J.A., Cheng, J., Xu, G., Lee, B.H., Laude, A., Teoh, S., Lim T.H., Wong D.W.K. and Liu, J. (2015), "A robust outlier elimination approach for multimodal retina image registration", Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany, October.
42 Long, J., Shelhamer, E. and Darrell, T. (2015), "Fully convolutional networks for semantic segmen- tation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, June.