Browse > Article
http://dx.doi.org/10.12989/sss.2022.29.1.153

One-step deep learning-based method for pixel-level detection of fine cracks in steel girder images  

Li, Zhihang (Department of Civil Engineering, Monash University)
Huang, Mengqi (Department of Civil Engineering, Monash University)
Ji, Pengxuan (Department of Civil Engineering, Monash University)
Zhu, Huamei (Department of Civil Engineering, Monash University)
Zhang, Qianbing (Department of Civil Engineering, Monash University)
Publication Information
Smart Structures and Systems / v.29, no.1, 2022 , pp. 153-166 More about this Journal
Abstract
Identifying fine cracks in steel bridge facilities is a challenging task of structural health monitoring (SHM). This study proposed an end-to-end crack image segmentation framework based on a one-step Convolutional Neural Network (CNN) for pixel-level object recognition with high accuracy. To particularly address the challenges arising from small object detection in complex background, efforts were made in loss function selection aiming at sample imbalance and module modification in order to improve the generalization ability on complicated images. Specifically, loss functions were compared among alternatives including the Binary Cross Entropy (BCE), Focal, Tversky and Dice loss, with the last three specialized for biased sample distribution. Structural modifications with dilated convolution, Spatial Pyramid Pooling (SPP) and Feature Pyramid Network (FPN) were also performed to form a new backbone termed CrackDet. Models of various loss functions and feature extraction modules were trained on crack images and tested on full-scale images collected on steel box girders. The CNN model incorporated the classic U-Net as its backbone, and Dice loss as its loss function achieved the highest mean Intersection-over-Union (mIoU) of 0.7571 on full-scale pictures. In contrast, the best performance on cropped crack images was achieved by integrating CrackDet with Dice loss at a mIoU of 0.7670.
Keywords
CNN; crack detection; data imbalance; feature extraction; loss function;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 Russell, B.C., Torralba, A., Murphy, K.P. and Freeman, W.T. (2008), "LabelMe: a database and web-based tool for image annotation", Int. J. Comput. Vision, 77(1-3), 157-173. https://doi.org/10.1007/s11263-007-0090-8   DOI
2 Salehi, S.S.M., Erdogmus, D. and Gholipour, A. (2017), "Tversky loss function for image segmentation using 3D fully convolutional deep networks", International Workshop on Machine Learning in Medical Imaging, pp. 379-387. https://doi.org/10.1007/978-3-319-67389-9_44   DOI
3 Spencer Jr., B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Engineering, 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030   DOI
4 Tversky, A. (1988), "Features of similarity", Readings in Cognitive Science, 84(4), 290-302. https://doi.org/10.1037/0033-295X.84.4.327   DOI
5 Ronneberger, O., Fischer, P. and Brox, T. (2015), "U-net: Convolutional networks for biomedical image segmentation", Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234-241. https://doi.org/10.1007/978-3-662-54345-0_3   DOI
6 Cha, Y.J., Choi, W. and Buyukozturk, O. (2017), "Deep learning- based crack damage detection using convolutional neural networks", Comput.-Aided Civil Infrastr. Eng., 32(5), 361-378. https://doi.org/10.1111/mice.12263   DOI
7 Abdel-Qader, I., Abudayyeh, O. and Kelly, M.E. (2003), "Analysis of edge-detection techniques for crack identification in bridges", J. Comput. Civil Eng., 17(4), 255-263. https://doi.org/10.1061/(ASCE)0887-3801(2003)17:4(255)   DOI
8 Bao, Y., Chen, Z., Wei, S., Xu, Y., Tang, Z. and Li, H. (2019), "The state of the art of data science and engineering in structural health monitoring", Engineering, 5(2), 234-242. https://doi.org/10.1016/j.eng.2018.11.027   DOI
9 Bao, Y., Li, J., Nagayama, T., Xu, Y., Spencer Jr, B.F. and Li, H. (2021), "The 1st International Project Competition for Structural Health Monitoring (IPC-SHM, 2020): A summary and benchmark problem", Struct. Health Monitor., 20(4), 2229-2239. https://doi.org/10.1177/14759217211006485   DOI
10 Cha, Y.J., Choi, W., Suh, G., Mahmoudkhani, S. and Buyukozturk, O. (2018), "Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types", Comput.-Aided Civil Infrastr. Eng., 33(9), 731-747. https://doi.org/10.1111/mice.12334   DOI
11 Kang, D., Benipal, S.S., Gopal, D.L. and Cha, Y.J. (2020), "Hybrid pixel-level concrete crack segmentation and quantification across complex backgrounds using deep learning", Automat. Constr., 118, 103291. https://doi.org/10.1016/j.autcon.2020.103291   DOI
12 He, K., Zhang, X., Ren, S. and Sun, J. (2014), "Spatial pyramid pooling in deep convolutional networks for visual recognition", IEEE Transact. Pattern Anal. Mach. Intell., 37(9), 1904-1916. https://doi.org/10.1007/978-3-319-10578-9_23   DOI
13 Koch, C., Georgieva, K., Kasireddy, V., Akinci, B. and Fieguth, P. (2015), "A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure", Adv. Eng. Inform., 29(2), 196-210. https://doi.org/10.1016/j.aei.2015.01.008   DOI
14 Zhang, L., Yang, F., Zhang, Y.D. and Zhu, Y.J. (2016), "Road crack detection using deep convolutional neural network", Proceedings of IEEE International Conference on Image Processing (ICIP), Phoenix, AZ, USA, pp. 3708-3712. https://doi.org/doi/10.1109/ICIP.2016.7533052   DOI
15 Goodfellow, I., Bengio, Y. and Courville, A. (2016), Deep learning, MIT press Cambridge.
16 Huang, M.Q., Ninic, J. and Zhang, Q.B. (2021), "BIM, machine learning and computer vision techniques in underground construction: Current status and future perspectives", Tunnell. Undergr. Space Technol., 108, 103677. https://doi.org/10.1016/j.tust.2020.103677   DOI
17 Dung, C.V. (2019), "Autonomous concrete crack detection using deep fully convolutional neural network", Automat. Constr., 99, 52-58. https://doi.org/10.1016/j.autcon.2018.11.028   DOI
18 Lattanzi, D. and Miller, G.R. (2014), "Robust automated concrete damage detection algorithms for field applications", J. Comput. Civil Eng., 28(2), 253-262. https://doi.org/10.1061/(asce)cp.1943-5487.0000257   DOI
19 Lei, B., Ren, Y., Wang, N., Huo, L. and Song, G. (2020), "Design of a new low-cost unmanned aerial vehicle and vision-based concrete crack inspection method", Struct. Health Monitor., 19(6), 1871-1883. https://doi.org/10.1177/1475921719898862   DOI
20 Li, L., Wang, Q., Zhang, G., Shi, L., Dong, J. and Jia, P. (2018), "A method of detecting the cracks of concrete undergo high-temperature", Constr. Build. Mater., 162, 345-358. https://doi.org/10.1016/j.conbuildmat.2017.12.010   DOI
21 Lin, T.Y., Goyal, P., Girshick, R., He, K. and Dollar, P. (2018), "Focal loss for dense object detection", Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2980-2988. https://doi.org/10.1109/ICCV.2017.324   DOI
22 Long, J., Shelhamer, E. and Darrell, T. (2015), "Fully convolutional networks for semantic segmentation", IEEE Transact. Pattern Anal. Mach. Intell., 39(4), 640-651. https://doi.org/10.1109/CVPR.2015.7298965   DOI
23 Lin, T.Y., Dollar, P., Girshick, R., He, K., Hariharan, B. and Belongie, S. (2017), "Feature pyramid networks for object detection", Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2117-2125. https://doi.org/10.1109/CVPR.2017.106   DOI
24 Zhao, H., Qin, G. and Wang, X. (2010), "Improvement of canny algorithm based on pavement edge detection", Proceedings of 2010 3rd International Congress on Image and Signal Processing, Yantai, China. https://doi.org/10.1109/CISP.2010.5646923   DOI
25 Hutchinson, T.C. and Chen, Z. (2006), "Improved image analysis for evaluating concrete damage", J. Comput. Civil Eng., 20(3), 210-216. https://doi.org/10.1061/(ASCE)0887-3801(2006)20:3(210)   DOI
26 Xu, Y., Li, S., Zhang, D., Jin, Y., Zhang, F., Li, N. and Li, H. (2018), "Identification framework for cracks on a steel structure surface by a restricted Boltzmann machines algorithm based on consumer-grade camera images", Struct. Control Health Monitor., 25(2), e2075. https://doi.org/10.1002/stc.2075   DOI
27 Yamaguchi, T. and Hashimoto, S. (2009), "Fast crack detection method for large-size concrete surface images using percolation-based image processing", Mach. Vision Applicat., 21(5), 797-809. https://doi.org/10.1007/s00138-009-0189-8   DOI
28 Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S. and Cardoso, M.J. (2017), "Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations", In: Deep learning in medical image analysis and multimodal learning for clinical decision support, pp. 240-248. https://doi.org/10.1007/978-3-319-67558-9_28   DOI
29 Yeum, C.M. and Dyke, S.J. (2015), "Vision-based automated crack detection for bridge inspection", Comput.-Aided Civil Infrastr. Eng., 30(10), 759-770. https://doi.org/10.1111/mice.12141   DOI
30 Yu, F. and Koltun, V. (2016), "Multi-scale context aggregation by dilated convolutions", arXiv preprint arXiv:1511.07122.
31 Yang, X., Li, H., Yu, Y., Luo, X., Huang, T. and Yang, X. (2018), "Automatic pixel-level crack detection and measurement using fully convolutional network", Comput.-Aided Civil Infrastr. Eng., 33(12), 1090-1109. https://doi.org/10.1111/mice.12412   DOI
32 Nayyeri, F., Hou, L., Zhou, J. and Guan, H. (2018), "Foreground-background separation technique for crack detection", Comput.- Aided Civil Infrastr. Eng., 34(6), 457-470. https://doi.org/10.1111/mice.12428   DOI
33 Prasanna, P., Dana, K., Gucunski, N. and Basily, B. (2012), "Computer-vision based crack detection and analysis", Proceedings of SPIE - The International Society for Optical Engineering, Vol. 8345, pp. 834542. https://doi.org/10.1117/12.915384   DOI
34 Milletari, F., Navab, N. and Ahmadi, S.A. (2016), "Fully convolutional neural networks for volumetric medical image segmentation", Proceedings of 2016 Fourth International Conference on 3D Vision (3DV), pp. 565-571. https://doi.org/10.1109/3DV.2016.79   DOI
35 Nash, W., Drummond, T. and Birbilis, N. (2018), "A review of deep learning in the study of materials degradation", npj Materials Degradation, 2(1), 1-12. https://doi.org/10.1038/s41529-018-0058-x   DOI
36 Ren, Y., Huang, J., Hong, Z., Lu, W., Yin, J., Zou, L. and Shen, X. (2020), "Image-based concrete crack detection in tunnels using deep fully convolutional networks", Constr. Build. Mater., 234, 117367. https://doi.org/10.1016/j.conbuildmat.2019.117367   DOI