Browse > Article
http://dx.doi.org/10.12989/sss.2022.29.1.207

Impacts of label quality on performance of steel fatigue crack recognition using deep learning-based image segmentation  

Hsu, Shun-Hsiang (NCREE-NTUCE Joint Artificial Intelligence Research Center)
Chang, Ting-Wei (Department of Civil Engineering, National Taiwan University)
Chang, Chia-Ming (Department of Civil Engineering, National Taiwan University)
Publication Information
Smart Structures and Systems / v.29, no.1, 2022 , pp. 207-220 More about this Journal
Abstract
Structural health monitoring (SHM) plays a vital role in the maintenance and operation of constructions. In recent years, autonomous inspection has received considerable attention because conventional monitoring methods are inefficient and expensive to some extent. To develop autonomous inspection, a potential approach of crack identification is needed to locate defects. Therefore, this study exploits two deep learning-based segmentation models, DeepLabv3+ and Mask R-CNN, for crack segmentation because these two segmentation models can outperform other similar models on public datasets. Additionally, impacts of label quality on model performance are explored to obtain an empirical guideline on the preparation of image datasets. The influence of image cropping and label refining are also investigated, and different strategies are applied to the dataset, resulting in six alternated datasets. By conducting experiments with these datasets, the highest mean Intersection-over-Union (mIoU), 75%, is achieved by Mask R-CNN. The rise in the percentage of annotations by image cropping improves model performance while the label refining has opposite effects on the two models. As the label refining results in fewer error annotations of cracks, this modification enhances the performance of DeepLabv3+. Instead, the performance of Mask R-CNN decreases because fragmented annotations may mistake an instance as multiple instances. To sum up, both DeepLabv3+ and Mask R-CNN are capable of crack identification, and an empirical guideline on the data preparation is presented to strengthen identification successfulness via image cropping and label refining.
Keywords
crack recognition; deep learning; image segmentation; label quality;
Citations & Related Records
연도 인용수 순위
  • Reference
1 Abdellatif, M., Peel, H., Cohn, A.G. and Fuentes, R. (2020), "Combining block-based and pixel-based approaches to improve crack detection and localization", Automat. Constr., 122, 103492. https://doi.org/10.1016/j.autcon.2020.103492   DOI
2 Albrecht, P. and Wright, W. (2000), "Fatigue and fracture of steel bridges", Eur. Struct. Integr. Soc., 26, 211-234. https://doi.org/10.1016/S1566-1369(00)80051-5   DOI
3 Badrinarayanan, V., Kendall, A. and Cipolla, R. (2017), "Segnet: a deep convolutional encoder-decoder architecture for image segmentation", IEEE Transact. Pattern Anal. Mach. Intell., 39(12), 2481-2495. https://doi.org/10.1109/TPAMI.2016.2644615   DOI
4 Feng, D. and Feng, M.Q. (2018), "Computer vision for SHM of civil infrastructure: From dynamic response measurement to damage detection-A review", Eng. Struct., 156, 105-117. https://doi.org/10.1016/j.engstruct.2017.11.018   DOI
5 He, K., Zhang, X., Ren, S. and Sun, J. (2016), "Deep residual learning for image recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NA, USA, June.
6 Breheret, A. (2017), "Pixel Annotation Tool", Retrieved from: https://github.com/abreheret/PixelAnnotationTool
7 Chen, L.-C., Papandreou, G., Kokkinos, I., Murphy, K. and Yuille, A.L. (2017), "Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs", IEEE Transact. Pattern Anal. Mach. Intell., 40(4), 834-848. https://doi.org/10.1109/TPAMI.2017.2699184   DOI
8 Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F. and Adam, H. (2018), "Encoder-decoder with atrous separable convolution for semantic image segmentation", Proceedings of the European Conference on Computer Vision, Munich, Germany, September.
9 Farrar, C.R. and Worden, K. (2007), "An introduction to structural health monitoring", Philos. Transact. Royal Soc. A: Mathe. Phys. Eng. Sci., 365(1851), 303-315. https://doi.org/10.1098/rsta.2006.1928   DOI
10 Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K. and Fei-Fei, L. (2009), "Imagenet: a large-scale hierarchical image database", IEEE Conference on Computer Vision and Pattern Recognition, Miami, FL, USA, June.
11 He, K., Gkioxari, G., Dollar, P. and Girshick, R. (2017), "Mask rcnn", Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy, October.
12 Wu, Y., Kirillov, A., Massa, F., Lo, W.-Y. and Girshick, R. (2019), "Detectron2", Retrieved from: https://github.com/facebookresearch/detectron2
13 Song, G., Wang, C. and Wang, B. (2017), "Structural health monitoring (SHM) of civil structures", Appl. Sci., 7(8), 789. https://doi.org/10.3390/app7080789   DOI
14 Song, Q., Wu, Y., Xin, X., Yang, L., Yang, M., Chen, H., Liu, C., Hu, M., Chai, X. and Li, J. (2019), "Real-time tunnel crack analysis system via deep learning", IEEE Access, 7, 64186-64197. https://doi.org/10.1109/ACCESS.2019.2916330   DOI
15 Wada, K. (2016), "labelme: Image Polygonal Annotation with Python", Retrieved from: https://github.com/wkentaro/labelme
16 Zhou, C. (2020), "Yolact++ better real-time instance segmentation", Ph.D. Dissertation; University of California, Davis, CA, USA.
17 Hsu, S.-H., Chang, T.-W. and Chang, C.-M. (2020), "Concrete Surface Crack Segmentation Based on Deep Learning", In: European Workshop on Structural Health Monitoring (Lecture Notes in Civil Engineering), Vol. 128, pp. 24-34. https://doi.org/10.1007/978-3-030-64908-1_3   DOI
18 Kalfarisi, R., Wu, Z.Y. and Soh, K. (2020), "Crack detection and segmentation using deep learning with 3D reality mesh model for quantitative assessment and integrated visualization", J. Comput. Civil Eng., 34(3), 04020010. https://doi.org/10.1061/(ASCE)CP.1943-5487.0000890   DOI
19 Kiefer, J. and Wolfowitz, J. (1952), "Stochastic estimation of the maximum of a regression function", Math. Statist., 23(3), 462-466.   DOI
20 Kirk, D. (2007), "NVIDIA CUDA software and GPU parallel computing architecture", Proceedings of the 6th International Symposium on Memory Management, New York, NY, USA, October.
21 Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollar, P. and Zitnick, C.L. (2014), "Microsoft coco: common objects in context", Proceedings of the European Conference on Computer Vision, Zurich, Switzerland, September.
22 Lin, T.-Y., Dollar, P., Girshick, R., He, K., Hariharan, B. and Belongie, S. (2017), "Feature pyramid networks for object detection", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA, July.
23 Long, J., Shelhamer, E. and Darrell, T. (2015), "Fully convolutional networks for semantic segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Boston, MA, USA, June.
24 Ren, S., He, K., Girshick, R. and Sun, J. (2015), "Faster r-cnn: towards real-time object detection with region proposal networks", IEEE Transact. Pattern Anal. Mach. Intell., 39(6), 1137-1149. https://doi.org/10.1109/TPAMI.2016.2577031   DOI
25 Jegou, S., Drozdzal, M., Vazquez, D., Romero, A. and Bengio, Y. (2017), "The one hundred layers tiramisu: fully convolutional densenets for semantic segmentation", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Honolulu, HI, USA, July.
26 Dung, C.V. (2019), "Autonomous concrete crack detection using deep fully convolutional neural network", Automat. Constr., 99, 52-58. https://doi.org/10.1016/j.autcon.2018.11.028   DOI
27 Meng, Z., Fan, X., Chen, X., Chen, M. and Tong, Y. (2017), "Detecting small signs from large images", IEEE International Conference on Information Reuse and Integration, San Diego, CA, USA, August.
28 Mohan, A. and Poobal, S. (2018), "Crack detection using image processing: a critical review and analysis", Alexandria Eng. J., 57(2), 787-798. https://doi.org/10.1016/j.aej.2017.01.020   DOI
29 Ozge Unel, F., Ozkalayci, B.O. and Cigla, C. (2019), "The power of tiling for small object detection", Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA, June.
30 Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., Antiga, L. and Desmaison, A. (2019), "PyTorch: an imperative style, high-performance deep learning library", Adv. Neural Inform. Process. Syst., 32, 8024-8035.
31 Redmon, J. and Farhadi, A. (2018), "Yolov3: an incremental improvement", arXiv preprint, arXiv: 1804.02767.