Semantic crack-image identification framework for steel structures using atrous convolution-based Deeplabv3+ Network |
Ta, Quoc-Bao
(Department of Ocean Eng., Pukyong National University)
Dang, Ngoc-Loi (Urban Infrastructure Faculty, Mien Tay Construction University) Kim, Yoon-Chul (Department of Civil and Environmental Eng., Yonsei University) Kam, Hyeon-Dong (Department of Ocean Eng., Pukyong National University) Kim, Jeong-Tae (Department of Ocean Eng., Pukyong National University) |
1 | Habtour, E., Cole, D.P., Riddick, J.C., Weiss, V., Robeson, M., Sridharan, R. and Dasgupta, A. (2016), "Detection of fatigue damage precursor using a nonlinear vibration approach", Struct. Control Health Monitor., 23(12), 1442-1463. https://doi.org/10.1002/stc.1844 DOI |
2 | He, K., Zhang, X., Ren, S. and Sun, J. (2015), "Spatial pyramid pooling in deep convolutional networks for visual recognition", IEEE Transact. Pattern Anal. Mach. Intell., 37(9), 1904-1916. https://doi.org/10.1109/TPAMI.2015.2389824 DOI |
3 | Jeong, S., Kim, H., Lee, J. and Sim, S.H. (2020), "Automated wireless monitoring system for cable tension forces using deep learning", Struct. Health Monitor., 20(4), 1805-1821. https://doi.org/10.1177/1475921720935837 DOI |
4 | Jin, S.S., Jeong, S., Sim, S.H., Seo, D.W. and Park, Y.S. (2021), "Fully automated peak-picking method for an autonomous stay-cable monitoring system in cable-stayed bridges", Automat. Constr., 126, 103628. https://doi.org/10.1016/j.autcon.2021.103628 DOI |
5 | Bhalla, S., Vittal, P.A. and Veljkovic, M. (2012), "Piezo-impedance transducers for residual fatigue life assessment of bolted steel joints", Struct. Health Monitor., Int. J., 11(6), 733-750. https://doi.org/10.1177/1475921712458708 DOI |
6 | Cha, Y.J., Choi, W. and Buyukozturk, O. (2017), "Deep learning-based crack damage detection using convolutional neural networks", Comput.-Aided Civil Infrastruct. Eng., 32(5), 361-378. https://doi.org/10.1111/mice.12263 DOI |
7 | Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K. and Yuille, A.L. (2014), "Semantic image segmentation with deep convolutional nets and fully connected crfs", Computer Vision and Pattern Recognition, arXiv preprint arXiv:1412.7062. https://doi.org/10.48550/arXiv.1412.7062 |
8 | Huynh, T.C. (2021), "Vision-based autonomous bolt-looseness detection method for splice connections: Design, lab-scale evaluation, and field application", Automat. Constr, 124, 103591. https://doi.org/10.1016/j.autcon.2021.103591 DOI |
9 | Barbedo, J.G.A. (2018), "Impact of dataset size and variety on the effectiveness of deep learning and transfer learning for plant disease classification", Comput. Electron. Agricult., 153, 46-53. https://doi.org/10.1016/j.compag.2018.08.013 DOI |
10 | Bastani, A., Amindavar, H., Shamshirsaz, M. and Sepehry, N. (2011), "Identification of temperature variation and vibration disturbance in impedance-based structural health monitoring using piezoelectric sensor array method", Struct. Health Monitor., Int. J., 11(3), 305-314. https://doi.org/10.1177/1475921711427486 DOI |
11 | Bao, Y., Li, J., Nagayama, T., Xu, Y., Spencer, B.F. and Li, H. (2021), "The 1st international project competition for structural health monitoring (IPC-SHM, 2020): a summary and benchmark problem", Struct. Health Monitor., 20(4), 2229-2239. https://doi.org/10.1177/14759217211006485 DOI |
12 | Sun, Y., Yang, Y., Yao, G., Wei, F. and Wong, M. (2021), "Autonomous crack and bughole detection for concrete surface image based on deep learning", IEEE Access, 9, 85709-85720. https://doi.org/10.1109/ACCESS.2021.3088292 DOI |
13 | He, K., Zhang, X., Ren, S. and Sun, J. (2016), "Deep residual learning for image recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR.2016.90 DOI |
14 | Hou, Y., Yue, P., Xin, Q., Pauli, T., Sun, W. and Wang, L. (2013), "Fracture failure of asphalt binder in mixed mode (Modes I and II) by using phase-field model", Road Mater. Pave. Des., 15(1), 167-181. https://doi.org/10.1080/14680629.2013.866155 DOI |
15 | Yang, X., Li, H., Yu, Y., Luo, X., Huang, T. and Yang, X. (2018), "Automatic pixel-level crack detection and measurement using fully convolutional network", Comput.-Aided Civil Infrastr. Eng., 33(12), 1090-1109. https://doi.org/10.1111/mice.12412 DOI |
16 | Di, J., Ruan, X., Zhou, X., Wang, J. and Peng, X. (2021), "Fatigue assessment of orthotropic steel bridge decks based on strain monitoring data", Eng. Struct., 228, 111437. https://doi.org/10.1016/j.engstruct.2020.111437 DOI |
17 | Chollet, F. (2017), "Xception: Deep learning with depthwise separable convolutions", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.48550/arXiv.1610.02357 DOI |
18 | Ta, Q.B. and Kim, J.T. (2020), "Monitoring of corroded and loosened bolts in steel structures via deep learning and hough transforms", Sensors, 20(23). https://doi.org/10.3390/s20236888 DOI |
19 | Xu, Y., Bao, Y., Chen, J., Zuo, W. and Li, H. (2018a), "Surface fatigue crack identification in steel box girder of bridges by a deep fusion convolutional neural network based on consumerg-rade camera images", Struct. Health Monitor, 18(3), 653-674. https://doi.org/10.1177/1475921718764873 DOI |
20 | Xu, Y., Li, S., Zhang, D., Jin, Y., Zhang, F., Li, N. and Li, H. (2018b), "Identification framework for cracks on a steel structure surface by a restricted Boltzmann machines algorithm based on consumer-grade camera images", Struct. Control Health Monitor., 25(2), 2075. https://doi.org/10.1002/stc.2075 DOI |
21 | Yao, L., Dong, Q., Jiang, J. and Ni, F. (2020), "Deep reinforcement learning for long-term pavement maintenance planning", Comput.-Aided Civil Infrastr. Eng., 35(11), 1230-1245. https://doi.org/10.1111/mice.12558 DOI |
22 | Csurka, G., Larlus, D., Perronnin, F. and Meylan, F. (2013), "What is a good evaluation measure for semantic segmentation?", In: Bmvc, Vol. 27, pp. 10-5244. http://dx.doi.org/10.5244/C.27.32 DOI |
23 | Zhao, X., Zhang, Y. and Wang, N. (2019), "Bolt loosening angle detection technology using deep learning", Struct. Control Health Monitor., 26(1), 2292. https://doi.org/10.1002/stc.2292 DOI |
24 | Li, M., Suzuki, Y., Hashimoto, K. and Sugiura, K. (2018), "Experimental study on fatigue resistance of rib-to-deck joint in orthotropic steel bridge deck", J. Bridge Eng., 23(2), 04017128. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001175 DOI |
25 | Ly, C.D., Vo, T.H., Mondal, S., Park, S., Choi, J., Vu, T.T.H., Kim, C.S. and Oh, J. (2021), "Full-view in vivo skin and blood vessels profile segmentation in photoacoustic imaging based on deep learning", Photoacoustics, 25, 100310. https://doi.org/10.1016/j.pacs.2021.100310 DOI |
26 | Lee, Y.F., Lu, Y. and Guan, R. (2020), "Nonlinear guided waves for fatigue crack evaluation in steel joints with digital image correlation validation", Smart Mater. Struct., 29(3), 035031. https://doi.org/10.1088/1361-665X/ab6fe7 DOI |
27 | Papadimitriou, C., Fritzen, C.P., Kraemer, P. and Ntotsios, E. (2011), "Fatigue predictions in entire body of metallic structures from a limited number of vibration sensors using Kalman filtering", Struct. Control Health Monitor., 18(5), 554-573. https://doi.org/10.1002/stc.395 DOI |
28 | Park, J.-H., Huynh, T.-C., Choi, S.-H. and Kim, J.-T. (2015), "Vision-based technique for bolt-loosening detection in wind turbine tower", Wind Struct., Int. J., 21(6), 709-726. https://doi.org/10.12989/was.2015.21.6.709 DOI |
29 | Pham, H.C., Ta, Q.B., Kim, J.T., Ho, D.D., Tran, X.L. and Huynh, T.C. (2020), "Bolt-loosening monitoring framework using an image-based deep learning and graphical model", Sensors, 20(12). https://doi.org/10.3390/s20123382 DOI |
30 | Shorten, C. and Khoshgoftaar, T.M. (2019), "A survey on image data augmentation for deep learning", J. Big Data, 6(1). https://doi.org/10.1186/s40537-019-0197-0 DOI |
31 | Huynh, T.C., Park, Y.H., Park, J.H., Hong, D.S. and Kim, J.T. (2015), "Effect of temperature variation on vibration monitoring of prestressed concrete girders", Shock Vib., 1-9. https://doi.org/10.1155/2015/741618 DOI |
32 | Dhivya, J.J. and Ramaswami, M. (2018), "A perusal analysis on hybrid spectrum handoff schemes in cognitive radio networks", International Conference on Intelligent Systems Design and Applications, pp. 312-321. https://doi.org/10.1007/978-3-030-16660-1_31 DOI |
33 | Ye, X., Jin, T. and Yun, C. (2019), "A review on deep learning-based structural health monitoring of civil infrastructures", Smart Struct. Syst.., Int. J., 24(5), 567-586. https://doi.org/10.12989/sss.2019.24.5.567 DOI |
34 | Fasl, J., Helwig, T. and Wood, S.L. (2016), "Fatigue response of a fracture-critical bridge at the end of service life", J. Perform. Constr. Facil, 30(5), 04016019. https://doi.org/10.1061/(ASCE)CF.1943-5509.0000871 DOI |
35 | Sim, H.B. and Uang, C.M. (2012), "Stress analyses and parametric study on full-scale fatigue tests of rib-to-deck welded joints in steel orthotropic decks", J. Bridge Eng., 17(5), 765-773. https://doi.org/10.1061/(ASCE)BE.1943-5592.0000307 DOI |
36 | Campbell, L.E., Connor, R.J., Whitehead, J.M. and Washer, G.A. (2020), "Benchmark for evaluating performance in visual inspection of fatigue cracking in steel bridges", J. Bridge Eng., 25(1), 04019128. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001507 DOI |
37 | Chen, L.C., Papandreou, G., Schroff, F. and Adam, H. (2017), "Rethinking atrous convolution for semantic image segmentation", arXiv preprint arXiv:1706.05587. https://doi.org/10.48550/arXiv.1706.05587 |
38 | Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K. and Yuille, A.L. (2018a), "DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs", IEEE Transact. Pattern Anal. Machine Intell., 40(4), 834-848. https://doi.org/10.1109/TPAMI.2017.2699184 DOI |
39 | Connor, R.J. (2012), "Manual for design, construction, and maintenance of orthotropic steel deck bridges", No. FHWA-IF-12-027; United States, Federal Highway Administration. https://rosap.ntl.bts.gov/view/dot/41395 |
40 | Dellenbaugh, L., Kong, X., Al-Salih, H., Collins, W., Bennett, C., Li, J. and Sutley, E.J. (2020), "Development of a distortion-induced fatigue crack characterization methodology using digital image correlation", J. Bridge Eng., 25(9), 04020063. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001598 DOI |
41 | Chen, L.C., Zhu, Y., Papandreou, G., Schroff, F. and Adam, H. (2018b), "Encoder-decoder with atrous separable convolution for semantic image segmentation", Proceedings of the European Conference on Computer Vision (ECCV). https://doi.org/10.48550/arXiv.1802.02611 DOI |
42 | Soh, C.K. and Lim, Y.Y. (2009), "Detection and characterization of fatigue induced damage using electromechanical impedance technique", Adv. Mater. Res., 79-82, 2031-2034. https://doi.org/10.4028/www.scientific.net/AMR.79-82.2031 DOI |
43 | Spencer, B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Engineering, 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030 DOI |
44 | Sun, L.M., Zhang, W. and Nagarajaiah, S. (2019), "Bridge real-time damage identification method using inclination and strain measurements in the presence of temperature variation", J. Bridge Eng., 24(2), 04018111. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001325 DOI |
45 | Ronneberger, O., Fischer, P. and Brox, T. (2015), "U-net: convolutional networks for biomedical image segmentation", In: International Conference on Medical image computing and Computer-assisted Intervention, 9351. https://doi.org/10.1007/978-3-319-24574-4_28 DOI |
46 | Huynh, T.C., Dang, N.L. and Kim, J.T. (2017), "Advances and challenges in impedance-based structural health monitoring", Struct. Monitor. Maint., Int. J., 4(4), 301-329. https://doi.org/10.12989/smm.2017.4.4.301 DOI |
47 | Ya, S., Yamada, K. and Ishikawa, T. (2011), "Fatigue evaluation of rib-to-deck welded joints of orthotropic steel bridge deck", J. Bridge Eng., 16(4), 492-499. https://doi.org/10.1061/(ASCE)BE.1943-5592.0000181 DOI |
48 | Huynh, T.C., Nguyen, T.T., Kim, J.T., Ta, Q.B., Ho, D.D. and Phan, T.T.V. (2021), "Deep learning-based functional assessment of piezoelectric-based smart interface under various degradations", Smart Struct. Syst., Int. J., 28(1), 69-87. https://doi.org/10.12989/sss.2021.28.1.069 DOI |
49 | Huynh, T.C., Dang, N.L. and Kim, J.T. (2018), "PCA-based filtering of temperature effect on impedance monitoring in prestressed tendon anchorage", Smart Struct. Syst., Int. J., 22(1), 57-70. https://doi.org/10.12989/sss.2018.22.1.057 DOI |
50 | Huynh, T.C., Park, J.H., Jung, H.J. and Kim, J.T. (2019), "Quasi-autonomous bolt-loosening detection method using vision-based deep learning and image processing", Automat. Constr., 105, 102844. https://doi.org/10.1016/j.autcon.2019.102844 DOI |
51 | Khuc, T. and Catbas, F.N. (2016), "Computer vision-based displacement and vibration monitoring without using physical target on structures", Struct. Infrastr. Eng., 13(4), 505-516. https://doi.org/10.1080/15732479.2016.1164729 DOI |
52 | Kim, H., Yoon, J. and Sim, S.H. (2020), "Automated bridge component recognition from point clouds using deep learning", Struct. Control Health Monitor, 27(9). https://doi.org/10.1002/stc.2591 DOI |
53 | Kong, X., Li, J., Collins, W., Bennett, C., Laflamme, S. and Jo, H. (2018), "Sensing distortion-induced fatigue cracks in steel bridges with capacitive skin sensor arrays", Smart Mater. Struct, 27(11), 115008. https://doi.org/10.1088/1361-665X/aadbfb DOI |
54 | Dung, C.V. and Anh, L.D. (2019), "Autonomous concrete crack detection using deep fully convolutional neural network", Automat. Constr., 99, 52-58. https://doi.org/10.1016/j.autcon.2018.11.028 DOI |
55 | Dong, C.Z. and Catbas, F.N. (2020), "A review of computer vision-based structural health monitoring at local and global levels", Struct. Health Monitor., 20(2), 692-743. https://doi.org/10.1177/1475921720935585 DOI |
56 | Dong, C.Z., Bas, S. and Catbas, F.N. (2020), "Investigation of vibration serviceability of a footbridge using computer vision-based methods", Eng. Struct., 224, 111224. https://doi.org/10.1016/j.engstruct.2020.111224 DOI |
57 | Dong, C., Li, L., Yan, J., Zhang, Z., Pan, H. and Catbas, F.N. (2021), "Pixel-level fatigue crack segmentation in large-scale images of steel structures using an encoder-decoder network", Sensors, 21(12), 4135. https://doi.org/10.3390/s21124135 DOI |
58 | Erdogan, Y.S. and Ada, M. (2020), "A computer-vision based vibration transducer scheme for structural health monitoring applications", Smart Mater. Struct., 29(8), 085007. https://doi.org/10.1088/1361-665X/ab9062 DOI |
59 | Ghahremani, K., Sadhu, A., Walbridge, S. and Narasimhan, S. (2013), "Fatigue testing and structural health monitoring of retrofitted web stiffeners on steel highway bridges", Transport. Res. Record: J. Transport. Res. Board, 2360(1), 27-35. https://doi.org/10.3141/2360-04 DOI |
![]() |