Browse > Article
http://dx.doi.org/10.12989/sss.2022.29.1.063

Convolutional neural network-based data anomaly detection considering class imbalance with limited data  

Du, Yao (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University)
Li, Ling-fang (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University)
Hou, Rong-rong (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University)
Wang, Xiao-you (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University)
Tian, Wei (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University)
Xia, Yong (Department of Civil and Environmental Engineering, The Hong Kong Polytechnic University)
Publication Information
Smart Structures and Systems / v.29, no.1, 2022 , pp. 63-75 More about this Journal
Abstract
The raw data collected by structural health monitoring (SHM) systems may suffer multiple patterns of anomalies, which pose a significant barrier for an automatic and accurate structural condition assessment. Therefore, the detection and classification of these anomalies is an essential pre-processing step for SHM systems. However, the heterogeneous data patterns, scarce anomalous samples and severe class imbalance make data anomaly detection difficult. In this regard, this study proposes a convolutional neural network-based data anomaly detection method. The time and frequency domains data are transferred as images and used as the input of the neural network for training. ResNet18 is adopted as the feature extractor to avoid training with massive labelled data. In addition, the focal loss function is adopted to soften the class imbalance-induced classification bias. The effectiveness of the proposed method is validated using acceleration data collected in a long-span cable-stayed bridge. The proposed approach detects and classifies data anomalies with high accuracy.
Keywords
class imbalance; data anomaly detection; focal loss; limited labelled data; transfer learning;
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 He, K., Zhang, X., Ren, S. and Sun, J. (2016), "Deep residual learning for image recognition", Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.
2 Hernandez-Garcia, M.R. and Masri, S.F. (2014), "Application of statistical monitoring using latent-variable techniques for detection of faults in sensor networks", J. Intell. Mater. Syst. Struct., 25(2), 121-136. https://doi.org/10.1177/1045389x13479182   DOI
3 Hou, R. and Xia, Y. (2020), "Review on the new development of vibration-based damage identification for civil engineering structures: 2010-2019", J. Sound Vib., 491, 115741. https://doi.org/10.1016/j.jsv.2020.115741   DOI
4 Iandola, F., Moskewicz, M., Karayev, S., Girshick, R., Darrell, T. and Keutzer, K. (2014), "Densenet: Implementing efficient convnet descriptor pyramids", arXiv preprint arXiv:1404.1869.
5 Ioffe, S. and Szegedy, C. (2015), "Batch normalisation: Accelerating deep network training by reducing internal covariate shift", Proceedings of International Conference on Machine Learning, pp. 448-456.
6 Fu, Y.G., Peng, C., Gomez, F., Narazaki, Y. and Spencer, B.F. (2019), "Sensor fault management techniques for wireless smart sensor networks in structural health monitoring", Struct. Control Health Monitor., 26(7). https://doi.org/10.1002/stc.2362   DOI
7 Jeong, S., Ferguson, M., Hou, R., Lynch, J.P., Sohn, H. and Law, K.H. (2019), "Sensor data reconstruction using bidirectional recurrent neural network with application to bridge monitoring", Adv. Eng. Inf., 42, 1009911. https://doi.org/10.1016/j.aei.2019.100991   DOI
8 Krizhevsky, A., Sutskever, I. and Hinton, G.E. (2012), "Imagenet classification with deep convolutional neural networks", Adv. Neural Inform. Process. Syst., 25, 1097-1105.
9 Kuznetsova, A., Rom, H., Alldrin, N., Uijlings, J., Krasin, I., Pont-Tuset, J., Kamali, S., Popov, S., Malloci, M. and Kolesnikov, A. (2020), "The open images dataset v4", Int. J. Comput. Vis., 1-26. https://doi.org/10.1007/s11263-020-01316-z   DOI
10 Rodriguez, J.D., Perez, A. and Lozano, J.A. (2009), "Sensitivity analysis of k-fold cross validation in prediction error estimation", IEEE Trans. Pattern Anal. Mach. Intell., 32(3), 569-575. https://doi.org/10.1109/TPAMI.2009.187   DOI
11 LeCun, Y., Bengio, Y. and Hinton, G. (2015), "Deep learning", Nature, 521(7553), 436-444. https://doi.org/10.1038/nature14539   DOI
12 Lei, X., Sun, L. and Xia, Y. (2020), "Lost data reconstruction for structural health monitoring using deep convolutional generative adversarial networks", Struct. Health Monitor., 20(4), 2069-2087. https://doi.org/10.1177/1475921720959226   DOI
13 He, K., Zhang, X., Ren, S. and Sun, J. (2015), "Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition", IEEE Trans. Pattern Anal. Mach. Intell., 37(9), 1904-1916. https://doi.org/10.1109/TPAMI.2015.2389824   DOI
14 Smarsly, K. and Law, K.H. (2014), "Decentralised fault detection and isolation in wireless structural health monitoring systems using analytical redundancy", Adv. Eng. Software, 73, 1-10. https://doi.org/10.1016/j.advengsoft.2014.02.005   DOI
15 Rawat, W. and Wang, Z. (2017), "Deep convolutional neural networks for image classification: A comprehensive review", Neural Comput., 29(9), 2352-2449. https://doi.org/10.1162/NECO_a_00990   DOI
16 Sun, Z., Zou, Z.L. and Zhang, Y.F. (2017), "Utilisation of structural health monitoring in long-span bridges: Case studies", Struct. Control Health Monitor., 24(10). https://doi.org/10.1002/stc.1979   DOI
17 Wu, R.T. and Jahanshahi, M.R. (2019), "Deep convolutional neural network for structural dynamic response estimation and system identification", J. Eng. Mech., 145(1). https://doi.org/10.1061/(asce)em.1943-7889.0001556   DOI
18 Yeum, C.M. and Dyke, S.J. (2015), "Vision-Based Automated Crack Detection for Bridge Inspection", Comput.-Aided Civ. Infrastruct. Eng., 30(10), 759-770. https://doi.org/10.1111/mice.12141   DOI
19 Sharifi, R., Kim, Y. and Langari, R. (2010), "Sensor fault isolation and detection of smart structures", Smart Mater. Struct., 19(10), 105001. https://doi.org/10.1088/0964-1726/19/10/105001   DOI
20 Yu, Y., Wang, C.Y., Gu, X.Y. and Li, J.C. (2018), "A novel deep learning-based method for damage identification of smart building structures", Struct. Health Monitor., 18(1), 143-163. https://doi.org/10.1177/1475921718804132   DOI
21 Spencer, B.F., Hoskere, V. and Narazaki, Y. (2019), "Advances in computer vision-based civil infrastructure inspection and monitoring", Eng., 5(2), 199-222. https://doi.org/10.1016/j.eng.2018.11.030   DOI
22 Szegedy, C., Ioffe, S., Vanhoucke, V. and Alemi, A. (2017), "Inception-v4, inception-resnet and the impact of residual connections on learning", Proceedings of the AAAI Conference on Artificial Intelligence.
23 Tang, Z.Y., Chen, Z.C., Bao, Y.Q. and Li, H. (2019), "Convolutional neural network-based data anomaly detection method using multiple information for structural health monitoring", Struct. Control Health Monitor., 26(1). https://doi.org/10.1002/stc.2296   DOI
24 Xia, Y., Chen, B., Weng, S., Ni, Y.-Q. and Xu, Y.-L. (2012), "Temperature effect on vibration properties of civil structures: a literature review and case studies", J. Civil Struct. Health Monitor., 2(1), 29-46. https://doi.org/10.1007/s13349-011-0015-7   DOI
25 Bengio, Y. and Grandvalet, Y. (2004), "No unbiased estimator of the variance of k-fold cross-validation", J. Mach. Learn. Res., 5, 1089-1105.
26 Kullaa, J. (2011), "Distinguishing between sensor fault, structural damage, and environmental or operational effects in structural health monitoring", Mech. Syst. Sig. Process., 25(8), 2976-2989. https://doi.org/10.1016/j.ymssp.2011.05.017   DOI
27 Xia, Y., Lei, X., Wang, P., Liu, G. and Sun, L. (2020), "Long-term performance monitoring and assessment of concrete beam bridges using neutral axis indicator", Struct. Control Health Monitor., 27(12). https://doi.org/10.1002/stc.2637   DOI
28 An, Y.H., Chatzi, E., Sim, S.H., Laflamme, S., Blachowski, B. and Ou, J.P. (2019), "Recent progress and future trends on damage identification methods for bridge structures", Struct. Control Health Monitor., 26(10). https://doi.org/10.1002/stc.2416   DOI
29 Bao, Y. and Li, H. (2020), "Machine learning paradigm for structural health monitoring", Struct. Health Monitor., 20(4), 1353-1372. https://doi.org/10.1177/1475921720972416   DOI
30 Bao, Y., Tang, Z., Li, H. and Zhang, Y. (2019a), "Computer vision and deep learning-based data anomaly detection method for structural health monitoring", Struct. Health Monitor., 18(2), 401-421. https://doi.org/10.1177/1475921718757405   DOI
31 Bao, Y.Q., Chen, Z.C., Wei, S.Y., Xu, Y., Tang, Z.Y. and Li, H. (2019b), "The State of the Art of Data Science and Engineering in Structural Health Monitoring", Engineering, 5(2), 234-242. https://doi.org/10.1016/j.eng.2018.11.027   DOI
32 Yi, T.H., Li, H.N., Song, G.B. and Guo, Q. (2016), "Detection of shifts in GPS measurements for a long-span bridge using CUSUM chart", Int. J. Struct. Stab. Dyn., 16(04), 1640024. https://doi.org/10.1142/S0219455416400241   DOI
33 Cha, Y.J., Choi, W. and Buyukozturk, O. (2017), "Deep learning-based crack damage detection using convolutional neural networks", Comput.-Aided Civ. Infrastruct. Eng., 32(5), 361-378. https://doi.org/10.1111/mice.12263   DOI
34 Chang, P.C., Flatau, A. and Liu, S. (2003), "Health monitoring of civil infrastructure", Struct. Health Monitor., 2(3), 257-267. https://doi.org/10.1177/1475921703036169   DOI
35 Fan, G., Li, J. and Hao, H. (2019), "Lost data recovery for structural health monitoring based on convolutional neural networks", Struct. Control Health Monitor., 26(10). https://doi.org/10.1002/stc.2433   DOI
36 Fan, G., Li, J. and Hao, H. (2020), "Dynamic response reconstruction for structural health monitoring using densely connected convolutional networks", Struct. Health Monitor., 20(4), 1373-1391. https://doi.org/10.1177/1475921720916881   DOI
37 Zhang, A., Wang, K.C.P., Li, B.X., Yang, E.H., Dai, X.X., Peng, Y., Fei, Y., Liu, Y., Li, J.Q. and Chen, C. (2017), "Automated Pixel-Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep-Learning Network", Comput.-Aided Civil Infrastruct. Eng., 32(10), 805-819. https://doi.org/10.1111/mice.12297   DOI
38 Lin, Y.Z., Nie, Z.H. and Ma, H.W. (2017b), "Structural damage detection with automatic feature-extraction through deep learning", Comput.-Aided Civ. Infrastruct. Eng., 32(12), 1025-1046. https://doi.org/10.1111/mice.12313   DOI
39 Liu, G., Li, L.L., Zhang, L.L., Li, Q. and Law, S.S. (2020), "Sensor faults classification for SHM systems using deep learning-based method with Tsfresh features", Smart Mater. Struct., 29(7). https://doi.org/10.1088/1361-665X/ab85a6   DOI
40 Mao, J.X., Wang, H. and Spencer, B.F. (2020), "Toward data anomaly detection for automated structural health monitoring: Exploiting generative adversarial nets and autoencoders", Struct. Health Monitor, 20(4), 1609-1626. https://doi.org/10.1177/1475921720924601   DOI
41 Zhang, Y.Q., Miyamori, Y., Mikami, S. and Saito, T. (2019), "Vibration-based structural state identification by a 1-dimensional convolutional neural network", Comput.-Aided Civ. Infrastruct. Eng., 34(9), 822-839. https://doi.org/10.1111/mice.12447   DOI
42 Lin, T.-Y., Goyal, P., Girshick, R., He, K. and Dollar, P. (2017a), "Focal loss for dense object detection", Proceedings of the IEEE International Conference on Computer Vision, pp. 2980-2988.
43 Mohtasham Khani, M., Vahidnia, S., Ghasemzadeh, L., Ozturk, Y.E., Yuvalaklioglu, M., Akin, S. and Ure, N.K. (2019), "Deep-learning-based crack detection with applications for the structural health monitoring of gas turbines", Struct. Health Monitor., 19(5), 1440-1452. https://doi.org/10.1177/1475921719883202   DOI
44 Li, J., Hao, H., Xia, Y. and Zhu, H.-P. (2014), "Damage detection of shear connectors in bridge structures with transmissibility in frequency domain", Int. J. Struct. Stab. Dyn., 14(02), 1350061. https://doi.org/10.1142/s0219455413500612   DOI
45 Ni, F.T., Zhang, J. and Noori, M.N. (2020), "Deep learning for data anomaly detection and data compression of a long-span suspension bridge", Comput.-Aided Civ. Infrastruct. Eng., 35(7), 685-700. https://doi.org/10.1111/mice.12528   DOI
46 Oh, B.K., Glisic, B., Kim, Y. and Park, H.S. (2019), "Convolutional neural network-based wind-induced response estimation model for tall buildings", Comput.-Aided Civ. Infrastruct. Eng., 34(10), 843-858. https://doi.org/10.1111/mice.12476   DOI
47 Pan, S.J. and Yang, Q.A. (2010), "A Survey on Transfer Learning", IEEE Trans. Knowl. Data. Eng., 22(10), 1345-1359. https://doi.org/10.1109/Tkde.2009.191   DOI
48 Ramachandran, P., Zoph, B. and Le, Q.V. (2017), "Searching for activation functions", arXiv preprint arXiv:1710.05941.
49 Rao, A.R.M., Kasireddy, V., Gopalakrishnan, N. and Lakshmi, K. (2015), "Sensor fault detection in structural health monitoring using null subspace-based approach", J. Intell. Mater. Syst. Struct., 26(2), 172-185. https://doi.org/10.1177/1045389x14522534   DOI
50 Li, L.L., Liu, G., Zhang, L.L. and Li, Q. (2019), "Sensor fault detection with generalised likelihood ratio and correlation coefficient for bridge SHM", J. Sound Vib., 442, 445-458. https://doi.org/10.1016/j.jsv.2018.10.062   DOI