Browse > Article
http://dx.doi.org/10.12989/smm.2020.7.4.345

Big data platform for health monitoring systems of multiple bridges  

Wang, Manya (Key Laboratory of C&PC Structures of the Ministry of Education, Southeast University)
Ding, Youliang (Key Laboratory of C&PC Structures of the Ministry of Education, Southeast University)
Wan, Chunfeng (Key Laboratory of C&PC Structures of the Ministry of Education, Southeast University)
Zhao, Hanwei (Key Laboratory of C&PC Structures of the Ministry of Education, Southeast University)
Publication Information
Structural Monitoring and Maintenance / v.7, no.4, 2020 , pp. 345-365 More about this Journal
Abstract
At present, many machine leaning and data mining methods are used for analyzing and predicting structural response characteristics. However, the platform that combines big data analysis methods with online and offline analysis modules has not been used in actual projects. This work is dedicated to developing a multifunctional Hadoop-Spark big data platform for bridges to monitor and evaluate the serviceability based on structural health monitoring system. It realizes rapid processing, analysis and storage of collected health monitoring data. The platform contains offline computing and online analysis modules, using Hadoop-Spark environment. Hadoop provides the overall framework and storage subsystem for big data platform, while Spark is used for online computing. Finally, the big data Hadoop-Spark platform computational performance is verified through several actual analysis tasks. Experiments show the Hadoop-Spark big data platform has good fault tolerance, scalability and online analysis performance. It can meet the daily analysis requirements of 5s/time for one bridge and 40s/time for 100 bridges.
Keywords
structural health monitoring; bridge; big data; Hadoop-Spark platform; data processing;
Citations & Related Records
Times Cited By KSCI : 10  (Citation Analysis)
연도 인용수 순위
1 Zhang, W.J. and Huang, Y.P. (2019), "Using big data computing framework and parallelized PSO algorithm to construct the reservoir dispatching rule optimization", Soft Computing, 24(11), 8113-8124. http://doi.org/10.1007/s00500-019-04188-9.   DOI
2 Zhao, H.W., Ding, Y.L., An, Y.H. and Li, A.Q. (2016), "Transverse dynamic mechanical behavior of hangers in the rigid tied-arch bridge under train loads", J. Perform. Constr. Fac., 31(1), 04016072. https://doi.org/10.1061/(ASCE)CF.1943-5509.0000932.   DOI
3 Zhao, H.W., Ding, Y.L., Geng, F.F. and Li, A.Q. (2018), "RAMS evaluation for a steel-truss arch highspeed railway bridge based on SHM system", Struct. Monit. Maint., 5(1), 79-92. https://doi.org/10.12989/smm.2018.5.1.079.   DOI
4 Zhao, H.W., Ding, Y.L., Li, A.Q., Ren, Z.Z. and Yang, K. (2019), "Live-load strain evaluation of the prestressed concrete box-girder bridge using deep learning and clustering", Struct. Health Monit., 147592171987563. https://doi.org/10.1177/1475921719875630.   DOI
5 Zhao, H.W., Ding, Y.L., Nagarajaiah, S. and Li, A.Q. (2019), "Longitudinal displacement behavior and girder end reliability of a jointless steel-truss arch railway bridge during operation", Appl. Sci., 9(11), 2222. http://doi.org/10.3390/app9112222.   DOI
6 Zhao, L. and Yin, A.J. (2015), "High-order partial differential equation de-noising method for vibration signal", Math. Method. Appl. Sci., 38(5), 937-947. https://doi.org/10.1002/mma.3119.   DOI
7 Guo, S., Luo, H. and Yong, L. (2015), "A big data-based workers behavior observation in china metro construction", Procedia Eng., 123, 190-197. https://doi.org/10.1016/j.proeng.2015.10.077.   DOI
8 Han, S.H. (2017), "Optimal safety valuation of high-speed railway bridges based on reliability assessment and life-cycle cost concept", Int. J. Steel Struct., 17(1), 339-349. https://doi.org/10.1007/s13296-014-0165-7.   DOI
9 Huang, Y., Beck, J.L., Wu, S. and Li, H. (2016), "Bayesian compressive sensing for approximately sparse signals and application to structural health monitoring signals for data loss recovery", Probabilist. Eng. Mech., 46, 62-79. http://dx.doi.org/10.1016/j.probengmech.2016.08.001.   DOI
10 Huang, Y.W. et al. (2010), "Lost strain data reconstruction based on least squares support vector machine", Measurement Control Technol., 29, 8-12. http://doi.org/10.2991/icacsei.2013.159.   DOI
11 Jagadish, H.V., Gehrke, J., Labrinidis, A. and Papakonstantinou, Y. (2014), "Big data and its technical challenges", Communications of the ACM, 57(7), 86-94. https://doi.org/10.1145/2611567.   DOI
12 Jeong, S., Zhang, Y.L., O'Connor, S., Lynch, J.P., Sohn, H. and Law, K.H. (2016), "A NoSQL data management infrastructure for bridge monitoring", Smart Struct. Syst., 17(4), 669-690. http://doi.org/10.12989/sss.2016.17.4.669.   DOI
13 Kawamura, K., Miyamoto, A., Frangopol, D. M. and Kimura, R. (2003), "Performance Evaluation of Concrete Slab of Existing Bridges Using Neural Networks", Eng. Struct., 25(12), 1455-1477. https://doi.org/10.1016/S0141-0296(03)00112-3.   DOI
14 Landset, S., Khoshgoftaar, T.M., Richter, A.N. and Hasanin, T. (2015), "A survey of open source tools for machine learning with big data in the Hadoop ecosystem", J. Big Data, 2(1), 1-36. http://doi.org/10.1186/s40537-015-0032-1.   DOI
15 Liu, H., Ding, Y.L., Zhao, H.W, Wang, M.Y. and Geng, F.F. (2020), "Deep learning-based recovery method for missing structural temperature data using LSTM network", Struct. Monit. Maint., 7(2), 109-124. https://doi.org/10.12989/smm.2020.7.2.109.   DOI
16 Memmolo, V., Pasquino, N. and Ricci, F. (2018), "Experimental characterization of a damage detection and localization system for composite structures", Measurement, 129, 381-388. https://doi.org/10.1016/j.measurement.2018.07.032.   DOI
17 Nguyen, C.U., Huynh, T.C. and Kim, J.T. (2018), "Vibration- based damage detection in wind turbine towers using artificial neural networks", Struct. Monit. Maint., 5(4), 507-519. https://doi.org/10.12989/smm.2018.5.4.507.   DOI
18 Nick, W., Asamene, K., Bullock, G., Esterline, A. and Sundaresan, M. (2015), "A study of machine learning techniques for detecting and classifying structural damage", Int. J. Machine Learning Comput., 5(4), 313-318. https://doi.org/10.7763/IJMLC.2015.V5.526.   DOI
19 Okasha, N.M. and Frangopol, D.M. (2012), "Integration of structural health monitoring in a system performance based life-cycle bridge management framework", Struct. Infrastruct. Eng., 8(11), 999-1016. https://doi.org/10.1080/15732479.2010.485726.   DOI
20 Cao, B.Y., Ding, Y.L., Zhao, H.W. and Song, Y.S. (2016), "Damage identification for high-speed railway truss arch bridge using fuzzy clustering analysis". Struct. Monit. Maint., 3(4), 315-333. https://doi.org/10.12989/smm.2016.3.4.315.   DOI
21 Chen, D., et al. (2017), "Real-time or near real-time persisting daily healthcare data into HDFS and ElasticSearch index inside a big data platform", IEEE T. Ind. Inform., 13(2), 595-606. https://doi.org/10.1109/TII.2016.2645606.   DOI
22 Chouliaras, S. and Sotiriadis, S. (2019), "Real-Time Anomaly Detection of NoSQL Systems Based on Resource Usage Monitoring", IEEE T. Ind. Inform., 16(9), 6042-6049, https://doi:10.1109/TII.2019.2958606.   DOI
23 Ding, Y.L., Wang, C., Zhao, H.W., Yue, Q. and Wu, L.Y. (2016), "Vehicle-bridge resonance analysis of dashengguan bridge based on vibration acceleration monitoring", J. Railway Eng. Soc., 33(9), 48-54. https://doi.org/10.1061/(ASCE)CF.1943-5509.0000932.   DOI
24 Ding, Y.L., Zhao, H.W., Deng, L., Li, A.Q. and Wang, M.Y. (2017), "Early Warning of Abnormal Train-Induced Vibrations for a Steel-Truss Arch Railway Bridge: Case Study", J. Bridge Eng., 22(11), 05017011.1-05017011.12. https://doi.org/10.1061/(ASCE)BE.1943-5592.0001143.   DOI
25 Ding, Z.Y., Mei, G., Cuomo, S., Li, Y.X. and Xu, N.X. (2018), "Comparison of estimating missing values in IOT time series data using different interpolation algorithms", Int. J. Parallel Program., 48(3), 534-548. https://doi.org/10.1007/s10766-018-0595-5.   DOI
26 Furuta, H., He, J.H. and Watanabe, E. (1996), "A Fuzzy Expert System for Damage Assessment Using Genetic Algorithms and Neural Networks", Comput. - Aided Civil Infrastruct. Eng., 11(1), 37-45. https://doi.org/10.1111/j.1467-8667.1996.tb00307.x.   DOI
27 Garcia-Valls, M., Abhishek, D. and Vicent, B. (2018), "Introducing the new paradigm of social dispersed computing: applications, technologies and challenges", Journal of Systems Architecture, 91, 83-102. https://doi.org/10.1016/j.sysarc.2018.05.007.   DOI
28 Garcia-Valls, M., Calva-Urrego, C. and Garcia-Fornes, A. (2018), "Accelerating smart eHealth services execution at the fog computing infrastructure", Future Generation Computer Systems, S0167739X17327425. https://doi.org/10.1016/j.future.2018.07.001.   DOI
29 Guo, J.Q., Xie, X.B., Bie, R.F. and Sun, L.M. (2014), "Structural health monitoring by using a sparse codingbased deep learning algorithm with wireless sensor networks", Personal & Ubiquitous Comput., 18(8), 1977-1987. https://doi.org/10.1007/s00779-014-0800-5.   DOI
30 Bao, Y.Q., Li, H., Sun, X.D., Yu, Y. and Ou, J.P. (2012), "Compressive sampling-based data loss recovery for wireless sensor networks used in civil structural health monitoring", Struct. Health Monit., 12(1), 78-95. https://doi.org/10.1177/1475921712462936.   DOI
31 Beltempo, A., Cappello, C., Zonta, D., Bonelli, A., Bursi, O.S., Costa, C. and Pardatscher, W. (2015), "Structural Health Monitoring of the Colle Isarco Viaduct", Workshop on Environmental Engergy and Structural Monitoring Systems, 7-11. https://doi.org/10.1109/EESMS.2015.7175843.   DOI
32 Praveen, D.S. and Raj, D.P. (2020), "Smart traffic management system in metropolitan cities", J. Ambient Intelligence and Humanized Comput., 1-13. https://doi.org/10.1007/s12652-020-02453-6.   DOI
33 Ren, D.M., Rahal, I. and Perrizo, W. (2004), "A vertical outlier detection algorithm with clusters as byproduct", International Conference on Tools with Artificial Intelligence, (2004), 22-29. https://doi.org/10.1109/ICTAI.2004.22.   DOI
34 Satti, Fahad Ahmed, et al. (2020), "Ubiquitous Health Profile (UHPr): a big data curation platform for supporting health data interoperability", Computing, 1-36. https://doi.org/10.1007/s00607-020-00837-2 .   DOI
35 Sayed, M.A., Kaloop, M.R., Kim, E. and Kim, D. (2017), "Assessment of acceleration responses of a railway bridge using wavelet analysis", KSCE J. Civil Eng., 21(5), 1844-1853. https://doi.org/10.1007/s12205-016-1762-0.   DOI
36 White, T. (2012), "Hadoop: the definitive guide", O'Reilly Media Inc. Gravenstein Highway North, 215(11), 1-4. http://dx.doi.org/10.9774/GLEAF.978-1-909493-38-4_2.
37 Yang, Y. and Nagarajaiah, S. (2016), "Harnessing data structure for recovery of randomly missing structural vibration responses time history: sparse representation versus low-rank structure", Mech. Syst. Signal Pr., 74, 165-182. http://dx.doi.org/10.1016/j.ymssp.2015.11.009.   DOI