Browse > Article
http://dx.doi.org/10.9708/jksci.2022.27.05.055

A Proposal of Sensor-based Time Series Classification Model using Explainable Convolutional Neural Network  

Jang, Youngjun (Dept. of Industrial and Management Engineering, Korea University)
Kim, Jiho (Dept. of Industrial and Management Engineering, Korea University)
Lee, Hongchul (Dept. of Industrial and Management Engineering, Korea University)
Publication Information
Journal of the Korea Society of Computer and Information / v.27, no.5, 2022 , pp. 55-67 More about this Journal
Abstract
Sensor data can provide fault diagnosis for equipment. However, the cause analysis for fault results of equipment is not often provided. In this study, we propose an explainable convolutional neural network framework for the sensor-based time series classification model. We used sensor-based time series dataset, acquired from vehicles equipped with sensors, and the Wafer dataset, acquired from manufacturing process. Moreover, we used Cycle Signal dataset, acquired from real world mechanical equipment, and for Data augmentation methods, scaling and jittering were used to train our deep learning models. In addition, our proposed classification models are convolutional neural network based models, FCN, 1D-CNN, and ResNet, to compare evaluations for each model. Our experimental results show that the ResNet provides promising results in the context of time series classification with accuracy and F1 Score reaching 95%, improved by 3% compared to the previous study. Furthermore, we propose XAI methods, Class Activation Map and Layer Visualization, to interpret the experiment result. XAI methods can visualize the time series interval that shows important factors for sensor data classification.
Keywords
Sensor Data; Time Series Classification; Pattern Recognition; Deep Learning; eXplainable Artificial Intelligence(XAI);
Citations & Related Records
Times Cited By KSCI : 1  (Citation Analysis)
연도 인용수 순위
1 K. He, X. Zhang, S. Ren and J. Sun, "Deep Residual Learning for Image Recognition." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 770-78.
2 T. Rojat, R. Puget , D. Filliat , J. Del Ser, R. Gelin and N. Diaz-Rodriguez "Explainable artificial intelligence (xai) on timeseries data: A survey." arXiv preprint arXiv:2104.00950, 2021.
3 S. H. Kwon, M. J. An, H. C. Lee, "Fault Detection and Classification of Process Cycle Signals Using Density-based Clustering and Deep Learning." Journal of the Korean Institute of Industrial Engineers 44.6, 2018, pp. 475-482.   DOI
4 K. S. Choi et al., "Prediction of IDH Genotype in Gliomas with Dynamic Susceptibility Contrast Perfusion MR Imaging Using an Explainable Recurrent Neural Network." Neuro-Oncology, vol. 21, no. 9, Oxford University Press US, 2019, pp. 1197-209.   DOI
5 D. Q. Goldin, and C. P. Kanellakis, "On Similarity Queries for Time-Series Data: Constraint Specification and Implementation." International Conference on Principles and Practice of Constraint Programming, Springer, 1995, pp. 137-53.
6 B. K. Iwana and S. Uchida, "An Empirical Survey of Data Augmentation for Time Series Classification with Neural Networks." Plos One, vol. 16, no. 7, Public Library of Science San Francisco, CA USA, 2021, p. e0254841.   DOI
7 W. Tang, G. Long, L. Liu, T. Zhou, J. Jiang and M. Blumenstein, "Rethinking 1d-Cnn for Time Series Classification: A Stronger Baseline." ArXiv Preprint ArXiv:2002.10061, 2020.
8 C. Oh, S. Han and J. Jeong, "Time-series data augmentation based on interpolation." Procedia Computer Science 175, pp. 64-71, 2020.   DOI
9 H. Ismail Fawaz et al., "Inceptiontime: Finding Alexnet for Time Series Classification." Data Mining and Knowledge Discovery, vol. 34, no. 6, Springer, 2020, pp. 1936-62.   DOI
10 F. Oviedo et al., "Fast and Interpretable Classification of Small X-Ray Diffraction Datasets Using Data Augmentation and Deep Neural Networks." Npj Computational Materials, vol. 5, no. 1, Nature Publishing Group, 2019, pp. 1-9   DOI
11 Y. S. Jeong, M. K. Jeong and O. A. Omitaomu, "Weighted dynamic time warping for time series classification." Pattern recognition 44.9, 2011, pp. 2231-2240.   DOI
12 D. Smirnov and E. M. Nguifo, "Time Series Classification with Recurrent Neural Networks." Advanced Analytics and Learning on Temporal Data, vol. 8, 2018.
13 C. L. Yang, Z. X. Chen and C. Y. Yang, "Sensor classification using convolutional neural network by encoding multivariate time series as two-dimensional colored images." Sensors 20.1, pp. 168, 2019.   DOI
14 M. Wenninger et al., "Timage-A Robust Time Series Classification Pipeline." International Conference on Artificial Neural Networks, Springer, 2019, pp. 450-61.
15 A. Preece et al., "Stakeholders in Explainable AI." ArXiv Preprint ArXiv:1810.00184, 2018.
16 J. Zhu et al., "Explainable AI for Designers: A Human-Centered Perspective on Mixed-Initiative Co-Creation." 2018 IEEE Conference on Computational Intelligence and Games (CIG), IEEE, 2018, pp. 1-8.
17 B. Zhou, A. Khosla, A. Lapedriza, A. Oliva and A. Torralba, "Learning Deep Features for Discriminative Localization." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2921-29.
18 B. Zhao et al., "Convolutional neural networks for time series classification." Journal of Systems Engineering and Electronics 28.1, 2017, pp. 162-169.   DOI
19 D. Kuang, "A 1d convolutional network for leaf and time series classification." arXiv preprint arXiv:1907.00069, 2019.
20 G. S. Ahn, H. C. Lee and S. Hur, "Feature Selection Method for Multivariate Time Series Data Classification." Journal of the Korean Institute of Industrial Engineers 43.6, 2017, pp. 413-421.   DOI
21 H. Ismail Fawaz et al., "Deep Learning for Time Series Classification: A Review." Data Mining and Knowledge Discovery, vol. 33, no. 4, Springer, 2019, pp. 917-63.   DOI
22 M. Van Lent, W. Fisher and M. Mancuso, "An explainable artificial intelligence system for small-unit tactical behavior." Proceedings of the national conference on artificial intelligence. Menlo Park, CA; Cambridge, MA; London; AAAI Press; MIT Press; 1999, 2004.
23 Z. Wang, W. Yan and T. Oates, "Time Series Classification from Scratch with Deep Neural Networks: A Strong Baseline." 2017 International Joint Conference on Neural Networks (IJCNN), IEEE, 2017, pp. 1578-85.
24 D. Gunning and D. Aha, "DARPA's Explainable Artificial Intelligence (XAI) Program." AI Magazine, vol. 40, no. 2, 2019, pp. 44-58.   DOI
25 N. Strodthoff et al. "Deep Learning for ECG Analysis: Benchmarks and Insights from PTB-XL." IEEE Journal of Biomedical and Health Informatics, vol. 25, no. 5, IEEE, 2020, pp. 1519-28.   DOI
26 A. A. Ismail, M. Gunady, H. Corrada Bravo and S. Feizi, "Benchmarking Deep Learning Interpretability in Time Series Predictions." ArXiv Preprint ArXiv:2010.13924, 2020.
27 G. Forestier, F. Petitjean, H. A. Dau, G. I. Webb, and E. Keogh, "Generating Synthetic Time Series to Augment Sparse Datasets." 2017 IEEE International Conference on Data Mining (ICDM), IEEE, 2017, pp. 865-70.
28 T. Um, F. M. Pfister, D. Pichler, S. Endo, M. Lang and D. Kulic, "Data Augmentation of Wearable Sensor Data for Parkinson's Disease Monitoring Using Convolutional Neural Networks." Proceedings of the 19th ACM International Conference on Multimodal Interaction, 2017, pp. 216-20.
29 Q. Wen, L. Sun, F. Yang, X. Song, J. Gao, X. Wang and H. Xu, "Time Series Data Augmentation for Deep Learning: A Survey." ArXiv Preprint ArXiv:2002.12478, 2020.
30 F. Rosenblatt, Principles of Neurodynamics. Perceptrons and the Theory of Brain Mechanisms. Cornell Aeronautical Lab Inc Buffalo NY, 1961.