산업경영시스템학회지 (Journal of Korean Society of Industrial and Systems Engineering)

  • 제46권3호
  • /
  • Pages.123-138
  • /
  • 2023
  • /
  • 2005-0461(pISSN)
  • /
  • 2287-7975(eISSN)
한국산업경영시스템학회 (Society of Korea Industrial and System Engineering)
권호 표지
DOI QR코드

DOI QR Code

ProphetNet 모델을 활용한 시계열 데이터의 열화 패턴 기반 Health Index 연구

A Study on the Health Index Based on Degradation Patterns in Time Series Data Using ProphetNet Model

  • 원선주 (경기대학교 일반대학원 산업시스템공학과) ;
  • 김용수 (경기대학교 산업시스템공학과)
  • Sun-Ju Won (Department of Industrial and Systems Engineering, Kyonggi University Graduate School) ;
  • Yong Soo Kim (Department of Industrial and Systems Engineering, Kyonggi University)
  • 투고 : 2023.07.21
  • 심사 : 2023.09.05
  • 발행 : 2023.09.30
PDF 다운로드
⟨ 이전 논문 다음 논문 ⟩

초록

The Fourth Industrial Revolution and sensor technology have led to increased utilization of sensor data. In our modern society, data complexity is rising, and the extraction of valuable information has become crucial with the rapid changes in information technology (IT). Recurrent neural networks (RNN) and long short-term memory (LSTM) models have shown remarkable performance in natural language processing (NLP) and time series prediction. Consequently, there is a strong expectation that models excelling in NLP will also excel in time series prediction. However, current research on Transformer models for time series prediction remains limited. Traditional RNN and LSTM models have demonstrated superior performance compared to Transformers in big data analysis. Nevertheless, with continuous advancements in Transformer models, such as GPT-2 (Generative Pre-trained Transformer 2) and ProphetNet, they have gained attention in the field of time series prediction. This study aims to evaluate the classification performance and interval prediction of remaining useful life (RUL) using an advanced Transformer model. The performance of each model will be utilized to establish a health index (HI) for cutting blades, enabling real-time monitoring of machine health. The results are expected to provide valuable insights for machine monitoring, evaluation, and management, confirming the effectiveness of advanced Transformer models in time series analysis when applied in industrial settings.

키워드

과제정보

This work was supported by the GRRC program of Gyeonggi province. [GRRC KGU 2023-B01, Research on Intelligent Industrial Data Analytics]

참고문헌

  1. Abiodun, O.I., Jantan, A., Omolara, A.E., Dada, K.V., Umar, A.M., Linus, O.U., Arshad, H., Kazaure, A.A., Gana, U., and Kiru, M.U., Comprehensive review of artificial neural network applications to pattern recognition, IEEE Access, 2019, Vol. 7, pp. 158820-158846. https://doi.org/10.1109/ACCESS.2019.2945545
  2. Ahn, H.J., Kim, H.B., Jung, D.W., Kim, D.J., and Lee, D.K., Development of Artificial Intelligence-Based Anomaly Detection Method for Aerospace Systems, Korea Aerospace Research Institute, 2019.
  3. Alqudsi, A. and El-Hag, A., Application of machine learning in transformer health index prediction, Energies, 2019, Vol. 12, No. 14, p. 2694.
  4. Bohatyrewicz, P. and Mrozik, A., The analysis of power transformer population working in different operating conditions with the use of health index, Energies, 2021, Vol. 14, No. 16, p. 5213.
  5. Chandar, S., Sankar, C., Vorontsov, E., Kahou, S.E., and Bengio, Y., Towards non-saturating recurrent units for modelling long-term dependencies, Proceedings of the AAAI Conference on Artificial Intelligence, 2019, Vol. 33, No. 01, pp. 3280-3287.
  6. Chen, G., A gentle tutorial of recurrent neural network with error backpropagation, arXiv preprint arXiv, 2016, 1610.02583.
  7. Fang, W., Chen, Y., and Xue, Q., Survey on research of RNN-based spatio-temporal sequence prediction algorithms, Journal on Big Data, 2021, Vol. 3, No. 3, p. 97.
  8. Jahromi, A., Piercy, R., Cress, S., Service, J., and Fan, W., An approach to power transformer asset management using health index, IEEE Electrical Insulation Magazine, 2009, Vol. 25, No. 2, pp. 20-34. https://doi.org/10.1109/MEI.2009.4802595
  9. Jana, R.K., Ghosh, I., and Wallin, M.W., Taming energy and electronic waste generation in bitcoin mining: Insights from Facebook prophet and deep neural network, Technological Forecasting and Social Change, 2022, Vol. 178, p. 121584.
  10. Jang, S.M., Moon, J-H., and Sohn, K-A., Comparison of Transformer and LSTM for threat detection and traffic prediction on long time-series data, Journal of Korean Institute of Next Generation Computing, 2021, pp. 18-21.
  11. Jin, Y-H., Ji, S-H., and Han, K-H., Time Series Data Analysis and Prediction System Using PCA, Journal of the Korea Convergence Society, 2020, Vol. 12, No. 11, pp. 99-107.
  12. Jung, I.K., Park, D.K., and Jun, D.B., Performance of Pairs Trading Algorithm with the Implementation of Structural Changes Detection Procedure, Journal of The Korean Operations Research and Management Science Society, 2017, Vol. 42, No. 3, pp. 13-24.
  13. Kim, Y.S. and Park, K.S., The Multivariate Sensor Data Classification using Time Series Imaging, Journal of Korean Institute of Information Scientists and Engineers, 2022, Vol. 49, No. 8, pp. 593-600. https://doi.org/10.5626/JOK.2022.49.8.593
  14. Lim, J., Kim, I.K., Lee, M.H., Ha, J.M., and Lee, J.K., Performance comparison of network anomaly detection using BERT, LSTM and GRU, Journal of Korean Institute of Communications and Information Sciences, 2022, pp. 1268-1269.
  15. Mizuno, T., Fujimoto, S., and Ishikawa, A., Generation of individual daily trajectories by GPT-2, Frontiers in Physics, 2022, p. 1118.
  16. Murugan, R. and Ramasamy, R., Understanding the power transformer component failures for health index-based maintenance planning in electric utilities, Engineering Failure Analysis, 2019, Vol. 96, pp. 274-288. https://doi.org/10.1016/j.engfailanal.2018.10.011
  17. Naderian, A., Cress, S., Piercy, R., Wang, F., and Service, J., An approach to determine the health index of power transformers, Conference Record of the 2008 IEEE International Symposium on Electrical Insulation, 2008, pp. 192-196.
  18. Oh, H-W. and Kim, W-S., A Study on the Small Motion Classification Model based on Time Serial Data, Journal of Korean Institute of Communications and Information Sciences, 2022, pp. 949-950.
  19. Qi, W., Yan, Y., Gonkg, Y., Liu, D., Duan, N., Chen, J., Zhang, R., and Zhou, M., Prophetnet: Predicting future n-gram for sequence-to-sequence pre-training, arXiv preprint arXiv, 2020, 2001.04063.
  20. Rediansyah, D., Prasojo, R.A., and Abu-Siada, A., Artificial intelligence-based power transformer health index for handling data uncertainty, IEEE Access, 2021, Vol. 9, pp. 150637-150648. https://doi.org/10.1109/ACCESS.2021.3125379
  21. Sandoval, R.M., Garcia-Sanchez, A.J., Garcia-Haro, J., and Chen, T.M., Optimal policy derivation for transmission duty-cycle constrained LPWAN, IEEE Internet of Things Journal, 2018, Vol. 5, No. 4, pp. 3114-3125. https://doi.org/10.1109/JIOT.2018.2833289
  22. Saqlain, M., Jargalsaikhan, B., and Lee, J.Y., A voting ensemble classifier for wafer map defect patterns identification in semiconductor manufacturing, IEEE Transactions on Semiconductor Manufacturing, 2019, Vol. 32, No. 2, pp. 171-182. https://doi.org/10.1109/TSM.2019.2904306
  23. Sherstinsky, A., Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM) network, Physica D: Nonlinear Phenomena, 2020, Vol. 404, p. 132306.
  24. Tan, C., Deep Reinforcement Learning with Copy-oriented Context Awareness and Weighted Rewards for Abstractive Summarizatio, Proceedings of the 2023 2nd Asia Conference on Algorithms, Computing and Machine Learning, 2023, pp. 84-89.
  25. Xiao, J. and Zhou, Z., Research progress of RNN language model, 2020 IEEE International Conference on Artificial Intelligence and Computer Applications (ICAICA), 2020, pp. 1285-1288.
  26. Yi, D-H., Yu, Y.S., Ko, Y.D., Jo, H.K., and Park, C.S., Classification of Building Energy Pattern based on Robust Regression and Time series clustering, Journal of the Architectural Institute of Korea, 2020, Vol. 40, No. 2, pp. 392-393.
  27. Yoo, I-J. and Park, D-H., Derivation of Digital Music's Ranking Change Through Time Series Clustering, Journal of Intelligence and Information Systems, 2020, Vol. 26, No. 3, pp. 171-191. https://doi.org/10.13088/JIIS.2020.26.3.171
  28. Yook, H.N., Kim, Y-J., Choi, Y.S., Oh, J.W., Lee, K., and Ha, S.L., Comprehensive Studies on Reliability of Cutting Blade According to the Material of Knife Mill Used in Biomass Pretreatment Process, Transactions of the Korean Society of Mechanical Engineers, 2022, p. 171.
  29. Zhang, M. and Li, J., A commentary of GPT-3 in MIT Technology Review 2021, Fundamental Research, 2021, Vol. 1, No. 6, pp. 831-833.  https://doi.org/10.1016/j.fmre.2021.11.011