The Korean Journal of Applied Statistics (응용통계연구)

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

Procedure for monitoring autocorrelated processes using LSTM Autoencoder

LSTM Autoencoder를 이용한 자기상관 공정의 모니터링 절차

  • Pyoungjin Ji (Department of Applied Statistics, Chung-Ang University) ;
  • Jaeheon Lee (Department of Applied Statistics, Chung-Ang University)
  • 지평진 (중앙대학교 응용통계학과) ;
  • 이재헌 (중앙대학교 응용통계학과)
  • Received : 2023.12.23
  • Accepted : 2024.02.28
  • Published : 2024.04.30
Download PDF
⟨ Previous Next ⟩

Abstract

Many studies have been conducted to quickly detect out-of-control situations in autocorrelated processes. The most traditionally used method is a residual control chart, which uses residuals calculated from a fitted time series model. However, many procedures for monitoring autocorrelated processes using statistical learning methods have recently been proposed. In this paper, we propose a monitoring procedure using the latent vector of LSTM Autoencoder, a deep learning-based unsupervised learning method. We compare the performance of this procedure with the LSTM Autoencoder procedure based on the reconstruction error, the RNN classification procedure, and the residual charting procedure through simulation studies. Simulation results show that the performance of the proposed procedure and the RNN classification procedure are similar, but the proposed procedure has the advantage of being useful in processes where sufficient out-of-control data cannot be obtained, because it does not require out-of-control data for training.

자기상관 공정에서 이상상태를 빠르게 탐지하는 절차에 대해 많은 연구가 진행되어 왔다. 가장 전통적인 절차는 관측된 데이터에 대해 적합한 시계열 모형에서 계산된 잔차를 이용하는 잔차 관리도이다. 그러나 최근에는 통계적 학습 방법을 이용하여 자기상관 공정을 모니터링하는 절차가 많이 제안되었다. 이 논문에서는 딥러닝에 기반한 비지도 학습 방법인 LSTM Autoencoder의 잠재 벡터를 이용한 모니터링 절차를 제안하고, 이를 모의실험을 통해 LSTM Autoencoder의 복원 오차를 이용한 절차, RNN 분류 모니터링 절차, 그리고 잔차 관리도 절차의 성능과 비교하였다. 모의실험 결과, 제안된 절차와 RNN 분류 모니터링 절차의 성능은 유사하지만, 제안된 절차는 학습에 이상상태의 데이터가 필요하지 않기 때문에 이상상태의 데이터를 충분하게 확보할 수 없는 공정에 유용하게 적용할 수 있다는 장점이 있다.

Keywords

References

  1. Chen S and Yu J (2019). Deep recurrent neural network-based residual control chart for autocorrelated processes, Quality and Reliability Engineering International, 35, 2687-2708. https://doi.org/10.1002/qre.2551
  2. Ho ES and Chang SI (1999). An integrated neural network approach for simultaneous monitoring of process mean and variance shifts a comparative study, International Journal of Production Research, 37, 1881-1901. https://doi.org/10.1080/002075499191049
  3. Hochreiter S and Schmidhuber J (1997). Long short-term memory, Neural Computation, 9, 1735-1780. https://doi.org/10.1162/neco.1997.9.8.1735
  4. Homayouni H, Ghosh S, Ray I, Gondalia S, Duggan J, and Kahn MG (2020). An autocorrelation-based LSTM-Autoencoder for anomaly detection on time-series data. In Proceedings of 2020 IEEE International Conference on Big Data, Atlanta, GA, 5068-5077.
  5. Hwarng HB (2004). Detecting process mean shift in the presence of autocorrelation: A neural-network based monitoring scheme, International Journal of Production Research, 42, 573-595. https://doi.org/10.1080/0020754032000123614
  6. Hwarng HB and Wang Y (2010). Shift detection and source identification in multivariate autocorrelated processes, International Journal of Production Research, 48, 835-859. https://doi.org/10.1080/00207540802431326
  7. Ji P and Lee J (2023). Performance comparison of procedures for monitoring autocorrelated processes based on classification and forecasting, Journal of the Korean Data & Information Science Society, 34, 775-789.
  8. Kramer MA (1991). Nonlinear principal component analysis using autoassociative neural networks, AIChE Journal, 37, 233-243. https://doi.org/10.1002/aic.690370209
  9. Kramer MA (1992). Autoassociative neural networks, Computers & Chemical Engineering, 16, 313-328.
  10. Lee S and Lee J (2022). Monitoring procedure of autocorrelated processes using the deep learning-based LSTM model, Journal of the Korean Data & Information Science Society, 33, 835-842.
  11. Lu CW and Reynolds MR JR (1999). EWMA control charts for monitoring the mean of autocorrelated processes, Journal of Quality Technology, 31, 166-188. https://doi.org/10.1080/00224065.1999.11979913
  12. Maged A, Lui CF, Haridy S, and Xie M (2023). Variational AutoEncoders-LSTM based fault detection of time-dependent high dimensional processes, International Journal of Production Research, 62, 1-16. https://doi.org/10.1080/00207543.2023.2175591
  13. Montgomery DC (2020). Introduction to Statistical Quality Control (8th ed), John Wiley & Sons, Inc., Hoboken, NJ.
  14. Prajapati DR and Singh S (2015). Application of ANN to monitor the correlated process using higher sample size, International Journal of Performability Engineering, 11, 395-404.
  15. Srivastava N, Mansimov E, and Salakhutdinov R (2015). Unsupervised learning of video representations using LSTMs. In Proceedings of the 32nd International Conference on Machine Learning, Lille, France, 843-852.
  16. Wei Y, Jang-Jaccard J, Xu W, Sabrina F, Camtepe S, and Boulic M (2022). LSTM-Autoencoder based anomaly detection for indoor air quality time series data, Available from: arXiv:2204.06701 [cs.LG]
  17. White AK and Safi SK (2016). The efficiency of artificial neural networks for forecasting in the presence of autocorrelated disturbances, International Journal of Statistics and Probability, 5, 51-58. https://doi.org/10.5539/ijsp.v5n2p51
  18. Yuangyai C and Abrahams R (2011). Statistical process control with autocorrelated data using neural networks. In Proceedings of 2011 IEEE International Conference on Quality and Reliability, Bangkok, Thailand, 283-287.