Nuclear Engineering and Technology

Korean Nuclear Society (한국원자력학회)
권호 표지
DOI QR코드

DOI QR Code

Robust transformer-based anomaly detection for nuclear power data using maximum correntropy criterion

  • Shuang Yi (College of Electrical Engineering & New Energy, China Three Gorges University) ;
  • Sheng Zheng (College of Electrical Engineering & New Energy, China Three Gorges University) ;
  • Senquan Yang (China Nuclear Industry Key Laboratory of Simulation Technology) ;
  • Guangrong Zhou (College of Science, China Three Gorges University) ;
  • Junjie He (College of Science, China Three Gorges University)
  • Received : 2023.06.16
  • Accepted : 2023.11.17
  • Published : 2024.04.25
Download PDF
⟨ Previous Next ⟩

Abstract

Due to increasing operational security demands, digital and intelligent condition monitoring of nuclear power plants is becoming more significant. However, establishing an accurate and effective anomaly detection model is still challenging. This is mainly because of data characteristics of nuclear power data, including the lack of clear class labels combined with frequent interference from outliers and anomalies. In this paper, we introduce a Transformer-based unsupervised model for anomaly detection of nuclear power data, a modified loss function based on the maximum correntropy criterion (MCC) is applied in the model training to improve the robustness. Experimental results on simulation datasets demonstrate that the proposed Trans-MCC model achieves equivalent or superior detection performance to the baseline models, and the use of the MCC loss function is proven can obviously alleviate the negative effect of outliers and anomalies in the training procedure, the F1 score is improved by up to 0.31 compared to Trans-MSE on a specific dataset. Further studies on genuine nuclear power data have verified the model's capability to detect anomalies at an earlier stage, which is significant to condition monitoring.

Keywords

Acknowledgement

This research was funded by the CNNC Key Laboratory of Nuclear Industry Simulation Technology External Open Fund Project (B220631).

References

  1. J. Ma, J. Jiang, Applications of fault detection and diagnosis methods in nuclear power plants: a review, Prog. Nucl. Energy 53 (3) (2011) 255-266. https://doi.org/10.1016/j.pnucene.2010.12.001
  2. D. Bailey, E. Wright, Practical SCADA for Industry, Newnes, Oxford, UK, 2003.
  3. E. Gursel, B. Reddy, A. Khojandi, M. Madadi, J.B. Coble, V. Agarwal, V. Yadav, R. L. Boring, Using artificial intelligence to detect human errors in nuclear power plants: a case in operation and maintenance, Nucl. Eng. Technol. 55 (2) (2023) 603-622. https://doi.org/10.1016/j.net.2022.10.032
  4. H. Kim, M.G. Na, G. Heo, Application of monitoring, diagnosis, and prognosis in thermal performance analysis for nuclear power plants, Nucl. Eng. Technol. 46 (6) (2014) 737-752. https://doi.org/10.5516/NET.04.2014.720
  5. M.G. Fernandez, K. Higley, A. Tokuhiro, K. Welter, W.K. Wong, H. Yang, Status of research and development of learning-based approaches in nuclear science and engineering: a review, Nucl. Eng. Des. 359 (2020), 110479.
  6. J. Contreras, R. Espinola, F.J. Nogales, A.J. Conejo, ARIMA models to predict next-day electricity prices, IEEE Trans. Power Apparatus Syst. 18 (3) (2003) 1014-1020. https://doi.org/10.1109/TPWRS.2002.804943
  7. B.N. Saha, N. Ray, H. Zhang, Snake validation: a PCA-based outlier detection method, IEEE Signal Process. Lett. 16 (6) (2009) 549-552. https://doi.org/10.1109/LSP.2009.2017477
  8. L.-J. Cao, K.-S. Chua, W.-K. Chong, H.-P. Lee, Q.-M. Gu, A comparison of PCA, KPCA and ICA for dimensionality reduction in support vector machine, Neurocomputing 55 (1-2) (2003) 321-336. https://doi.org/10.1016/S0925-2312(03)00433-8
  9. T. Kanungo, D.M. Mount, N.S. Netanyahu, C.D. Piatko, R. Silverman, A.-Y. Wu, An efficient k-means clustering algorithm: analysis and implementation, IEEE Trans. Pattern Anal. Mach. Intell. 24 (7) (2002) 881-892. https://doi.org/10.1109/TPAMI.2002.1017616
  10. T. Cover, P. Hart, in: Nearest Neighbor Pattern Classification, IEEE T. Inform. Theory, IT, 1967, pp. 21-27, 13.
  11. M.M. Breunig, H.P. Kriegel, R.T. Ng, J. Sander, LOF: identifying density-based local outliers, in: ACM SIGMOD Int. Conf. On Management of Data, 2000. Dalles, USA, May 15-18.
  12. A. Banerjee, P. Burlina, C. Diehl, A support vector method for anomaly detection in hyperspectral imagery, IEEE Trans. Geosci. Electron. 44 (8) (2006) 2282-2291. https://doi.org/10.1109/TGRS.2006.873019
  13. K. Hundman, V. Constantinou, C. Laporte, I. Colwell, T. Soderstrom, Detecting spacecraft anomalies using LSTMs and nonparametric dynamic thresholding, in: KDD 2018 Deep Learning Day, 2018. London, UK, August 19-23.
  14. Z.Z. Darban, G.I. Webb, S.-R. Pan, C.C. Aggarwal, M. Salehi, Deep Learning for Time Series Anomaly Detection: A Survey, 2022 arXiv: 2211.05244vol. 2.
  15. B. Zong, Q. Song, M.R. Min, W. Cheng, C. Lumezanu, D. Cho, H.-F. Chen, Deep autoencoding Gaussian mixture model for unsupervised anomaly detection, in: 6th International Conference on Learning Representations, 2018. Vancouver, Canada, April 30 - May 3.
  16. J. Audibert, P. Michiardi, F. Guyard, S. Marti, M.A. Zuluaga, USAD: unsupervised anomaly detection on multivariate time series, in: The 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (Virtual), 2020. July 6-10.
  17. Y. Su, Y.-J. Zhao, C.-H. Niu, R. Liu, W. Sun, D. Pei, Robust anomaly detection for multivariate time series through stochastic recurrent neural network, in: 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2019. Anchorage, USA, August 4-8.
  18. D. Li, D.-C. Chen, L. Shi, B.-H. Jin, J. Goh, S.K. Ng, MAD-GAN: multivariate anomaly detection for time series data with generative adversarial networks, in: 28th International Conference on Artificial Neural Networks, 2019. Munich, Germany, September 17-19.
  19. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A.N. Gomez, L. Kaiser, I. Polosukhin, Attention is all you need, in: The 31st International Conference on Neural Information Processing Systems, 2017. Long Beach, USA, December 4-8.
  20. J.-H. Xu, H.-X. Wu, J.-M. Wang, M.-S. Long, Anomaly Transformer: time series anomaly detection with association discrepancy, in: The 10th International Conference on Learning Representations (Virtual), 2022. April 25-29.
  21. X.-X. Wang, D.-C. Pi, X.-Y. Zhang, H. Liu, C. Guo, Variational transformer-based anomaly detection approach for multivariate time series, Measurement 191 (2022), 110791.
  22. S. Tuli, G. Casale, N.R. Jennings, TranAD: deep transformer networks for anomaly detection in multivariate time series data, in: 48th International Conference on Very Large Data Bases, 2022. Sydney, Australia, September 5-9.
  23. W.-F. Liu, P.P. Pokharel, J.C. Principe, Correntropy: a localized similarity measure, in: 2006 International Joint Conference on Neural Networks, 2006. Vancouver, Canada, July 16-21.
  24. N. Khentout, G. Magrotti, Fault supervision of nuclear research reactor systems using artificial neural networks: a review with results, Ann. Nucl. Energy 185 (2023), 109684.
  25. E.Y. Chow, A. S Willsky, Analytical redundancy and the design of robust failure detection systems, IEEE Trans. Automat. Control 29 (1984) 603-614. https://doi.org/10.1109/TAC.1984.1103593
  26. M. Basseville, Detecting changes in signals and systems-A survey, Automatica 24 (1988) 309-326. https://doi.org/10.1016/0005-1098(88)90073-8
  27. R. Isermann, Process fault detection based on modeling and estimation methods-A survey, Automatica 20 (1984) 387-404. https://doi.org/10.1016/0005-1098(84)90098-0
  28. R.V. Beard, Failure Accommodation in Linear Systems through Self-Reconfiguration, MIT, Cambridge, MA, USA, 1971.
  29. S.S. Godbole, Applications of kalman filtering technique to nuclear reactors, IEEE Trans. Nucl. Sci. 20 (1973) 661-667. https://doi.org/10.1109/TNS.1973.4326977
  30. N. Khentout, H. Salhi, G. Magrotti, D. Merrouche, Fault monitoring and accommodation of the heat exchanger parameters of Triga-Mark II nuclear research reactor using model-based analytical redundancy, Prog. Nucl. Energy 109 (2018) 97-112. https://doi.org/10.1016/j.pnucene.2018.02.019
  31. J.W. Hines, E. Davis, Lessons learned from the US nuclear power plant on-line monitoring programs, Prog. Nucl. Energy 46 (3-4) (2005) 176-189. https://doi.org/10.1016/j.pnucene.2005.03.003
  32. B.M. Wise, N.B. Gallagher, The process chemometrics approach to process monitoring and fault detection, J. Process Control 6 (1996) 329-348. https://doi.org/10.1016/0959-1524(96)00009-1
  33. J.W. Hines, R.E. Uhrig, D.J. Wrest, Use of autoassociative neural networks for signal validation, J. Intell. Rob. Syst. 21 (1998) 143-154. https://doi.org/10.1023/A:1007981322574
  34. N. Kaistha, B.R. Upadhyaya, Incipient fault detection and isolation of field devices in nuclear power systems using principal component analysis, Nucl. Technol. 136 (2001) 221-230. https://doi.org/10.13182/NT01-A3240
  35. N. Zavaljevskl, K.C. Gross, Sensor fault detection in nuclear power plants using multivariate state estimation technique and support vector machines, in: The Third International Conference of the Yugoslav, Nuclear Society, Belgrade, Yugoslavia, 2000. Oct 2-5.
  36. A. Papaoikonomou, J. Wingate, V. Verma, et al., Deep learning techniques for in-core perturbation identification and localization of time-series nuclear plant measurements, Ann. Nucl. Energy 178 (2022), 109373.
  37. S. Ryu, B. Jeon, H. Seo, et al., Development of deep autoencoder-based anomaly detection system for HANARO, Nucl. Eng. Technol. 55 (2) (2023) 475-483. https://doi.org/10.1016/j.net.2022.10.009
  38. J.C. Principe, Information Theoretic Learning: Renyi's Entropy and Kernel Perspectives, Springer, New York, 2010.
  39. P.P. Pokharel, W.-F. Liu, J.C. Principe, A low complexity robust detector in impulsive noise, Signal Process. 89 (10) (2009) 1902-1909. https://doi.org/10.1016/j.sigpro.2009.03.027
  40. X.-B. Chen, J. Yang, J. Liang, Q.-L. Ye, Recursive robust least squares support vector regression based on maximum correntropy criterion, Neurocomputing 97 (2012) 63-73. https://doi.org/10.1016/j.neucom.2012.05.004
  41. W.-C. Lu, J.-D. Duan, P. Wang, W.-T. Ma, S. Fang, Short-term wind power forecasting using the hybrid model of improved variational mode decomposition and maximum mixture correntropy long short-term memory neural network, Int. J. Elec. Power. 144 (2023), 108552.
  42. B.W. Silverman, Density Estimation for Statistics and Data Analysis, Chapman and Hall, London, 1986, pp. 48-49.
  43. S. Kollias, M. Yu, J. Wingate, et al., Machine learning for analysis of real nuclear plant data in the frequency domain, Ann. Nucl. Energy 177 (2022), 109293.
  44. X. Li, T. Huang, K. Cheng, et al., Research on anomaly detection method of nuclear power plant operation state based on unsupervised deep generative model, Ann. Nucl. Energy 167 (2022), 108785.
  45. H.-W. Xu, W.-X. Chen, N.-W. Zhao, Z.-Y. Li, J.-H. Bu, Z.-H. Li, Y. Liu, Y.J. Zhao, D. Pei, Y. Feng, J. Chen, Z.-G. Wang, H.-L. Qiao, Unsupervised anomaly detection via variational auto-encoder for seasonal KPIs in web applications, in: The 2018 World Wide Web Conference, April 23-27, 2018. Lyon, France.
  46. A. Siffer, P.A. Fouque, A. Termier, et al., Anomaly detection in streams with extreme value theory, in: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Halifax NS, 2017. Canada, August 13-17.