Nuclear Engineering and Technology

Korean Nuclear Society (한국원자력학회)
권호 표지
DOI QR코드

DOI QR Code

Fault-tolerant control system for once-through steam generator based on reinforcement learning algorithm

  • Received : 2021.09.16
  • Accepted : 2022.04.18
  • Published : 2022.09.25
Download PDF
⟨ Previous Next ⟩

Abstract

Based on the Deep Q-Network(DQN) algorithm of reinforcement learning, an active fault-tolerance method with incremental action is proposed for the control system with sensor faults of the once-through steam generator(OTSG). In this paper, we first establish the OTSG model as the interaction environment for the agent of reinforcement learning. The reinforcement learning agent chooses an action according to the system state obtained by the pressure sensor, the incremental action can gradually approach the optimal strategy for the current fault, and then the agent updates the network by different rewards obtained in the interaction process. In this way, we can transform the active fault tolerant control process of the OTSG to the reinforcement learning agent's decision-making process. The comparison experiments compared with the traditional reinforcement learning algorithm(RL) with fixed strategies show that the active fault-tolerant controller designed in this paper can accurately and rapidly control under sensor faults so that the pressure of the OTSG can be stabilized near the set-point value, and the OTSG can run normally and stably.

Keywords

References

  1. Peiwei Sun, Xianshan Zhang, Modeling and simulation of feedwater control system with multiple once-through steam generators in a sodium-cooled fast reactor (II): control system simulation, Ann. Nucl. Energy 149 (2020), https://doi.org/10.1016/j.anucene.2020.107780.
  2. wei Li, minjun Peng, yongkuo Liu, Research on condition monitoring technique of sensors in NPPs based on PCA, Nucl. Power Eng. 39 (1) (2018) 136-139, https://doi.org/10.13832/j.jnpe.2018.01.0136.
  3. Simani Silvio, Zattoni Elena, Advanced control design and fault diagnosis, Energies 14 (18) (2021) 5699, https://doi.org/10.3390/en14185699.
  4. Alain Yetendje, Maria Seron, Jose De Don a, Robust multisensor fault tolerant model-following MPC design for constrained systems, Int. J. Appl. Math. Comput. Sci. 22 (1) (2012) 211-223, https://doi.org/10.2478/v10006-012-0016-7.
  5. Mogens Blanke, Michel Kinnaert, Lunze Jan, et al., Diagnosis and FaultTolerant Control, Springle Press, 2004.
  6. Wu Hn, H.Y. Zhang, Reliable fuzzy control for continuous-time nonlinear systems with actuator failures, IEEE Trans. Fuzzy Syst. 14 (5) (2006) 609-618, https://doi.org/10.1109/TFUZZ.2006.876723.
  7. M. Gopinathan, J.D. Boskovic, R.K. Mehra, et al., A Multiple Model Predictive Scheme for Fault-Tolerant Flight Control Design, in: IEEE Conference on Decision & Control, IEEE, Tampa, FL, USA, 1998, pp. 1376-1381.
  8. A. Paoli, M. Sartini, L. Stephane, Active fault tolerant control of discrete event systems using online diagnostics, Automatica 47 (4) (2011) 639-649, https://doi.org/10.1016/j.automatica.2011.01.007.
  9. Z. Wang, L. Liu, H. Zhang, et al., Fault-tolerant controller design for a class of nonlinear MIMO discrete-time systems via online reinforcement learning algorithm, IEEE Trans. Syst. Man Cybern. Syst. 46 (5) (2016) 611-622, https://doi.org/10.1109/TSMC.2015.2478885.
  10. Farivar Faezeh, Ahmadabadi Majid Nili, Continuous reinforcement learning to robust fault tolerant control for a class of unknown nonlinear systems, Appl. Soft Comput. 37 (2015) 702-714, https://doi.org/10.1016/j.asoc.2015.08.049.
  11. I.I. Belyakov, et al., Static instability of once-through steam generators with convective heating, Sov. Atom. Energy 56 (5) (1984) 347-350. https://doi.org/10.1007/BF01124329
  12. M. Osakabe, Thermal-hydraulic study of integrated steam generator in PWR, J. Nucl. Sci. Technol. 26 (2) (1989) 66-74. https://doi.org/10.1080/18811248.1989.9734304
  13. Ellen Namalomba, Hu Feihu, Haijie Shi, Agent based simulation of centralized electricity transaction market using bi-level and Q-learning algorithm approach, Int. J. Electr. Power Energy Syst. 134 (2022), https://doi.org/10.1016/J.IJEPES.2021.107415.
  14. Liu Wentao, Su Shuai, Tao Tang, Xi Wang, A DQN-based intelligent control method for heavy haul trains on long steep downhill section, Transport. Res. Part C 129 (2021), https://doi.org/10.1016/J.TRC.2021.103249.
  15. Li Haipeng, Huang Xiaojin, Zhang Liangju, A lumped parameter dynamic model of the helical coiled once-through steam generator with movable boundaries, Nucl. Eng. Des. 238 (7) (2008) 1657-1663, https://doi.org/10.1016/j.nucengdes.2008.01.009.
  16. D. Silver, A. Huang, C.J. Maddison, et al., Mastering the game of go with deep neural networks and tree search, Nature 529 (7587) (2016) 484-489, https://doi.org/10.1038/nature16961.