The Korean Journal of Applied Statistics (응용통계연구)

The Korean Statistical Society (한국통계학회)
권호 표지
DOI QR코드

DOI QR Code

A study on the accuracy of a numerical iteration for Markov processes by using reliability models

신뢰도 모형을 이용한 마코프 과정의 수치적 반복법의 정확성에 대한 연구

  • Hyeonah Park (Division of Data Science, Yonsei University) ;
  • Seongryong Na (Division of Data Science, Yonsei University)
  • 박현아 (연세대학교 데이터사이언스학부) ;
  • 나성룡 (연세대학교 데이터사이언스학부)
  • Received : 2024.02.06
  • Accepted : 2024.04.11
  • Published : 2024.08.31
Download PDF
⟨ Previous Next ⟩

Abstract

For Markov processes whose stationary probabilities are difficult to obtain in the analytical form, approximate solutions can be considered using numerical methods such as a matrix operation method or an iterative calculation method. In this paper we perform the study to verify the accuracy of a numerical iteration formula which calculate the stationary probabilities of Markov chains or processes. Especially, the convergence and accuracy of the numerical method are investigated by using Markov models for system availability. We compare the values of the system availability based on the numerical calculation and those based on the complicated but analytical solutions. We also calculate the iteration numbers necessary for the convergence of the numerical solutions. The accuracy and usefulness of the numerical iterative calculation method can be ascertained through this study.

해석적 형태의 정상확률을 얻기 어려운 마코프 과정에 대하여 행렬 연산 방법 또는 반복 계산 방법 등의 수치적 방법을 이용한 근사 해를 고려할 수 있다. 이 논문에서는 마코프 체인 또는 마코프 과정의 정상확률을 계산하는 수치적 반복 공식의 정확성을 규명하는 연구를 수행한다. 특별히 시스템 가용도를 위한 마코프 모형을 이용하여 수치적 방법의 수렴과 정확성을 검토한다. 수치적 계산에 의한 시스템 가용도와 복잡하지만 해석적 수식에 의한 시스템 가용도를 비교한다. 그리고 수치적 해의 수렴에 필요한 반복 회수를 조사한다. 이 연구를 통하여 수치적 반복 계산 방법의 정확성과 유용성을 확인할 수 있다.

Keywords

References

  1. Na S (2011). Reliability analysis of repairable systems considering failure detection equipments, The Korean Journal of Applied Statistics, 24, 515-521.
  2. Na S (2014). Numerical iteration for stationary probabilities of Markov chains, Communications for Statistical Applications and Methods, 21, 513-520.
  3. Na S and Bang S (2013). The effect of failure detection equipment on system reliability, The Korean Journal of Applied Statistics, 26, 111-118.
  4. Na S and Bang S (2022). Repair policies of failure detection equipments and system availability, Communications for Statistical Applications and Methods, 29, 151-160.
  5. Nesterov Y and Nemirovski A (2015). Finding the stationary states of Markov chains by iterative methods, Applied Mathematics and Computation, 255, 58-65.
  6. O'Leary DP (1993). Iterative methods for finding the stationary vector for Markov chains, In Linear Algebra, Markov Chains, and Queueing Models (Meyer CD and Plemmons RJ (Eds)), The IMA Volumes in Mathematics and Its Applications, 48, (pp.125-136), Springer, New York.
  7. Ross SM (1996). Stochastic Processes (2nd ed), Wiley, New York.
  8. Stewart WJ (2000). Numerical methods for computing stationary distributions of finite irreducible Markov chains, In Computational Probability (Grassmann WK (Eds)), International Series in Operations Research and Management Science, 24, (pp. 81-111), Springer, New York.
  9. Zhao D, Li H, and Su D (2012). A numerical algorithm on the computation of the stationary distribution of a discrete time homogeneous finite Markov chain, Mathematical Problems in Engineering, 2012, 1-10.