[KSCI] Korea Science Citation Index Service

http://dx.doi.org/10.5351/CSAM.2014.21.6.513

Numerical Iteration for Stationary Probabilities of Markov Chains

Na, Seongryong (Department of Information and Statistics, Yonsei University)

Publication Information

Communications for Statistical Applications and Methods / v.21, no.6, 2014 , pp. 513-520 More about this Journal

Abstract

We study numerical methods to obtain the stationary probabilities of continuous-time Markov chains whose embedded chains are periodic. The power method is applied to the balance equations of the periodic embedded Markov chains. The power method can have the convergence speed of exponential rate that is ambiguous in its application to original continuous-time Markov chains since the embedded chains are discrete-time processes. An illustrative example is presented to investigate the numerical iteration of this paper. A numerical study shows that a rapid and stable solution for stationary probabilities can be achieved regardless of periodicity and initial conditions.

Keywords

Markov chain; embedded chain; periodicity; power method; stationary probability; numerical iteration; balance equation;

Citations & Related Records

Times Cited By KSCI : 1 (Citation Analysis)

Reference
Cited By KSCI

1	Ross, S. M. (1996). Stochastic Processes, 2nd edition, Wiley, New York.
2	Na, S. (2010). Markov modeling of multiclass loss systems, The Korean Journal of Applied Statistics, 23, 747-757. 과학기술학회마을 DOI
3	Nesterov, Y. and Nemirovski, A. (2014). Finding the stationary states of Markov chains by iterative methods, Applied Mathematics and Computation, Available from: http:/www.sciencedirect.com/science/article/pii/s0096300314005931, In press.
4	Stewart, W. J. (2000). Numerical methods for computing stationary distributions of finite irreducible Markov chains, Computational Probability (W. K. Grassmann (ed.)), International series in operations research and management science, 24, 81-111, Springer, New York. DOI
5	Zhao, D., Li, H. and Su, D. (2012). A numerical algorithm on the computation of the stationary distribution of a discrete time homogeneous finite Markov chain, Mathematical Problems in Engineering, 2012, Article ID 167453, 10 pages.
6	O'Leary, D. P. (1993). Iterative methods for finding the stationary vector for Markov chains, Linear Algebra, Markov Chains, and Queueing Models (C. D. Meyer and R. J. Plemmons (ed.)), The IMA volumes in mathematics and its applications, 48, 125-136, Springer, New York.