CYCLIC STATIONARY PROBABILITY DISTRIBUTION OF SECOND ORDER MARKOV CHAINS AND ITS APPLICATIONS

In this paper, we define a system of cyclic stationary probability distribution equations for a second order Markov chain process in case that all states are independent each other, which improves the system of equations in [W. Li, and M.K. Ng, On the limiting probability distribution of a transition probability tensor, Linear and Multilinear Algebra. 62(2014): 362-385]. There are two applications for the new model. First, the proposed model can be seen as a rank-3 approximation of a second order Markov chain with non-independent states. Second, unlike the previous tensor model, if the fixed point algorithm for solving the new model is convergent, the second order Markov chain process in the independent state cyclic-converges. Furthermore, we investigate properties of the solutions for the proposed stationary equation.

Automated summary

This paper introduces a system of cyclic stationary probability distribution equations for a second order Markov chain process, assuming all states are independent, which improves upon existing equations. The new model has two applications and properties of the solutions for the proposed stationary equation are investigated.