Steady-State Approximation for a Vector Valued Markov Chain

The problem addressed is that of a condensed steady-state solution for the Markov Chain (X(t), S(t)). The steady state marginal distribution of S(t) is known; we desire only the steady state marginal distribution of X(t). Such a case frequently arises when the supplementary random variable S(t) is required in the state description solely to satisfy the Markovian assumption. An iterative algorithm is presented which makes use of an approximation to the conditional distribution for S(t) given X(t).