The use of Markov processes (or Markov chains) has become widespread in dynamic stochastic modeling. For example, its use is ubiquitous in macroeconomics (dynamic stochastic general equilibrium), finance (dynamic asset pricing), and areas of microeconomics (dynamic programming). As we discuss...