This paper studies a Monte Carlo algorithm for computing distributions of state variables when the underlying model is a Markov process. It is shown that the $L_1$ error of the estimator always converges to zero with probability one, and often at a parametric rate. A related technique for...