This paper provides series expansions of the stationary distribution of a finite Markov chain. This leads to an efficient numerical algorithm for computing the stationary distribution of a finite Markov chain. Numerical examples are given to illustrate the performance of the algorithm.

This paper provides strong bounds on perturbations over a collection of independent random variables, where ‘strong’ has to be understood as uniform w.r.t. some functional norm. Our analysis is based on studying the concept of weak differentiability. By applying a fundamental result from the...

This paper provides strong bounds on perturbations over a collection of independent random variables, where ‘strong’ has to be understood as uniform w.r.t. some functional norm. Our analysis is based on studying the concept of weak differentiability. By applying a fundamental result from the...

This paper provides series expansions of the stationary distribution of a finite Markov chain. This leads to an efficient numerical algorithm for computing the stationary distribution of a finite Markov chain. Numerical examples are given to illustrate the performance of the algorithm.

In this paper, we consider deterministic (both fluid and discrete) polling systems with N queues with infinite buffers and we show how to compute the best polling sequence (minimizing the average total workload). With two queues, the best polling sequence is always periodic when the system is...

In this paper we analyse a closed queueing network in which customers have to be assigned to parallel queues. The routing decision may not depend on the numbers of customers in the queues. We present an algorithm and we show that it computes an average optimal policy in case of exponential...

In this companion paper of [10] we introduce the combinatorial notion of unbalance for a routing pattern. Using this unbalance we derive an upper bound for the total average expected waiting time of jobs which are routed to parallel queues according to a periodic routing rule. A billiard...

In this paper we introduce the combinatorial notion of unbalance for a periodic zero-one splitting sequence. Using this unbalance we derive an upper bound for the average expected waiting time of jobs which are routed to one queue according to a periodic zero-one splitting sequence. In the...

This paper is the first part of a study of Blackwell optimal policies in Markov decision chains with a Borel state space and unbounded rewards. We prove here the existence of deterministic stationary policies which are Blackwell optimal in the class of all, in general randomized, stationary...