This work concerns controlled Markov chains with denumerable state space and discrete time parameter. The reward function is assumed to be≤0 and the performance of a control policy is measured by the expected total-reward criterion. Within this context, sufficient conditions are given so that...

Both the static and the dynamic single-leg revenue management problem are studied from the perspective of a risk-averse decision maker. Structural results well-known from the risk-neutral case are extended to the risk-averse case on the basis of an exponential utility function. In particular,...

For sequential decision processes with countable state spaces, we prove compactness of the set of strategic measures corresponding to nonrandomized policies. For the Borel state case, this set may not be compact (Piunovskiy, Optimal control of random sequences in problems with constraints....

In this paper we consider the weighted reward Markov decision process, with perturbation. The “weighted reward” refers to appropriately normalized convex combination of the discounted and the long-run average reward criteria. This criterion allows the controller to trade-off short-term costs...

We present a novel simulation-based algorithm, as an extension of the well-known policy iteration algorithm, by combining multi-policy improvement with a distributed simulation-based voting policy evaluation, for approximately solving Markov Decision Processes (MDPs) with infinite horizon...

We consider a problem of dynamic replenishment of parts in the supply chain consisting of single class of customers, company, and supplier. Customers request a service via the WEB-based ordering system and the company supports service using parts which are procured from the supplier. The...

This paper studies the discrete time Markov decision processes (MDP) with expected discounted total reward, where the state space is countable, the action space is measurable, the reward function is extended real-valued, and the discount rate may be any real number. Two conditions (GC) and (C)...

We consider discrete time Markov Decision Process (MDP) with finite state and action spaces under average reward optimality criterion. The decomposition theory, in Ross and Varadarajan [11], leads to a natural partition of the state space into strongly communicating classes and a set of states...

We propose a new approach to accelerate the convergence of the modified policy iteration method for Markov decision processes with the total expected discounted reward. In the new policy iteration an additional operator is applied to the iterate generated by Markov operator, resulting in a...

This paper addresses an optimal inventory control in a supply chain in which customers arrive at a facility according to a Poisson process and the facility provides service which takes exponential amounts of time, using items supplied by an outside supplier with exponential lead time process....