This paper addresses an optimal inventory control in a supply chain in which customers arrive at a facility according to a Poisson process and the facility provides service which takes exponential amounts of time, using items supplied by an outside supplier with exponential lead time process....

In this paper we consider a singularly perturbed Markov decision process with finitely many states and actions and the limiting expected average reward criterion. We make no assumptions about the underlying ergodic structure. We present algorithms for the computation of a uniformly optimal...

Both the static and the dynamic single-leg revenue management problem are studied from the perspective of a risk-averse decision maker. Structural results well-known from the risk-neutral case are extended to the risk-averse case on the basis of an exponential utility function. In particular,...

We consider the problem of control for continuous time stochastic hybrid systems in finite time horizon. The systems considered are nonlinear: the state evolution is a nonlinear function of both the control and the state. The control parameters change at discrete times according to an underlying...

One-armed bandit processes with continuous delayed responses are formulated as controlled stochastic processes following the Bayesian approach. It is shown that under some regularity conditions, a Gittins-like index exists which is the limit of a monotonic sequence of break-even values...

We consider Markov Decision Processes under light traffic conditions. We develop an algorithm to obtain asymptotically optimal policies for both the total discounted and the average cost criterion. This gives a general framework for several light traffic results in the literature. We illustrate...

We present a novel simulation-based algorithm, as an extension of the well-known policy iteration algorithm, by combining multi-policy improvement with a distributed simulation-based voting policy evaluation, for approximately solving Markov Decision Processes (MDPs) with infinite horizon...

We consider a problem of dynamic replenishment of parts in the supply chain consisting of single class of customers, company, and supplier. Customers request a service via the WEB-based ordering system and the company supports service using parts which are procured from the supplier. The...

This work concerns finte-state Markov decision chains endowed with the long-run average reward criterion. Assuming that the optimality equation has a solution, it is shown that a nearly optimal stationary policy, as well as an approximation to the optimal average reward within a specified error,...

We consider discrete time Markov Decision Process (MDP) with finite state and action spaces under average reward optimality criterion. The decomposition theory, in Ross and Varadarajan [11], leads to a natural partition of the state space into strongly communicating classes and a set of states...