This paper addresses an optimal inventory control in a supply chain in which customers arrive at a facility according to a Poisson process and the facility provides service which takes exponential amounts of time, using items supplied by an outside supplier with exponential lead time process....

In this paper we consider a singularly perturbed Markov decision process with finitely many states and actions and the limiting expected average reward criterion. We make no assumptions about the underlying ergodic structure. We present algorithms for the computation of a uniformly optimal...

We consider the problem of control for continuous time stochastic hybrid systems in finite time horizon. The systems considered are nonlinear: the state evolution is a nonlinear function of both the control and the state. The control parameters change at discrete times according to an underlying...

In this paper we consider Markov Decision Processes with discounted cost and a random rate in Borel spaces. We establish the dynamic programming algorithm in finite and infinity horizon cases. We provide conditions for the existence of measurable selectors. And we show an example of...

One-armed bandit processes with continuous delayed responses are formulated as controlled stochastic processes following the Bayesian approach. It is shown that under some regularity conditions, a Gittins-like index exists which is the limit of a monotonic sequence of break-even values...

We consider Markov Decision Processes under light traffic conditions. We develop an algorithm to obtain asymptotically optimal policies for both the total discounted and the average cost criterion. This gives a general framework for several light traffic results in the literature. We illustrate...

This paper considers an assembly system where a firm produces a single product which is assembled using two types of components (component 1 and component 2). The components are provided by individual suppliers (supplier 1 and supplier 2). We assume that the firm makes different procurement...

The control problem of controlling ruin probabilities by investments in a financial market is studied. The insurance business is described by the usual Cramer-Lundberg-type model and the risk driver of the financial market is a compound Poisson process. Conditions for investments to be...

This work concerns finte-state Markov decision chains endowed with the long-run average reward criterion. Assuming that the optimality equation has a solution, it is shown that a nearly optimal stationary policy, as well as an approximation to the optimal average reward within a specified error,...

This paper deals with denumerable-state continuous-time controlled Markov chains with possibly unbounded transition and reward rates. It concerns optimality criteria that improve the usual expected average reward criterion. First, we show the existence of average reward optimal policies with...