This paper addresses Markov Decision Processes over compact state and action spaces. We investigate the special case of linear dynamics and piecewise-linear and convex immediate costs for the average cost criterion. This model is very general and covers many interesting examples, for instance in...

Operational Research (OR) techniques have been applied, from the early stages of the discipline, to a wide variety of issues in education. At the government level, these include questions of what resources should be allocated to education as a whole and how these should be divided amongst the...

Stroke disease places a heavy burden on society, incurring long periods of time in hospital and community care, and associated costs. Also stroke is a highly complex disease with diverse outcomes and multiple strategies for therapy and care. Previously a modeling framework has been developed...

This paper provides a unified framework to study monotone optimal control for a class of Markov decision processes through D-multimodularity. We demonstrate that each system in this class can be classified as either a substitution-type or a complement-type system according to the possible...

This paper presents a methodology to find near-optimal joint inventory control policies for the real case of a one-warehouse, n-retailer distribution system of infusion solutions at a University Medical Center in France. We consider stochastic demand, batching and order-up-to level policies as...

This paper presents a basic formula for performance gradient estimation of semi-Markov decision processes (SMDPs) under average-reward criterion. This formula directly follows from a sensitivity equation in perturbation analysis. With this formula, we develop three sample-path-based gradient...

In this work the problem of obtaining an optimal maintenance policy for a single-machine, single-product workstation that deteriorates over time is addressed, using Markov Decision Process (MDP) models. Two models are proposed. The decision criteria for the first model is based on the cost of...

We study an (s,S) production inventory system where the processing of inventory requires a positive random amount of time. As a consequence a queue of demands is formed. Demand process is assumed to be Poisson, duration of each service and time required to add an item to the inventory when the...

In this paper we analyze the interval availability of a two-echelon, multi-item spare part inventory system. We consider a scenario inspired by a situation that we encountered at Thales Netherlands, a manufacturer of naval sensors and naval command and control systems. Modeling the complete...

The present paper proposes a non-homogeneous multivariate Markov manpower system in the general category of mathematical human resource planning. More specifically, we suggest a model, which takes into account the divisions existing in an organization categorizing its employees into several...