We consider a finite-state, finite-action, infinite-horizon, discounted reward Markov decision process and study the bias and variance in the value function estimates that result from empirical estimates of the model parameters. We provide closed-form approximations for the bias and variance,...

We consider a finite two-player zero-sum game with vector-valued rewards. We study the question of whether a given polyhedral set D is "approachable," that is, whether Player 1 (the "decision maker") can guarantee that the long-term average reward belongs to D, for any strategy of Player 2 (the...

Regret minimization in repeated matrix games has been extensively studied ever since Hannan's seminal paper [Hannan, J., 1957. Approximation to Bayes risk in repeated play. In: Dresher, M., Tucker, A.W., Wolfe, P. (Eds.), Contributions to the Theory of Games, vol. III. Ann. of Math. Stud., vol....

We propose simple randomized strategies for sequential prediction under imperfect monitoring, that is, when the forecaster does not have access to the past outcomes but rather to a feedback signal. The proposed strategies are consistent in the sense that they achieve, asymptotically, the best...

We provide yet another proof of the existence of calibrated forecasters; it has two merits. First, it is valid for an arbitrary finite number of outcomes. Second, it is short and simple and it follows from a direct application of Blackwell's approachability theorem to carefully chosen...

We consider a Cournot oligopoly model where multiple suppliers (oligopolists) compete by choosing quantities. We compare the social welfare achieved at a Cournot equilibrium to the maximum possible, for the case where the inverse market demand function is convex. We establish a lower bound on...

Deciding who should receive a mail-order catalog is among the most important decisions that mail-order-catalog firms must address. In practice, the current approach to the problem is invariably myopic: firms send catalogs to customers who they think are most likely to order from that catalog. In...

We study a single product inventory system with nonnegative setup cost in which the demand is a continuous random variable but orders are restricted to be integer valued. Optimal policies, when there are no setup costs, have a nice form. However, we show that, when the setup costs are nonzero,...

A problem that often arises in the process of searching for a job or for a candidate to fill a position is that applicants do not know if they will receive an offer from any given firm with which they interview, and, conversely, firms do not know whether applicants will definitely take positions...

Includes bibliographical references (p. 48-50).