We consider a cross-calibration test of predictions by multiple potential experts in a stochastic environment. This test checks whether each expert is calibrated conditional on the predictions made by other experts. We show that this test is good in the sense that a true expert - one informed of...

Abstract A nonparametric method for comparing multiple forecast models is developed and implemented. The hypothesis of Optimal Predictive Ability generalizes the Superior Predictive Ability hypothesis from a single given loss function to an entire class of loss functions. Distinction is drawn...

This paper introduces the model confidence set (MCS) and applies it to the selection of models. An MCS is a set of models that is constructed so that it will contain the best model with a given level of confidence. The MCS is in this sense analogous to a confidence interval for a parameter. The...

In this paper, we compare two fundamentally different judgmental demand forecasting approaches used to estimate demand and their corresponding demand distributions. In the first approach, parameters are obtained from a linear regression and maximum likelihood estimation (MLE) based on team...

This paper uses a normative method to compare the performance of the composite leading economic indicators (CLI) after the measure was revised by the Conference Board in 1996 and 2001 with its prior design to check for a claim that the new design improves its performance in predicting a downturn...

We develop a forecasting method for the manufacturer and online seller of a product collection that changes periodically and radically. The firm, an industry leader in technology and quality, has experienced double-digit annual sales growth. In seeking to minimize supply-demand mismatch costs...

This paper applies the model confidence sets (MCS) procedure to a set of volatility models. A MSC is analogous to a confidence interval of parameter in the sense that the former contains the best forecasting model with a certain probability. The key to the MCS is that it acknowledges the...

This paper presents a methodology for producing a probability forecast of a turning point in U.S. economy using Composite Leading Indicators. This methodology is based on classical statistical decision theory and uses information-theoretic measurement to produce a probability. The methodology is...

Predictions about the future are commonly evaluated through statistical tests. As shown by recent literature, many known tests are subject to adverse selection problems and cannot discriminate between forecasters who are competent and forecasters who are uninformed but predict strategically. We...

Forecasts are useless whenever the forecast error variance fails to be smaller than the unconditional variance of the target variable. This paper develops tests for the null hypothesis that forecasts become uninformative beyond some limiting forecast horizon h. Following Diebold and Mariano (DM,...