Categorical forecasts and non-categorical loss functions

This paper introduces a new test of the predictive performance and market timing for categorical forecasts based on contingency tables when the user has non-categorical loss functions. For example, a user might be interested in the return of an underlying variable instead of just the direction. This new test statistic can also be used to determine whether directional forecasts are derived from non-directional forecasts and whether point forecast have predictive value when transformed into directional forecasts. The tests are applied to the categorical exchange rate forecasts in the ifo-Institute's World Economic Survey and to the point forecasts for quarterly GDP in the Philadelphia Fed's Survey of Professional Forecasters. We find that the loss function matters as exchange rate forecasters perform better under non-categorical loss functions, and the GDP forecasts have value up to two quarters ahead.