A Rank-Order Test on the Statistical Performance of Neural Network Models for Regional Labor Market Forecasts

Using a panel of 439 German regions we evaluate and compare the performance of various Neural Network (NN) models as forecasting tools for regional employment growth. Because of relevant differences in data availability between the former East and West Germany, the NN models are computed separately for the two parts of the country. The comparisons of the models and their ex post forecasts are carried out by means of a non-parametric test: viz. the Friedman statistic. The Friedman statistic tests the consistency of model results obtained in terms of their rank order. Since there is no normal distribution assumption, this methodology is an interesting substitute for a standard analysis of variance. Furthermore, the Friedman statistic is indifferent to the scale on which the data are measured. The evaluation of the ex post forecasts suggests that NN models are generally able to correctly identify the fastest-growing and the slowest-growing regions, and hence predict rather well the correct ranking of regions in terms of their employment growth. The comparison among NN models – on the basis of several criteria – suggests that the choice of the variables used in the model may influence the model's performance and the reliability of its forecasts