Treatment of Outliers in Business Surveys: The Case of Short-term Economic Survey of Enterprises in Japan (Tankan)

This paper discusses the appropriate treatment of outliers in Tankan. Under the current population estimate method, if one of the data which greatly diverges from the rest of the data (the so-called "outlier") does not represent the population, the estimates could also greatly diverge from the real value for the population. Since the real picture of the population when the outlier occurred is unknown, it is difficult to choose the optimum method through empirical analyses. Considering 1) the accuracy of Tankan, 2) the maintenance of its "easy to understand" quality, and 3) the reduction of its work load, we assume that outliers in Tankan are those which have extremely large impact on the rate of change from the previous year or the rate of revision from the previous survey of the estimated values for the population. The method of detecting and treating outliers is suitable for the characteristics of Tankan through empirical analyses of actual Tankan data. Specifically, detection method is based on the adjusted "range edit" method, which uses 1 percentile and 99 percentile, so that it can be applied to non-parametric data, for which the incidence of "0" (zero) is very high. The outliers thus detected are regarded as missing values and should be augmented by applying the missing value imputation, which is already being used for Tankan.