Model risk in backtesting risk measures

Under the Basel II regulatory framework non-negligible statistical problems arise when backtesting risk measures. In this setting backtests often become infeasible due to a low number of violations leading to heavy size distortions. According to Escanciano and Olmo (2010, 2011) these problems persist when incorporating estimation and model risk by adjusting the asymptotic variance of the test statistics. In this paper, we analyze backtests based on hit and duration sequences in a univariate framework by running a simulation study in order to identify the problems of backtests that examine the adequacy of Value at Risk measures. One main finding indicates that backtests of all classes show heavy size distortions. These problems for the relevant Basel II set-up, however, cannot be alleviated by modifying backtests in a way that accounts for estimation risk or misspecification risk.