It is showed that the magnitude of the initial condition of a highly persistent process affects the properties of stationarity tests applied to the first differences, with a "relatively large" initial condition implying non-negligible oversizing. An empirical illustration is provided.