We propose a new test for the stability of parameters in a Markov switching model where regime changes are driven by an unobservable Markov chain. Testing in this context is more challenging than testing in structural change and threshold models because, besides the presence of nuisance parameters that are not identified under the null hypothesis, there is the additional difficulty due to the singularity of the information matrix under the null. We derive a class of information matrix-type tests and show that they are equivalent to the likelihood ratio test. Hence, our tests are asymptotically optimal. Besides their optimality properties, these tests are more general than the competing tests proposed by Garcia (1998) and Hansen (1992). Indeed, the underlying Markov chain driving the regime changes may have a finite or continuous state space, as long as it is exogenous. It is not restricted to linear models either. Therefore, our technique applies for instance to testing stability in random coefficient GARCH models. We use this test to investigate the presence of rational collapsing bubbles in stock markets. There is bubble if the stock price is disconnected from the market fundamental value. We regress the stock price on dividends and use the residual as proxy for the bubble size. Using US data, we find that the residuals are stationary, which could be hastily interpreted as evidence against the presence of bubbles. However, our Markov switching test strongly rejects the linearity, suggesting that at least two regimes should be used to fit the data. Estimating a two-state Markov switching model (Hamilton, 1989) reveals that one regime has a unit root, while the other is mean reverting, which is consistent with periodically collapsing bubbles