We introduce two online backtest overfitting tools: BODT simulates the overfitting of seasonal strategies (typical of technical analysis), and TMST simulates the overfitting of econometric strategies (typical of academic journals). We show that econometric methods lend themselves to extreme...

In mathematical finance, backtest overfitting relates to the usage of historical market data (a backtest) to develop an investment strategy, where the strategy profits from random patterns rather than variables' signals. Backtest overfitting is now thought to be a primary reason why quantitative...

We carry out several test cases to illustrate how the Probability of Backtest Overfitting (PBO) performs under different scenarios. We also assess the accuracy of PBO using two alternative approaches (Monte Carlo Methods and Extreme Value Theory).The paper "The Probability of Backtest...

Proofs to the propositions in "Stop-Outs Under Serial Correlation".The paper "Stop-Outs Under Serial Correlation and 'The Triple Penance Rule" to which these Appendices apply is available at the following URL: "http://ssrn.com/abstract=2201302" http://ssrn.com/abstract=2201302

In the field of mathematical finance, a “backtest” is the usage of historical market data to assess the performance of a proposed trading strategy. It is a relatively simple matter for a present-day computer system to explore thousands, millions or even billions of variations of a proposed...

We prove that high simulated performance is easily achievable after backtesting a relatively small number of alternative strategy configurations, a practice we denote “backtest overfitting”. The higher the number of configurations tried, the greater is the probability that the backtest is...

The False Strategy theorem tells us that the optimal outcome of an unknown number of historical simulations is right-unbounded — with enough trials, there is no Sharpe ratio sufficiently enough to reject the hypothesis that a strategy is false. Given the ease with which one can use a computer...

Many investors rely on market experts and forecasters when making investment decisions, such as when to buy or sell securities. Ranking and grading market forecasters provides investors with metrics on which they may choose forecasters with the best record of accuracy for their particular market...

Several features of financial research make it particularly prone to the occurrence of false discoveries. First, the probability of finding a positive (profitable investment strategy) is very low, due to intense competition. Second, true findings are mostly short-lived, as a result of the...

Most firms and portfolio managers rely on backtests (or historical simulations of performance) to select investment strategies and allocate them capital. Standard statistical techniques designed to prevent regression over-fitting, such as hold-out, tend to be unreliable and inaccurate in the...