We develop a novel ranking methodology to rank the market forecaster. In particular, we distinguish forecasts by their specificity, rather than considering all predictions and forecasts equally important, and we also analyze the impact of the number of forecasts made by a particular forecaster....

Many investors rely on market experts and forecasters when making investment decisions, such as when to buy or sell securities. Ranking and grading market forecasters provides investors with metrics on which they may choose forecasters with the best record of accuracy for their particular market...

In mathematical finance, backtest overfitting relates to the usage of historical market data (a backtest) to develop an investment strategy, where the strategy profits from random patterns rather than variables' signals. Backtest overfitting is now thought to be a primary reason why quantitative...

We carry out several test cases to illustrate how the Probability of Backtest Overfitting (PBO) performs under different scenarios. We also assess the accuracy of PBO using two alternative approaches (Monte Carlo Methods and Extreme Value Theory).The paper "The Probability of Backtest...

Most firms and portfolio managers rely on backtests (or historical simulations of performance) to select investment strategies and allocate them capital. Standard statistical techniques designed to prevent regression over-fitting, such as hold-out, tend to be unreliable and inaccurate in the...

We prove that high simulated performance is easily achievable after backtesting a relatively small number of alternative strategy configurations, a practice we denote “backtest overfitting”. The higher the number of configurations tried, the greater is the probability that the backtest is...

We demonstrate a computer program that designs a portfolio consisting of common securities, such as the constituents of the S&P 500 index, that achieves any desired profile via in-sample backtest optimization. Unfortunately, the program also shows that these portfolios typically perform...

We introduce two online backtest overfitting tools: BODT simulates the overfitting of seasonal strategies (typical of technical analysis), and TMST simulates the overfitting of econometric strategies (typical of academic journals). We show that econometric methods lend themselves to extreme...