In a number of applications, data may be anonymized, obfuscated, or highly noisy. In such cases, it is difficult to use domain knowledge or low-dimensional visualizations to engineer the features for tasks such as machine learning, instead, we explore dimensionality reduction (DR) as a...

Several features of financial research make it particularly prone to the occurrence of false discoveries. First, the probability of finding a positive (profitable investment strategy) is very low, due to intense competition. Second, true findings are mostly short-lived, as a result of the...

Investing can be characterized as a data science problem. While investment firms have attracted scientific talent, they have done a poor job at developing it. Firms hire specialists, but entice them to become generalists (e.g., portfolio managers). Under the ubiquitous silo/platform structure,...

Finance cannot become a rigorous science (in the Popperian or Lakatosian sense), however it can still operate as an “industrial science”. This article describes the scientific method by which industrial finance discovers through experimentation, and avoids false discoveries

The ‘flash crash' of May 6th 2010 was the second largest point swing (1,010.14 points) and the biggest one-day point decline (998.5 points) in the history of the Dow Jones Industrial Average. For a few minutes, $1 trillion in market value vanished. In this paper, we argue that the ‘flash...

A substitution effect takes place when two or more explanatory variables share a substantial amount of information (predictive power).Under the presence of substitution effects, feature importance methods may not be able to determine robustly which variables are significant.This presentation...

Many problems in finance require the clustering of variables or observations. Despite its usefulness, clustering is almost never taught in Econometrics courses. In this seminar we review two general clustering approaches: partitional and hierarchical

Two random variables are codependent when knowing the value of one helps us determine the value of the other. This should not me confounded with the notion of causality.Correlation is perhaps the best known measure of codependence in econometric studies. Despite its popularity among economists,...

I have divided this testimony into four sections, which discuss: (1) several types of automation currently being deployed in capital markets and the financial sector, and how they affect decision-making; (2) how machine learning (ML) and automation can help and hurt workers by disruption of the...

Machine learning (ML) is changing virtually every aspect of our lives. Today ML algorithms accomplish tasks that until recently only expert humans could perform. As it relates to finance, this is the most exciting time to adopt a disruptive technology that will transform how everyone invests for...