We have seen in the past decade a sharp increase in the extent that companies use data to optimize their businesses. Variously called the `Big Data' or `Data Science' revolution, this has been characterized by massive amounts of data, including unstructured and nontraditional data like text and...

The last 40 years have seen huge innovations in computing technology and data availability. Data derived from millions of administrative records or by using (as we do) new methods of data generation such as text mining are now common. New data often requires new methods, which in turn can...

We study a model where firms accumulate data as a valuable intangible asset. Data accumulation affects firms' dynamics. It increases the skewness of the firm size distribution as large firms generate more data and invest more in active experimentation. On the other hand, small data- savvy firms...

Policymakers can take actions to prevent local conflict before it begins, if such violence can be accurately predicted. We examine the two countries with the richest available sub-national data: Colombia and Indonesia. We assemble two decades of fine-grained violence data by type, alongside...

A key challenge for research on many questions in the social sciences is that it is difficult to link historical records in a way that allows investigators to observe people at different points in their life or across generations. In this paper, we develop a new approach that relies on millions...

We document the degree of price dispersion and the similarities as well as differences in pricing and promotion strategies across stores in the U.S. retail (grocery) industry. Our analysis is based on "big data" that allow us to draw general conclusions based on the prices for close to 50,000...

Text data is ultra-high dimensional, which makes machine learning techniques indispensable for textual analysis. Text is often selected--journalists, speechwriters, and others craft messages to target their audiences' limited attention. We develop an economically motivated high dimensional...

Modern investors face a high-dimensional prediction problem: thousands of observable variables are potentially relevant for forecasting. We reassess the conventional wisdom on market efficiency in light of this fact. In our model economy, which resembles a typical machine learning setting, N...

This paper combines a data rich environment with a machine learning algorithm to provide new estimates of time-varying systematic expectational errors ("belief distortions") embedded in survey responses. We find that distortions are large on average even for professional forecasters, with all...

In this paper we present and describe a large quarterly frequency, macroeconomic database. The data provided are closely modeled to that used in Stock and Watson (2012a). As in our previous work on FRED-MD, our goal is simply to provide a publicly available source of macroeconomic "big data"...