A typical empirical study involves choosing a sample, a research design, and an analysis path. Variation in such choices across studies leads to heterogeneity in results that introduce an additional layer of uncertainty not accounted for in reported standard errors and condence intervals. We...

Placebo tests, where a null result is used to support the validity of the research design, is common in economics. Such tests provide an incentive to underreport statistically significant tests, a form of reversed p-hacking. Based on a pre-registered analysis plan, we test for such...

A fundamental question to the scientific enterprise is to what extent published scientific findings are credible. This question is related to the reproducibility and replicability of scientific findings where reproducibility is defined as testing if the results of an original study can be...

This article reviews and summarizes current reproduction and replication practices in political science. We first provide definitions for reproducibility and replicability. We then review data availability policies for 28 leading political science journals and present the results from a survey...

Robustness reproductions and replicability discussions are on the rise in response to concerns about a potential credibility crisis in economics. This paper proposes a protocol to structure reproducibility and replicability assessments, with a focus on robustness. Starting with a computational...

We estimate the robustness reproducibility of key results from 17 non-experimental AER papers published in 2013 (8 papers) and 2022/23 (9 papers). We find that many of the results are not robust, with no improvement over time. The fraction of significant robustness tests (p0.05) varies between...

This study pushes our understanding of research reliability by reproducing and replicating claims from 110 papers in leading economic and political science journals. The analysis involves computational reproducibility checks and robustness assessments. It reveals several patterns. First, we...

Naidu and Yuchtman (2013) find that labor demand shocks in 19th-century Britain had an impact on master and servant prosecutions, as breaking an employee contract was a criminal offense until 1875. We first reproduce all regression tables in Naidu and Yuchtman (2013) and then test for robustness...

Berger, Easterly, Nunn and Satyanath (2013) find that increased US political influence, arising from Cold War interventions, was used to create a larger export market for American products. They find that after CIA interventions, US imports increased dramatically, and the authors rule out other...

Cloyne (2013) constructs a novel dataset documenting fiscal tax shocks in the United Kingdom using the narrative approach developed by Romer and Romer (2010), and estimates the impact of tax changes on GDP. He finds that a tax cut of one percent of GDP causes a 0.6 percent increase in output in...