Why Some Acute Health Effects of Air Pollution Could Be Inflated

Hundreds of studies have shown that air pollution affects health in the very short-run. This played a key role in setting air quality standards. Yet, estimated effect sizes can vary widely across studies. Analyzing the results published in epidemiology and economics, we find that publication bias and a lack of statistical power could lead some estimates to be inflated. We then run real data simulations to identify the design parameters causing these issues. We show that this exaggeration may be driven by a small numbers of exogenous shocks, instruments with limited strength or sparse outcomes. Other literatures relying on comparable research design could also be affected by these issues. Our paper provides a principled workflow to evaluate and avoid the risk of exaggeration when conducting an observational study.