Improving the Productivity of Education Experiments: Lessons from a Randomized Study of Need-Based Financial Aid

Given scarce resources for evaluation, we recommend that education researchers more frequently conduct comprehensive randomized trials that generate evidence on how, why, and under what conditions interventions succeed or fail in producing effects. Recent experience evaluating a randomized need-based financial aid intervention highlights some of our arguments and guides our outline of the circumstances under which the examination of mechanisms and heterogeneous impacts is particularly important. Comprehensive experiments can enhance research productivity by increasing the number of theories both tested and generated and can advance policy and practice by exploring the conditions under which interventions will be most successful in scale up. Paradoxically, while the emphasis on average treatment effects is typically associated with efficiency-minded economists, we argue that the approach is often inefficient from the standpoints of science and policy. © 2012 Association for Education Finance and Policy