Comparing Treatments across Labor Markets: An Assessment of Nonexperimental Multiple-Treatment Strategies

We consider the problem of using data from several programs, each implemented at a different location, to compare what their effect would be if they were implemented at a specific location. In particular, we study the effectiveness of nonexperimental strategies in adjusting for differences across comparison groups arising from two sources. First, we adjust for differences in the distribution of individual characteristics simultaneously across all locations by using unconfoundedness-based and conditional difference-in-difference methods for multiple treatments. Second, we explicitly adjust for differences in local economic conditions. We stress the importance of analyzing the overlap of, and adjusting for, local economic conditions after program participation. Our results suggest that the strategies studied are valuable econometric tools for the problem we consider, as long as we adjust for a rich set of individual characteristics and have sufficient overlap across locations for both individual and local labor market characteristics. Our results show that the overlap analysis of these two sets of variables is critical for identifying non-comparable groups and they illustrate the difficulty of adjusting for local economic conditions that differ greatly across locations.