Data Issues in Developing Valid ROI Estimates

This paper discusses issues associated with using readily available administrative data in estimating ROI for vocational rehabilitation services. It starts with a discussion of longitudinal outcomes data. The discussion is divided up into labor market outcomes data, other types of outcomes data, necessary sample sizes (power analysis), and ways to deal with people systematically excluded from the outcomes data. Next, the paper focuses on services data. The topics covered include the need for control groups, using service cohort data, different sources of service, and merging service data with outcomes data. Finally, the paper moves to the need for other controlling explanatory variables including discussions of inclusion of demographic explanatory variables and data from local labor markets. Two online appendices to this paper provide additional details through (a) an example of a power analysis to illustrate sample size issues and (b) a discussion of Institutional Review Board issues associated with conducting empirical investigations using administrative data