In recent years, industrial policy evaluation has been a theme under much debate internationally much of which has been spurred on through the significant EU structural fund transfers of the 1990s. Despite this, there are still few well-established methodologies to aid the evaluation process and this remains a serious issue with which researchers in this field continue to grapple. An important question in any evaluation is what would have happened in the absence of assistance. One of the key questions regarding the effectiveness of industrial policy is the extent to which growth in individual firms can be attributed to the financial assistance provided by the appropriate agency. In reality, all or part of the employment created for example might have come into existence anyway. This is referred to as deadweight. In this paper, two methodological approaches for measuring the impact of grant assistance are developed. The data employed in the evaluative frameworks were gathered during face-to-face interviews with the relevant personnel of 40 firms that received grants from Enterprise Ireland (EI) in the Republic of Ireland (ROI) in the years 2000 and 2001. An important dimension of the analysis will be to investigate whether there are regional deadweight effects in Ireland by structuring the analysis in terms of the Dublin Region, City Regions and Rural areas. Approach 1: The Case Study Approach This approach involved the use of a questionnaire during face-to-face interviews. The aim of this technique was to utilize the views of individuals (firms) directly affected by policy in an attempt to identify a 'counter-factual position'. To assess deadweight directly respondents were asked to answer the hypothetical question of what would most likely have happened if they had not received assistance from EI. This definition of deadweight accounts for the various degrees or levels of deadweight as measured by time location and scale. The paper thus distinguishes between the various degrees of deadweight ranging from 'pure' ('full') and 'partial' to 'zero' deadweight (a dimension which has been overlooked in similar research to date). Approach 2: The Control Group Approach A counter-factual position may be achieved with the use of control groups of non-assisted businesses in Ireland and elsewhere. This approach is based on the principle that any observed difference in the growth performance of assisted and non-assisted businesses can be used to construct estimates of the net additionality of public policy support to assisted businesses. The implication here is that the difference in growth rates, whether measured in terms of employment, turnover, productivity or profitability, can be interpreted as the scale of the impact of the policy intervention - in this case financial assistance by EI. The main problem, as articulated by Storey (2000) and Roper et al (2001) , is that to interpret the difference in the growth rates between assisted and a non-assisted control group as the scale of the impact of financial assistance is fraught with methodological difficulties. The solution is essentially the need to construct a methodology which seeks to explore whether any observed differences between assisted and non-assisted control groups of businesses are due to differences in the characteristics of these groups or can be directly attributed to the effects of assistance. Further, there is a need to accurately isolate the effects of 'selection' and 'assistance' on those assisted businesses that had grown faster than non-assisted businesses. For example, where it is found that assisted small businesses grew faster than non-assisted businesses it is not clear whether their faster growth reflects: · the benefits of assistance; · a tendency for faster growing firms to be keener to apply for assistance; · or, whether assistance was successfully targeted on faster growing firms Within the terms of reference for this study it was not possible to construct a suitable methodology to undertake the necessary econometric modelling to address the separation of 'selection' and 'assistance' effects. Nevertheless, whilst recognising the constraints of the approach we would, however, argue that the use of control groups in this case do provide an important element in the definition of the counter-factual policy position. Finally, a comparison of the results obtained using the two methodological approaches is provided. Of key concern here is to assess the value of proceeding with evaluations which rely solely on asking policy recipients what they would have done in the absence of assistance. Do the results of such studies provide similar or different results from approaches which rely on large-scale databases of assisted and non-assisted controls? Further, is there a need for more elaborate evaluation methodologies in order to guide the nature and scale of industrial policy as operated by development agencies such as EI? Although the evaluative frameworks developed in this paper have been 'tested' in an Irish context, the logic is clear and the evaluative frameworks have a much wider international applicability regarding the evaluation of industrial policy interventions. Industrial policy evaluation is very much in its infancy in Irish academic and policymaking circles. As the adoption of evaluative approaches in the Irish context is set to increase (largely EU driven), the findings of this paper should provide timely and valuable lessons to those charged with the task of carrying out such evaluations.
