100641 1 Finance & PSD Impact NOVEMBER 2015 The Lessons from DECFP Impact Evaluations ISSUE 34 Our latest note provides evidence from the first successfully implemented randomized experiment of a World Bank matching grant program, showing how it can spur innovation. Spurring Innovation with Matching Grants: Evidence from Yemen David McKenzie, Nabila Assaf and Ana Paula Cusolito Matching grants are one of the most online or in paper; and the program was well- common tools used in private sector advertised. development programs in developing countries and have been included in more High demand for the grants than 60 World Bank projects totaling over The combination of an easy US$1.2 billion, funding over 100,000 micro, application process and the need among firms small and medium enterprises. for the program meant that low demand was Yet public subsidies to private firms not an issue in Yemen. In total 820 can be controversial, and credible evidence as applications were received, slightly more to whether these grants induce firms to than four times the number of grants undertake innovative activities that they available for the first round (200). would not otherwise do (additionality) is Firms were selected for the program limited. Seven previous attempts to use from among the eligible applicants in public randomized experiments to evaluate World randomization events held in January 2014. Bank matching grant programs in sub- A reserve list was chosen in case originally Saharan Africa failed, raising questions about selected firms withdrew from the program, the way these projects are implemented in giving a treatment group size of 216 firms. practice and whether it is feasible to use We chose a random sample of 200 of the experimental methods to evaluate them applicants to be the control group for the (Campos et al, 2014). purpose of follow-up surveys. The median firm has 5 employees, but A matching grant for innovation in Yemen there is considerable variability in firm size. The Enterprise Revitalization and The most common industries are retail sales, Employment Pilot (EREP) was designed as a food production, clothing, education, and two year pilot project aimed at improving computer services. firm capabilities and the employability of recent graduates. The matching grant Data Collection component provided firms with a matching Unfortunately civil war broke out in grant of up to $10,000 as a 50 percent subsidy Yemen towards the start of 2015. This lead to towards the cost of innovation and business the second year of the program being services like finance and accounting systems, canceled. In order to measure short-term website creation, training, marketing, impacts from the first year we quickly participation in exhibitions, and some conducted a phone follow-up survey of firms associated goods. in March 2015. The response rate was 54%, The program implementation was with the security situation and gas shortages designed with the lessons of other matching preventing further follow-up. However the grant programs in mind in order to overcome treatment and control group interviewed are their problems with take-up: eligibility similar in terms of baseline observable criteria were kept broad; the application form characteristics, enabling comparison of the was not complex and could be done either two groups. Do you have a project you want evaluated? DECRG-FP researchers are always looking for opportunities to work with colleagues in the Bank and IFC. If you would like to ask our experts for advice or to collaborate on an evaluation, contact us care of the Impact editor, David McKenzie (dmckenzie@worldbank.org) Results The matching grants do appear to have resulted in additional innovation activities being undertaken by firms. Figure 1 shows that firms receiving the grants were 30.3 percentage points (p.p.) more likely to introduce a new product, which is more than a doubling of the 26.5 percent rate in the control group; they do more marketing (19.8 p.p. increase), introduce new accounting systems (48.5 p.p. increase), and overall are 37.1 p.p. more likely to have undertaken at least one innovative activity. Firms are also more likely to have made a capital investment or used a consultant. However, we do not have sufficient statistical power to detect impacts on employment or sales. Figure 1: The Matching Grants Increased Innovation 1.2 1 0.8 Proportion of Firms 0.6 0.4 0.2 0 Introduced new Introduced new Marketing New Accounting Any innovation product process campaign system Control Treatment Note: 95% confidence interval shown on LATE estimate of treatment effect. Policy Implications While there are several important caveats in this analysis due to the short time-frame of follow-up, attrition, and sample size, the evaluation still does provide some encouraging news for learning about matching grants. It demonstrates that attention in design to making the application process better for firms can lead to high take-up and make it feasible to evaluate through a randomized oversubscription design. In this context at least, these grants don’t just appear to be subsidizing firms to do what they would plan on doing anyway. Further evaluation of such programs on a larger scale is needed to measure other key outcomes such as employment and sales, and whether the grants also involve externalities for other firms. For further reading see: McKenzie, David, Nabila Assaf and Ana Paula Cusolito (2015) “The Additionality Impact of a Matching Grant Program for Small Firms: Experimental Evidence from Yemen”, World Bank Policy Research Working Paper no. 7462 Do you have a project you want evaluated? DECRG-FP researchers are always looking for opportunities to work with colleagues in the Bank and IFC. If you would like to ask our experts for advice or to collaborate on an evaluation, contact us care of the Impact editor, David McKenzie (dmckenzie@worldbank.org)