54005 Finance & PSD Impact SEPTEMBER 2009 The Lessons from DECRG-FP Impact Evaluations ISSUE 8 The eighth in our impact series summarizes the results of an evaluation that didn't go as planned. Nevertheless, we believe there are lessons to be had from this experience for Bank and IFC finance and private sector work in other countries, and that it is important to not lose the information generated. The results of a pilot financial literacy and business planning training program for women in Uganda David McKenzie and Michaela Weber Around the world a growing minority of small To understand the role that the training and medium enterprises are owned by women. provision played within this context, evaluations Understanding which policies work for getting can play a helpful role. Two evaluation female-owned enterprises to grow is critical for strategies were pursued. In the first, undertaken enabling women to have a bigger impact on with financing from the Gender Action Plan, a private sector development. randomized experiment was designed for a 2009 Over the past five years the World Bank training group; 75 women were selected from a and IFC have placed more emphasis on list of 200 DFCU clients to whom training programs increasing access to finance and other would be offered. However, only half of those policies needed for female-owned businesses to offered the trained attended, and other firms grow. To date, a small portfolio of operations is from the experimental control group ended up beginning to form, which can provide first being invited to make up the target number to be indications on the relevance and performance of trained. Given the small sample size and other interventions in this area. project limitations, this led to us having to abandon the randomized experiment. Training and Credit in Uganda However, the experiment is still useful Women in Uganda receive only 9 percent of for understanding the take-up decision among commercial credit, despite owning 39 percent of clients offered the training. Contrary to what one registered businesses. IFC put together a might expect, the main determinant of the take- comprehensive program for DFCU, a local bank; up decision was the firm manager's existing the program was comprised of a $6 million SME level of formal education: 77 percent of those credit line (including $2 million targeted to attending the training had some university women entrepreneurs), and of a tailor-made education. advisory program that combined training for the Bank and for its potential female SME Ex-post Evaluation borrowers. The second evaluation methodology The courses were dedicated to the implemented was a difference-in-differences bank's potential loan clients, spanned 12 days, methodology, comparing the change in firm and covered banking requirements and outcomes over the 2007 to mid-2009 period for processes, financial literacy, separation of firms in the first training group in 2007 business and personal accounts, and provision of (treatment group) to that in firms whose owners opportunities for business networking and peer started training in mid-2009 (control group). mentoring. 51 women were trained in a first The 2007 training was implemented group in 2007. At the same time, DFCU drew without an impact evaluation design or baseline down the women entrepreneur specific credit survey. We therefore had to design an ex-post line rapidly (within 3 months instead of 2 years) evaluation which attempted to find and contact and reported providing loans to 297 women the firm owners from the 2007 training group, entrepreneurs, with an outstanding default rate and ask both them and the 2009 group about the of less than 0.5% conditions of their firm today and in 2007, and about their training experience. Do you have a project you want evaluated? DECRG-FP researchers are always looking for opportunities to work with colleagues in the Bank and IFC. If you would like to ask our experts for advice or to collaborate on an evaluation, contact us care of the Impact editor, David McKenzie (dmckenzie@worldbank.org) There were some differences between be that the training has beneficial effects on firm the two years' training groups. Those trained in growth for this subgroup, but we are unable to 2007 had existed longer, had several more detect this with the sample size which we have. employees in 2007, and were somewhat less in traditionally female sectors of the economy. Lessons Survey tracking rates were lower for the 2007 1. Targeting. Best uptake seems to come from training group, perhaps due to less successful women with some university training; businesses closing or being less willing to featuring this into marketing strategies may participate in the survey. save cost and effort. However, reaching out to less educated women with high potential Self-assessed results are high businesses may require a different approach 72% of the 2007 training group said the than through the commercial banks. training has significantly improved the 2. Continuously refine content based on performance of the business and reports entrepreneur segment and beneficiaries from those who agreed to participate in a feedback. The material on business focus group following the survey were planning and accounting and financial positive. management practices, including separating business and personal accounts was practical Is training the key to success? and useful, other content was thought too Training did lead firms to be more likely to conceptual. The material is specifically hold separate business and personal targeted at a narrow range of relatively accounts, to specify a salary they draw each educated women with perceived ability to month, and to have budgets they monitor. grow their businesses, and achieving the There were no changes in other business right level for content to meet their needs practices such as willingness to delegate and time constraints appears to require business tasks, payment of taxes, and how further refinement and testing of content and they employ relatives. delivery mechanisms. Training is not associated with greater use of 3. Further impact studies are needed, to test professional accountants, or more access to whether training matters for those finance. receiving credit. Such a design should be Those trained in 2007 did not experience implemented ex ante. Budget constraints significantly higher sales or employment and the pilot nature of the project limited on- growth over 2007-mid 2009 than those the-ground-presence, and as a result, sample trained in 2009. size and evaluation methodology. Randomization is always difficult to explain The main goal of the training was to create a to development partners, in particular to a pool of women entrepreneurs who would then commercially operating partner. More face- qualify for commercial financial services to to face contact with all partners can help achieve their maximum growth potential; the making it happen right. training was intended to have most benefit when coupled with access to commercial financial services. In our sample only half the firms trained in 2007 had bank loans in 2009. It may Recent impact notes are available on our website: http://econ.worldbank.org/programs/finance/impact