Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005100 IMPLEMENTATION COMPLETION AND RESULTS REPORT < IDA-52560 > ON A CREDIT IN THE AMOUNT OF SDR 33.4 MILLION (US$50 MILLION EQUIVALENT) TO THE Burkina Faso FOR THE BF-Youth Employment & Skills Development Project September 15, 2020 Social Protection & Jobs Global Practice Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective Dec 16, 2019) Currency Unit = CFA Franc 589 FCFA = US$1 US$ 1.382 = SDR 1 FISCAL YEAR July 1 - June 30 Regional Vice President: Hafez M. H. Ghanem Country Director: Soukeyna Kane Regional Director: Dena Ringold Practice Manager: Jehan Arulpragasam Task Team Leader(s): Rebekka E. Grun ICR Main Contributor: Kirsten Schuettler ABBREVIATIONS AND ACRONYMS ANPE National Employment Agency (Agence Nationale Pour l’Emploi) BF Burkina Faso BUNEE National Office for Environmental Evaluations (Bureau National des Evaluations Environnementales) CAS Country Assistance Strategy CCI-BF Chamber of Industry and Commerce (Chambre de Commerce et de l'Industrie du Burkina Faso) CFBTP Vocational Training Center in the sector of building / public works (Centre de Formation en Bâtiment et Travaux Publics) CFTH Vocational Training Center in the sector of tourism / hotel / restaurant (Centre de Formation en Tourisme et Hôtellerie) CMA-BF Chamber of Crafts and Trade of Burkina Faso (Chambre des Métiers de l'Artisanat du Burkina Faso) CMU Country Management Unit CPF Country Partnership Framework DGESS Statistical Department of the Ministry of Youth (Direction Générale des Etudes et des Statistiques Sectorielles du Ministère de la Jeunesse et de la Promotion de l’Enrepreneuriat des Jeunes) EO4SD Earth Observation for Sustainable Development ESA European Space Agency FONAENF Fund for Literacy Training and Non-Formal Education GP Global Practice GRM Grievances redress mechanism ICR Implementation Completion Report IDA International Development Association IGA Income-generating activities IRR Internal rates of return ISR Implementation Status and Results Report LIPW Labor Intensive Public Works M&E Monitoring and Evaluation MEBF private sector promotion body Maison de l'Entreprise du Burkina Faso MIT Ministry of Infrastructure and Transport MJFPE Ministry of Youth, Vocational Training and Employment (Ministère de la Jeunesse, la Formation professionnelle et de l'Emploi) MJPEJ Ministry of Youth and Youth Entrepreneurship Promotion (Ministère de la Jeunesse et de la Promotion de l’Entrepreneuriat des Jeunes) MTR Mid-term Review NGO Non-governmental organization NPV Net present value PAD Project Appraisal Document PCU Project Coordination Unit PDO Project Development Objective PEJDC Youth Employment and Skills Development Project (Projet Emploi des Jeunes et Développement des Compétences) PIU Project Implementation Unit PTR-HIMO Special Labor-intensive Public Works unit in the Ministry of Infrastructure and Transport (Projet de Travaux Routiers-Haute Intensité de Main d'Œuvre) RF Results Framework RPF Resettlement Policy Framework SCADD Stratégie de Croissance Accélérée et de Développement Durable 2012-2016 TTL Task Team Leader UNDP United Nations Development Program TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) .....................................10 II. OUTCOME .................................................................................................................... 12 A. RELEVANCE OF PDOs ............................................................................................................12 B. ACHIEVEMENT OF PDOs (EFFICACY) ......................................................................................14 C. EFFICIENCY ...........................................................................................................................23 D. JUSTIFICATION OF OVERALL OUTCOME RATING ....................................................................26 E. OTHER OUTCOMES AND IMPACTS (IF ANY)............................................................................27 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 29 A. KEY FACTORS DURING PREPARATION ...................................................................................29 B. KEY FACTORS DURING IMPLEMENTATION .............................................................................30 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 31 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................31 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE .....................................................32 C. BANK PERFORMANCE ...........................................................................................................34 D. RISK TO DEVELOPMENT OUTCOME .......................................................................................36 V. LESSONS AND RECOMMENDATIONS ............................................................................. 37 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 41 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 52 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 54 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 55 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 60 ANNEX 6. BENEFICIARES OF THE ENTREPRENEURSHIP SUB-COMPONENT ............................. 61 ANNEX 7. METHODOLOGY OF THE TRACER STUDY ............................................................... 62 ANNEX 8. MAPS OF INFRASTRUCTURE BUILT THROUGH RURAL LIPW .................................. 64 ANNEX 9. EXECUTIF SUMMARY OF GOVERNMENT ICR (TRANSLATION) ................................ 70 The World Bank BF-Youth Employment & Skills Development (P130735) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P130735 BF-Youth Employment & Skills Development Country Financing Instrument Burkina Faso Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Burkina Faso Ministry of Youth, Professional Training, and Employment Project Development Objective (PDO) Original PDO The project development objective (PDO) is to increase access to temporary employment and skills development opportunities for out-of- school youth. Page 1 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 50,000,000 48,170,357 45,006,523 IDA-52560 Total 50,000,000 48,170,357 45,006,523 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 50,000,000 48,170,357 45,006,523 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 23-May-2013 03-Mar-2014 28-Jul-2014 15-Dec-2018 15-Dec-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 14-Jun-2017 15.29 Change in Results Framework Change in Components and Cost Reallocation between Disbursement Categories 19-Nov-2018 32.87 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Modest Page 2 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 17-Nov-2013 Satisfactory Satisfactory 0 02 06-Jul-2014 Satisfactory Moderately Satisfactory 1.77 03 02-Apr-2015 Satisfactory Moderately Satisfactory 2.25 04 09-Oct-2015 Satisfactory Moderately Satisfactory 4.90 Moderately 05 13-Jun-2016 Moderately Unsatisfactory 9.03 Unsatisfactory Moderately 06 04-Jan-2017 Moderately Unsatisfactory 10.66 Unsatisfactory 07 22-Jun-2017 Moderately Satisfactory Moderately Satisfactory 15.29 Moderately 08 08-Jan-2018 Moderately Unsatisfactory 18.48 Unsatisfactory 09 28-Jun-2018 Moderately Satisfactory Moderately Satisfactory 29.32 10 11-Jan-2019 Moderately Satisfactory Moderately Satisfactory 35.16 11 13-Jun-2019 Satisfactory Satisfactory 38.48 SECTORS AND THEMES Sectors Major Sector/Sector (%) Education 40 Adult, Basic and Continuing Education 20 Workforce Development and Vocational Education 20 Social Protection 60 Social Protection 50 Public Administration - Social Protection 10 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Page 3 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Private Sector Development 100 Jobs 100 Human Development and Gender 100 Education 24 Access to Education 6 Science and Technology 6 Teachers 6 Standards, Curriculum and Textbooks 6 Labor Market Policy and Programs 76 Labor Market Institutions 38 Active Labor Market Programs 38 ADM STAFF Role At Approval At ICR Regional Vice President: Makhtar Diop Ousmane Diagana Country Director: Madani M. Tall Soukeyna Kane Director: Ritva S. Reinikka Dena Ringold Practice Manager: Peter Nicolas Materu Jehan Arulpragasam Task Team Leader(s): Hamoud Abdel Wedoud Kamil Rebekka E. Grun ICR Contributing Author: Kirsten Schuettler Page 4 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. At the time of appraisal of the Youth Employment and Skills Development Project in 2013, lack of jobs for youth was a key challenge in Burkina Faso. Ranking 182th out of 188 countries in the United Nations Development Program (UNDP) Human Development Index, Burkina Faso is one of the poorest countries in the world with an undiversified economic structure prone to shocks. In spite of sustained growth, the economy has failed to produce the kind of jobs needed to pull large numbers of people out of poverty. The vast majority of the population in Burkina Faso is young (over 80 percent are under the age of 35) and the labor force is made up of predominantly young people. 2. Evidence shows the importance of providing this key population with improved skills and employment opportunities so as to increase their opportunity for future employment. The World Bank (2012) report entitled "Burkina Faso: Employment and Skills Development" shows that the skill level of the workforce is too low to support diversification and competitiveness of the economy and the national skills development system is weak and faces major challenges.1 About three-quarters of the labor force has never participated in formal education. Poverty remains high because most people are employed in low productivity, low earning, and highly vulnerable jobs. Thus, for the large majority of the work force, the main problem is not job creation per se but rather the type of jobs that are available and the lack of skills to work in jobs of higher productivity and quality. Those vulnerable to unemployment are primarily a small group of non-poor, educated, and urban youth, for which the unemployment rate exceeds 25 percent due to the mismatch between the education system and market demand. The educational system performs sub-optimally, with low rates of return and ineffective allocation of resources. Without a well- functioning educational system, the population cannot acquire the necessary skills to participate in productive employment. Unless they engage in productive employment, they cannot earn adequate incomes. 3. The social unrest observed in 2011 led the Government to place even more urgency on addressing the issue of job creation, particularly for youth. Investment in human capital to provide the economy with the relevant skills was one of the three strategic areas in its poverty reduction strategy at the time of appraisal (Stratégie de Croissance Accélérée et de Développement Durable 2012-2016 - SCADD). 4. The project was also fully in line with the Bank's assistance strategy to Burkina Faso at the time of appraisal, as detailed in its 2010-2012 and 2013-16 Country Assistance Strategies (CAS), which focused on: (i) minimizing economic vulnerability and promoting growth through economic transformation by increasing productivity; and (ii) promoting shared growth through effective social service delivery, in particular in education and training and health. Lastly, the project, by building fostering employment opportunities for youth, was in line with the World Bank's Africa Strategy at the time of appraisal. 1 World Bank. 2012. "Burkina Faso: Employment and Skills Development." World Bank: Washington, DC. Page 5 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Theory of Change (Results Chain) 5. The PAD did not include a formalized Theory of Change. 2 For this Implementation Completion and Results Report (ICR), the Project’s Theory of Change is constructed ex-post and has been retrofitted from the Project’s Results Framework (RF) and its detailed description. Figure 1 below presents project activities, outputs, outcomes and its (long-term) expected impact. Activities under capacity building and project management had an overall impact on all outcomes. They are hence not connected to a single outcome in Figure 1. Figure 1: Theory of Change Source: ICR Author. NB: The theory of change was constructed for the ICR based on the PAD. Critical Assumptions: - A1: Chamber of Industry and Commerce builds vocational training centers and Ministry of Youth funds heavy equipment - A2: Quality of infrastructure can be assured in rural areas - A3: Government develops operational manual and staffs the Job Counseling Youth Centers - A4: Government has resources to continue LIPW accompanying measures (trainings and creches mobiles) - A5: LIPW wages will allow participants to save enough to start income-generating activities; Demand for labor exists 2As per OPCS guidelines, including the theory of change became mandatory for PADs as of May 2018. The PAD for the Burkina Faso Youth Employment and Skills Development Project was approved by the World Bank’s Board of Directors in 2013. Page 6 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) - A6: Youth have capital to create/sustain enterprises; Demand for labor exists - A7: Sustainable funding for training is assured after the end of the project Project Development Objectives (PDOs) 6. The PDO, as stated in the Financing Agreement and the Project Appraisal Document, was to increase access to temporary employment and skills development opportunities for out-of-school youth. Key Expected Outcomes and Outcome Indicators 7. The original key outcome indicators, as described in the PAD were: - PDO Indicator 1: 46,800 direct beneficiaries (out-of-school youth between the ages of 16 to 35 years), of which 30 percent female - PDO Indicator 2: Number of man days in Labor Intensive Public Works created - PDO Indicator 3: Youth who completed training programs offered by the project - PDO Indicator 4: Proportion of youth trained who are employed or self-employed (one year after completion of training) Components 8. The project comprised of three components. Component 1: Labor Intensive Public Works (estimated US$25 million, of which US$ 15m for urban LIPW and US$ 10m for rural LIPW; actual US$25.83 million, of which US$ 14.02m for urban LIPW and US$ 11.81m for rural LIPW)3 9. The short-term objective of this component was to provide immediate employment for youth with no or little education through Labor Intensive Public Works (LIPW) in urban and rural areas. While the LIPWs in urban areas mainly cleaned public spaces, the LIPWs in rural areas were meant to rehabilitate or construct productive infrastructure that would have secondary positive impacts on access to economic opportunities and social services. In terms of inputs, the project organized communications campaigns and provided wages, insurance, tools and small equipment as well as, in the rural areas, the non-salary costs of the investments. 10. The LIPW were combined with training and a savings mechanism to facilitate a more permanent integration into the labor market. In urban areas, the training comprised social trainings (on HIV/AIDS, civic behavior etc.), entrepreneurship training and coaching for business plan development, literacy training and training in seven types of manual jobs. Savings were encouraged in all communes (between 200 and 500 FCFA per day) and made obligatory in two communes. Savings accounts were opened for beneficiaries. In rural areas, the training was limited to on-the-job training. In each urban commune, a Job Counseling Youth Center was to be established or strengthened to assist participants in identifying permanent employment or employment enhancing opportunities at the end of their LIPWs 3 Actual spending has been compiled by the PIU and converted with the average exchange rate of 1US$=578.15 FCFA. Page 7 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) contract, such as IGAs or vocational/apprenticeship programs (with a direct link to activities under Component 2). The project aimed to promote the participation of women by selecting activities that are supportive of women employment and reserving a percentage of the LIPW jobs specifically for women. It piloted mobile childcare at the worksite in one commune. 11. Because LIPW Programs should be playing a major role in employment creation in Burkina Faso for many years to come, the component also supported the mainstreaming of LIPWs into national policies, strategies and procedures and the establishment of a permanent capacity for LIPWs in the country (together with component 3). The project supported the government in developing a database with all LIPW participants (from the project and beyond) as a contribution to a unified social registry. The project helped accompany the new LIPW program of the government (PTR-HIMO) with training (on social aspects and entrepreneurship), savings program, mobile childcare and supervision of safeguards. 12. The component was implemented by a multitude of actors. The responsibility of the implementation of this component was given to the special LIPW unit (Projet de Travaux Routiers à Haute Intensite de Main d’Oeuvre (PTR- HIMO)) of the Ministry of Infrastructure and Transport (MIT) under a delegated management contract signed with the Project Coordination Unit (PCU). The urban LIPW were implemented by the municipalities (Manga, Ougadougou, Bobo-Dioulasso, and Ouahigouya). The literacy trainings were implemented by the Fund for Literacy Training and Non- Formal Education (FONAENF) and the entrepreneurship trainings by the private sector promotion body Maison de l’Entreprise. The rural LIPW were eventually implemented by the NGO HELVETAS, as the municipalities did not have the required capacities. After coordination with the government, six rural communes were selected (Kain, Tangaye, Barga, Rambo (in North Region/Yatenga Province), and Béré and Guiba (in Center-South Region/Zoundwéogo Province)). Component 2: Skills development (estimated US$20m, actual US$14.02 million) 4 13. The component aimed to improve youth employability by offering youth four different types of skills development opportunities: (i) initial vocational training, (ii) on-demand training (initial and on-the-job training), (iii) apprenticeships, and (iv) entrepreneurship training. At least 30 percent of the beneficiaries of each type of training should be female. 14. Component 2.1.a aimed to develop initial vocational training through a dual training approach for economic sectors with strong growth and employment creation. It aimed to establish two training centers and dual training of about 1,800 young people aged 16-24. The Chamber of Industry and Commerce (Chambre de Commerce et d’Industrie du Burkina Faso, CCI-BF) agreed to finance the construction of the two training centers and the dual training of the beneficiaries in the sectors of building / public works and tourism / hotel / restaurant. The project focused its support on (i) studies on the design and dimensioning of these two centers, (ii) the necessary equipment of centers’ laboratories and workshops, (iii) training of trainers, (iv) the development of specific curricula, (v) the external technical assistance required for the start-up of these two centers as well as for the establishment of a dual training system and a management and guidance approach, and (vi) the development of the surveys for the tracer study of trained youth. 4 Actual spending has been calculated with the average exchange rate of 1US$=578.15 FCFA. Page 8 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) 15. Component 2.1.b aimed to establish a demand-driven training system for private sector companies and provide training for 2,000 youth. The project provided technical assistance for (i) the establishment of the system and the preparation of manual of procedures; (ii) implementation of an information campaign; (iii) training of training counselors; (iv) funding of training contracts negotiated with selected training providers; (v) logistic support, operating costs and equipment; and (vi) training costs of 2,000 youth incl allowances. As for component 2.1a, component 2.1b was as well implemented by the CCI-BF. 16. Component 2.2. aimed to establish an apprenticeship program and enroll 4,000 youth. The project financed (i) a communications campaign to promote the new approach to apprenticeships to youth and employers; (ii) consultations with private sector organizations; (iii) capacity building of ANPE and the Chamber of Crafts and Trade; (iv) equipment/tools and pedagogical materials for training; (v) identification and training of participating master- craftsmen; (vi) training of ANPE’s staff participating in the program and complementary equipment for its centers; (vii) provision of allowances and toolkits to apprentices; (viii) development of professional qualification standards for 10 trades; (ix) basic skills entrepreneurship training and follow up support to apprentices after the completion of their apprenticeship; (x) funding of training contracts with selected training providers; (xi) TA for the review of the national regulatory framework for apprenticeship, and (xii) (vii) the cost of the sub-component M&E system. The component 2.2. was implemented by the National Employment Agency (Agence Nationale Pour l’Emploi-ANPE) and the Chamber of Crafts and Trade in Burkina Faso (Chambre des Métiers de l’Artisanat du Burkina Faso, CMA-BF). 17. Component 2.3. aimed to provide entrepreneurship training to 8,000 youth and provision follow up support to entrepreneurs. The sub-component supported: (i) an entrepreneurship promotion campaign; (ii) the design/ adaptation of suitable entrepreneurship training modules; (iii) the training/retraining of 50 trainers in entrepreneurship, among Maison de l'Entreprise (MEBF) staff and its accredited service providers (iv) the selection of participating youth, recruitment of external trainers and the organization of training and coaching sessions; (v) the costs of training and coaching of the participating youth, (vi) capacity building activities for the ME and other structures supporting the creation of enterprises; and (vii) the cost of the sub-component M&E system. The component 2.3 was implemented by the private sector promotion body Maison de l’Entreprise du Burkina Faso (MEBF). Component 3: Institutional Capacity Strengthening and Project Management (estimated US$5m, actual US$5.15 million) 5 18. Component 3 aimed to increase institutional capacity and strengthen project management. The purpose of this component was to (a) strengthen the capacity of private and public sector institutions to engage in an informed policy dialogue on skills and employment on a regular basis, e.g. through technical capacity strengthening and the creation of a mechanism for consultations and collaboration; and (b) improve the knowledge base on employment and youth. In order to achieve these objectives, the component funded: (i) the Project Coordination Unit (PCU); and activities in (ii) capacity- building; and (iii) M&E. 5 Actual spending has been calculated with the average exchange rate of 1US$=578.15 FCFA. Page 9 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) 19. The project was restructured twice, in June 2017 and November 2018. Revised PDOs and Outcome Targets 20. The first restructuring in June 2017 revised targets for PDO indicators downwards to bring targets in line with realistically achievable progress of the project and assure quality given some overly ambitious targets, delays and implementation challenges. For an overview of the original and revised targets see table 1. The first PDO indicator target was revised upwards by 700 beneficiaries in the operations portal after the restructuring to correct for the omission of the vocational training beneficiaries in the total number of training beneficiaries. Table 1: Changes in Outcome Targets PDO Indicator Original target Target after restructuring 1. direct beneficiaries (out-of-school youth between the ages of 16 to 35 46,800 34,200 6 years), of which 30 percent female, based on: 31,000 (16,000 20,500 (13,000 number of participants in LIPW urban and 15,000 urban and rural) 15,000 rural) 1,800 700 number or participants in initial vocational training 2,000 2,000 number of participants in on demand training 4,000 3,000 number of participants in apprenticeships 8,000 8,000 number of participants in entrepreneurship training 2. Number of man days in Labor Intensive Public Works created 3,900,000 (in text 3,174,194 7 and ISR_2 onwards) 4,800,000 (in results framework annex in PAD and in first ISR) 6 As noted in the text, this target was 33,500 in the restructuring paper. It was later corrected in the Operations Portal to 34,200, to include the vocational training beneficiaries which had been omitted in the sum of the overall beneficiaries. In this ICR we use the correctly calculated aggregated target of 34,200 for the time period after the restructuring. 7 In the second ISR after restructuring the target was changed to 2,808,860. This seems to better correspond to the revised number of beneficiaries, if the number of man days per beneficiary is not to change significantly. For the ICR we nevertheless use the target as set in the restructuring paper. Page 10 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Of which urban 2,400,000 (in PAD, not RF) Of which rural 1,500,000 (in PAD, not RF) 3. Youth who completed training programs offered by the project 100% 80% 4. Proportion of youth trained who are employed or self-employed (one 100% 60% year after completion of training) Revised PDO Indicators 21. The June 2017 restructuring only changed the PDO indicator targets but not the indicators themselves. Revised Components 22. Component names, and the broad component content remained unchanged throughout the project, but the first restructuring reallocated between disbursement categories and components. It cancelled heavy equipment for the construction training center due to the delay in construction and the inability of the CCI-BF to apply the equipment in time to good use (component 2.1.a). A business plan competition was added as part of component 2.3., financing start- up grants for 200 graduates and 1000 youth with little or no schooling ($3m in total), as beneficiaries had mentioned the lack of funding as key obstacle. Graduate winners were scheduled to receive a grant of $5,100 each, and out of school youth of $1500 each - amounting to about 30 percent of the average size of their start-up projects. This amount was chosen so that it could serve as the ‘own contribution’ to a commercial credit, which demands up to 30 percent down payment in Burkina Faso. The new business plan competition did not increase the total number of beneficiaries but contributed to the quality of the program as the beneficiaries were chosen among the participants of the subcomponent. 23. The first restructuring also revised downwards the cost projections for all components. These downward revisions were based on the downward revision of the targets and a devaluation of the FCFA against the US dollar (freeing up additional resources of over US$4m). The Government requested to keep the saved project funds as an unallocated amount. The project was progressing more continuously at the time of restructuring, and the Government intended to exceed the new endline targets. Other Changes 24. The first restructuring revised intermediated indicator targets, disbursement estimates and the internal rate of return downwards. Together with the outcome indicator targets, the first restructuring revised intermediate indicators downwards. 8 This included the number of beneficiaries in the different training programs as well as the infrastructure 8As noted for two outcome indicators, there were also additional changes in the targets for some intermediate indicators in ISRs after restructuring. These revised targets were in part also used by the PIU in their monitoring. In the ICR we use the targets as Page 11 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) build as part of the rural LIPW. Based on the downward revision of the indicator targets and lower disbursement rates than expected, the first restructuring also changed disbursement estimates and the economic and financial analysis. The internal rate of return projections was revised downwards from 15 percent to 12 percent in the most conservative scenario (based on the downward revision of the indicator targets but keeping the original costs per component) (see Annex 4). 25. The first restructuring also made slight changes to activities under component 1, which did not signify a change in component. A new activity accompanied informal mobile ‘creches’ set up by mothers working in LIPW for their children with a sensitization program on health, nutrition and education for the mothers, and equipped them with educational material for mothers and children. These relatively light add-ons were covered by the current legal agreement that provided for ‘sensitization and communication campaigns’ as well as provision of goods for the beneficiaries under the LIPW component. The restructuring also strengthened the training programs offered to beneficiaries of the urban LIPW (basic entrepreneurship training by the “Maison de l’Entreprise” also offered to LIPW beneficiaries, and short technical trainings for specific income generating activities). 26. A second restructuring extended the project by one year. In November 2018 the end date of the project was extended for one year to December 15, 2019. 27. Implementation responsibility was shifted between stakeholders due to weak implementation capacities. The actual implementation of component 2.2 was shifted from ANPE to the Chamber of Crafts and Trades for implementation of component 2.2. due to slow implementation. Because of the weak implementation capacities of the rural communes, the implementation responsibility for the rural LIPW was fully entrusted in May 2016 (through a management contract) to HELVETAS, a Swiss NGO already operating similar activities in Burkina Faso at the time. Rationale for Changes and Their Implication on the Original Theory of Change 28. The changes were a response to slow project implementation and did not change the original theory of change. The project restructurings did not touch the fundamental logic of the Theory of Change. The two main parts of the objective remained the same, and the outputs and outcomes were not modified. The first restructuring did, however, make changes to activities to react to implementation capacity and speed after the slow start of the project, due amongst others to the political situation. For the same reason it also adapted the results framework and revised the targets of the main indicators downwards. The extension was needed to allow for the completion of important activities and better achievement of the results, as implementation speed clearly picked up after 2017. II. OUTCOME A. RELEVANCE OF PDOs 29. Relevance of PDO is rated high. set in the restructuring paper. Page 12 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Assessment of Relevance of PDOs and Rating 30. The PDO remains highly relevant to the World Bank’s current development priorities. The World Bank Group’s (WBG) 2017 Systematic Country Diagnostic (SCD) for Burkina Faso identified promoting skills development as one of the three top tier priorities. Among the seven second-tier priorities the project contributes to encouraging financial inclusion and increasing resilience through improved social protection to the poorest. The project contributes to the Focus Area 2: Invest in human capital and social protection systems of the current Country Partnership Framework (CPF) (2018-2023), notably its objective 2.1: Support inclusive, high- quality education and skills development, as well as objective 2.3: Objective 2.3: Expand social protection to the most vulnerable. The operation thus remained important to achieving CPF development priorities. 31. The project is also aligned with the BF government’s current strategic priorities, as reflected in the National Economic and Social Development Plan (Plan National de Développement Economique et Social – PNDES 2016-2020), Burkina Faso’s medium-term national development policy framework, which succeeded the 2010-2015 Strategy for Accelerated Growth and Sustainable Development (SCADD) during appraisal. A sectoral policy (politique sectorielle Travail Emploi et Protection Sociale – TEPS) was elaborated to contribute to the implementation of the PNDES. The PNDES promotes a vision of structural transformation of the economy, inclusive growth and decent jobs for youth and women. The project contributes to two of its three priorities: human capital development (which includes education and employment) and economic transformation for growth and job creation. In the World Bank’s 2016 Country Survey education and jobs figured among the most important development priorities. Interviews conducted by the Statistical Division of the Ministry of Youth (DGESS) for the final evaluation of the project confirmed that the objectives of the project were aligned with the priorities of the Ministry of Youth. The relevance of the rural infrastructure constructed with LIPW was confirmed in interviews with the mayors and were taken from the municipal development plans (PCD). 99 percent of beneficiaries affirmed that the LIPW were adapted to the socio-economic context of their municipality. 32. During the course of the project, its PDO gained even further in relevance, due to increasing fragility and insecurity. Since the end of 2016, the security situation in North/Eastern Burkina Faso has deteriorated and remains highly volatile, with regular military interventions, terrorist attacks, hijacking of vehicles, targeted assassinations and kidnappings. Terrorist attacks have also taken place in Ouagadougou. After the events in 2011 and 2014-2015 further civil unrest is also likely. The project is active in the areas affected by fragility and insecurity and can contribute to stability through providing youth with a perspective and employment that address their economic and social aspirations.9 33. The target group of the project was well selected. The project focused on out-of school youth (defined as 16-35 years) with little or no schooling, which make up the vast majority of youth in jobs with low productivity. Given the high visibility of unemployed university graduates in urban areas and their 9 World Bank. 2020. Supporting Jobs in Situations of Fragility, Conflict and Violence (FCV). Jobs Solutions Note. Page 13 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) impacts on social and political stability, the project also offered entrepreneurship training to a small number of young university graduates. 34. Evidence-based project design and adaptations during implementation helped assure relevance from project start to closure. Project-design was based on best practices from other projects in the region as well as recommendations from research and analytical work (see section III.A on Key Factors During Preparation). This assured relevance of the project from the outset. Relevance was maintained through by adjusting the project during implementation (see section III.B on Key Factors during Implementation and IV. C on Bank Performance). 35. In conclusion, the relevance of the PDO can thus be rated as high. B. ACHIEVEMENT OF PDOs (EFFICACY) 36. Efficacy is rated modest before restructuring and substantial after restructuring. 37. Overall, the project reached 43,703 beneficiaries, exceeding the target after restructuring and nearly meeting the original target. The overall number of beneficiaries (PDO indicator 1) of 43,703 nearly met the original target (46,800) and exceeded the target after restructuring (34,200). The project reached 56 percent women, much higher than the original target of 30 percent. 38. The PDO statement for the project has two clearly identifiable objectives which will be assessed separately for efficacy: (i) to provide temporary employment; and (ii) to provide access to skills development opportunities for out-of-school youth. The overall number of beneficiaries (PDO indicator 1) aggregates beneficiaries across these two objectives (seen annex 1). The number of LIPW beneficiaries and the number of trainings beneficiaries, contributing to the PDO indicator, will be discussed in each section respectively. 39. As there were changes to project outcome targets, a split evaluation (before and after the changes) for this ICR is necessary as per the World Bank Guidance 10, even if the PDO remained the same throughout the life of the project and no indicators were removed. The restructuring happened in June 2017, 39 months after project effectiveness, with 33 percent disbursed. The project was still implemented for 30 months after the restructuring and 63 percent of disbursements took place during this time period. The total disbursement rate in SDR was 96 percent In total, US$45 out of US$50 million were disbursed, i.e. a disbursement rate of 90 percent in US$. 11 40. The evaluation of Efficacy is based on a tracer study conducted by the Statistical Division of the Ministry of Youth and maps developed in collaboration with the European Space Agency (ESA), in addition to the project’s M&E data and documentation as well as discussions during the ICR mission. The Statistical Division in the Ministry of Youth (DGESS) conducted their own project evaluation, quantitative and qualitative, which was turned into a tracer study through World Bank Technical Assistance (for the methodology of the study see annex 7). The World Bank team also worked with the European Space Agency to use satellite data in order to asses the 10 Bank Guidance on Implementation Completion and Results Report (ICR) for Investment Project Financing (IPF) Operations, Catalogue Number OPS5.03-GUID.156, issued March 2, 2020. 11 The difference between the disbursement rates in US$ and SDR are due to changes in the SDR-US$ exchange rate over the time of the project. Page 14 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) quantity and quality of the infrastructure constructed through rural LIPW as well as their potential impact (see annex 8). Assessment of Achievement of Each Objective/Outcome Objective 1: Provide access to temporary employment 41. The first part of the PDO aimed to increase access to temporary employment for out-of-school youth, but the project also aimed to increase beneficiaries’ employability and productivity as well as to build institutional capacity. Temporary employment to youth with little or no education in rural and urban areas was provided through labor-intensive public works. The provision of temporary employment was measured at the level of the PDO by the number of LIPW man days and the number of beneficiaries. The urban labor-intensive public works were combined with trainings and a savings mechanism to increase participants’ nutrition, human capital as well as financial capital and financial inclusion, which should in turn increase their employability and productivity and provide them with increased opportunities for future, more permanent income generation (see I.A Theory of Change). Beyond its target indicators at the level of direct project beneficiaries, the project also aimed to enhance the capacity of existing institutions to benefit further youth beyond the direct project beneficiaries. Technical assistance was provided to the government on improving its own LIPW program (Mesures sociales) in order to increase its positive impacts on participants beyond the direct project beneficiaries. The infrastructure constructed or rehabilitated during the projects’ LIPWs was intended itself to contribute to secondary employability and productivity impacts beyond the direct beneficiary youth in as far as it facilitates access to social services and markets. The Job Counseling Youth Centers should provide counseling to youth beyond the project. 42. The project largely exceed the target for beneficiaries employed through LIPW after restructuring and nearly reached the original target. The project employed 29,332 beneficiaries, of which 64 percent were women. 12 43. The project achieved its objective of providing temporary employment in urban areas. The targets for the component had been revised downwards during the restructuring, as project implementation had been slow amongst others due to the political transition (see section I.B). Eventually, however, the number of man days and beneficiaries for urban LIPW nearly achieved the original targets and exceeded the targets after restructuring (see table 2.1a). The urban LIPW provided 2,098,200 man days of work and reached 13,998 beneficiaries. The urban LIPW beneficiaries were employed for 150 days on average, as originally foreseen in the PAD and around US$5.4 million were paid out in wages. 44. Beyond providing temporary employment, the project also implemented extensive accompanying measures for the urban LIPW which increased beneficiaries’ human capital for higher future productivity and 12 Helvetas only monitored the number of tasks/jobs created. The number of rural LIPW beneficiaries was estimated for the ICR by the Statistical Division of the Ministry of Youth using the handwritten lists with the names of those who worked on each task. They counted those cases with the same name and identification card number as one beneficiary. Of the 5 721 attendance sheets, 25 where not readable and the number of participants per sheet was conservatively estimated. It is also important to note that of the LIPW beneficiaries surveyed 11.7 percent were non-youth. Page 15 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) employability, as intended in the PAD. The accompanying measures corresponded to the expectations of more than a quarter of the participants: While 65 percent of the tracer study sample said they participated to meet their basic needs, 15 percent aimed to save to develop, grow or diversify their income-generating activity and nearly 12 percent aimed to enhance their skills (DGESS 2020). 13 These additional measures for LIPW beneficiaries were included in the PAD but no related indicators were included in the original results framework. 14 The project’s monitoring results show that the beneficiaries in urban areas benefitted from a substantial amount of training (see table 2.1a); in rural areas limited training was provided on the job. Of those surveyed, 88 percent said that their self-esteem and confidence on the labor market increased due to their participation in the LIPW (DGESS 2020). 15 Among the illiterate beneficiaries participating in the literacy training, 71 percent of those surveyed were able to read and write after the training (DGESS 2020). The only downside was that due to implementation challenges the training programs in urban communes took place long after LIPW participation, which diminished potential impacts in terms of synergies with the income from LIPWs. 45. The urban LIPW also increased financial capital and inclusion through promoting savings and opening up bank accounts for beneficiaries. With the urban LIPW, 78 percent of beneficiaries surveyed opened a bank account, while only 20 percent had a bank account before the LIPW (DGESS 2020). 16 Available data from the urban communes for some cohorts show that the percentage of those saving varied widely (between 10 percent and 100 percent) and the amount saved ranged from 10 to 19 percent of the salary received. The tracer study found that 65 percent of beneficiaries said they were able to save some amount of their income during the LIPW. Two communes (Manga and Bobo-Dioulasso) forced participants to save by withholding part of their salary, which led to higher savings compared to voluntary savings. Voluntary savings were lower among men than women. Table 2.1 a: Beneficiary-level indicators for objective 1: Provide access to temporary employment Indicator Original Target Target after Actual achieved restructuring PDO Indicator Number of man days in LIPW 3,900,000 / 3,174,194 2,277,092 created 4,800,000 17 13 The tracer study was conducted by the Statistical Department (Direction Générale des Etudes et des Statistiques Sectorielles - DGESS) in the Ministry of Youth with support from the World Bank. Results are not representative for all beneficiaries and cannot be interpreted causally, as there is no counterfactual (see annex 7 for a description of the methodology of the study). 14 Paragraph 77 of the PAD states: “Labor Intensive Public Works under Component 1 are accompanied by the following measures to increase participants' employability and improve the sustainability of employment beyond the six months provided by the project: (i) basic skills development both in "soft" skills (e.g. Technical, Health, Environment) and in business related skills (e.g. functional literacy and numeracy, basic accounting, business planning).” The detailed project description also included the voluntary savings program (paragraph 17, Annex 2). 15 Direction Générale des Etudes et des Statistiques Sectorielles (DGESS) Ministère de la Jeunesse et de la Promotion de l’Entrepreneuriat des Jeunes. 2020. Evaluation Finale de la Mise en Œuvre du Projet Emplois des Jeunes et Développement des Compétences (PEJDC). 16 As described in the PAD, the savings and financial inclusion measures were limited to urban LIPW. Beneficiaries of rural LIPW mostly had no access to bank accounts (only 12 percent) and none of the participants opened accounts during the LIPWs. Voluntary savings were low. 17 3.9 million in text, 4.8 million in PAD results framework annex and in first ISR. Page 16 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) - Urban LIPW - 2,400,000 18 - 1 950 000 20 - 2,098,200 - Rural LIPW - 1,500,000 19 - 1,074,194 / - 178,892 750,000 21 Intermediate Indicators Number of youth employed through 31,000 (30% 20,500 (30% 29,332 (64% female) LIPW (of which female) female) female) - Urban LIPW - 16,000 - 13,000 - 13,988 - Rural LIPW - 15,000 - 7,500 - 15,344 Additional indicators Amount of wages distributed (in 6bn 22 - 3.4bn FCFA) - Urban LIPW - 3.7bn - 3.1bn - Rural LIPW - 2.3bn - 0.3bn Training for urban LIPW - - beneficiaries (completed, of which female) - Literacy - 6,628 (88% female) - Professional - 849 (84% female) - Entrepreneurship - 12,619 (85% female) - Coaching - 1,627 - Social themes - 10,629 (73% female) Opening of bank accounts for urban - 78% (baseline: 20%) LIPW beneficiaries 46. The results of the tracer study suggest positive effects on employment and income after the end of the LIPW, mainly through the savings acquired. Among the LIPW beneficiaries surveyed, unemployment declined slightly after the LIPW (48 percent compared to 50.5 percent before the LIPW). At the same time, a higher percentage were self-employed, employed or employers (nine percent higher) and less were family aides (seven percent lower). Directly after the LIPW, only 38 percent had been unemployed, 18 percent started a new activity or employment and 44 percent returned to their old activity. Among those who were (self-)employed immediately after the LIPW, 45 percent estimated that it was due to their project participation thanks to savings and skills acquired. Around half of the participants surveyed also estimate that their incomes increased after the LIPW participation. Income rose by around 40 percent on average. These results of the tracer study, which cannot 18 In PAD main text, not included in RF in annex. 19 In PAD main text, not included in RF in annex. 20 Based on borrower’s evaluation report; the division of the new target for LIPW man days between rural and urban was not included in the restructuring paper. 21 If the same number of 100 days per beneficiary is kept as in the PAD, the new amount for rural LIPW man days would be 750,000. To reach the overall target after restructuring, however, the number would be 1,224,194, leading to 163 days per beneficiary. 22 Target set in the PAD main text but not included in the RF. Page 17 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) be causally interpreted due to the lack of counterfactual, thus suggest positive effects on quality and quantity of employment after the end of the LIPW even if these effects do not seem to have been fully sustainable. One confounding factor might have been the increase in insecurity during the period. The positive effects seem to be mainly associated with savings accumulated through the LIPW and less with the trainings. The tracer study shows that those with a bank account (77 percent) made up 95 percent of those employed after the end of the LIPW. A logit regression analysis also shows that the duration of the participation in the LIPW had a significant impact on the likelihood to be employed, probably through allowing higher savings. 47. The project also made important contributions to build government capacity for LIPW, as intended in the PAD, amongst others accompanying a new round of government LIPW, which reached an additional 10 255 beneficiaries. As part of the operational objective of component 1 the PAD states that: “Because LIPW Programs should be playing a major role in employment creation in Burkina Faso for many years to come, the component will also (with Component 3) support the mainstreaming of LIPWs into national policies, strategies and procedures and the establishment of a permanent capacity for LIPWs in the country.” 23 The project accompanied the government’s self-financed LIPW (Mesures sociales) with training, mobile childcare and supervision of safeguards. 9718 beneficiaries were trained in social topics, 9411 in income-generating activities and 1953 benefitted from technical training. The project also worked on a database with all LIPW participants (from the project itself and others) as contribution to a national registry. Three of the four Job Counseling Youth Centers were rehabilitated and, even if not yet made operational during the time of the project, will be able to counsel youth on employment opportunities beyond LIPW in the future. 48. The rural LIPW were less successful in providing temporary employment, due to delays in implementation because of the political situation and a stronger focus on infrastructure quality than labor intensity. Rural LIPW exceeded even the original target for the number of beneficiaries, providing temporary employment to 15,344 youth, of which 60 percent were female. 24 The number of man days and wages paid, however, were much lower than planned, even during restructuring (see table 2.1a). The rural LIPW participants were hired per task and only employed for around 12 days on average instead of 100 days as calculated in the PAD. Delays in the start of the implementation of the rural LIPW, mainly due to the political transition, led to time pressure and the need to work during the agricultural season, which, combined with additional difficulties to find labor, forced Helvetas to hire more beneficiaries for less days (see section V. Lessons Learnt). This explains a change in focus from labor intensity to the quality of the infrastructure provided. 49. The infrastructure constructed through rural LIPW had additional positive impacts, as intended in the PAD. 25 The project achieved most of the lower targets after restructuring for the intermediate indicators on the amount of infrastructure constructed or rehabilitated (see table 2.1.b). 26 The security situation, notably in the 23 See paragraph 32 in the PAD. 24 Helvetas only monitored the number of tasks/jobs created. The number of rural LIPW beneficiaries was estimated for the ICR by the Statistical Division of the Ministry of Youth using the handwritten lists with the names of those who worked on each task. They counted those cases with the same name and identification card number as one beneficiary. Of the 5 721 attendance sheets, 25 where not readable and the number of participants per sheet was conservatively estimated. It is also important to note that of the LIPW beneficiaries surveyed 11.7 percent were non-youth. 25 See paragraph 74 in the PAD, as well as paragraph 11 in Annex 2 (Detailed project description). 26 The target for the community woodlands indicator was also not met, but this seems less important from an impact point of view. It had already been strongly revised downwards during restructuring not only due to high costs and implementation difficulties, but also low impact on employment creation, income and environment (see AM 03/2017). Page 18 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) commune of Kain, as well as the insistence of two communes on tax contributions by the hired firms hampered construction or rehabilitation of infrastructure (see section III.B). - There is a large body of evidence showing the positive impact of this type of rural infrastructure on rural livelihoods. While the results framework had no indicators on the potential impacts of the infrastructure on productivity and employment, the rural infrastructure project sheets as well as the maps that were developed for this ICR as part of the World Bank’s partnership with the European Space Agency (ESA) provide some evidence in this regard. They show that the roads decreased transport costs and provided access to markets, agricultural land, financial services and social services (schools and primary health facilities), thus improving the productivity of the population (see annex 8). The LIPW also reclaimed floodable land and rehabilitated or constructed vegetable plots and wells amongst others, increasing resources for agricultural activities. Soil conservation projects reduced soil losses and improve soil fertility and water retention, thus increasing productivity in particular in low rainfall years. Small water retention structures permit dry season irrigation, provide water for livestock and reduce the distance women and children travel to fetch water. The wells also decreased time needed to fetch water. Table 2.1 b: Infrastructure-level indicators for objective 1: Provide access to temporary employment Indicator Original Target Target after Actual achieved restructuring Intermediate Indicators Roads maintained/rehabilitated 240km 120km 133.329 km (rural) Roads maintained/rehabilitated 60km 60km 60km (urban) Community woodlots 2400ha 355ha 124.80ha maintained Reclaiming floodable areas - 200 107.19 Large diameter wells built - 37 41 Additional indicators Dug-outs/ponds 16 27 13 28 12 Vegetable plots - - 29 50. Efficacy of objective 1 is rated modest before restructuring and substantial after restructuring. The urban LIPW were modest before restructuring and high after restructuring, exceeding all restructured targets. They also successfully implemented the extensive accompanying measures foreseen in the PAD which increased beneficiaries’ human and financial capital and even had positive impacts on the quality of employment beyond the LIPW. The project also contributed to mainstreaming of LIPWs into national policies, and building up a permanent capacity for LIPWs in the country, as intended in the PAD. Rural LIPW were modest before and after restructuring. The project reached the intended number of beneficiaries, but the number of man days for rural LIPW stayed well below the original and revised targets. The intermediary indicators on infrastructure nearly met 27 In PAD text, not in results framework. 28 In AM 05/2018 and following. Page 19 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) all targets after restructuring and the infrastructure had positive secondary effects on productivity and employment, as intended in the PAD. Objective 2: Provide access to skills-development opportunities 51. The second part of the PDO aimed to increase access to skills development opportunities, in order to increase their employability and allow them to find higher quality employment or create their own income- generating activity or start-up. The project aimed to enroll 13,700 youth in four different forms of training systems, setting targets for completion rates and employment after the training. The project also aimed to benefit youth beyond the direct project beneficiaries through access to improved skills development opportunities by improving the institutional framework for initial vocational training and increasing capacity for on-demand, apprenticeship and entrepreneurship training that corresponds to market demands. 52. The project clearly exceeded the targets in terms of beneficiaries and completion rates set after restructuring and nearly met the original targets set at appraisal. The project trained a total of 15,061 beneficiaries and 14,371 beneficiaries completed their training of which 43 percent were women. 29 The completion rate was thus 95 percent (see table 2.2). The numbers do not include the beneficiaries trained as part of the LIPW in component 1, which were also provided with access to skills development opportunities in high numbers (see table 2.1a). The on-demand training and entrepreneurship training subcomponents very successfully both exceeded even the original targets. The entrepreneurship sub-component also reached its goals for the business plan competition included at restructuring, as well as internal targets set for coaching and B2B beneficiaries (see annex 6). The apprenticeships exceeded the beneficiary and completion rate targets set at restructuring but stayed below the original targets. Delays in paying apprentices had negative impacts on participation. The initial vocational training stayed well below even the beneficiary target set after restructuring, due to the delay in the construction of both training centers by the CCI-BF (after litigation during the procurement process) and their inability to recruit participants using other premises (see section IV.C). Table 2.2 Indicators for objective 2: Provide access to skills-development opportunities Indicator Original Target Target after Actual achieved restructuring Out of school youth enrolled in skills 15,800 (30% 13,700 (30% 15,061 30 (43% development programs (of which female) female) female) female) Vocational training On-demand training - 1,800 (30%) - 700 (30%) - 48 (2% female) Apprenticeships - 2,000 (30%) - 2,000 (30%) - 3,825 (27% female) Entrepreneurship training - 4,000 (30%) - 3,000 (30%) - 3,471 (43% female) 29It is important to note that of the component 2 beneficiaries surveyed 2.7 percent were non-youth. 30 Aggregated from the number of beneficiaries who were enrolled in vocational and on demand training, apprenticeships as well as entrepreneurship training, without counting those twice who benefited from apprenticeship as well as entrepreneurship training and without counting those that benefited from entrepreneurship training as part of component 1. Page 20 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) - 8,000 (30%) - 8,000 (30%) - 8,142 31 (51% female) / 20,761 32 Youth who completed training programs 100% (30% 80% (30% 95% (43% female) offered by the Project (of which female) female) female) Vocational training - 100% (30%) - 80% (30%) - 100% (2% female) On-demand training - 100% (30%) - 80% (30%) - 96% (27% female) Apprenticeships - 100% (30%) - 80% (30%) - 84% (44% female) Entrepreneurship training - 100% (30%) - 80% (30%) - 100% (51% female) Youth trained who are employed or self- 100% (30% 60% (30% T: 60% (50% employed one year after completion of female) female) female) 33 training (of which female) Vocational training - 100% (30%) - 60% (30%) - I: 56% (0% female) On-demand training - 100% (30%) - 60% (30%) - I: 61% (23% female) / T: 50% (20% female) Apprenticeships - 100% (30%) - 60% (30%) - T: 44% (50% female) Entrepreneurship training - 100% (30%) - 60% (30%) - I: 35% (61% female)/ T: 73% (57% female ) Number of prize winners of a business - 1,200 1,239 (63% female) plan competition 53. In terms of the insertion into the labor market after the training, the project did not achieve the overly ambitious target set at appraisal, but reached the target set at restructuring. The project set a very high target of 100 percent for employment after the training (PDO indicator 3), which was unrealistic even in less difficult environments, and was rightfully revised downwards to 60 percent during restructuring. Two sources are used to measure this indicator: data collected by the implementing agencies (I) and the tracer study (T) by the Ministry of Youth (see table 2.2). The implementing agencies only directly measured the labor market insertion of a subsection of those participating in the on-demand training (61 percent insertion rate) as well as of those who benefitted from the entrepreneurship training and coaching (35 percent insertion rate).34 The tracer study shows that the numbers of those who integrated the labor market after the training reached 60 percent at the time of 31 Including 425 apprentices who also benefitted from entrepreneurship training. 32 Including apprentices and those that benefited from entrepreneurship training as part of component 1. 33 T = measured by tracer study and I = measured by implementing agencies. The tracer study sample does not cover all sub- components and regions of the project (see annex 7). 34 For the on-demand training, only the insertion of those that participated in initial training and not those that benefited from on-the-job training was measured. For the entrepreneurship sub-component, the monitoring followed only those who benefitted from coaching after the training. The insertion rate for out-of-school youth (excluding the small number of university graduates) was 32 percent. Page 21 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) the survey, the target set at restructuring. Women benefitted as much from the insertion into the labor market as their male counterparts. 54. Quality of employment increased as well, which was a key intention of the project according to the PAD, even if not captured in the Results Framework, and an outstanding contribution to structural transformation in Burkina Faso. The project aimed to increase the employability and productivity of youth to help lift them out of poverty. 35 In this regard, the quality of employment is a much better indicator than employment itself in a poor country like Burkina Faso with little safety nets, where few people can afford not to work. The tracer study showed notable changes in the composition of employment: a strong reduction of those working as family aides notably for apprentices (-32 percent), and an increase in the number of employers, notably due to a change from self-employed to employers (+26 percent) in the entrepreneurship training (see table 3). Both are signs of improvements in the quality of employment, with the latter pointing to even some additional indirect employment impacts of the training. A short movie commissioned by the PIU at the end of the project also showcased stories of several beneficiaries who strongly increased their productivity and employed others after receiving the business plan competition grant. On average, income rose by 38 percent for surveyed beneficiaries of component 2. The tracer study showed that over 90 percent of the participants perceived that the training had a positive impact on their employability and self-esteem. It also shows that the participants were very satisfied with the quality of the training (97 percent on average), that it increased their skills and that it corresponded to market needs (both over 90 percent) (DGESS 2020). Table 3: Tracer Study before-after comparison (at time of survey) Type of training Unemployment Employed Self- Employer Family Aid/ Total (change) (change) Employed (change) apprentice/ change (change) volunteer employed, (change) self- employed or employer On-demand -8.3 13.6 -2.5 3.4 -12.3 14.5 training Apprenticeships 3.3 18.1 7.3 4.6 -32.2 30 Entrepreneurship -13.7 6.5 -10.1 26.3 -9 22.7 training Source: ICR author’s calculation based on DGESS (2020) 55. The project also build much needed capacity, contributing to setting up four different systems to grant access to skills development opportunities which will benefit youth beyond the project. It supported the construction of two new vocational training centers, the development of curricula, and the training of trainers for the centers. Even if delayed during project implementation, the centers will benefit youth beyond the project 35 See section I.B Sectoral Context and paragraph 30 on project beneficiaries in the PAD. Page 22 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) (see section IV.D). A new on-demand training was introduced, based on direct demand from the private sector. The CCI-BF engaged in concertation with the private sector and has permanently integrated the personnel for the on-demand training in their organizational chart. The novel approach to dual apprenticeships included the development of professional qualification standards and training of 1,199 master craftsmen and 68 ANPE trainers. The project also trained 50 young trainers of the Maison de l’Entreprise and the existing modules were translated to local languages to allow the training to reach youth with lower levels of education. 56. Overall, Efficacy of objective 2 is rated modest before restructuring and substantial after restructuring. Targets before restructuring were not all met as a result of challenges in implementation and overly ambitious initial targets. Targets set after restructuring were met or exceeded, with varying efficacy of the subcomponents. In addition, important effects on quality of employment and institutional capacity building were achieved, which were part of objective 2 in the PAD even if not included in the RF. Justification of Overall Efficacy Rating 57. Overall project efficacy is rated as modest before restructuring and substantial after restructuring. Following restructuring the majority of the indicators where met or exceeded, even if in some cases they did not fully reach the original targets as a result of implementation challenges (see III. Key Factors). C. EFFICIENCY 58. Efficiency is rated modest. Assessment of Efficiency and Rating 59. The ICR assesses project efficiency in three ways. The first is by comparing estimates for the net present value (NPV) and internal rate of return (IRR) at completion with the estimates made at appraisal and the downward revisions in the restructuring paper. Second, the ICR evaluates the efficiency of some activities by comparing the costs of the project’s activities with relevant comparator projects drawing from a World Bank cost effectiveness analysis of economic inclusion projects globally. Third, the ICR discusses aspects of design and implementation that either contributed to or reduced efficiency. Economic Analysis 60. The Economic Analysis shows that it was worth investing in the project, even if the returns were lower than expected at project preparation and restructuring, due to implementation challenges. The economic analysis builds on that of the PAD by assessing the net present value (NPV) and internal rates of return (IRR) of the LIPW and trainings with a similar methodology in the interest of comparison (see annex 4 for more details). Based on a discount rate of 10 percent, the tangible cost-benefit analysis shows a net present value (NPV) of US$9.2 million, lower than the US$25.2 million estimated in the PAD and the US$16.12 million estimated at restructuring (see table 4). 36 The overall cost benefits analysis result (including costs but excluding benefits of intangible investments) leads to a net present value (NPV) of US$0.9 million, compared to US$18.1 million in the 36The tangible cost-benefit analysis excludes costs and benefits of intangible investments as defined in the PAD, notably component 2.3 (entrepreneurship training) and component 3. Page 23 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) PAD and US$9 million at restructuring. 37 The lowest overall internal rate of return (IRR) is 10 percent, compared to 15 percent in the PAD and 12 percent after restructuring. Therefore, the analysis shows that it was worth investing in the project, even if the returns were lower than expected at project preparation and restructuring. 61. The infrastructure constructed or rehabilitated as part of the LIPW in rural communes, not included in the Economic Analysis, increased project benefits. The investments through rural LIPW were similar to those undertaken under the First and Second Community Based Rural Development Projects (CBRDP 1 and 2). An ex- post economic and financial evaluation of sub-projects financed under the Second Phase of CBRD-2 carried out during the preparation of the third phase of the Program found very high rate of returns (74 percent overall for the mix of sub-projects) and positive benefit-to-cost ratios (ranging from 3.7 for water supply and small multipurpose water works to 1.8 for soil and water conservation) and health infrastructure. Table 4: Summary of NPV and IRR estimates at completion for the overall project Probability post Present Present Incremental Internal Net Value of Value Combination of realist intervention Present benefit Rate of Total of assumption of component 1 of Benefit ration/Project Return Cost Benefits and component 2 employment cost ratio (IRR) Tangible cost benefit analysis (excludes costs and benefits of intangible investments) High probability of Project employment target 38 $29.9 $59.9 $30.0 2.0 19% Moderate probability of 10% less employment than target $29.9 $48.0 $18.1 1.6 15% Less probability of 20% less employment than target $29.9 $37.0 $7.1 1.2 12% Overall cost benefits analysis result (includes costs but excludes benefits of intangible investments) High probability of Project employment target $37.0 $59.9 $22.9 1.6 15% Moderate probability of 10% less employment than target $37.0 $48.0 $10.9 1.3 13% Less probability of 20% less employment than target $37.0 $37.0 $.0 1.0 10% 37 We used the scenario with less probability of employment (i.e. 30 percent), given the results of the tracer study on employment and increases in income after participation in LIPW (see section on efficacy above). The realistic scenario is estimated calculating the difference in annual earning between no education category and the average of some primary education and TVET annual earnings. 38 In the economic analysis the project target was assumed to be 50 percent, even though in the results framework the project aimed at 100 percent of youth trained who are employed or self-employed one year after completion of training in the PAD and 60 percent after restructuring. In the interest of comparability, the table is reproduced exactly as in the PAD. Page 24 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Comparison with similar projects 62. Comparisons with other projects show slightly higher costs for delivery and staff as well as LIPW wages paid and higher costs for the business grants. A recent analysis of the costs of 35 economic inclusion projects worldwide indicates that the project’s costs for delivery and staff were slightly above the mean (16 vs. 14 percent) (World Bank 2020). 39 In component 1, the wages paid for LIPW were slightly above the average but corresponded to the minimum wage in Burkina Faso. They made up 14 percent of the average consumption per household per annum for the bottom 20 percent. In component 2, the grant paid out per beneficiary as part of the business plan competition (US$3,639 in PPP 2011) was much higher than the average grant size for other jobs and livelihoods programs analyzed (US$416). The higher grant amount was, however, deliberately chosen to allow the beneficiaries to strongly increase productivity and move from self-employment to entrepreneurship. The amount also allowed beneficiaries to make the needed contribution to a commercial credit, which demands up to 30 percent down payment in Burkina Faso. The existing evidence discussed in the section on Efficacy above seems to indicate that the intended effect was achieved at least for part of the beneficiaries. We do, however, lack data to assess cost-effectiveness. Design and Implementation Efficiency 63. The complexity of the project decreased design efficiency. Besides differently designed LIPW in urban and rural areas, the project aimed to set up four different kinds of training systems mostly from scratch. It implemented its activities through a multitude of government and private sector entities who then signed additional implementing contracts with other entities. Setting up such a complex governance system required time, and the resulting long decision chains together with additional procurement challenges strongly increased transaction costs and further delayed project implementation (see III. Key Factors). 64. Collaboration with the private sector increased efficiency, even if delays were recorded. The training component was designed to exploit synergies with the private sector. This collaboration strongly increased efficiency, notably through the financing of the construction of the training centers by the CCI-BF, even if it also led to delays in implementation of component 2.1.A. 65. Management expenses were low and Bank supervision efficient, but implementation efficiency was decreased by implementation of the rural LIPW, difficulties to build up a functional PIU and procurement delays, also due to the political situation. The actual administrative costs of the project were US$3.9 million, less than 10 percent of project costs. During implementation, the project also used synergies between its (sub- )components to increase the quality of results for the same beneficiaries in an efficient way (e.g. Maison de l’Entreprise also training 425 apprentices and 12,619 participants of LIPW). On the World Bank side, there was only one change in TTL from project preparation to project closure. Joint participation in the mid-term review mission of the project as part of the handover helped ease the transition and reduce potential inefficiencies. Actual spending per component compared to the estimated component cost at appraisal and the revisions at restructuring was much lower for initial vocational training (in line with the much lower number of beneficiaries and the cancelled heavy equipment) but higher than expected for rural LIPW (which is notable given the fact that actual wages paid were only 10 percent of the originally calculated amount). Delays in hiring and high staff turnover in the PCU (see III.B) as well as delays in procurement and signature of contracts (see IV.B), had a 39 Costs were converted using PPP 2011. Page 25 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) negative impact on implementation efficiency. It is important to note, however, that delays during the political transition period 2014-2016 also helped prevent potential misuse of money. The project had to be granted a one- year extension to achieve the initially expected outcomes (see section on restructuring). As implementation speed had strongly picked up at the time of decision at the end of 2018, the extension did, nevertheless, lead to an increase in efficiency through the achievement of increased outcomes. 66. Overall, efficiency is rated as modest. The Economic Analysis shows that it was worth investing in the project, costs were mostly comparable to other projects, with particularly low administrative costs, and collaboration with the private sector increased efficiency. Efficiency was, however, somewhat below expectations in the operation’s sector due to complexity of project design, implementation of the rural LIPW with a focus on infrastructure instead of labor-intensity, difficulties to build up a functional PIU, and delays in procurement and contract signatures, with the latter partly due to the political situation. D. JUSTIFICATION OF OVERALL OUTCOME RATING With high relevance of its objectives at closing, modest efficacy before restructuring which changed to substantial after restructuring, and modest efficiency, the overall outcome of the project is rated moderately satisfactory. Table 5: Overall Outcome Rating Relevan Efficacy by Efficiency Overall Value of Share of Weighte Final ce of Objective* Outcom the Disbursemen d Value Overall PDOs e Disburse t Outcom Rating 40 d IDA e Rating Credit (US$) Before 1) Modest MU (3) US$ 33% (3*0.33) 0.99 + Restructuring 2) Modest 15.29m = 0.99 2.52 = Overall: 3.51, High Modest rounding it to 4.0 After 1) Substantia MS (4) US$ 63% (4*0.63) Restructuring l Modest 29.72m 41 = 2.52 = MS 2) Substantia l Overall: Substanti al *Objective 1: Provide access to temporary employment; Objective 2: Provide access to skill development opportunities 40 Following the formula outlined in Appendix H of the World Bank’s Guidance for ICR for IPF Operations, the Outcome rating for the period before restructuring is Moderately Unsatisfactory (MU) while it is Moderately Satisfactory (MS) after restructuring. 41 Disbursement data from the credit account shows US$1.85M undisbursed by the time of the ICR writing. Further, US3.14 million is recorded as a loss due to the variations in the SDR to US$ exchange rates over the life of the credit. Page 26 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 67. More than two thirds of the LIPW beneficiaries and an even higher percentage of accompanying training beneficiaries were women. A percentage of LIPW jobs was reserved specifically for women. The lotteries to select LIPW beneficiaries had two separate baskets, one for women and one for men. This proved not to be necessary, as those willing to work were mostly women, affirming existing knowledge that public works are not biased toward male labor. 42 The project achieved a much higher number of female participants in LIPW than originally targeted (64 instead of 30 percent). They also made up an even higher percentage of those trained as part of the LIPW (between 73 and 88 percent). The high number of female participants was achieved even if from a safeguards point of view the project did not allow those to participate that were visibly pregnant when signing up. When registered, pregnant women were fully paid and asked to watch children of other women working. 68. The project made efforts to take the specific needs of women in LIPW into account. To strengthen the childcare arrangements the women had set up during the LIPW, the project developed mobile childcare pilots. Besides positive effects on children, the mobile childcare sites allowed pregnant women and women with small, breastfed children to participate in the LIPW. A survey among beneficiaries confirmed that around 14 percent of women surveyed had no one to take care of their children (DGESS 2020). The mobile childcare was subsequently scaled up to 20 sites as part of the accompanying measures of the new round of government LIPW (Mesures sociales). The average LIPW work hours also left women enough time for household chores (DGESS 2020). With financing from the Umbrella Trust Fund for Gender Equality, the project also conducted extensive qualitative research to better understand the context and specific constraints that women face in LIPW. 43 The research provided insights into gender-specific impacts of the participation in LIPW, including increases in self-esteem, financial independence and decision-making power in the household. 69. In the trainings of component 2, women made up 43 percent of the participants. The participation varied by type of training: Women only made up a quarter of participants in the on-demand training, but around half of the participants of the apprenticeships and entrepreneurship trainings. The project made sure to include training activities and sectors of interest to women and favored female candidates. Institutional Strengthening 70. The project set-up strengthened the capacity of all the institutional stakeholders involved in the implementation of the LIPW and the trainings. Using a PCU, which coordinates the implementation of the project by other national institutions and agencies, instead of implementing it itself, has the added value of capacity building and institutional strengthening in the country. The implementation set-up of the LIPW strengthened the responsible unit in the Ministry of Infrastructure and Transportation as well as the communes. In the last year of the project, a new round of LIPWs of the government were implemented with technical assistance from the project, including a better monitoring of safeguards as well as trainings and mobile childcare 42 See for example: del Ninno, C., K. Subbarao and A. Milazzo. 2009. How to Make Public Works Work: A Review of the Experiences. 43 Kantiono, Florence, Ayaba Gilberte Kedote and Nathalie Ouangroua. 2020. Tonnoma. Une histoire vraie, ed. by Rebekka Grun and Irene Jillson, Washington, DC : World Bank. Page 27 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) (see section on efficacy above). This technical assistance assured the transfer of the lessons learnt during the project to the next round of LIPW at the central level. LIPW are now also integrated in the communal development programs in Bobo-Dioulasso, one of the urban communes. As described above in the section on efficacy, the second component increased capacity for four different types of training, by developing curricula and new training materials, adapting and translating existing modules to reach youth with lower levels of education, providing extensive training of trainers and supporting the construction of two new vocational training centers. South-South learning was promoted, through training of trainers in Tunisia and a study visit of a team from Burkina Faso to learn from a similar Youth Employment and Skills Development project in Côte d’Ivoire. A film was produced to capitalize the results of the project. Mobilizing Private Sector Financing 71. The two training centers under component 2.1.a (see para 14) were built entirely by the Chamber of Industry and Commerce (Chambre de Commerce et de l'Industrie du Burkina Faso) at their own cost. Poverty Reduction and Shared Prosperity 72. The project contributed to eliminating extreme poverty and boosting shared prosperity. Labor intensive public works are known to efficiently target the poor through self-selection based on the low wages and the types of tasks conducted. The selection of the participants was considered transparent by nearly 90 percent of the beneficiaries (DGESS 2020). Participation helped poor households to overcome income vulnerability, paying out a total of 3.4bn FCFA (US$5.8 million) in wages. The accompanying measures increased beneficiaries’ human and financial capital. They also helped them obtain more and better jobs. Over half of those surveyed in the tracer study reported that their income increased and 9 percent more were employed compared to before the LIPW. The skills development opportunities of the project were also mostly targeted to youth with little or no schooling. The results of the tracer study showed that more beneficiaries were in employment after the training (+25 percent compared to before) and saw their income increase significantly (see section II.B Efficacy) . Other Unintended Outcomes and Impacts 73. Thanks to its technical assistance to the government, the project reached additional beneficiaries. The accompanying measures of the government LIPW reached an additional 9,718 youth beyond the original project targets with training, mobile childcare and improved safeguards. The project also supported the successful organization of a large employment fair in Ougadougou and Bobo-Dioulasso (Salon de l’Emploi) in November 2019, which reached 14,542 youth. 7,712 youth were trained in on how to search for a job and 5,131 youth received unemployment cards. Participating firms announced 4,708 internships and 578 jobs. As of March 10, 2020, 813 youth have effectively been placed in an internship and 74 youth have been hired. The Covid-19 pandemic has unfortunately had a negative impact on further follow-up activities after March 2020. Page 28 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 74. Integration of best practices from relevant projects. The project design integrated best practices from relevant International Development Association (IDA) projects in the region. These included youth employment and skills projects in Cote d'Ivoire, Egypt, Yemen, Sierra Leone, Kenya, Liberia as well as private sector projects in post-conflict environments, such as South Sudan. It incorporated in particular lessons learned from the recently closed and on-going IDA operations in Côte d'Ivoire, such as the LICUS Youth Employment Pilot Program, the Post-Conflict Assistance Project and the Young Entrepreneurs and Urban Job Creation Project. 75. Project design based on research and analytics. The project design was also based on analysis and recommendations of research as well as analytical work of the World Bank. 44 Two of the most salient recommendations reflected in the project design include (i) moving away from inefficient supply-driven programs targeted at youth who have spent many years in the education system and move toward training programs for youth without other opportunities, teaching skills that not only clearly lead to salaried but also self-employment, given that most jobs in Burkina Faso are not in the formal sector. Evidence on skills programs shows that programs focusing only on skills have not been very successful. Results may be better when training aligns with labor demand and combines with other support. Along these lines, the project included the private sector in the design and implementation of the training programs, and provided life skills and entrepreneurship training, as well as coaching for those starting a business. (ii) strengthening/improving the existing LIPW programs, an area in which Burkina Faso has already had significant experience, to address income vulnerability among youth and adults, combining them with training in basic life skills and technical training for selected workers to help them find better earning jobs in the future. 76. PDO at the right level of ambition. Evidence on LIPW in general shows that they are widely used to boost short-run employment and that they can have positive impacts, at least in the short-term, on income and consumption and encourage savings even in very poor, remote and fragile regions. In their recent review of experimental or quasi-experimental evaluations of 15 public works programs in developing countries, Gehrke and Hartwig (2018) do, however, find that LIPW are often insufficient to allow participants to make productive investments afterwards and that accompanying trainings might not have positive impacts on employability or business earnings. 45 In terms of setting expectations right, formulating the PDO as access to temporary employment while including measures to increase the likelihood of finding higher-earning employment or starting an income-generating activity afterwards through trainings and savings mechanisms seems pertinent. 77. Complexity of project design. While the PDO of the project was set at the right level of ambition given the country context, the set-up of the project was very complex. Besides LIPW in urban and rural areas, the project aimed to set up four different kinds of training systems mostly from scratch. It implemented its activities 44This notably included World Bank. 2012. "Burkina Faso: Employment and Skills Development." World Bank: Washington, DC. 45Esther Gehrke and Renate Hartwig. 2018. “Productive effects of public works programs: What do we know? What should we know?” World Development 107, 111-124. Page 29 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) through a multitude of government and private sector entities who then signed additional implementing contracts with other entities. Setting up such a complex governance system required time, and the resulting long decision chains together with additional procurement challenges further delayed project implementation. Reducing this complexity during implementation would have required to cut sub-components during restructuring. This solution was discussed by the Bank and borrower but proved not to be possible due to political economy issues, as it would have meant to cut out the respective implementing partners as well. 78. Appropriate assessment and selection of stakeholders. During preparation it was decided to implement the project through a Project Coordination Unit (PCU), which coordinates the implementation of the project by other national institutions and agencies, instead of implementing it itself. Institutional assessments were conducted. Some of the stakeholders nevertheless did not have the required capacity to implement the respective activities. The PCU and World Bank team addressed this challenge by further delegating implementation to other entities throughout the course of the project. The PCU nevertheless struggled to coordinate all the stakeholders, implement sub-components with lower performing implementing agencies, monitor progress and quality and obtain reliable M&E data. B. KEY FACTORS DURING IMPLEMENTATION Factors subject to the control of the government and/or implementing agencies 79. Delays in hiring and high staff turnover in PCU. While the project was approved by the Board in May 2013, it only become effective in March 2014. A new PCU was set up for the project, which took additional time, also due to the socio-political situation in the country. The PCU was only fully staffed at the end of 2015, a training specialist in the PCU was only hired in June 2017, after 3 years of implementation. The project had four coordinators in six years. Due to high staff turnover essential functions (like safeguards, M&E, internal auditor) were missing during extended time periods all through project implementation. One reason for the high staff turnover was a lack of motivation due to salaries. After salaries had been increased for six months without prior non-objection of the World Bank, they were lowered to the initial level which led to departures. 80. Commitment and leadership: Employment is a cross-cutting theme which requires collaboration with different Ministries and other stakeholders that can only be achieved through strong leadership. While youth employment was a strategic priority for the government, the Ministry of Youth, where the project was based, lacked the convening power to assemble all stakeholders behind the objectives of the project. The Ministry itself only named a focal point for the project in the last two years of project implementation. The project relied on contributions from national counterparts that were not always made. Some expenses needed for the implementation and supervision of activities were not eligible. The government co-financing, which was needed to cover theses expenses was only a small amount and not always paid as needed. The municipalities lacked resources to implement and monitor the LIPW. The lack of support for LIPW in two rural municipalities (Béré and Guiba), which asked the firms to pay taxes, led to a stop of the LIPW in these communes. The operations manual and staffing for the Youth Jobs Counseling Centers were not finalized during the time of the project. Page 30 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Factors outside the control of government and/or implementing agencies 81. Difficult political and security situation. The popular uprising in October 2014 and the low responsiveness of national institutions under the ensuing transition government until May 2016 together with an unsuccessful coup in September 2015 delayed project implementation. The political situation notably delayed hiring of staff for the PCU and hampered decisions on the selection of the rural communes for LIPW and the signature between PTR-HIMO and the communes. Legitimate regional and local government were finally elected in April 2016 and this opened the door for the selection of six rural communes (four in the North and two in the Center-South). The convention with Helvetas was approved in September 2015 but only signed by the Government in March 2016. The security situation deteriorated since 2015, with an exponential increase since the beginning of 2019. The deteriorated security situation has notably affected the LIPW in the rural commune of Kain, which had to be partially cancelled and less infrastructure could be constructed than originally planned, as well as in Barga and Tangaye later. 82. Foreign exchange rate. The significant appreciation of the US dollar against the CFA Franc (from CFAF 510 at appraisal to CFAF 610 at restructuring, and CFAF 590 at project closure) decreased actual disbursement. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) 83. The quality of Monitoring and Evaluation (M&E) is rated modest. M&E Design 84. The projects result framework was well designed but some PDO indicator targets were too ambitious and some aspects not covered. Even if not formulated in a separate theory of change section in the PAD at the time, the theory of change of the operation was clear and adequate indicators were identified to monitor progress toward the PDOs. Some of the indicator endline targets in the PAD, however, were overly ambitious. Notably the indicator target of 100 percent of youth trained who are employed or self-employed one year after completion of training was unrealistic, given the average impacts of training interventions on employment even in less difficult environments, as well as the indicator on 100 percent of those enrolled also completing the trainings. These indicator targets were rightfully subsequently revised downwards in the first restructuring. The Results Framework did also not encompass the ambition of the project vis-à-vis an increase in the quality of employment through both components and secondary employment effects through the infrastructure built as part of the rural LIPW. Indicators on the trainings and savings program for the urban LIPW were missing as intermediary indicators but were monitored by the project. The M&E arrangements relied on implementing organizations providing regular M&E inputs to the PCU. These inputs were included in the contracts signed with the different entities, covering the indicators included in the results framework as well as additional indicators. Page 31 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) M&E Implementation 85. There was systematic updating of the Project’s RF through regular World Bank supervision missions and VCs/phone calls, but it proved difficult to rely on the M&E inputs provided by implementing agencies. With the exception of the Maison de l’Entreprise, the implementing entities showed deficiencies in terms of M&E and other stakeholders involved did not have the required budget for supervision (PTR-HIMO and municipalities). The PCU organized meetings with the implementing agencies to monitor progress and address open questions but did not schedule them on a regular basis. The M&E position in the PCU was vacant for over a year (April 2018 to June 2019) and no IT software was used to systematically register and check the data reported by the implementing agencies. This led to corrections of results reported from one ISR to the next. The impact evaluations and tracer studies planned during preparation were also not implemented and no international M&E firm was hired to support the PIU as described in the PAD. 86. Substantial efforts were subsequently conducted by the World Bank team and the project to improve the M&E data. These efforts included hiring a consultant to verify the results of the apprenticeship sub- component, innovative efforts to monitor and evaluate the rural infrastructure constructed or rehabilitated through LIPW in the commune of Kaine that could not be visited in person due to the security situation,and intensive discussions with and requests to the implementing agencies to fill in data gaps notably after the hiring of the M&E specialist in June 2019. The project also commissioned a final evaluation of the project by the Statistical Department in the Ministry of Youth (Direction Générale des Etudes et des Statistiques Sectorielles du Ministère de la Jeunesse et de la Promotion de l’Enrepreneuriat des Jeunes) which was turned into a tracer study with support from the World Bank, in order to cover existing data gaps. The quantitative survey was triangulated with qualitative focus group discussions and semi-structured individual interviews. At project closing, the Government submitted a comprehensive Project Completion Report and Borrower’s ICR contribution (Annex 5 and Annex 9). For the ICR, the World Bank team also used the partnership with the European Space Agency (ESA) to develop maps to assess the infrastructure built or rehabilitated through the rural LIPW. An impact evaluation of the mobile childcare pilots, financed by a Trust Fund, is still underway. M&E Utilization 87. During each World Bank supervision mission, progress was tracked through the RF and actions were agreed to improve the lagging indicators. The M&E results were used as a basis to decide to restructure the project. The results were also used to inform decisions on changes of implementing agencies as well as decisions on adding or changing activities (see section on quality of Bank supervision below). Justification of Overall Rating of Quality of M&E 88. The overall quality of M&E can be rated as modest. The design of Project’s M&E framework was robust, but some key aspects intended in the PAD were not covered, some targets were overly ambitious and there were shortcomings in M&E implementation. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Page 32 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Environmental and social safeguards 89. The project was classified as Environmental Category B, three safeguard policies were triggered under component 1 and the appropriate safeguard instruments were disclosed in-country. The project was classified as Environmental Category B since potential adverse environmental and social impacts associated with LIPW are generally less adverse, small-scale and site specific; thus, manageable to an accepted level. Potential limited negative impacts may include loss of vegetation (linked to the rehabilitation of rural roads) and an increase in solid waste (linked to street and drainage systems cleaning). Three safeguard policies were triggered by the project under component 1 (LIPW): Environmental Assessment OP/BP 4.01, Natural Habitats OP/BP 4.04 and Involuntary Resettlement OP/BP 4.12. Since the location of future component's activities were not known at appraisal, the government prepared and, after approval, disclosed in-country the appropriate safeguards instruments: an Environmental and Social Management Framework (ESMF) that outlines an environmental and social screening process for component's activities and a Resettlement Policy Framework (RPF) to address potential land acquisition or loss of economic activity on the part of individuals or groups of individuals in project intervention zones, even if the project was not expected to entail land take or restriction of access to assets or sources of livelihood. 90. The project faced some safeguards challenges during implementation, which were noted by World Bank supervision but could only be partially resolved: • A change of the location of one of the vocational training centers (CFBTP center), financed by the CCI- BF, rendered the environmental and social impact assessment obsolete, and construction started without a new assessment. The ToR for the construction of the second center were not validated by the National Office for Environmental Evaluations (Bureau National des Evaluations Environnementales- BUNEE). Social and environmental clauses were not included in the tender documents for the vocational centers. The Youth Center in Manga was reconstructed instead of rehabilitated as agreed, but the safeguards compliance was not adapted accordingly. • Neither PTR-HIMO nor Helvetas had an environmental or social safeguards expert throughout the implementation of the project, even if this position was required to implement the tasks included in the respective conventions. Thus, the infrastructure constructed or rehabilitated through LIPW was not subject to environmental and social screening. Assessments of the environmental and social impacts have been developed for rural roads and water retention basins (boulis), but have not been validated by the BUNEE or the World Bank. The BUNEE did also not externally monitor the implementation of the ESMF, as no convention with BUNEE was signed. In terms of mitigation measures included in the ESMF, only the reforestation was implemented as compensation for roads and water retention basins. Increases of solid waste did prove challenging for the urban municipalities, but the project made sure that they had the equipment needed, as noted during the March-April 2017 supervision mission. • The social safeguards specialist of the PCU was only hired six months before project closure. No social assessment was done and no Resettlement Action Plans (RAP) or Abbreviated Resettlement Plans (ARPs) were put in place for the roads and reclaiming floodable areas, as required by the Resettlement Policy Framework (RPF). No formal grievances redress mechanism (GRM) was established. However, social mediation firms were hired for the rural LIPW to mediate between firms and the population, prevent conflicts and provide transparency and equity in division of plots (reclaimed floodable areas, vegetable plots etc.). In some rural communities minutes of land transfers for the communal activities Page 33 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) like vegetable plots or reclaiming floodable areas were established (six out of 32). Social safeguard issues during the first batch of LIPW led the team to stop implementation of the LIPW until these issues were resolved. To address remaining safeguards issues, an environmental and social safeguards audit was done before the end of the project. Follow-up on observations from the Bank and the National Office for Environmental Evaluations (Bureau National des Evaluations Environnementales- BUNEE) was not finalized before project closure. BUNEE did, however, eventually confirm that the project conformed with environmental safeguards. 46 Fiduciary compliance 91. Financial management proved challenging but key issues were resolved before project closure. Financial management was rated as satisfactory at the beginning of the project but consistently rated as unsatisfactory since the May 2018 supervision mission. A number of challenges arose during project implementation but were solved before the end of the project: Ineligible expenses of 44 million FCFA for the salary increase were reimbursed. An audit of HELVETAS was conducted. The financial statements for the 2018 financial year audit were corrected, a final audit of the project realized and general activities linked to project closure conducted in due manner. 92. The project faced important procurement delays throughout implementation. Procurement was rated moderately unsatisfactory throughout most of project implementation. Government procedures were relied upon in the beginning, even if IDA procedures were foreseen in the PAD. The Government eventually authorized the project to use the IDA procedures in the second half of 2015, which helped to accelerate the implementation of the project. The PCU also received support from the procurement specialist of a similar youth employment project in Cote d’Ivoire and benefitted from technical support and regular supervision by the Bank. Nevertheless, considerable delays persisted in the execution of procurement activities, whether for contracts awarded, or those already executed. One reasons was that the post of the procurement specialist was vacant for longer time periods during project implementation and previous experience with Bank projects was not taken into account when hiring a replacement (even though reiterated by the Bank team during supervision missions). All materials were archived in STEP before project closure. C. BANK PERFORMANCE 93. Bank performance is rated as satisfactory. Quality at Entry 94. The project was well prepared but too complex. The preparation team helped to design a project with high strategic relevance, that responded to the Government’s needs and built on lessons learned from other related World Bank-financed projects as well as analytics (see III.A Key factors during preparation). The results framework was logical and straightforward, albeit with some overly ambitious targets and some missing indicators (see IV. Quality of M&E). The risk assessment appropriately rated the overall project risk as substantial 46Arrêté no 2020-309/MEEVCC//CAB portant émission d’avis conforme sur la conformité environnementale des investissements réalisés dans le cadre du Projet Emplois des Jeunes et Développement des Compétences (PEJCD). Page 34 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) given the fragile environment and low institutional capacity. The project was prepared within a year. While the PDO of the project was set at the right level of ambition given the country context, the set-up of the project was, however, very complex. Besides LIPW in urban and rural areas, the project aimed to set up four different kinds of training systems mostly from scratch. The PCU had to work with a multitude of government and private sector entities who then signed additional implementing contracts with other entities. The transactions costs of such a complex governance system were high. While not attributable to preparation quality, per se, the need for the first restructuring reflects the preparation team’s over estimation of the Government’s implementation capacity. Quality of Supervision 95. The team was not able to address project complexity at MTR but intensive supervision helped turn the project around from moderately unsatisfactory to moderately satisfactory. The supervision team produced twelve ISRs (on average one for each semester of project implementation) and carried out regular supervision missions in line with World Bank guidelines. Supervision missions comprised the necessary team members, including fiduciary and safeguards specialists, and findings and agreements of the missions were detailed in quality Aide-Memoires and management letters. Supervision was proactive and provided realistic feedback to management, as well as guidance to counterparts. Project reporting was thorough, candid, and ratings in ISRs were mostly adjusted to reflect challenges and progress in implementation, except before the restructuring when the team was slow in downgrading project ratings. The World Bank’s implementation support benefited from continuity in Task Management, with only one change in TTL from preparation to closure. The change of TTL took place during the Mid-Term Review with both TTLs participating in the mission, which facilitated hand-over. The project was restructured to respond to delays in implementation, mainly due to the political transition, which made the achievement of the original targets unrealistic. Instead of simply decreasing targets across the board, restructuring should have tackled the complexity of the project, but that would have meant cutting out whole sub-components with their respective implementing agencies. Based on consultations with the respective GP and CMU, these political economy considerations were taken into account in the restructuring. Fully aware of the difficulties to implement the project even after restructuring given its complexities, the Bank supervision team started weekly phone calls with the PCU, first by the TTL from headquarters and then by team members in the country office. These regular phone calls together with in-person meetings where needed in between missions, helped to speed up implementation of the project and contributed to turning the project to a much more successful result. 96. Besides restructuring, the Bank team adapted to an evolving context through continuous adaptations based on intensive supervision to proactively get implementation on track. Adaptations implemented by the team included the following: • Implementation arrangements were adapted to the actual capacity of the government entities: Because of weak implementation capacity of the rural municipalities that became apparent in the first year, the project utilized partial delegation of project implementation to the Swiss NGO Helvetas to leverage their comparative advantage in labor-intensive public works (LIPW) at the regional level. As a response to slow implementation of component 2.2, implementation was further delegated from ANPE to CMA-BF. Page 35 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) • Shortcomings in the urban LIPW were corrected: Due to shortcomings with the first batch of urban LIPW, the start of the second batch was conditioned on improving conditions for implementation. Mobile childcare was developed as a response to childcare challenges seen at LIPW sites during supervision. • Solutions to challenges in component 2 were proposed and implemented: Given the delays in the construction of the vocational training centers, due to litigations in the procurement process, the Bank team recommended to start the vocational training in a different location. As the Maison de l’Entreprise reported that entrepreneurship training beneficiaries needed additional coaching and lack of financing was a key obstacle to start their activity, a business plan competition was introduced with the first restructuring. • Seizing arising opportunities, additional beneficiaries were reached and activities performing well were expanded: Making use of new opportunities, the project provided training, insurance and safeguards supervision for the new wave of government LIPW, which allowed it to reach nearly 10 000 additional beneficiaries, supported the government in the organization of a large National Employment Fair and developed pilots on dual apprenticeships for 158 beneficiaries. The satisfaction of the beneficiaries with the dual apprenticeships was witnessed during the ICR mission. The team also expanded activities that were working well, like the entrepreneurship trainings which were also provided to LIPW beneficiaries. • Measures were undertaken to improve the M&E, financial management, procurement and safeguards of the project. Action plans were developed and supervised. To speed up project implementation, the Bank team send a team of the PCU to Côte d’Ivoire to learn from a similar Youth Employment and Skills Development project and the procurement specialist received assistance from their procurement specialist. Justification of Overall Rating of Bank Performance 97. The Overall Rating of World Bank Performance is Satisfactory. The Project was well prepared, except for the fact that it was too complex (Quality at Entry). The team was not able to address project complexity at MTR due to political economy considerations but intensive supervision, including weekly phone calls with the PIU, quick reactions and amendments helped turn the project to an Outcome in the Satisfactory range (Quality of Supervision). D. RISK TO DEVELOPMENT OUTCOME 98. The Risk to Development Outcome is High. Risk to Development Outcome is evaluated from two angles. First, the ICR looks at the risk that the works financed by the project would not be maintained. Second the ICR considers the likelihood that the training systems developed and supported by the project would continue after project closure. 99. The risk that works financed by the Project will not be maintained is somewhat mitigated by the high ownership of the subprojects by the communities, as they were identified as priorities by them. Global Page 36 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) experiences show that community involvement in project selection creates ownership of the assets created through LIPW and may lead to better maintenance of the assets. 47 Projects selected in rural areas were all taken from their communal development plans. In urban areas, the activities were defined by the municipalities themselves. There are, however, maintenance plans or monitoring of sub-projects after the works were completed and maintenance would need to rely completely on voluntary engagement of community members. Given the deteriorating economic conditions, community members involved in voluntary maintenance of the infrastructure are increasingly forced to devote all their time to finding livelihood options, and may not have the time or the capacity to work to maintain community assets without pay, as the opportunity cost would be too high. Even if the Youth Jobs Centers have not been officially opened yet, progress has been made since project closure with respect to staff and organization of the Centers. 100. The engagement of the private sector increases the likelihood that the training systems supported by the project will continue to be implemented after project closure, even if there are several risks. The sustainability of the vocational training centers will depend on the finalization of the construction by the CCI-BF, the acquisition of the heavy equipment for the construction center by the Ministry and the development of a sustainable business plan. The CCI-BF has continued to make progress on construction after project closure and the Centers are scheduled to officially open next year. With regards to the on-demand training, the CCI-BF has permanently integrated the necessary personnel in their organizational chart. The future of the on-demand training will depend on the private sector’s willingness to pay for the trainings and hire those trained. 101. In general, as youth employment is key for the government, this is likely to positively influence their willingness to continue project’s activities after project closure, but Covid-19 and the security situation inhibit future. The sustainability of project's activities will depend on Government's willingness to devote sufficient priority and resources for their continuation after the project period. Job creation has taken center stage in Burkina Faso's economic and political debate and the government has placed high importance on the issues of job creation in its recent poverty reduction strategy (SCADD). The social unrest of 2011 has given even more urgency to addressing the issue of job creation for the youth (see section on Relevance). Government is fully aware that youth employment and increasing the productivity of the labor force will be key structural issues for many years to come and, although fiscal space is limited and further squeezed due to Covid-19, it is committed to support skill development and job creation over the long term. Overall, however, Covid-19 and the deterioration of the security situation will inhibit beneficiaries’ prospects of getting a job or establishing a viable business or income generating activity and increased the risk of whittling away their precious savings. V. LESSONS AND RECOMMENDATIONS 102. Ambition of PDO and complexity of project design need to be adapted to the context to facilitate implementation. The PDO of the project was set at the right level of ambition, but the set-up of the project was very complex. Greater selectivity at project design, notably a focus on one or two types of training, likely decreases transaction costs and facilitates project implementation in low-capacity environments. 47 del Nino, C., K. Subbarao and A. Milazzo. 2009. How to Make Public Works Work: A Review of the Experiences. Page 37 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) 103. Careful assessment of the number and the capacity of all stakeholders involved is critical when using a PCU, which coordinates project implementation by other national institutions and agencies. The PCU worked with a multitude of government and private sector entities who then signed additional implementing contracts with other entities. None of the implementing agencies were under the umbrella of the Ministry of Youth in charge of the project. Transactions costs of such a complex governance system are high: setting it up delays project implementation, decision chains become very long, and additional procurement challenges emerge. These costs need to be weighed against the benefits of capacity building among the entities involved. A thorough institutional assessment of implementing partners would have been important and could have prevented additional delegations of implementation throughout the course of the project. The PCU struggled to coordinate all the stakeholders, implement sub-components with lower performing implementing agencies and obtain reliable M&E data. When using an NGO to implement activities monitoring cannot be delegated and the activities still need to be actively supervised by the PCU. Each agency was invested in its own sub-component, so it was impossible to cut sub-components and simplify at restructuring due to political economy issues, which may have been better than simply decreasing targets across the board but not tackling this underlying problem. 104. Satellite data and close collaboration with statistical divisions of the government can help close gaps in M&E in FCV and low-capacity settings. The collaboration with the ESA allowed the use of satellite data to measure and ascertain infrastructure quantity, quality and impact in remote rural and insecure areas that partly became inaccessible during project implementation. The Statistical Division in the Ministry of Youth undertook their own tracer survey, quantitative and qualitative, with dedication and rigor despite the difficulties in the field. With intense Bank TA, but also with high own motivation, hard work and own financing they managed to provide key information for the ICR, filling gaps due to low M&E capacity of implementing actors. 105. Skills development opportunities can contribute to structural transformation and measuring the quality of employment (and not only employment itself) allows to capture this impact. The project aimed to lift youth into more productive, better paying jobs. In a country as poor as Burkina Faso, with little safety nets, most youth are forced to work, so the main challenge is not employment per se but the quality of employment. The tracer study conducted at the end of the project provided evidence for noticeable effects on the quality of employment (type of employment and income). In order to capture this important effect and monitor it throughout project implementation, it would have been important to not only include an indicator on employment after the training but also on employment quality in the Results Framework. 106. An integrated approach to promote youth employment is needed, if ambitious targets for employment after the training are set. In order to integrate youth into the labor market, projects do not only need to increase skills of youth (supply side of the labor market) but also make sure there is sufficient matching labor demand, for employees or for the labor, products or services of those self-employed. Otherwise it is difficult to reach ambitious targets for employment after training participation, as the ones set by the Youth Employment project in Burkina Faso. Recent systematic reviews show that stand-alone supply interventions have had mixed results; less than 30 percent of programs evaluated have had a positive impact and successful programs have had only small impacts. 48 Integrating supply and demand components in the same project has proven to be difficult, but projects across the World Bank portfolio can and should complement each other in this regard. 48 Kluve, J., Puerto. S, Robalino. D, Romero. J.R, Rother. F, Stöterau. J, Weidenkaff. F, Witte. W. 2016. „Do Youth Employment Page 38 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) 107. Combining skills training with access to capital is key when promoting self-employment and entrepreneurship. Within the same project, trainings should be combined with access to capital when self- employment and entrepreneurship is the objective, to make sure that those starting an activity are able to obtain funding. In the case of the Youth Employment project in Burkina Faso, lack of funding was cited by beneficiaries as the key obstacle for their labor market integration after participation in the project (DGESS 2020). The focus group discussions also showed that lack of financing after the trainings was a source of frustration for the participants. B2B session were organized for entrepreneurship training participants, as originally planned in the PAD, but the financial institutions involved in these sessions usually do not give out start-up loans to micro- enterprises that lack collateral. The inclusion of the business plan competition in the project as part of the first restructuring was an important step to provide the needed funding to beneficiaries. The tracer study results also show, that assuring that enough LIPW man days are provided to allow beneficiaries to save and accompany the LIPW with financial inclusion and savings promotion measures is important to allow them to start or expand an activity. 108. Labor-intensive public works are more effective as a social safety net than an employment promotion tool and are complex to implement. Only if implemented at large scale, LIPW can have noticeable impacts on the demand side of the labor market through increasing wages for unskilled labor and multiplier effects through increased spending by beneficiaries (Blattmann and Ralston 2015). 49 The infrastructure built or rehabilitated through LIPW can contribute to these demand side effects, but only if it is productive infrastructure, increasing access to markets and relevant services as well as agricultural production. The PIU also needs the competencies to monitor the quality of the infrastructure built. At the same time, the PIU needs to make sure that delivering the infrastructure does not replace providing temporary youth employment as the key objective, as was the case for the rural LIPW in Burkina Faso (where Helvetas provided high quality infrastructure at the expense of labor intensity and hired workers over 35 years to deliver on the infrastructure targets under time pressure). On the supply side, LIPW can have noticeable impacts by allowing households to invest in productive assets and potentially more rewarding but also more risky activities, but only if providing a certain level of income over time. In the case of the Youth Employment project in Burkina Faso, the urban LIPW seem to have had a certain impact on future self-employment through the facilitation of savings, while the rural LIPW employed a high number of beneficiaries for a low number of days each to make up for implementation delays, which diminished potential positive impacts on the productive capacity of the beneficiaries. Evidence on the acquisition of skills through LIPW in general is mixed (Gehrke and Hartwig 2018). In any case, LIPW and training would need to be implemented at the same time to exploit the synergies between increased income and skills to allow participants to launch a new activity. During appraisal, it also seems important to conduct a thorough assessment of the respective labor market to understand who will be attracted by LIPW and what their potential is to integrate beneficiaries into the labor market. In Burkina Faso, for example, rural LIPW were less attractive for male beneficiaries as jobs in gold mines became available. Programs Improve Labor Market Outcomes? A Systematic Review.” IZA DP No. 10263.; and Cho, Y.; Robalino, D; Watson, S. 2014. Supporting Self-Employment and Small-Scale Entrepreneurship: Potential programs to improve livelihoods for vulnerable workers. Washington, DC: World Bank Group. 49 Blattman, Christopher, and Laura Ralston. 2015. “Generating Employment in Poor and Fragile States: Evidence from Labor Market and Entrepreneurship Programs.” Available at SSRN 2622220. Page 39 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) . Page 40 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Coverage of out-of-school youth with temporary employment and skills developement opportunities Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 46800.00 34200.00 43703.00 31-Mar-2013 15-Dec-2018 15-Dec-2019 15-Dec-2019 Female beneficiaries Percentage 0.00 30.00 57.00 30-Apr-2018 Comments (achievements against targets): The number of direct project beneficiaries was revised downwards during the first restructuring to 33,500. The target was later corrected in the Operations Portal to 34,200, to include the vocational training beneficiaries which had been omitted in the sum of the overall beneficiaries. Objective/Outcome: Access to temporary employment for out-of-school youth Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of man days in Labor Number 0.00 3900000.00 3172194.00 2277092.33 Page 41 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Intensive Public Works 31-Mar-2013 15-Dec-2018 15-Dec-2019 15-Dec-2019 created Comments (achievements against targets): The number of man days in Labor Intensive Public Works was revised downwards during the first restructuring. Objective/Outcome: Skills development opportunities for out-of-school youth Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Youth who completed Percentage 0.00 100.00 80.00 95.00 training programs offered by the Project (disaggregated by 31-Mar-2013 15-Dec-2018 15-Dec-2019 15-Dec-2019 program and gender) Comments (achievements against targets): The target was revised downwards during the first restructuring. Overall, of those who completed the training, 43% were women. Vocational training: Completion rate 100%, of which 2% women On-demand training: Completion rate 96%, of which 27% women Apprenticeships: Completion rate 84%, of which 44% women Entrepreneurship training: Completion rate 100%, of which 51% women Page 42 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Youth trained who are Percentage 0.00 100.00 60.00 60.00 employed or self-employed one year after completion of 31-Mar-2013 15-Dec-2018 15-Dec-2019 31-Mar-2019 training (overall and by sector, in project areas) Comments (achievements against targets): The target was revised downwards during the first restructuring. A.2 Intermediate Results Indicators Component: Component 1: Labor Intensive Public Works Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Roads Kilometers 0.00 240.00 120.00 133.33 maintained/rehabilitated (rural) 31-Mar-2013 15-Dec-2018 15-Dec-2019 15-Dec-2019 Roads Kilometers 0.00 60.00 60.00 maintained/rehabilitated (urban) 30-Apr-2018 Page 43 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Community woodlots Hectare(Ha) 0.00 2400.00 355.00 124.80 maintained 30-Apr-2018 reclaiming floodable areas Hectare(Ha) 0.00 0.00 200.00 107.19 (amenagement des bas- fonds) large diameter wells built Kilometers 0.00 0.00 37.00 41.00 Comments (achievements against targets): The target was revised downwards during the first restructuring. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of youth employed Number 0.00 31000.00 20500.00 29332.00 through LIPW 31-Mar-2013 15-Dec-2018 15-Dec-2019 15-Dec-2019 Proportion of female Percentage 0.00 30.00 64.00 participants of LIPW 30-Apr-2018 Comments (achievements against targets): Page 44 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) The target was revised downwards during the first restructuring. The number of rural LIPW beneficiaries was estimated by the Statistical Division of the Ministry of Youth using the handwritten lists with the names of those who worked on each task. They counted those cases with the same name and identification card number as one beneficiary. Of the 5 721 attendance sheets, 25 where not readable and the number of participants per sheet was conservatively estimated. Component: Component 2: Skills development Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Out of school youth enrolled Number 0.00 15800.00 13700.00 15061.00 in skills development programs 31-Mar-2013 15-Dec-2018 15-Dec-2019 15-Dec-2019 Of which female (out of Percentage 0.00 30.00 43.00 school youth who are enrolled in skills 30-Apr-2018 development programs) Out of school youth Number 0.00 3800.00 2700.00 3873.00 enrolled in professional training 31-Mar-2013 15-Dec-2018 15-Dec-2019 15-Dec-2019 Out of school youth Number 0.00 4000.00 3000.00 3471.00 enrolled in apprenticeships 31-Mar-2013 15-Dec-2018 15-Dec-2019 15-Dec-2019 Page 45 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Out of school youth Number 0.00 8000.00 20761.00 enrolled in entrepreneurship training 31-Mar-2013 15-Dec-2018 15-Dec-2019 Number of prize winners of Number 0.00 0.00 1200.00 1239.00 a business plan competition 28-Feb-2017 15-Dec-2018 15-Dec-2019 15-Dec-2019 Comments (achievements against targets): Aggregated from the number of beneficiaries who were enrolled in vocational and on demand training (=professional training), apprenticeships as well as entrepreneurship training, without counting those twice who benefited from apprenticeship as well as entrepreneurship training and without counting those that benefited from entrepreneurship training as part of component 1. . The target was revised downwards during the first restructuring to 13,000. It was later corrected to 13,700 as those enrolled in the vocational training had been accidentally omitted. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Out of school youth who Percentage 0.00 100.00 80.00 96.00 completed in professional training 31-Mar-2013 15-Dec-2018 15-Dec-2019 15-Dec-2019 of which female (Out of Percentage 0.00 30.00 26.00 school youth who completed in professional 30-Apr-2018 Page 46 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) training) Comments (achievements against targets): The target was revised downwards during the first restructuring of the project. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Out of school youth who Percentage 0.00 100.00 80.00 84.00 completed in apprenticeships 31-Mar-2013 15-Dec-2018 15-Dec-2019 15-Dec-2019 of which female (out of Percentage 0.00 30.00 44.00 school youth who completed in 30-Apr-2018 apprenticeships) Comments (achievements against targets): The target was revised downwards during the first restructuring of the project. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Out of school youth who Percentage 0.00 100.00 80.00 100.00 completed in business development programs 31-Mar-2013 15-Dec-2018 15-Dec-2019 15-Dec-2019 Page 47 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Of which female (Out of Percentage 0.00 30.00 51.00 school youth who completed 30-Apr-2018 entrepreneurship /business development programs) Comments (achievements against targets): The target was revised downwards during the first restructuring of the project. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of youth enrolled in Number 0.00 1800.00 700.00 48.00 the two new training institutions supported by the 31-Mar-2013 15-Dec-2018 15-Dec-2019 15-Dec-2019 project in key economic growth sectors Proportion of female (youth Percentage 0.00 30.00 2.00 enrolled in the two new training institutions 30-Apr-2018 supported by the project in key economic growth sectors) Comments (achievements against targets): The target was revised downwards during the first restructuring of the project. Page 48 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Component: Component 3: Institutional Capacity Strengthening and Project Management Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Impact Evaluation Studies Yes/No N Y N Conducted 31-Mar-2013 15-Dec-2018 15-Dec-2019 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Beneficiary surveys Yes/No N Y N completed to support on- going training improvement 31-Mar-2013 15-Dec-2018 15-Dec-2019 Comments (achievements against targets): Page 49 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 Access to temporary employment for out-of-school youth 1. Direct beneficiaries and proportion of female Outcome Indicators 2. Number of man days in Labor Intensive Public Works created 1. Roads maintained/rehabilitated (rural) 2. Roads maintained/rehabilitated (urban) 3. Community woodlots maintained Intermediate Results Indicators 4. Reclaiming floodable areas 5. Large diameter wells built 6. Number of youth employed through LIPW and proportion of female 1. 20,500 youth enrolled in rural and urban LIPW incl. trainings and savings programs Key Outputs by Component 2. Critical infrastructure established or rehabilitated for productive (linked to the achievement of the Objective/Outcome 1) activities, access to services & markets 3. 4 Job Counseling Youth Centers constructed and equipped 4. Technical Assistance for Government on LIPW provided Objective/Outcome 2 Skills development opportunities for out-of-school youth 1. Direct beneficiaries and proportion of female 2. Youth who completed training programs offered by the Project Outcome Indicators 3. Youth trained who are employed or self-employed one year after completion of training 1. Out of school youth enrolled in skills development programs (professional training, apprenticeships, entrepreneurship training), of Intermediate Results Indicators which female 2. Number of prize winners of a business plan competition 3. Out of school youth who completed professional training, of which Page 50 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) female 4. Out of school youth who completed in apprenticeships, of which female 5. Out of school youth who completed in business development programs, of which female 6. Number of youth enrolled in the two new training institutions supported by the project in key economic growth sectors, of which female 1. Two dual vocational centers built and equipped and 700 beneficiaries enrolled 2. 2000 beneficiaries enrolled in newly established on-demand Key Outputs by Component training system (linked to the achievement of the Objective/Outcome 2) 3. 3000 beneficiaries enrolled in newly established apprenticeship program 4. 8000 beneficiaries enrolled in an entrepreneurship training program Page 51 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Hamoud Abdel Wedoud Kamil Task Team Leader(s) Mamata Tiendrebeogo Procurement Specialist(s) Edith Atioumoutio Zannou Tchoko Financial Management Specialist Lucienne M. M'Baipor Social Specialist Africa Eshogba Olojoba Environmental Specialist Juliana Victor-Ahuchogu Monitoring and Evaluation Specialist Supervision/ICR Rebekka E. Grun Task Team Leader(s) Mathias Gogohounga Procurement Specialist(s) Sandrine Egoue Ngasseu Financial Management Specialist Suzane Kabore Rayaisse Procurement Team Yolande Bougouma-Zagre Procurement Team Ayaba Gilberte Kedote Team Member Catherine Marie Z. Compaore Procurement Team Fatoumata Diallo Social Specialist Djeneba Bambara Sere Team Member Lydie Anne Billey Team Member Bougadare Kone Environmental Specialist Julian Alexander Koschorke Team Member Ousseini Ouedraogo Team Member Leandre Yameogo Environmental Specialist Roland Berenger Berehoudougou Team Member Page 52 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Kirsten Schuettler ICR author B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 2.100 40,352.28 FY13 21.262 218,028.75 FY14 13.982 56,431.58 FY15 6.927 51,692.74 FY16 9.170 58,845.20 FY17 0 929.83 FY18 0 0.00 Total 53.44 426,280.38 Supervision/ICR FY14 0 6.96 FY15 .360 4,640.40 FY16 2.802 36,322.62 FY17 22.779 189,821.29 FY18 23.960 199,832.73 FY19 25.028 175,646.06 FY20 27.226 106,335.49 Total 102.16 712,605.55 Page 53 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Component 1: Labor 25.00 25.83 103% Intensive Public Works Component 2: Skills 20.00 14.02 70% development Component 3: Institutional Capacity Strengthening and 5.00 5.15 103% Project Management Total 50.00 45.01 90% Page 54 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) ANNEX 4. EFFICIENCY ANALYSIS 1. In the interest of comparison, the efficiency analysis conducted for the ICR builds on the economic analysis conducted at appraisal by assessing the net present value (NPV) and internal rates of return (IRR) of the LIPW and trainings with a similar methodology and assumptions. The benefits of the project consist of quantifiable and nonquantifiable attributes. The cost-benefit analysis focused on the quantifiable benefits and provided the economic rationale for the intangible benefit portions. Since the components of the project vary in terms of the target population, and implementation mechanism, we calculate the cost-benefit for each tangible component separately, just like in the PAD. At the end, we combine the cost-benefit estimation of all tangible components based on plausible assumptions as well as the cost side of intangible portions of components to envision the project's benefit at the lower spectrum. In order to conduct the cost-benefit analysis, the model makes several assumptions about the project and the associated costs and benefits. The following presents the model assumptions, net present values (NPV) and internal rate of return (IRR) for the project and its components. 2. The cost-benefit analysis used different wage and employment scenarios. The cost-benefit analysis and its different scenarios are based on the fact that in Burkina Faso wages/earnings increase with the education level. For component 1, the incremental benefit-cost analysis is estimated using three scenarios: (i) pessimistic – estimated as the difference in annual earning between no education and some primary education categories; ii) realistic- estimated as the difference in annual earning between no education category and the average of some primary education and TVET annual earnings, and (iii) optimistic – difference in annual earning between no education and TVET categories. For component 2, it is estimated based on earnings difference between primary and TVET categories. For both components, 50 percent chance of employment after the completion of the project was taken as the target and relaxed to 40 percent and 30 percent for sensitivity analysis. 50 3. The intangible components comprise sub-component 2.3 - entrepreneurship training and component 3-capacity building. Component 1 and sub-components 2.1 A, 2.1 B and 2.2 signify the tangible portion of the project, accounting for over 80 percent of the total project costs. Although short term trainings such as entrepreneurship training may have a huge social and long-term economic impact, the PAD assessed that it was impossible to quantify the opportunity costs and benefits of such short-term trainings. In other words, the entrepreneurship-training sub-component is too short to quantify the benefit (3 days to 3 weeks training), although in most of the cases, short training can easily invent the wheel leading to the enormous benefit. As the result, the cost-benefit analysis only captures the cost side of this category (i.e. the opportunity cost of training and benefit is not estimated) as part of the project cost-benefit summary but not as part of the component 2. As component 3 purely targeted capacity building, standard cost-benefit analysis could not be performed. However, since the component aimed to improve the general framework for skill development and employment it is likely that it will have tangible value added for the future. 4. The original economic analysis in the PAD was based on the following general assumptions: 50The only exception was component 2.2. (apprenticeships) which assumed 75 percent as target. The other target that was set lower in the cost-benefit analysis compared to the Results Framework was the number of LIPW man days used, 3.5 million instead of the indicator target of 3.9 million. Page 55 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) • The analysis will take into account only quantifiable benefits. As a result, we assume that the internal rate of return (IRR) we compute is a lower bound for the impact of the project. • We use 10 percent discount rate to compute future returns. • We assume 5 percent annual inflation rate growth on expected benefit or nominal wage growth based average wage growth by age. • We assume the project will partially be implemented in 2013 • We estimated a wage rate for different levels of education and by age using EICVM 2009/2010 (as shown on figure 1 above). • We assume post intervention employment wage is the entry wage for cohorts to their average age of youth and growth by the age cohort wage plus 5 percent inflation adjustment. 5. It also made specific assumptions for Component 1: • Since the project targets unskilled and unemployed youth in urban areas and during off agricultural pick seasons in rural areas, there is no opportunity cost associated for the program participation. Hence, the total cost is the project cost, US$25 million. • We assume there is no incremental cost for the project as there will be no activity after the end of the six months employment contract as part of the component (although they can participate in a subsequent component). • We assume that the beneficiaries of the programs work for a period of 35 years after completion of the program given that the average age of the youth is 23 year-old. • Benefits of the infrastructure constructed through rural LIPW cannot be quantified ex-ante and are thus not included. 6. For component 2, the analysis at appraisal focused on the first three sub-components and made a number of additional assumptions. • Under this component, the total costs account for: (i) project cost for quantifiable subcomponents, (ii) forgone opportunity costs, and (iii) project running and maintenance costs accosted to the two training centers. • The project cost is the component share of the project's budget while the opportunity costs are the unearned wages due to participation in the program. • The first subcomponent has an intervention plan that will go beyond the project's life cycle in a sense that project complements with the existing plan for building two training centers. Thus, unlike component 1, this component involves costs after the project life cycle including trainer's salaries, subsidies for the beneficiaries and maintenance and running costs for the two centers. • 1.5 percent annual maintenance costs of the two training centers are assumed with 35 years life time. • 20 permanent trainers are assumed with some higher education training level based on 30:1 trainees to trainer ratio. Since trainers could be any age, the salary is estimated on the average earning of some higher education level with 5 percent inflation adjustment overtime. • Other running costs including equipment costs were assumed to be depreciated every 10 years and replaced by the original cost less 10 percent salvation value. In addition, 1.5 percent annual maintenance cost is applied for the life term of the running costs. • For the first subcomponent, the entry wage is measured as the average wage earned by an entry wage level for the primary education category. • The incremental benefits for the short term on demand training are assumed to be the earning difference between primary education wage level and TVET education level. This is after the completion of the nine Page 56 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) months training, which is possibly the lower bound given that primary education is required for the participation in the training program and firms may show interest in better educated people for demand driven training. • The incremental benefits for the apprenticeship training are assumed to be the earning difference between no education wage level and TEVT level. No education level is assumed based on the targeted population as described in the main section of the PAD. The training is, on average, 18 months, and the estimation from EICVM 2009/2010 shows that TVET wage is somewhere between lower secondary and upper secondary earning ranges, and earning level of TVET is more likely to be attained. 7. For the actual calculation for the ICR we used the same methodology and assumptions but adapted the following: The exchange rate used, the actual budget disbursed per (sub-)component, the actual number of beneficiaries and the actual start date of the activities. 8. The cost-benefit analysis shows that it was worth investing in the project, even if the net present value (NPV) and internal rates of return (IRR) for the overall project were lower than expected. The first restructuring of the project in June 2017 revised the projected Internal Rates of Return moderately downwards. The revised IRR were deliberately conservative. The downgrade was overestimated because the team conservatively kept the original cost estimates but combined them with lower overall beneficiary numbers reflecting the new targets. The net present value and internal rate of return estimated at completion were lower than expected at project preparation and at restructuring. Based on a discount rate of 10 percent, the tangible cost-benefit analysis shows a present discounted value of benefits of the overall project of US$34.5 million while the present discounted value of costs is estimated to be US$25.3 million, with a net present value (NPV) of US$9.2 million, lower than the 25.2 million estimated in the PAD and the $16.12 million estimated at restructuring (see tables A4-1 and A4-2). The overall cost benefits analysis result (including tangible costs but net benefit) leads to a net present value (NPV) of $0.9 million, compared to $18.1 million in the PAD and $9 million at restructuring. The lowest overall internal rate of return (IRR) is 10 percent, compared to 15 percent in the PAD and 12 percent after restructuring. In the PAD, the sensitivity analysis showed that the IRR may range from 14 percent at the low end (worst case) to 54 percent at the high end. Overall, the analysis therefore shows that it was worth investing in the project, but the returns were lower than expected at project preparation and restructuring. Table A4-1: Net Present Value (NPV) and Internal Rate of Return (IRR) Under Different Scenarios in the PAD and after restructuring (in italic) Combination of realist Probability of Present Present Net Increment Internal assumption of component 1 employment Value of Value of Present al Benefit Rate and component 2 Total Benefits benefit /Cost ratio of Return Cost (IRR) Tangible cost benefit analysis (excludes costs and benefits of intangible investment) High probability of Project target (50 $39.1 $97.2 / $58.1 / 2.5 / 28% / employment percent) $81.0 $41.8 2.1 23% Moderate probability of 10% less than $39.1 $80.3 / $41.1 / 2.1 / 23% / employment target $67.8 $28.6 1.7 19% Page 57 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Less probability of 20% less than $39.1 $64.4 / $25.2 / 1.6 / 18% / employment target $55.2 $16.12 1.4 16% Overall cost benefits analysis result (includes costs but excludes benefits of intangible investments) High probability of Project target (50 $46.3 $97.2 / $51.0 / 2.1 / 23% / employment percent) $81.0 $34.7 1.4 19% Moderate probability of 10% less than $46.3 $80.3 / $34.0 / 1.7 / 19% / employment target $67.8 $21.5 1.3 16% Less probability of 20% less than $46.3 $64.4 / $18.1 / 1.4 / 15% / employment target $55.2 $9.0 0.7 12% Table A4-2: Net Present Value (NPV) and Internal Rate of Return (IRR) under different scenarios at project completion Combination of realist Probability of Present Present Net Increment Internal assumption of component 1 employment Value of Value of Present al Benefit Rate and component 2 Total Benefits benefit /Cost ratio of Return Cost (IRR) Tangible cost benefit analysis (excludes costs and benefits of intangible investment) High probability of Project target $25.3 $55.6 $30.3 2.2 20% employment Moderate probability of 10% less than $25.3 $45.0 $19.8 1.8 17% employment target Less probability of 20% less than $25.3 $34.5 $9.2 1.4 13% employment target Overall cost benefits analysis result (includes costs but excludes benefits of intangible investments) High probability of Project target $33.6 $55.6 $22.0 1.7 15% employment Moderate probability of 10% less than $33.6 $45.0 $11.4 1.3 13% employment target Less probability of 20% less than $33.6 $34.5 $.9 1.0 10% employment target 9. For the LIPW of component 1, the actual NPV and IRR were lower than expected ($2 million instead of $14.5 million, and 11 vs. 19 percent in the realistic scenario ) but infrastructure benefits were not included. This is mostly due to the rural LIPW which paid out a much lower amount on wage payments but nevertheless spent more than originally planned. The benefits of the infrastructure constructed as part of the LIPW in rural communes were, however, not included in the efficiency analysis. The investments were similar to those undertaken under the First and Second Community Based Rural Page 58 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Development Projects (CBRDP 1 and 2). An ex-post economic and financial evaluation of sub-projects financed under the Second Phase of CBRD-2 carried out during the preparation of the third phase of the Program found very high rate of returns (74 percent overall for the mix of sub-projects) and positive benefit-to-cost ratios (ranging from 3.7 for water supply and small multipurpose water works to 1.8 for soil and water conservation) and health infrastructure. Component 1 Net Present Value (NPV) and Internal Rate of Return (IRR) under different scenarios at project completion Probability post Present Value of Net Present Incremental benefit Present Value of Total Cost intervention Benefits benefit ration/Project cost Internal Rate of Post intervention earning assumptions employment ratio Return (IRR) Project target $17.7 $10.3 -$7.4 0.6 6% Pessimistic: Entry wage of some primary education level 10% less than target $17.7 $8.0 -$9.6 0.5 4% 20% less than target $17.7 $5.8 -$11.9 0.3 2% Project target $17.7 $33.4 $15.7 1.9 17% Realistic: Entry wage of average of primary and TVET level 10% less than target $17.7 $26.5 $8.8 1.5 14% 20% less than target $17.7 $19.6 $2.0 1.1 11% Project target $17.7 $56.4 $38.8 3.2 27% Optimistic: Entry wage of TVET level 10% less than target $17.7 $45.0 $27.3 2.5 23% 20% less than target $17.7 $33.5 $15.8 1.9 18% 10. For the trainings in component 2, the actual NPV was lower but the IRR higher than expected ($7.6 instead of $10.7 million, and 20 vs. 17 percent in the realistic scenario ). Project activities started much later than expected for all sub-components and could not be actually implemented for the vocational training sub-component 2.1 A due to delays in the construction of the training centers. On the other hand, actual costs were also much lower than expected for the sub-component 2.1 A, due to payment of allowances for a much smaller number of beneficiaries (less than 3 percent of original target) and the cancellation of heavy equipment for the training centers and training costs. The latter might have a negative impact on sustainability if the Ministry is not able to set aside the required budget (see section D. Risk to Development Outcome). Component 2 Net Present Value (NPV) and Internal Rate of Return (IRR) under different scenarios at project completion Probability post Present Value of Net Present Incremental benefit Present Value of Total Cost intervention of Benefits benefit ration/Project cost Internal Rate of Post Intervention entry wage of TVET level employment ratio Return (IRR) High probability of employment Project target $7.6 $22.9 $15.3 3.0 26% Moderate probability of employment 10% less than target $7.6 $19.0 $11.5 2.5 23% Less probability of employment 20% less than target $7.6 $15.2 $7.6 2.0 20% Page 59 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Translated comments from the borrower (see original version below): I would like to send you our amendments to the World Bank’s implementation completion and results report of the PEJDC project. We went through the document with a focus on the statistical data. Overall, we have no particular comments in form nor substance. The only observation we have is on page 30: for the Job Fair, it should be noted that 7,712 young people were trained in Job Search Techniques and 5,131 jobseeker cards were issued for the benefit of young people. You will find these amendments in track changes. I would like to commend the quality of the report which faithfully reflects the results and effects of the Project. Thank you to you and to the World Bank team. From: Amidou BANCE Sent: Wednesday, September 9, 2020 10:54 AM To: Rebekka E. Grun Cc: KOUDOUGOU Lamoussa ; Roland Berenger Berehoudougou ; Ayaba Gilberte Kedote ; Kirsten Schuettler ; Julian Alexander Koschorke Subject: Re: Draft: Rapport d'evaluation du PEJDC, Banque Mondiale [External] Bonjour Madame Rebekka, Je voudrais vous transmettre nos amendements sur le rapport d’évaluation du projet PEJDC par la Banque Mondiale. Nous avons parcouru le document en mettant l'accent sur les données statistiques. Dans l'ensemble, nous n'avons pas d'observations particulières dans la forme et le fonds. La seule observation que n'avons c'est à la page 30 : pour le Salon de l'emploi, il conviendrait de préciser que 7 712 jeunes ont été formés en Technique de Recherches d'Emplois et 5 131 cartes de demandeurs d'emplois ont été établies au profit des jeunes. Vous trouverez ces amendement en suivi de modification. Je voudrais saluer la qualité du rapport qui rend fidèlement compte des résultats et des effets du Projet . Merci à vous et à l'équipe de la Banque Mondiale. Page 60 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) ANNEX 6. BENEFICIARES OF THE ENTREPRENEURSHIP SUB-COMPONENT Beneficiaries Number of Percentage Activity enrolled women of women Entrepreneurship training, of which 8142 4113 51% - youth with little to no schooling 5702 3419 60% - university graduates 2015 487 24% - CMA-BF apprentices 425 207 49% Coached, of which 5663 - youth with little or no schooling 4345 - coached university graduates 1115 - coached apprentices 203 B2B 5309 - youth with little or no schooling 4150 - university graduates 960 - apprentices 199 Business plan competition grants 1239 785 63% - youth with little or no schooling 1026 733 71% - university graduates 213 52 24% Page 61 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) ANNEX 7. METHODOLOGY OF THE TRACER STUDY A tracer study was conducted for the Burkina Faso Youth Employment and Skills Development Project. It was part of a broader evaluation of the project conducted by the Statistical Department (Direction Générale des Etudes et des Statistiques Sectorielles - DGESS) in the Ministry of Youth at the end of the project with support from the World Bank. Quantitative and qualitative data was collected. For the qualitative data collection, 35 focus groups were conducted with 245 beneficiaries of both components, as well as 34 individual interviews with project stakeholders. A survey collected quantitative data among 1316 beneficiaries. The questionnaire traces the professional situation of each beneficiary at three different moments: (i) Before the project; (ii) Immediately after participation in the project; and (iii) at the time of the survey. As the ICR draws on the results of the tracer study, this annex summarizes the methodology of the survey. Sample frame. The sample frame for the quantitative data collection excluded an important number of project beneficiaries. For the rural LIPW the lists of beneficiaries had to first be digitized, as they were only available as scans or photos of the paper versions. Some lists were missing and 25 were not readable. 51 When inputting the data, beneficiaries which seemed to appear more than once were only included once. For the component 2 sample frame, only the initial on demand training, apprenticeships and the entrepreneurship training (only 5290 of 7717 beneficiaries) were included. The sample frame was cleaned, excluding all those where not only the name but also the identity card number and phone number were the same. In terms of bias, it is also important to note, that all those without contact information were excluded. The latter strongly reduced the number of LIPW beneficiaries in the sample frame. Table A7-1: Sample frame tracer study Components Before cleaning After cleaning LIPW 26839 2870 LIPW rural 14414 750 LIPW urban 12425 2120 Skills Development 11182 10592 On-demand training (initial only) 2420 1842 Apprenticeships 3472 3460 Entrepreneurship training 5290 5290 Total 38021 13462 Source: DGESS (2020) 51 After the survey, following a request of the World Bank and the PCU, the DGESS also worked on estimating the total number of rural LIPW beneficiaries. The PCU shared the missing lists. They counted those cases with the same name and identification card number as one beneficiary. Of the 5 721 attendance sheets, 25 where not readable and the number of participants per sheet was conservatively estimated. This led to an estimate of 15344 beneficiaries. Page 62 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Sampling. The sampling stratified by sub-components for component 2 but rural and urban LIPW were included in one stratum. When drawing the sample, some additional duplicates were discovered. The communes of Kain and Barga, where the rural LIPW were implemented, had to be excluded due to security reasons. Ouagadougou, where urban LIPW were implemented, was excluded as all beneficiaries lacked contact information. The results of the study are thus only representative for those surveyed and not for all beneficiaries of the project. Results can also not be interpreted causally, as there is no comparison or control group. Data collection. Interviewers were trained for two days. Data was collected using CAPI (Computer Assisted Personal Interviews) through tablets with CSPro (Census and Survey Processing System). Data was collected in seven days. Non-response rates were highest for LIPW (23 percent) and lowest for initial on- demand training participants (3 percent) (see table A7-2 below). The main reason for non-response was phone numbers not working. Table A7-2: Sample tracer study Non- response Non- with response regards to with Increased Actual increased regards to Sample Sample size sample size sample sample size sample Component frame target target size target size target Component 1: LIPW LIPW 2 870 592 642 456 29% 23% Component 2: Skills Development On-demand training (initial only) 1842 290 354 281 21% 3% Apprenticeships 3460 314 411 273 34% 13% Entrepreneurship training 5290 324 411 306 26% 6% Total 13462 1520 1818 1316 28% 13% Source: DGESS (2020) Page 63 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) ANNEX 8. MAPS OF INFRASTRUCTURE BUILT THROUGH RURAL LIPW Figure 1: Rural LIPW infrastructure assessment Barga commune Source: Earth Observation for Sustainable Development (EO4SD)-fragility (SIRS for ESA/World Bank, 2020) Page 64 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Rural LIPW infrastructure assessment Kain commune Source: EO4SD-fragility (SIRS for ESA/World Bank, 2020) Page 65 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Rural LIPW infrastructure assessment Rambo commune Source: EO4SD-fragility (SIRS for ESA/World Bank, 2020) Page 66 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Example of infrastructure validation for water retention bassins (boulis) in Barga Source: EO4SD-fragility (SIRS for ESA/World Bank, 2020) Page 67 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Example of before-after comparisons for water retention bassins (boulis) in Rambo Source: EO4SD-fragility (SIRS for ESA/World Bank, 2020) Page 68 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) Example of detailed analysis of roads in the commune of Kain Source: EO4SD-fragility (SIRS for ESA/World Bank, 2020) Page 69 of 70 The World Bank BF-Youth Employment & Skills Development (P130735) ANNEX 9. EXECUTIF SUMMARY OF GOVERNMENT ICR (TRANSLATION) The Government of Burkina Faso has signed a financing agreement with the International Development Association (IDA) of the World Bank Group, to implement the "Youth Employment and Skills Development Project (PEJDC)". The official launch took place on June 20, 2014. Originally scheduled to close on December 15, 2018, the project was closed on December 15, 2019. This implementation completion and results report describes the overall performance and immediate effects of the project. It reviews the factors that affected the implementation and the results and explains the extent to which the procedures described in the procedure manuals have been followed. According to the final evaluation of the project and additional studies, the results achieved testify to the performance of project implementation. Despite the difficulties linked to the late start and the insecurity in certain intervention areas, the PEJDC has produced results beyond the objectives initially set. The number of young people, in line with the development objective of the Project, who benefited directly from the project's intervention at the end of December 15, 2019 is 43,703 beneficiaries (with 56.91% women), i.e. a rate of performance of 127.78% on the revised target of 34,200 youth and 93.38% on the initial target of 46,800 youth. The cumulative expenditure statement (IDA and government funds) is 26,319,738,745 FCFA at project closing date, i.e. a financial execution rate of 105.29% of the project cost at formulation. The project achieved unanticipated results which are mainly the implementation of the accompanying measures to the labor intensive public works (“mesures sociales”) of the government, which made it possible to train 9,718 beneficiaries on societal themes and Income Generating Activities (IGA), and the organization of a National Employment Fair in Bobo and Ouagadougou, which helped strengthen the employability of 14,542 young people. In addition, 20 mobile childcare sites (crèches mobiles) were set up in 20 communes and districts and parental education sessions in 18 communes were organized for the benefit of the government LIPW beneficiaries. In total, 67,963 young people benefited directly from the Project (Project development objective and unintended results). In addition to the jobs created and the income distributed under the project, youth benefited from technical training (initial training, training at the request of companies, entrepreneurship trainings and apprenticeships), basic training (functional literacy, training on societal topics, and initiation to entrepreneurship and IGA). Urban and rural communes, implementing agencies, private companies and public administration staff have had their capacities strengthened. Lessons learned, good practices and experiences to be continued and scaled up have been identified. Page 70 of 70