Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) Report Number : ICRR0020585 1. Project Data Project ID Project Name P121686 LR: Youth, Employment, Skills Project Country Practice Area(Lead) Liberia Social Protection & Labor L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-H5940,TF-97110 30-Jun-2013 19,400,000.00 Bank Approval Date Closing Date (Actual) 24-Jun-2010 30-Jun-2016 IBRD/IDA (USD) Grants (USD) Original Commitment 6,000,000.00 10,000,000.00 Revised Commitment 6,000,000.00 10,000,000.00 Actual 6,204,188.65 10,000,000.00 Prepared by Reviewed by ICR Review Coordinator Group Anthony Martin Tyrrell Judyth L. Twigg Joy Behrens IEGHC (Unit 2) PHPROJECTDATATBL Project ID Project Name P147967 YES Project Additional Financing ( P147967 ) L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) 3,400,000.00 Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) Bank Approval Date Closing Date (Actual) 01-Dec-2014 IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 3,400,000.00 Revised Commitment 0.00 3,369,406.58 Actual 0.00 3,369,406.58 2. Project Objectives and Components a. Objectives The project development objective (PDO) is consistently stated across relevant documentation (Financing Agreement, p. 5; Project Appraisal Document, p. 5) as follows: "to expand access of poor and young Liberians to temporary employment programs and to improve youth employability." For the purposes of this review, IEG will assess two separate objectives: (i) to expand access of poor and young Liberians to temporary employment programs; and (ii) to improve youth employability. The PDO remained unchanged for the duration of the project; however, there were three restructurings. The second, in June 2013, involved changes to Key Performance Indicators (KPIs) and associated outcome targets; therefore, this review takes a split rating approach (assessing against both original and revised targets). At the time of the June 2013 restructuring, US$12.74 million was disbursed, representing 65 percent of the total amount disbursed at closing. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? Yes d. Components The project was structured under the following two components, each of which had two sub-components. 1. Community Livelihoods (changed wording in December 2014 restructuring from the original, "Community Works") (Appraisal US$8.5 million; Actual: US$12.8 million). The first and Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) major subcomponent built on an existing cash-for-work program supported by the World Bank, the Cash- for-Work Temporary Employment Program (CfWTEP). Both the existing program and its elaboration through this project were implemented by the Liberian Agency for Community Empowerment (LACE). Community Works operated two sub activities -- public works and youth community farms -- in all of Liberia's 15 counties, offering access to income-generating activities to youth. Public works interventions were designed to be targeted, on a geographic basis country-wide, at the most vulnerable populations and poor households. Investments in rural areas were designed to create jobs, increase food production, and provide income as part of the economic recovery. The sub-component also included a non-cognitive skills module to reinforce basic life skills and workforce readiness behaviors. The second sub-component supported capacity building for the coordination and monitoring of public works with the longer term objective of building the capacity within government -- the Ministry of Planning and Economic Affairs in coordination with the Ministry of Labor and of Public Works -- to coordinate, monitor, and orient a public works program as part of a full system of protection for the poor against systemic crises. It was also envisaged that the sub-component would finance: (i) an independent impact assessment of Community Works, covering, for example, net income gains of beneficiary households, the impact and use of these additional resources, targeting effectiveness of the project, and potential job substitutions effects; and (ii) a feasibility study to assess the possibility of and potential benefits from including private contractors. 2. Employment through Skills Training (Appraisal US$8.5 million; Actual: US$6.59 million). The first sub- component was to finance skills development activities toward enhancing employability and improving employment of youth through the provision of key skills in high demand in what is referred to in the PAD as the informal sector (Small and Medium Enterprises (SMEs) in manufacturing and services as well as small- scale agribusinesses and fisheries), and in the formal sector (medium to large enterprises mostly in mining and forestry, as well as in agriculture). The second sub-component was to assist in building an institutional and policy framework with the long-term objective of creating an efficient demand-driven Technical Vocational Education and Training (TVET) system that integrates both public and private sector service providers within a certification system that recognizes competency and skills development linked to job demand. Specifically, this would involve: (i) provision of technical support to planned policies for TVET, including the establishment of a crosssectoral TVET Commission to steer TVET programs and the TVET Fund, as well financing studies and holding conferences to better understand the needs of youth in the labor market; and (ii) provision of capacity building support for the Agricultural Industrial Training Bureau (AITB) to create a system of certification and to set standards and certification for a number of TVET skills and courses. The subcomponent also allowed for support to AITB to improve its physical facility, including equipment for running efficient testing and certification services. On the level of the training providers, the subcomponent allowed for training of trainers and training for improving institutional management. This component also included support for project management. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost Actual project cost was US$19.59 million, against original costing and commitments of US$16 million. The increase primarily reflects an additional US$3.4 million introduced by the Africa Catalytic Growth Fund (ACGF) on 12/11/14 and, marginally, XDR exchange rate fluctuations. Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) Financing The project was financed by an IDA (Crisis Response Window) credit of US$6 million, and an original allocation of US$10 million from the ACGF, which was supplemented by a further US$3.4 million in Additional Financing on 12/11/14. Borrower Contribution There was no planned or actual borrower contribution. Dates The project became effective on 09/13/2011, and the originally planned closing date was 06/30/2013. The project underwent three restructurings, one of which was associated with Additional Financing (AF). The first restructuring, on 11/26/2012, was technical and definitional in nature. The second, 06/19/2013, was material, and involved a reallocation of budget from Component 2 to Component 1, dropping one key indicator, rewording another, and changing some targets associated with key indicators. It also involved a project extension to 06/30/2014. The third restructuring, on 12/11/2014, was associated with the introduction of AF and involved a further extension of the project to its ultimate closing to 06/30/2016. 3. Relevance of Objectives & Design a. Relevance of Objectives The PDO was highly relevant in the context of the post-conflict situation in Liberia and remains so at project close, under both the original and revised outcome targets. The PAD (p. 3) notes that, in 2009, 63.8 percent of Liberia’s 3.5 million people (75 percent of whom were under 35 years of age) lived below the poverty line, with 47.9 percent living in extreme poverty. Many of these youth came of age through the second civil war and suffered its effects, including a lack of education, skills, and training. Although the economy had performed well at a macro level post-2003, it was estimated that, in 2009, between 20 and 30 percent of youth (defined as those between 16 and 35 years of age) were either unemployed or underemployed. The country was also hard hit by the unfolding global financial and economic crises. The ICR notes (p. 11) that the project sought to address issues (youth employment, income generation, and improved employability) consistent with the Government’s Poverty Reduction Strategy (PRS) and with the joint World Bank and the African Development Bank Country Assistance Strategy (CAS) at the time of appraisal. Following the Ebola outbreak in late 2013 / early 2014, during the extension of the project associated with AF, Liberian youth faced significant additional employment shocks. At project closing, the PDO continued to be relevant and was linked to the cross-cutting issues (Pillar V) and Human Development (Pillar III) pillars of Liberia’s Agenda for Transformation (AfT), and was aligned with the Bank Group’s Country Partnership Strategy (CPS) for Liberia FY13–FY17 that focuses on increasing resilience among poor and vulnerable households. For example, the CPS (p. 8) notes that, despite relatively robust economic growth, less than one-fifth of the labor force is in paid employment, and nearly 80 percent of the labor force is in vulnerable employment with the level at around 94 percent for rural women. As such, the PDO continues to be highly relevant to country context, national strategy, and Bank strategy. Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) Rating Revised Rating High High b. Relevance of Design The project's design was only modestly relevant under both the original and revised outcome targets. The use of a Specific Investment Loan (SIL) was appropriate given the concrete nature of the government- requested intervention. Planned activities were built around two components that reflected the thrust of the PDO. The first component, promoting a productive public works model, was simple and appropriate in context. It built on an existing program supported by the World Bank -- the Cash-for-Work Temporary Employment Project (CfWTEP) -- that was implemented by a tried and tested agency, the Liberia Agency for Community Empowerment (LACE). The first component was essentially concerned with ramping up and expanding the CfWTEP in the face of the challenge of youth unemployment. The second component was far more ambitious and was set up in the absence of a lead agency. For it to be successfully delivered as designed, it would have required levels of sequencing and coordination that were unlikely given existing capacity. The ICR (p. 12) acknowledges that elements of the design proved to be overambitious. Rating Revised Rating Modest Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To expand access of poor and young Liberians to temporary employment programs (Substantial under both original and revised outcome targets) Rationale The achievement of this objective, which relates to Component 1 of the project (community works/community livelihoods), is rated Substantial. PDO indicators 3 (beneficiaries of community livelihoods program) and 4 (net income gain of targeted participants in year of participation), though listed as outcome indicators in the PAD and ICR, are in fact measures of output rather than outcome. Starting from a zero baseline, the original target for PDO indicator 3 was 45,000 beneficiaries. This was increased to 47,500 through the June 2013 restructuring, and to 56,500 through the AF adjustment in December 2014. The target was effectively met (56,250) by project close, noting the AF had a target of reaching 9,000 rural youth, which was effectively reached (8,750 beneficiaries). PDO 4 set out to measure the proportion of beneficiaries of Component 1 experiencing an income gain in the year of participation, but the ICR notes that there is insufficient data to report on achievement. Under Component 1, 79 percent of beneficiaries were Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) youth, exceeding the target of 50 percent. However, lack of data meant it was not possible to i) establish the number of beneficiaries below the poverty line (against a revised target of 85 percent), or ii) to establish the number of beneficiaries who were unemployed before participating in the project. That said, the poverty rate in Liberia was so high (estimated to be greater than 80% 2007-11) that it is likely that many beneficiaries were poor. Net income gain per direct beneficiary was recorded as US$150. The ICR notes that additional income is expected to be gained through the harvest and consumption or sale of crops produced through LACE's Community Livelihoods Project activities, but the absence of a beneficiary assessment means it is not possible to calculate additional gains. A planned impact evaluation was not completed due to the Ebola outbreak. Outputs • 2,337,500 work days were created against an original target of 1,800,000 and formally revised target of 2,337,500. • Youth represented 79 percent of participants against a target of 50 percent. • 100 percent of public works schemes were completed with satisfactory quality against a target of 85 percent. • 97 percent of public works schemes were completed within a specific period of time against a target of 85 percent. • The target of 56,500 beneficiaries trained in basic life skills was effectively met in that all actual beneficiaries (see outcome indicator below) received training in life skills. • 553 trainers were educated in basic life skills against an original target of 200 and formally revised target of 300. • 7,400 beneficiaries were trained in business skills against a target of 5,250. • 116 trainers were educated in business skills training against a target of 50. Outcomes • 56,250 people were beneficiaries of the cash-for-work, Community Livelihoods Program, against an original target of 45,000 and revised target of 56,500. • These 56,500 beneficiaries comprise 96 percent of the 58,581 direct project beneficiaries overall (exceeding the original target for direct beneficiaries of 49,500 and almost reaching the formally revised target of 59,800). • 47.5 percent of project beneficiaries were female, against a target of 48 percent. Rating Substantial PHREVDELTBL Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) PHEFFICACYTBL Objective 2 Objective To improve youth employability (Modest under both original and revised outcome targets) Rationale The achievement of this objective, which relates to Component 2 of the project (Employment through Skills Training) is rated Modest under both the original and revised outcome targets, given a lack of data to substantiate "improvement in employability" and the failure to meet formally revised targets for participation in skills training. PDO indicator 6 (establishment of a Board of Certification and Accreditation for TVET) was dropped as part of the June 2013 restructuring, leaving the reworded (June 2013 restructuring) PDO indicator 5 (persons participating in and completing skills development programs) as the sole outcome indicator exclusively associated with this objective. It should be noted, however, that this indicator is in fact a measure of output and does not demonstrate achievement of the objective. Starting from a zero baseline, the original target of 4,500 participants in training was reduced to 3,300 as part of the December 2014 restructuring. However, even that reduced target was not met by project closing, as the actual number of participants trained was 2,331. Outputs • 240 business owners participated in and completing business development training against a target of 250. • 25 percent of participants in business development training were women, in line with the target. • 75 percent of participants were under 35 years of age, in line with the target. • 250 trainers and institutional leaders trained against a target of 200. • 100 percent of trainers meet specified performance criteria against an 80 percent target. • 74 percent of employers were satisfied with the quality of students against an 80 percent target. Outcomes • 2,331 persons participated in and completed skills development programs, not meeting the original target of 4,500 or the formally revised target of 3,300. • No data are provided on employability of trained youth, measured by success in finding and/or sustaining employment. Rating Modest PHREVDELTBL PHREVISEDTBL Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) 5. Efficiency Efficiency is rated Modest due to insufficient information, as duly acknowledged in the ICR. The absence of data does not allow for any meaningful comparison of efficiency between this and other similar projects in other contexts, although the ICR draws some comparisons with "unit" costs associated with other projects in that regard. Furthermore, as the project was conceptualized as a response to crisis, "time" may be regarded as a useful measure of efficiency. In that regard, initial delays and associated extensions can be interpreted as a sign of overall inefficiency. The fact that the project was extended also meant that implementation, including the conduct of planned impact evaluation, was impacted. Finally, the ICR finds that administrative costs as a proportion of overall project cost (at 12.9%) are relatively competitive. However, the fact that these costs supported an operation for which it is not possible to establish a meaningful overall level of efficiency renders that measure less valuable than might otherwise be the case. In the absence of a robust certification system, the value of training carried out under Component 2 cannot be established. On the positive side, the project created 2,337,500 work days under the public works program, which is higher than the revised target by 62,500 work days, suggesting some element of efficiency in that regard. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Under both the original and revised outcome targets, relevance of objectives is rated High, responding to originally critical priorities in Liberia that remained relevant at project closure. Relevance of design is rated Modest reflecting sound design for Component 1 (involving use of an existing agency to build on an existing program) but shortcomings in design for Component 2, where there was over-ambition coupled with the fact that there was no implementing agency in place at approval, resulting in delays. The first objective was substantially achieved, and the second modestly so, under both the original and revised outcome targets. Efficiency is rated Modest given implementation delays and the absence of data regarding the profile of participants to allow for meaningful comparisons with other, similar projects. Taken together, these ratings are Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) indicative of significant shortcomings in the project's preparation and implementation under both the original and revised outcome targets, and therefore an Outcome rating of Moderately Unsatisfactory is assigned. a. Outcome Rating Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating Continued low global commodity prices and post-Ebola decline in official inflows created significant challenges for the Liberian economy (e.g., GDP growth in 2015 was flat, compared to 0.7% in 2014) and the government's capacity to provide continued support for either component at the level supported by the project. A possible public health threat like Ebola also adds to the risk. a. Risk to Development Outcome Rating Substantial 8. Assessment of Bank Performance a. Quality-at-Entry The World Bank team responded to Government requests in the context of critical need and worked diligently to prepare the project under tight timelines. However, the ICR notes that the nine-month turnaround from concept to approval was average in this type of context, despite the relative simplicity of the project. There were only two components, and a significant part of the project (Component 1) was being built on an existing program (CfWTEP) with an existing implementing agency (LACE) in place; the PAD (p. 3) refers to the Bank's comparative advantage in that regard. Also, as earlier noted, there were shortcomings in ensuring quality at entry, with particular reference to the over-ambition evident in Component 2 and the associated underestimation of implementing complex change in the context of weak existing capacity. As a result, one PDO indicator, critical to the thrust of Component 2, was dropped, and lesser adaptation - target refinement, redefinition of indicators - was necessary. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision The World Bank conducted regular supervision missions and addressed emerging issues as evidenced by the Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) restructurings and the AF. That said, given the shortcomings with some of the performance indicators as originally formed, adjustments to the results framework and design were the minimum professional response that could be expected. The ICR notes that supervision missions focused on the performance of the implementing agencies in managing the project, and that the World Bank provided the necessary support to facilitate and smooth disbursements as well as support for the development of procurement capacity. However, it is notable that supervisory missions ultimately did not ensure adequate monitoring and evaluation with a view to supporting the evidence base for the realization of targets in support of the PDO indicators - i.e., to profile project beneficiaries, and/or to ensure the employability of participants in training. The ICR also notes the project was designed as a cooperative effort between the education (Component 2) and the social protection (Component 1) sectors of both the World Bank and of the recipient, adding a potential element of synergy that ultimately went unrealized as the components were effectively implemented in mutual isolation. Finally, efforts to tighten and focus the project in the context of AF were positive and have led to enhanced technical capacity on the basis of which the World Bank has developed a follow-on project. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Unsatisfactory 9. Assessment of Borrower Performance a. Government Performance The ICR notes that the Government of Liberia actively participated in the project, being involved, for example, in facilitating public works activities under Component 1, allocating funding to communities, and mobilizing beneficiaries. Local government authorities also provided land for use by beneficiaries for communal farm activities. Under Component 2, the Government hired an international management agency to implement training activities and anchored fiduciary management at the Public Financial Management Unit (PFMU), which was headed by an international financial management specialist committed to achieving project objectives within a prudently managed model. That said, initial delays were associated with ratification and signing on the Government's part, and the scope of the project was curtailed by limited progress on TVET policy implementation. Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance The two main implementing agencies were LACE (for Component 1) and the Ministry for Youth and Sport (MYS, for Component 2). Overall project financial management and logistics were handled by the PFMU at the Ministry of Finance and Development Planning. The ICR notes that, throughout implementation, the PFMU and LACE had satisfactory performance. Because capacity was limited at the MYS, implementation under Component 2 was outsourced to a private management agency under MYS supervision. However, Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) the MYS also lacked the capacity to effectively supervise and monitor that agency, which itself had limited staff, all of which resulted in delayed implementation and suboptimal achievement of the planned results. Ultimately, the MYS discontinued the agency's contract following the June 2013 restructuring and managed to bring the component to completion itself and to further strengthen interaction with LACE and the PFMU. Implementing Agency Performance Rating Moderately Satisfactory Overall Borrower Performance Rating Moderately Satisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design The ICR notes that adequate resources were allocated under both components to ensure regular monitoring of activities and to undertake periodic impact evaluations of project components and outcomes. M&E responsibilities and activities rested with LACE, the Ministry of Planning and Economic Affairs, and the PFMU for Component 1, and the MYS for Component 2. The majority of project indicators were simple and were designed to allow monitoring and disaggregation of data, although they were output- rather than outcome- oriented. The PAD (p. 16) notes that, for Component 1, facilitators/NGOs were to report on a monthly basis to LACE on all project implementation indicators; LACE was to report on progress on a quarterly basis to its Board of Directors and to a (to be established) Social Protection Steering Committee; and M&E data were to be available on the LACE website. LACE planned to recruit a consulting firm to analyze the impact of the project on poor households. For Component 2 (PAD, p. 17), the plan was for the (to be selected) management agent to develop a project monitoring plan to include details on monitoring and evaluation activities, and to report on indicators at least twice every year. It was also anticipated that the management agent would conduct tracer studies on a regular basis to establish the indicators of employment and employability. Associated indicators and targets, measuring outputs, were simple and realistic. b. M&E Implementation The June 2013 restructuring resulted in the dropping of KPI 6, Board of Certification and Accreditation for TVET is Established, and the re-wording of KPI 5 to reflect that change as follows: the original KPI, Persons participating and completing skills development program and receiving certification was changed to Persons participating and completing skills development programs. Targets associated with KPI 1 (Direct project beneficiaries), and KPI 3 (Beneficiaries of Community Works Program) were increased, and targets associated with KPI 5 were decreased. A final restructuring associated with Additional Financing (December 2014) involved a further increase of output targets associated with KPI 1 and KPI 2. The ICR notes that local partners contracted by LACE conducted regular monitoring of implementation activities of subprojects at the community level, while LACE and the Ministry of Planning and Economic Affairs conducted quarterly field visits to subprojects; however, a plan to introduce electronic data collection at sub-project level and upload to Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) a central database for near real-time review of project activities was not successful because of the lack of infrastructure and inadequate technical capacity to use the technology. One wave of impact evaluation (baseline) was conducted between September 2012 and March 2013, but this was underpinned by poorly designed instruments and subject to vagaries in implementation, making it unreliable. A planned second wave of impact evaluation was prevented by the Ebola outbreak. The M&E Framework for Component 2 was not submitted to the World Bank until August of 2012, due in part to a disagreement between the MYS and the management contractor regarding the definition of employment. Two semi-annual progress updates (submitted in December 2011 and on August 28, 2012) were delayed. c. M&E Utilization Overall, the M&E data available was limited in scope and value, due primarily to the lack of planned impact evaluation. The data was therefore of limited utility. On the other hand, three surveys conducted in 2012 to inform the implementation of Component 2 have potential ongoing value in policy making and implementation: Youth and Employer Attitude towards Vocational Training in Liberia (identified gaps in the vocational training market and assessed youth and employer attitudes toward TVET); Employer and Labor Demand Survey (collected in-depth information concerning the labor market situation of youth); and Labor Market Transition of Young Women and Men in Liberia (focused on the transition of youth from school to work). M&E Quality Rating Modest 11. Other Issues a. Safeguards The project was expected to have limited and reversible adverse impacts on human populations or environmentally areas, which were anticipated to be site-specific and relatively easily mitigated in most cases. The project triggered safeguard OP 4.01 (Environmental Assessment - Category B), and the PAD noted (p. 29) that an Environmental and Social Management Framework was already available and being used by LACE in the implementation of other projects. The ICR (p. 10) notes that the overall safeguards rating remained satisfactory throughout implementation and that, where applicable, environmental issues were identified and made known to the beneficiaries, and communities were trained in monitoring environmental safeguards. ICR spot checks and anecdotal evidence confirmed availability of the Memoranda of Understanding between beneficiaries and landowners for three-year land use, as well as availability of environmental mitigation plans. b. Fiduciary Compliance Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) The PAD (p. 10) states that LACE was to be responsible for financial management (FM) for Component 1, capturing project transactions separately, maintaining records of those transactions, and preparing reports in accordance with the relevant Financing Agreement for submission to the Bank. For Component 2, the Ministry of Finance was designated with responsibility to ensure adequate FM capacities through a contract with a management agent. The PAD also notes that the World Bank had previously conducted financial assessments of LACE, reconfirming that LACE’S FM system met the minimum standards required by the Bank. Procurement of goods and works and selection of consultants was to be conducted in accordance with relevant World Bank guidelines. The ICR does not comment on fiduciary compliance and, in fact, provides very little information on financial management. c. Unintended impacts (Positive or Negative) The ICR (p. 17) suggests that the project had positive institutional impact at both the national and local levels. It states that LACE emerged as a relatively effective implementing agency, evidenced by its continued funding from both the World Bank and the Government, although this much seems to be already established in the PAD. It also claims that community-level implementing capacity was strengthened through support to local partners that, in turn, provided technical and implementation support to subprojects, noting that local partners in the form of Community Facilitators were unlikely to be supported post-project. The ICR also states that the project provided the opportunity for capacity and institution building, and that it funded the preparation of a TVET policy, which it claims as a significant achievement in the Liberian institutional context. In addition, the public works aspect of the project absorbed additional funds to provide additional, timely support during the Ebola outbreak, in cooperation with other donors and government. Finally, the ICR states that the project, especially the productive public works under the AF, stimulated local business based, for example, on procuring materials locally, including basics such as food and drink from local vendors, thereby generating a localized multiplier effect. d. Other --- 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome --- Unsatisfactory Unsatisfactory Risk to Development Substantial Substantial --- Outcome The ICR gives credit to the Moderately Moderately Bank for being available to the Bank Performance Satisfactory Unsatisfactory Government of Liberia at a time of need, for being Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) innovative, and for managing through the Ebola crisis. This review notes that a significant part of the project was relatively simple, appropriately so, and built on an existing project, but the second thrust of the project was simply unrealistic, rather than ambitious, in context. This review also notes shortcomings in design and M&E (meaning disaggregation of beneficiaries was not possible) and failure to rectify that problem over the life of the project. Despite improved efforts toward the end of the project period, and particularly with the input of AF, shortcomings in the original design meant that the project was never viable as envisaged. Moderately Moderately Borrower Performance --- Satisfactory Satisfactory Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons The ICR presents a significant number of lessons at policy and implementation levels. Foremost among the former is the need to take account of capacity limitations and, where capacity is limited, to avoid complex solutions, as well as the challenges involved in a fragile context in ensuring basic integration and connection between interventions at programmatic and administrative levels. With reference to lessons on implementation, the ICR emphasizes the need for transfer or progression routes between related interventions in Components 1 and 2 and, in the case of Component 2, which it describes, at heart, as an Active Labor Market Program, the need for more concrete implementation mechanisms linking demand and supply. IEG notes these lessons and emphasizes the following: • Although the objectives of the project were clearly relevant in a narrowly defined sense, the Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review LR: Youth, Employment, Skills Project (P121686) assessment of relevance, to be meaningful, has to be more than the desire to achieve something as set out in official policy or strategy. It must also take into account the issue of feasibility or "deliverability," as attempts to achieve an unrealistic objective, such as originally articulated under the second objective of this project, can result in significant wasted time and effort. This is particularly the case in fragile contexts and a crisis response. In those instances, simplicity and incremental concentration on making things work are appropriate. 14. Assessment Recommended? No 15. Comments on Quality of ICR The ICR is well written, concise, and easy to read, providing a solid overview of the context for the project and its relevance, design, and implementation, with the exception of discussion of certain basic elements such as procurement and financial management (which are under-reported). While ultimately arriving at a Moderately Unsatisfactory development outcome rating, the narrative of the ICR is, at times, overly benign in relation to project shortcomings (e.g., in relation to the lack of hard evidence and data to support social, poverty, and gender impact) (p. 16). The ICR also introduces the Ebola outbreak as a mitigating factor, but the outbreak was declared in early 2014, almost a year after the originally agreed closing date for the project. The fact that the project ran into the Ebola crisis was a function of, in the first instance, weak design followed by shortcomings in implementation. a. Quality of ICR Rating Substantial Page 15 of 15