Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036) Report Number: ICRR0023074 1. Project Data Project ID Project Name P157036 Employment Opportunities for Vuln. Youth Country Practice Area(Lead) Togo Social Protection & Jobs L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-D1730 31-Dec-2020 15,529,364.06 Bank Approval Date Closing Date (Actual) 21-Mar-2017 31-Dec-2021 IBRD/IDA (USD) Grants (USD) Original Commitment 15,000,000.00 0.00 Revised Commitment 15,000,000.00 0.00 Actual 15,529,364.06 0.00 Prepared by Reviewed by ICR Review Coordinator Group Judith Hahn Gaubatz Salim J. Habayeb Eduardo Fernandez IEGHC (Unit 2) Maldonado 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Financing Agreement (page 5) and the Project Appraisal Document (PAD, page 14), the project objective was as follows:  To provide access to income generating opportunities to targeted poor and vulnerable youth in the Recipient's territory. Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036) Note: The wording for one of the key project indicators was revised at the time of project restructuring; however, it was a clarification of the targeting method, not a change to the original intent or scope of the indicator. Therefore a split evaluation is not applicable. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components 1. Community Service and Training (Appraisal: US$ 8.5 million; Actual: US$ 8.25 million): The aim of the component was to give poor and vulnerable youth, who have generally never held a structured work opportunity, the chance to develop good work habits and gain civic values through working on community public works projects. The community works would be selected and managed by Village Development Committees. The project would finance stipends to beneficiary youth, and limited funds to cover inputs, training and administrative costs. Beneficiaries would be provided with a bank account, in which a specified portion of their earnings would be retained to provide part of the seed capital for launching an income generating activity. On-the-job training included technical skills and knowledge, as well as life skills training and micro-entrepreneurial training. 2. Support for Income-Generating Activities (Appraisal: US$ 2.5 million; Actual: US$ 2.62 million): The aim of the component was to provide those beneficiaries who complete the above phase of the program with a matching grant to top-up the savings generated during the community service, as well as mentoring to establish or scale up income generating activities. 3. Capacity building (Appraisal: US$ 1.0 million; Actual: US$ 0.99 million): The aim of this component was to help the government design and strengthen key operational tools for implementing youth employment programs, including geographic targeting, community-based targeting and proxy-means testing, management information and payment systems, program evaluation, and a grievance redress mechanism. The component also aimed to help communities implement the public works projects and promote accountability, including training in the Grassroots Management Training approach, 4. Project management (Appraisal: US$ 3.0 million; Actual: US$ 2.91 million); The aim of this component was to support activities related to project management and coordination. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost  The project cost at appraisal was US$ 15.0 million, and the actual amount disbursed was US$ 15.5 million. Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036)  In 2020, funds were reallocated from Component 4 to Component 3, due to the increase in the number of beneficiary communities from 140 to 200. In 2021, funds were reallocated from Project Preparation Funds (PPF) to Category 1 funds to cover exchange rate losses, to ensure that there would be enough resources to sustain the project operation after the one-year project extension, and to purchase personal protective equipment and handwashing devices at all project sites due to COVID. Financing  The project was financed by an IDA grant of US$15.0 million, which disbursed in its entirety. Borrower contribution  There was no planned Borrower contribution. Dates  March 21, 2017: Project approval.  November 29, 2017: The legal covenants were revised to reflect the delay in recruiting the safeguards specialists (original date was November 2017; revised date was three months later).  November 30, 2017: Project effectiveness.  October 4, 2019: Mid-Term Review.  March 17, 2020: The project was restructured to revise the wording of one key project indicator (to reflect the actual targeting method used) and to extend the closing date from December 2020 to December 2021 (due to additional time needed to complete the mentoring activities).  December 31, 2021: Project closing. 3. Relevance of Objectives Rationale Togo is a small country in Africa with a population of about 7.3 million, ranked 162 out of 188 countries in the Human Development Index (2015). The poverty headcount rate was 55.1 percent in 2015. The country has a high proportion of youth - 75 percent below the age of 35 and 60 percent below the age of 25. 54 percent of the working age population is 35 years old or younger. While the overall unemployment rate is low at 2.4 percent, the real underlying issue is underemployment, which is at 35 percent. Poorer and vulnerable youth tend to not meet the formal definition of unemployed as they often perform ad hoc work to earn a survival livelihood at very low levels of productivity. For underemployed youth, their lower level of earnings and productivity has a longer-lived impact on the economy, compared to older individuals. In response to these issues, the government adopted several strategic documents that aim to improve youth employment indicators, in order to contribute to the overall socioeconomic development of the country. The various strategies aim to improve youth entrepreneurship, strengthen labor market-adapted Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036) vocational skills, and improve education quality and access to finance, alongside labor market interventions to increase the supply of jobs. The project objectives are well-aligned with these strategic directions, with a particular focus on poor and vulnerable youth. The Bank's Country Partnership Framework for FY17- FY20, which was under preparation at the time of project appraisal and then subsequently extended to FY22, included private sector performance and job creation as a Focus Area, with improved business and employment opportunities as an objective and "improved employment options for Togolese youth" as an indicator. In order to improve productivity and competitiveness, the Bank framework aimed to address key gaps in skills and employment opportunities. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To provide access to income generating opportunities to targeted poor and vulnerable youth in Togo. Rationale The theory of change was clear. The planned activities to hire poor and vulnerable youth to participate in community public works projects, provide them with on-the-job training and life skills/ entrepreneurship training, and provide seed capital and mentoring to launch income generating activities were likely to lead to the outcome to increase access to income generating activities. The targeting mechanisms - geographic targeting, community identification of potential beneficiaries, and confirmation of beneficiaries through proxy- means test - were straightforward to ensure that poor and vulnerable youth were beneficiaries. This project supported an extension of the government's national volunteer initiative, which aimed to provide opportunities for youth to gain work experience and skills, through community service projects. The government program was initially designed for university graduates, with this project expanding the program to reach youth with little or no education, specifically those with a disability or those who did not finish primary school. The program consisted of two phases: (1) participation in a community service project which provided a platform for on-the-job training in technical skills, life skills training, and micro-entrepreneurship training; and (2) financial and mentoring support for launching and operating an income generating activity. Only those participants who completed the first phase received the financial grants in the second phase. Outputs Using a community-driven approach, public works projects were selected and managed by Village Development Committees, with financing for stipends for youth, on-the-job training, and administrative costs Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036) (maximum of 40% of total sub-project cost). Depending on the predominant economic activity in the village, projects would either be agricultural (on land allocated by the community) or other (i.e. road maintenance, rehabilitation of public buildings, drainage, environmental rehabilitation).  Community public works projects implemented in 200 villages (target: 140) in the 150 poorest cantons. Realized assets included 116 rehabilitated or created rural roads, 40 water reservoirs to support vegetable gardening and livestock activities, 20 reforestation works and community plantations, 6 quarries, 15 agriculture sub-projects, and sanitation sub-projects. As noted in the ICR (page 15), anecdotal evidence indicated a strong sense of ownership of realized public works by community members, and their commitment to work on maintenance after the end of the project. In addition, community capacity building provided an important entry point for women to participate in sub-projects (note: to ensure the participation of at least 50% women beneficiaries, village development committees prioritized sub-projects suitable to and appealing for women in the village).  Stipends paid to youth beneficiaries for their community service (Note: the total amount paid in stipends was 67% of total subproject costs, achieving the target of 60%). Beneficiaries were also provided with bank accounts, with a specified portion of their earnings retained in the account to provide part of the seed capital for launching an income-generating activity in the second phase.  Life-skills and entrepreneurial training provided to beneficiaries. The training was based on the existing FODA model, including discussions on entrepreneurial behaviors, generation of ideas, rudimentary market assessment, evaluation of inputs and equipment required, and basic financial literacy, analysis and business plan.  Training provided to Village Development Committees on identifying beneficiaries.  Matching grants and mentoring provided to second phase beneficiaries (top-up to savings in bank account) to provide start-up capital for income generating activities. These activities included trading activities and small businesses (46%), food processing (20%) and livestock raising (17%).  Capacity building support to government to design and implement youth employment programs including targeting mechanisms, management information system, payment system, program evaluation, and a grievance mechanism.  Capacity building support to villages, using the grassroots management training approach, including information, communication, organizing, poverty and needs assessment, planning and procurement, and infrastructure maintenance. The ICR (page 22) noted that difficulties were reported in properly implementing FODA training in certain regions, due to the lack of qualified trainers. Given the target group of the project, mostly made up of youth who were not literate or did not complete primary school, the FODA training was to be delivered in the local language, a requirement which proved to be a challenge in certain regions. Also, due to COVID-19 and associated shutdowns, an abbreviated FODA module was created and delivered in the remaining 44 villages. However, there were challenges at the start of the project period that delayed the launch of activities: delays in the study of existing employment opportunities and the large share of youth in the villages did not have identity documents which were necessary to open bank accounts. These challenges meant that the new agricultural season had already started without the beneficiaries being able to take full advantage of income- generating activities right away. Outcomes Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036)  14,415 beneficiaries in the 150 poorest cantons participated in the community sub-projects, achieving the target of 14,000. Of these, 60% were female (target: 50%). 95% were living with a disability or had not finished primary school, surpassing the target of 90%.  1,120,219 person-days were worked in community service projects, achieving the target of 1,120,000.  14,120 beneficiaries received training on business development (target: 12,500). Of these, 99% prepared a business plan with the support of a coach.  11,495 beneficiaries (including 124 people with disabilities) were able to meet the savings requirement and received start up grants and 5-6 months of coaching to help launch or expand their income generating activities (target: 10,000). As reported in the ICR (page 14), beneficiaries reported that access to mentorship helped them overcome common challenges faced when setting up or scaling up their business, such as identifying available opportunities or marketing strategies that could be relevant in the local context. Also, the ICR mission noted the continuation of the income generating activities even after the six months of coaching, which have become their main source of income to provide for daily expenses such as food, education of children, and health services.  10,805 beneficiaries were employed or self-employed six months after receiving the start-up grant (of these, 430 beneficiaries were approved for the grant but chose not to initiate an income-generating activity due to the opportunity for salaried employment elsewhere).  52% of stipend payments were made within one week of due dates, falling short of the target of 80%. The ICR (page 13) noted that this shortcoming was due to delays in payments to villages during the first phase of implementation.  97% of beneficiaries felt that the community sub-projects reflected their needs, surpassing the target of 80%.  83% of community sub-projects were of satisfactory technical quality, achieving the target of 80%. In addition, anecdotal evidence gathered through the ICR mission (reported in the ICR, page 18) suggested that gender dynamics have positively changed in households and villages. Women’s participation in the program led to enhanced self-confidence and increased respect within the household (as they are now perceived as breadwinners and less dependent on their husbands), and increased self-confidence due to participation in village meetings to discuss sub-project activities. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Achievement of the objective to increase access to income generating opportunities for poor and vulnerable youth is rated Substantial due to evidence of participation in community sub-projects, followed by launching of income generative activities, including meeting of targets in the number of participants. Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036) Overall Efficacy Rating Substantial 5. Efficiency At appraisal, the PAD did not provide a cost benefit analysis due to limited availability of data. Instead, the PAD provided the economic rationale for the project, including increasing productivity and earnings for youth, and economic (and social) benefits from the public works sub-projects. The appraisal also provided a comparison of wage earnings for poor and vulnerable youth who would most likely find work in an economic activity with very low productivity versus those with higher productivity (due to improved skills and access to capital). At completion (ICR, Annex 4), a cost-benefit analysis was conducted that estimated the social rates of return and also the efficiencies gained in pairing the two phases of the project. Benefits of the first phase (community service projects) included income generated from stipends and for the second phase, increased wages from the income-generating activities (subtracting the foregone income which represent opportunity costs). Costs were actual projects for phase one and two activities. The estimated social rate of return ranged from 53% to 88% for the first phase and 97% to 150% for the second phase. The analysis also noted that the cost-efficiency of the sub-projects (US$3.50 spent to generate US$1.00 of income benefits) was on par with other labor intensive public works projects implemented elsewhere, citing an earlier study which found that on average it takes US$3.30 to US$5.72 to generate US$1.00 of income benefits. Implementation efficiencies were likely realized by pairing the two phases of the project, by incentivizing beneficiaries to only spend what is needed from their stipend payments and having the community service projects act as a targeting mechanism for beneficiaries who then participated in the second phase. The increase in efficiency from pairing the two types of interventions in this project’s design was shown by estimating the returns to each component had they been implemented separately and comparing to the returns of the project as it was implemented. Having beneficiaries use their savings as seed capital and providing the entrepreneurship support/training during the subprojects yielded an increase of 30 to 40 percent greater return as compared to separate implementation. Further implementation efficiency was gained by sharing administrative functions with the ongoing Safety Nets and Basic Services Project (P157038) - both projects were designed, launched, and implemented parallel with a view of using the same implementing agency (ANADEB) from the start. Lastly, the actual targeting mechanisms were simplified as the original design intended to use a proxy means test followed by community validation, however, in actuality, once geographical targeting of poorest villages was carried out, additional poverty targeting to identify beneficiaries was not needed, because self-selection through registration was deemed to be appropriate in a context of widespread poverty. However, there were several challenges that affected project implementation: delays in key procurement activities (including the study to identify employment opportunities), complaints about the grassroots management training specialists that had to be resolved before continuing with implementation, delays in conducting safeguards screening for all of the subprojects due to limited staff, and delays in signing of the contract for delivery of training. There were also some inefficiencies when the planned sequencing of subprojects around the harvest and lean seasons was not actualized in all cases due to the above implementation delays, and lack of identity documents for a large number of beneficiaries also created obstacles Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036) to and delays in enrollment. However, these challenges were resolved by project closing such that outputs were delivered as planned. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of objectives is rated High due to strong alignment with country conditions, government strategy, and Bank strategy. Achievement of the objective to increase access to income generating opportunities for poor and vulnerable youth is rated Substantial due to evidence of participation in activities that led to launching of income generating activities. Efficiency is rated Substantial due to favorable cost comparisons and implementation efficiencies realized by the project. a. Outcome Rating Satisfactory 7. Risk to Development Outcome Institutional capacity has been strengthened through the project and is likely to be sustained, both at the central level and at the local level. The main implementing agency (ANADEB) has become the lead institution for all youth employment programs in the country. 200 local communities received grassroots management training. Poor and vulnerable youth have been given access to financial institutions and instilled with a culture of savings. However, the sustainability of community public works assets remains unclear, with the ICR (page 31) noting that the budgeting of the necessary resources for these purposes has yet to be approved. Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036) 8. Assessment of Bank Performance a. Quality-at-Entry The project design built on the government's experience with existing youth employment programs and on the Bank's experience with labor-intensive public works programs, including the lessons learned that integrating multiple interventions (technical and soft skills) for youth employment programs is more effective than technical skills alone. The project design featured a sequencing of activities to ensure beneficiaries’ continuous engagement and avoid dropouts, The project also took into consideration local context such as timing of subprojects due to harvest and lean seasons and predominant economic activity. Institutional arrangements were designed jointly with the Safety Nets and Basic Services project (P157038). Despite the above, there were some minor shortcomings in project design (lack of identification documents, migration of youth to urban areas and other countries, and burdensome administrative and transportation costs for decentralized financial system) which led to initial implementation delays. Risk was assessed as Moderate, with mitigation measures drawing from existing institutional arrangements and experience. The results framework was clear and straightforward, albeit with some minor shortcomings in two intermediate results indicators (community service sub-projects with satisfactory technical quality and beneficiaries who are employed or self-employed six months after receiving the start up grant) which led to a downgrade in the M&E rating in the final ISR from satisfactory to moderately satisfactory. Taking into account the whole context of operational, sectoral, and country aspects, quality at entry is rated Satisfactory due to minor shortcomings that had a limited impact on the ultimate achievement of intended outcomes. Quality-at-Entry Rating Satisfactory b. Quality of supervision The Bank team was highly proactive, with key staff based in-country and remaining on the task team from preparation to closing. The Bank team addressed numerous implementation challenges: delays in the procurement of a firm to conduct the study on existing employment opportunities and the firm that would deliver the training modules, difficulties in properly implementing the job skills training in certain regions due to lack of qualified trainers in local language, the large number of illiterate beneficiaries, and the lack of proper identification documents. These challenges led to delays in launching subprojects (thus the new agricultural season had already started without beneficiaries having been able to benefit from support for the creation and expansion of their income generating activities in the second phase) and delays in stipend payments which caused an initial lack of trust by beneficiaries. However, the project team was proactive in addressing these challenges, for example by establishing literacy centers, increasing the initial advance amount on the designated account to allow the pace of stipend payments to pick up, agreeing to cover Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036) account opening and travel costs for savings accounts, and setting up judiciary auditions to start the formal process for obtaining identity cards. In addition, a health state of emergency was declared in March 2020 due to the COVID-19 pandemic, which led to the temporary halt of project activities. However, the project team took measures to enable project activities to resume in the summer of 2020, such as the provision of personal protection equipment to beneficiaries and project staff, additional financial support to beneficiaries whose income generating activity in progress or whose business plan had been validated before the COVID-19 lockdown, a modification of the FODA training modules in the remaining 44 villages where the training had not been yet implemented. M&E activities were largely carried out as planned, with the establishment of the management information system as a key activity. Initial fiduciary challenges were addressed and no major shortcomings were noted. Quality of Supervision Rating Highly Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The results framework was clear and straightforward, with indicators measuring both key outputs and intended outcomes such as quality of subprojects and sustainability of income-generating opportunities. However, as noted above, two intermediate results indicators (community service sub-projects with satisfactory technical quality and beneficiaries who are employed or self-employed six months after receiving the start up grant) were deemed questionable by the final supervision team. Also, capacity building for government to design and implement youth employment programs did not have an indicator. Monitoring arrangements were simple and feasible, including developing a management information system as part of the project design. Targets were realistic given the newness of the operation and the implementing agency. Planned evaluative activities included technical and process evaluations at mid-term and an impact evaluation. b. M&E Implementation The management information system was put in place at project launch, which tracked data on beneficiaries (gender, age, employment status) and enabled regular project monitoring, including participation of beneficiaries in the multiple activities, payments made, and results such as income generated by businesses. The ICR (page 25) noted that the structure of the system, which would draw Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036) from the regional systems to inform the central one, posed at times some challenges in terms of data consistency between the different regions as well as timeliness of updates. At the time of project restructuring, one of the key project indicators was revised to reflect the actual targeting method used, which was done by identifying the poorest cantons followed by a random selection of beneficiaries, and not by a proxy means test. The planned impact evaluation was dropped given that the methodology for targeting beneficiaries was changed from proxy-means testing to random selection and self- targeting (hence, a control group could not be established). Instead, a final evaluation was conducted on a sample of 20% of the 200 beneficiary villages, randomly selected, with 560 beneficiaries surveyed through a mobile application. The evaluation also included individual/focus group meetings with community development committees, intermediary NGOs, M&E firms/NGOs and coaches. c. M&E Utilization The ICR (pages 25-26) reported that implementation progress was regularly tracked, with actions agreed upon at each supervision mission to improve the lagging indicators and/or their measurement. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as a Category "B" project due to small-scale public works. The project developed an Environmental and Social Management Framework (ESMF), adapted from a prior labor project (P127200), which provided guidance on the screening process for the sub-projects. Some delay was reported in carrying out the environmental and social screening, due to the limited number of environmental and social safeguards staff. This was resolved with the recruitment of junior environmental and social consultants in all five regions, along with additional consultants to support the two safeguards specialists on project staff. OP 4.12 on Involuntary Resettlement was triggered. Given that the exact location and type of sub-project that would be selected by the beneficiary communities were not known at the start of implementation phase, the project prepared and disclosed a Resettlement Policy Framework (RPF). The RPF included templates for sub-project screening to identify any adverse social impacts and guided the preparation of subsequent Resettlement Action Plans as and when necessary. Many of the sub-projects required land acquisition through voluntary land donations, which were documented. OP/BP 4.11 on Physical Cultural Resources was triggered because physical cultural resources may be found during civil works activities, thus leading to a "chance find procedure". Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036) OP/BP 4.09 on Pest Management was triggered; the ESMP had a dedicated section on pest management. The project conducted two environmental and social audits to assess the level of compliance with the provisions of the environmental and social safeguards frameworks. Both audits concluded that environment and social safeguards requirements were complied with in a satisfactory manner. The closing environmental and social audit however, pointed to a few weaknesses including delays in the recruitment of junior environmental and social safeguards consultants in each region, some weaknesses in the implementation of measures recommended by the screening of sub-projects, the absence of compensatory reforestation of certain sub-project sites, and insufficient maintenance of a few sub-project sites. The final overall safeguards rating is reported in the Operations Portal as Satisfactory. As reported in the ICR (page 28), there were two severe incidents which were reported by ANADEB and documented through memos to Bank management. Both were classified as severe as they involved fatalities resulting from road accidents. While the first one was closed (with investigations concluding that it was not project related), the second incident is still ongoing as a judicial case was opened by the involved insurance companies. According to the ICR, the incident was not caused by any deficiency in the project design or implementation. The police report prepared following the accident could not establish that the incident was caused by a project vehicle; however, the case was sent to the local court for further investigation and decision, and the matter was still open at the time of preparation of the ICR. Following this case, and to prevent as much as possible future similar situations, the task team recommended to ANADEB to revise its procedure manuals to ensure proper guidelines are included regarding project vehicles use, including speed limitations, defensive driving, etc. b. Fiduciary Compliance Financial management: The project team was adequately staffed with an experienced financial management person and accountants located in each of the five regional offices. As the fiduciary risk had been assessed as Substantial, mitigation measures were identified to strengthen internal controls. The ICR (page 28) reported that financial records were continuously and regularly updated, internal audit functions were performed regularly, and the accounting software had all the required functionalities. Two minor weaknesses were identified during the final mission (request for withdrawal of funds in advance on the designated account was refused by the Department of Public Debt and Financing due to limited availability, therefore the Bank suggested decreasing the requested amount; incorrect advice given to ANADEB regarding opening an escrow account to secure the amount of the final audit fees) but subsequently resolved. The overall financial management performance was rated as Satisfactory during the final mission. The project team confirmed that there were no qualifications to any project audits. Procurement: There were procurement issues which led to initial project implementation delays: the procurement of a firm to conduct the study on existing employment opportunities in project areas was delayed, with approval taking five months instead of two months; due to two complaints about the recruitment of project staff, the recruitment for key positions in the project was delayed by a few months. After the hiring of a full-time procurement officer, the pace of implementation picked up. The July 2021 procurement review identified some deviations, though none involving legal remedies or misprocurement (delays in the deadlines for the award of effective contracts and the signing of Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036) contracts; missing information and documentation, causing limited understanding of the contract execution phase, including how late payment penalties should be managed; extension of contract performance deadlines without a relevant reason leading to the non-application of late payment penalties to the successful tenderer). These minor shortcomings were resolved and at closing, the procurement performance was rated Satisfactory. c. Unintended impacts (Positive or Negative) As reported in the ICR (page 20), project activities triggered a positive dynamic within some communities by strengthening social cohesion. Anecdotal evidence suggested that participation in sub-projects as well as training sessions created regular interactions among beneficiaries, who now feel less isolated and perceive a sense of solidarity within the community. This was particularly observed among female beneficiaries, who traditionally had less opportunities for interactions outside their household. In addition, the joint work of two villages on one single project had the positive effect of creating interactions and links between the two participating communities. These observations were confirmed in the final evaluation, as surveyed village/village development committee chiefs claimed that the overall project implementation approach triggered: (i) strengthened community union; (ii) enhanced social cohesion in the village; (iii) strengthened community solidarity; (iv) a reorganization in the management of the village with regard to the organization of meetings and the management of conflicts; and (v) a reduction in the rural exodus. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment This ICR Review rated efficacy as Substantial in view of Outcome Highly Satisfactory Satisfactory moderate shortcomings that were reported, such as in stipend payments and training Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons Lessons drawn from the ICR (pages 32-33) and adapted by IEG:  Youth employment programs that integrate multiple interventions can have an increased impact compared to single-focus interventions. In the case of this project, interventions Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036) addressed both supply side (skills development for youth) and demand side (focus on self- employment opportunities, as well as soft skills (promoting a culture of savings) and technical needs (local market assessment) were more responsive to the range of constraints to increased youth employment.  Identifying synergies with existing operations and partners can generate efficiencies in project implementation. In the case of this project, the twinning of projects (with the parallel safety nets project) enabled sharing of some project specialist staff and administrative support staff, and partnering with GIZ enabled use of their existing business skills training curriculum. Also,  The tightly-linked sequencing of key project interventions can also generate efficiencies in project implementation and monitoring. In the case of this project, the savings requirement from the first phase of the project triggered the matching grants and hence provided the seed capital for the second phase. 13. Assessment Recommended? Yes ASSESSMENT_TABLE Please Explain To verify whether income generating opportunities and public works projects have been sustained beyond the project period and to learn lessons to apply to future youth employment programs. 14. Comments on Quality of ICR The ICR was results-oriented, reasonably concise, internally consistent, and consistent with guidelines. The quality of the evidence was satisfactory, with detailed data on the participation of beneficiaries in the various interventions supported by the project. However, there was less information on the longer-term impact on livelihoods for the beneficiaries. The detailed description of project implementation experience that was particularly informative for operations of this kind. Lessons were well-drawn from project experience. a. Quality of ICR Rating Substantial Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Employment Opportunities for Vuln. Youth (P157036) Page 15 of 15