Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) Report Number : ICRR0021063 1. Project Data Project ID Project Name P126339 EG Emergency Labor Intensive Investment Country Practice Area(Lead) Egypt, Arab Republic of Social Protection & Labor L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-81730 31-Dec-2015 200,000,000.00 Bank Approval Date Closing Date (Actual) 28-Jun-2012 30-Jun-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 200,000,000.00 0.00 Revised Commitment 200,000,000.00 0.00 Actual 194,884,898.13 0.00 Prepared by Reviewed by ICR Review Coordinator Group Rasmus Heltberg Judyth L. Twigg Joy Behrens IEGHC (Unit 2) 2. Project Objectives and Components a. Objectives The objectives of the project were (i) to create short-term employment opportunities for unemployed unskilled and semi-skilled workers, and (ii) to provide access to basic infrastructure services to the target population in poor areas in the Borrower's territory. The objectives were identical in the Project Document and the Legal Agreement and remained unchanged throughout the life of the project. Some outcome targets were revised at a June 25, 2015 restructuring, and therefore a split rating methodology will be applied. Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? Yes d. Components Component 1: Employment-Intensive Sub-projects (Appraised at US$198 million; accounted for 99.2% of total project expenditures in local currency): This component had two subcomponents: • Infrastructure (85.78% of total project local currency expenditures): provided grants to the sponsoring agencies (local governments) for sub-projects consisting of small-scale infrastructure works such as canal cleaning and rehabilitation of schools, housing, rural roads, canals, youth centers, and social units. • Community services (13.45% of total project local currency expenditures): provided grants to implementation partners (non-governmental organizations, NGOs) for sub-projects consisting of community services, such as early childhood education services, outreach for maternal and child health, nutrition, population services, solid waste collection, and youth initiatives in rural and urban settings. The responsibility for sub-project implementation was contracted out to the sponsoring agencies and implementing partners, with technical support, supervision, overall management, and oversight from the Social Fund for Development (SFD). Targeting emphasized rural areas and the poor Upper Egypt region. Component 2: Implementation Support (appraised at US$2 million; accounted for 0.8% of total project local currency expenditures): This component included provision of project implementation support, including project management, audits, public information and communication, technical verifications, quality assurance, monitoring and evaluation, and citizen engagement/social accountability. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost and financing: The project was fully financed by the World Bank through an International Bank for Reconstruction and Development loan processed under emergency procedures. The project disbursed US$194,884,898 against US$200,000,000 appraised. The European Union provided parallel financing through a Bank-administered and Recipient-executed Trust Fund (Emergency Employment Investment Project, P146143), in the amount of €67.6 million to replicate key elements of the project. The Danish International Development Agency provided some parallel grant funding of an unknown amount. Exchange rate fluctuations increased the budget available for the project in Egyptian Pounds by 35%. This allowed for a larger number of subprojects. The share of project funds allocated for community services was reduced, and the share allocated for infrastructure was increased proportionally. Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) Borrower contribution: No Borrower contributions were mentioned in the Implementation Completion and Results Report (ICR). Dates: The project was approved by the World Bank on 28 June 2012 and became effective 27 December 2012. The mid-term review was done on 4 November 2014. A 25 June 2015 level "2" restructuring revised some outcome targets and extended the original closing data of 31 December 2015 to 30 June 2017. 3. Relevance of Objectives Rationale The project's objectives were highly relevant to the Bank's Interim Strategy Note that was discussed at the same time as the project was approved, and where job creation as a way to mitigate the impact of economic crisis and help stabilize the post-revolutionary political transition was an explicit objective. The Project Document and the ICR contained a thorough discussion of the rationale for supporting labor-intensive works as a way to complement Egypt's then-existing but rudimentary and poorly targeted social protection system in a context of high poverty, high youth unemployment, high inflation, and lack of quality public social services. Labor intensive works have been used in crisis settings in many countries as a way to mitigate the adverse short-term impacts of spikes in unemployment and poverty stemming from crisis. The project was also aligned with the strategic objectives of the Bank’s Country Partnership Framework for Egypt for 2016- 19, which supports the three pillars of the Government’s development strategy: (i) improving governance; (ii) supporting private sector job creation; and (iii) improving social inclusion. This review therefore concurs with the ICR's rating of the relevance of the project's objectives as High. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To create short-term employment opportunities for unemployed unskilled and semi-skilled workers (original outcome targets) Rationale According to the project's theory of change, financing public works and provision of social services would create short-term employment opportunities. These opportunities would activate unemployed and underemployed people, who would also benefit from some on-the-job training. In this way, the project would Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) achieve temporary employment and income earning opportunities. The project deployed a variety of mechanisms to ensure targeting to youth, female, and the unemployed, such as establishing criteria for who could be hired, self-targeting through low wages, geographic targeting by placing subprojects in poorer locales selected using a poverty map, and communication efforts to allow unemployed individuals to register for public works activities. Targeting to women was ensured in the social services subcomponent. Mechanisms to ensure high labor intensity would help translate project spending into earnings for beneficiaries employed on subprojects (instead of expenses on materials and capital). High labor intensity was supported by making only small-scale local projects eligible and by making the wage share one of the selection criteria for contractors. This ICRR applies a split rating methodology in accordance with IEG-OPCS guidelines. Prior to restructuring, the project had aimed to create 194,800 direct job opportunities. This target was reduced substantially at restructuring, to 123,000, even as other targets were increased and some additional indicators were brought in. The reduction in the target for direct job opportunities amounts to a reduction in the level of ambition for the project on an indicator that directly corresponds to the first objective of the project, to create short-term employment opportunities for unemployed unskilled and semi-skilled workers. It is this reduction in the level of ambition that requires the split rating (which the ICR did not perform). The failure to fully meet the employment target was caused by a shift away from the most labor-intensive types of activities toward more capital-intensive ones. Different factors necessitated this shift, including low profitability of manual weeding and similar activiites, leading to low appetite among contractirs and few bids. Another factor at play was saturation of demand from civil society organizations, funded also under the parallel EU-financed project. The project achieved the following results against intermediate outcome indicators: • The wage share of total sub-projects cost was 49%, indicating a higher labor intensity than the 40% targeted. • US$ 14.47 million person/days of employment were created, against US$ 19.48 million originally targeted. • US$ 94.67 million was paid to workers: US$ 15 million was paid to female workers, and US$ 79.7 million to male workers. The project failed to meet the original target set for direct job creation while exceeding targets set for the other PDO and intermediate outcome indicators for this objective. According to the ICR, the project achieved the following results against the original PDO indicators: • 144,669 direct job opportunities were created, against the original target of 194,800. • 96,971 indirect job opportunities were created, against the original target of 64,000. • 34% of direct job opportunities were for women, against an original target of 25%. • 69% of jobs created were taken by youth below age 30, against an original target of 50%. Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) Rating Modest PHREVDELTBL PHINNERREVISEDTBL Objective 1 Revision 1 Revised Objective To create short-term employment opportunities for unemployed unskilled and semi-skilled workers (revised outcome targets) Revised Rationale The project more than fully met the revised targets set for the PDO and intermediate outcome indicators for this objective, justifying a rating of High under the revised targets for the first objective. According to the ICR, the project achieved the following results against the revised PDO indicators: • 144,669 direct job opportunities were created, against the revised target of 123,000. • 96,971 indirect job opportunities were created, against the revised target of 90,000. • 34% of direct job opportunities were for women, against the revised target of 20%. • 69% of jobs created were taken by youth below age 30, against the revised target of 50%. According to an impact evaluation being done by the World Bank (presented to Bank staff in draft, after the completion of the ICR), there are significant positive benefits in terms of wage employment, income, expenditures, and propensity to save over a 3-6 month period to the beneficiaries of the social services sub- projects (mostly women). Similar benefits are not evident to the (male) beneficiaries of the infrastructure subprojects. The impact evaluation also reviewed outcomes on rates of crime and violence in the community (which was not an objective of the project) and found small, mixed, and insignificant impacts. These findings are noted here for completeness but have not materially impacted the ratings assigned by IEG (source: Employment and Violence in Fragile States: Evidence and Insights from Recent Studies. Eric Mvukiyehe, Research Department. January 30, 2018. Presentation). Revised Rating High PHEFFICACYTBL Objective 2 Objective To provide access to basic infrastructure services to the target population, in poor areas in the Borrower’s territory. Rationale According to the project's theory of change, the project would build and rehabilitate public infrastructure (rural roads, schools, canals, housing, etc.) and provide community services (early childhood education, maternal and child health outreach, nutrition, street cleaning, etc.). These activities would improve access to basic infrastructure and community services in targeted areas. However, none of the project's PDO-level Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) indicators corresponded to this objective, and no indicator directly measured access to services in targeted areas. The indicators measured quantities of rehabilitated infrastructure, with also some metrics of community services outputs. Furthermore, although the project did use poverty targeting, none of the official results indicators directly measured the poverty rate of project areas. With no direct measurement of achievement of the objective, it is difficult to assess to what degree the objective was achieved. We have no evidence on access to services – how many more people are using these canals, roads, classrooms, etc.? According to IEG's rating methodology, absence of evidence on achievement is a cause for lowering the rating. Several of the intermediate outcome indicators relevant to this objective were revised at restructuring, in both directions, as noted below. A split rating is not required in this case, because the revision of the targets for the intermediate outcome indicators was more reflective of adjustments between types of subprojects and went both up and down, and therefore did not necessarily correspond to a lowering of the project's level of ambition or scope. The project inputs were facilitated through 132 infrastructure framework agreements and 219 community services subproject framework agreements with private providers completed during the project life. The infrastructure framework agreements financed the infrastructure outputs listed below under the intermediate outcome indicators, and the community services subproject framework agreements financed the following outputs: • Training of 10,011 young men and women on various topics and trades. • Implementation of 7,359 health and environmental training campaigns and seminars. • Implementation of 3,451 medical convoys. • Implementation of 18.7 million home visits related to health and the environment. • Opening of 1,037 illiteracy eradication classes for 16,734 beneficiaries and 337 kindergarten classes for 9,521 children, and equipping of 281 kindergarten classes. • Numbering of 48,553 streets and houses, and painting of 31,486 building facades and 3,578 street lighting poles. • Rehabilitation and painting of 40,650 meters of street sidewalks. • Levelling of 103,392 meters of streets and compaction of 2 kilometers of roads. • Planting of 24,640 trees. • Improvement of 72 village entrances. • Purchasing of 2,119 solid waste containers and 1,826 wheel carriers for waste collection. • Delivery of bread and gas cylinders to 23,425 families. There was substantial progress against most of the intermediate outcome indicators. The project achieved or exceeded most of the revised and many of the original targets. It fell short, however, against some of the original targets, especially canals cleaned and houses rehabilitated: • 75% of sub-projects were of quality, as measured by Client Satisfaction (against a target also of 75%). • 78% of sub-projects are reported as completed with high technical quality (against a target of 75%). Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) • 3,148 km of canals were cleaned (meeting the revised target of 3,148 km, but not the original target of 10,200 km). • 79 km of River Nile banks were protected (against a revised target of 78 km and an original target of 47 km). • 5,015 houses of the poor were rehabilitated (meeting the revised target of 3,791, but not the original target of 7,591). • 304 km of rural roads were upgraded (against a revised target of 198 km and an original target of 122 km). • 11,995 classrooms were rehabilitated (against an original/revised target of 12,000). • 1088 km of small canals were rehabilitated (against a target of 950 km; this was a new indicator introduced at restructuring). • 335 youth centers were rehabilitated against (against a target of 340; this was a new indicator introduced at restructuring). • 427 social units were rehabilitated (against a target of 370; this was a new indicator introduced at restructuring). Rating Substantial PHREVDELTBL PHOVRLEFFRATTBL Rationale Overall efficacy is rated Modest under the original outcome targets, due to the shortcomings in achievement of the original target for direct job creation, and substantial evidence only on outputs under the second objective; no evidence is provided on achievement of the actual objective of access to the basic infrastructure services that were built and enhanced. Overall efficacy is rated Substantial under the revised outcome targets, with achievement exceeding targets for short-term job creation. Overall Efficacy Rating Substantial 5. Efficiency The Project Paper convincingly explained why a social safety net can be an effective use of public funds as a way to insure against poverty risks and improve equity. The Project Paper also offered a detailed economic analysis comparing the benefits to the poor with the total outlays by the government to the program. The ICR updated this analysis. Appropriate for social safety nets, the analysis was not a sectoral rate of return estimation. It estimated how much it costs the government to transfer one Egyptian Pound (EGP) to a poor Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) unemployed citizen under different assumptions about how hard or easy it is to find other employment and under different assumptions about the value to the poor of the public assets created by the works. The estimation used a method pioneered by Martin Ravallion in an analysis of a similar workfare program in Argentina. Depending on assumptions, it costs between 1.02 to 2.43 EGP to transfer one EGP to the poor. The results for Egypt compared favorably with the similar findings for Argentina, which serve as a sort of benchmark. The ICR used this analysis to argue that the project was an efficient use of public resources, a conclusion with which this review concurs. Another way to think about the efficiency of the project -- not considered explicitly in the Project Paper or the ICR -- is to compare the relatively high targeting efficiency of labor-intensive public works with alternative social protection instruments. At the time of project approval, Egypt's alternative social protection instruments were energy and bread subsidies and rationing cards, all highly mistargeted and wasteful. Using those instruments to transfer an equivalent level of benefits to the poor would have cost the government far more than what this project cost. There were some delays in project implementation for various reasons that included: compliance with safeguards requirements; frequent need to retender because of low capacity among contractors and intermediary agencies, or because of too few bids received; need to adjust away from some subsectors such as manual weeding that did not see many bids; inflexible regulations and procedures causing delays in pipeline preparation and implementation; multiple layers of subproject management; under-resourced regional offices of the SFD; and a requirement under the increased allocation to the infrastructure subcomponent of identification of new sectors and sponsoring agencies. Furthermore, project implementation was delayed by Egypt's political turmoil during 2013-14. Overall, however, these delays are considered minor relative to the project's economic and targeting efficiency, resulting in a Substantial rating for overall efficiency. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) 6. Outcome Under IEG - OPCS harmonized guidelines, the Outcome rating for a project with revised outcome targets is derived from the percentage of the Bank's disbursements under each set of targets. In this case, 74% of disbursements was under the original targets, and the overall Outcome rating is Moderately Satisfactory. Dimension Original targets Revised targets Relevance of objectives High High Efficacy Objective 1: To create short-term employment Modest High opportunities for unemployed unskilled and semi-skilled workers Objective 2: to provide access to basic infrastructure Substantial Substantial services to the target population, in poor areas in the Borrower’s territory. Overall efficacy Modest Substantial Efficiency Substantial Substantial Outcome rating Moderately Satisfactory Unsatisfactory % of project funds disbursed 74 26 Weighted numerical score 2.22 (numerical 1.30 (numerical outcome=3; outcome=5; weight=0.74) weight=0.26) Total weighted numerical scores 3.52 (rounds up to 4) Total Outcome Rating Moderately Satisfactory (4.0) a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The risks to development outcomes are hard to assess. As a crisis intervention, the project was never intended to be financially sustainable. The ICR stated that the project helped build the capacity of the SFD to implement this type of labor-intensive works, which it has also done under the parallel and very similar European Union- financed project, and that the government has signaled commitment to continue the safety net operations of the SFD. The project helped build the capacity of SFD, including its Environment Unit, M&E capacity, and knowledge of social accountability, and the project also supported capacity of local NGOs. Likewise, small infrastructure constructed under the project has been handed over to relevant ministries for maintenance. Yet Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) the willingness of the government to continue funding the program is not known. Furthermore, the previously mentioned impact evaluation findings highlight that benefits in terms of higher incomes and expenditures accrue in the short term and dissipate quickly, within a few months, of beneficiaries having completed work under the program. In addition, SFD has lost autonomy as part of a merger of three existing institutions (the Social Fund for Development the Industrial Modernization Center and the Council of Technology and Innovation Centers) into a newly created Medium, Small, and Micro Enterprise Development Agency. The implications of this revised institutional framework on the SFD's program and public works is not clear. 8. Assessment of Bank Performance a. Quality-at-Entry The project design was appropriate for the context and built explicitly on lessons learned from previous experience in Egypt and elsewhere with safety nets in situations of crisis and emergency. The context at the time of appraisal was an observed and expected upsurge in unemployment and underemployment, increasing poverty, and regional disparities. Loss of employment is the most prominent cause of households falling into poverty. The Bank team was concerned that this social stress would further complicate the political transition then ongoing in Egypt. Egypt's safety net system at the time was dominated by food and energy subsidies, and there was no public works program. The project was to build a public works program that could become part of Egypt's larger social protection system while simultaneously responding to the emergency situation. Similar approaches have been taken in many other countries facing economic crises, and have been extensively studied and documented. The project design can be considered in line with good international practice, as it was similar to public works projects in other countries. The international lessons leading the Bank team to propose labor-intensive public works and that guided decisions about targeting approaches were clearly articulated in the Project Paper. The Bank team assessed appropriately the political and implementation risks and took reasonable measures to address them. The Bank team also enabled the Project to move toward greater efficiency, targeting accuracy, labor intensity, and citizen engagement through design choices. For example, efficiency was supported through a well- designed results-based M&E system that was actively used during implementation. Appraisal paid explicit attention to SFD's M&E capabilities, which had been upgraded with external support so as to address previously recognized weaknesses. The Project Paper foresaw the use of M&E as an essential management and planning tool that would be helpful also for targeting. Targeting accuracy was supported through a combination of self-targeting (low wages); geographic targeting (placing subprojects in poorer locales selected using a poverty map); community targeting; and communication efforts to allow unemployed individuals to register for public works activities. The planned use of poverty mapping is a noteworthy positive contribution by the Bank team. The high labor intensity helped translate public spending into earnings for beneficiaries and was supported through the small scale of the projects and criteria for contractors selected to execute subprojects (making the wage share one of the selection criteria for contractors). Several mechanisms underpinned the citizen engagement/social accountability dimensions of the project: a Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) grievance redress mechanism that allowed citizens to provide feedback to the SFD about project implementation and allows potential beneficiaries who were not included in the program to seek redress; public information; quality assurance surveys for community infrastructure services; and independent verification of results of sub-project implementation using NGOs. The simple design of the project was another design strength. Simple goals and simple implementation arrangements are appropriate for emergency operations. The Project Paper is short and clearly articulates the how and why of the chosen design, rejecting several more complex options put forward. According to the Project Paper, the Bank team considered but did not choose the following options: • A Development Policy Loan: appropriate for policy reforms but not to improve short-term employment opportunities as requested by government. • A small and medium enterprise program: was already being done in another Bank-financed project. • Support to design a broader safety net program: would not be feasible within the context of an emergency operation, as it would be more complex in design and therefore would take longer to prepare and would involve several agencies (this has since been done). • Support for labor-intensive public works implemented by line ministries: as stated elegantly by the Project Paper, "while such programs could be implemented relatively quickly, they were rejected for the following reasons: (i) any short term response to the crisis should be consistent with the longer term vision for Egypt, and given that in the long run the private sector has to be the engine of employment generation, hiring more people into an already too large public sector would be inconsistent with the vision, so an approach using private contractors and NGOs is better; (ii) the labor intensity of ministry-run programs is too low; and (iii) ministry programs do not correspond as well to local needs as programs with a more demand-driven approach." Quality-at-Entry Rating Satisfactory b. Quality of supervision Implementation support focused on maximizing development impacts and was impressively data intensive. The Bank team made active use of the M&E system in dialogue with SFD. Aides-memoire indicate discussions about ways to improve targeting efficiency and labor intensity based on M&E data. Continuous adjustments to the project occurred, including a formal (level 2) restructuring. Adjustments were sparked by opportunities to transfer funds to subprojects and regions where they could maximize impacts, aided by M&E data. Capacity building was a part of implementation support, with staff from SFD benefiting from Bank-sponsored trainings, South-South learning, and study tours. Missions also used data from independent verification, independent quality assurance, and various surveys done as part of the impact evaluation associated with the project. Aides-memoire seen by IEG are clear and action oriented. Implementation support also appears to have adequately handled procurement, financial management, citizen engagement, and social accountability aspects of the project. There were some initial implementation delays stemming from compliance with safeguards requirements that were handled proactively by the Bank. Overall, compared to many other investment projects, problems with fiduciary compliance appear to have consumed Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) less of the Bank team's supervisory energy, allowing it to focus on development impacts. The grievance redress mechanism played an active role in discussions between the Bank team and the SFD, as did the citizen engagement pilots, according to aides-memoire. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project results framework omitted indicators to directly capture infrastructure and services access (the second objective), but otherwise the M&E system was well designed. The Project Paper explained in clear language the rationale for the project and supported it with a results framework with relevant and intuitive indicators. The theory of change and chosen PDO and intermediate outcome indicators were appropriate given the PDO and the project design. Inclusion of indicators for the gender and age disaggregation of the employment created under the project was relevant. Independent verification and quality assurance was foreseen as part of a package of social accountability activities. As previously noted, no indicator directly measured achievement of the second objective, access to local infrastructure and community services. Intermediate outcome indicators instead measured project outputs in terms of, for example, infrastructure creation/rehabilitation. Admittedly, improved access would be hard to measure directly, but the lack of congruence between objective and indicators is a shortcoming of the M&E design. b. M&E Implementation The M&E system was implemented largely as designed. Some indicators were added later, in response to modest course corrections. The SDF collected and reported data on the indicators at regular intervals during implementation, as evidenced by the Implementation Status Reports (ISRs) and aides-memoire. The social accountability activities were carried out. An impact evaluation with various surveys was carried out, apparently funded by the Danish International Development Agency, with support/supervision by the Bank. c. M&E Utilization Aides-memoire and ISRs demonstrate active use of M&E data. The mid-term review made extensive and explicit use of the M&E data. Third party verification was used to verify contract documents, payroll, and other aspects of work performed. Adjustments to the project were made to improve targeting based on poverty maps and project M&E data. The ICR relied on the M&E data to document results and to update the economic analysis done as part of project preparation. Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project's environmental category was B, triggering OP/BP 4.01. Environmental Safeguards. Guidelines were prepared for weed reduction and rehabilitation of houses and schools, and Environmental Management Plans were prepared for rural road paving and other small works. The environment unit of the SFD monitored compliance with the Environmental Guidelines and Plans. The Bank supported the SFD in setting up Environment Units at central and governorate levels and establishing a management information system for reporting on environmental safeguards implementation. The ICR reported that safeguards compliance was satisfactory, although with challenges related to irregular reporting, incomplete databases, and inadequate analysis. Overall, the safeguards compliance requirements were met. b. Fiduciary Compliance The ICR reported that financial management and reporting was satisfactory, with no issues noted. Project accounts were audited annually, following international standards on auditing, by qualified independent external auditors. Internal control arrangements were in place, and a financial management system and disbursements were maintained adequately. The Procurement Department of the SFD ensured compliance with the Bank's Procurement Manual, conducted ex-post reviews, and assessed the procurement capacity of intermediary agencies. Because the project was prepared under the Bank's procedure for Rapid Response to Crises and Emergency, higher thresholds were allowed for procurement of the labor intensive sub-projects. SFD followed long-standing local procurement procedures acceptable to the Bank. c. Unintended impacts (Positive or Negative) Positive gender impacts from offering paid employment to women were noted in the ICR, and the draft impact evaluation found more positive impacts on beneficiaries of social services subprojects, who were women, compared to those for infrastructure projects, who were men. This finding merits further study. d. Other Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment As per OPCS/IEG guidelines, split rating against the original Moderately and revised targets has to be Outcome Satisfactory Satisfactory performed. The project achievements fell short of the original outcome targets. Bank Performance Satisfactory Satisfactory --- Quality of M&E Substantial Substantial --- Quality of ICR Substantial --- 12. Lessons The ICR offered several "lessons" and "recommendations" that need not be repeated verbatim here. In the following, some of the more novel/interesting lessons from the ICR are rephrased in a synthesized fashion, while also adding from an IEG perspective: • Simple design, simple objectives, and the use of one implementing agency only is appropriate for emergency operations. • Using an experienced implementing agency with a track record of implementing Bank-financed projects can help a project achieve its goals. For the Bank to have such a relationship--and, in this case, for the SFD to have built this experience--does not happen by chance. An element of continuity over the long term in some of the Bank's social protection engagements contributed to this favorable situation. • Adapting targeting mechanisms during implementation can help to address and reach different vulnerabilities. In this case, as new data became available, it was used, for example, to update the allocation of funds to different types of subprojects, thereby enabling the project to fine tune what categories of vulnerable people it reached. Adjustments to the length of work contracts and the timing of when work is performed can also be used to fine-tune targeting. • Results-based monitoring and active use of data during supervision enhances achievement of project objectives. In this case, framing supervision discussions around the results framework, using the collected data, and identifying the underlying causes of implementation challenges helped steer actions toward outcomes rather than inputs and outputs. It also helped steer World Bank technical support. Actions plans developed this data-driven way during supervision missions helped the project improve its performance. • There is a clear role for citizen engagement and social accountability in labor-intensive public works. These factors can help foster trust and community engagement in public works programs. A number of different citizen engagement mechanisms were used and showed potential: consultations and stakeholder Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EG Emergency Labor Intensive Investment (P126339) mapping; community score cards for benchmarking performance; community voluntary committees with a role in design and oversight of subprojects; and social audits of subprojects. There was also use of independent verification. However, these things are not free. Adequate capacity building for civil society is needed to reap the full benefits of citizen engagement. 13. Assessment Recommended? Yes Please explain Further study of this project might entail an opportunity to explore the differential positive gender impacts from offering paid employment to women that were noted in the ICR. Also, a draft impact evaluation suggested more positive impacts on beneficiaries of social-services subprojects, who were women, compared to beneficiaries of infrastructure projects, who were men. 14. Comments on Quality of ICR The ICR was of substantial quality. The writing was clear and had an adequate amount of detail. It presented in a succinct fashion the rationale for the project and the theory of change. (The task presumably was made easier for the ICR authors by the clarity of the Project Paper.) The clarity of the exposition was supported by: (1) clear logic underpinning the project's rationale and theory of change, and (2) avoiding repetition and unnecessary material. Implementation issues were mostly well described, with one exception: not much detail is given to the implementation of the citizen engagement and social accountability activities. It would have been good to understand how well they worked and in what ways feedback obtained through these activities was used to inform project implementation. In other ways, the ICR made good use of data, most obviously related to indicators and targets but also, for example, in discussions of labor intensity/wage share, in the economic analysis, and in the Borrower comments in Annex 5. However, there were some minor shortcomings. The ICR did not perform the split rating as required by OPCS/IEG guidelines. It also did not define well the outcome indicators for gender and youth participation, and it omitted mention of the impact evaluation findings. Annex 3 on project costs was not completed properly. a. Quality of ICR Rating Substantial Page 15 of 15