Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) Report Number: ICRR0023479 1. Project Data Project ID Project Name P145268 Niger DRM and Urban Development Project Country Practice Area(Lead) Niger Urban, Resilience and Land L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-53400,IDA-64130 30-Jun-2020 120,774,065.98 Bank Approval Date Closing Date (Actual) 11-Dec-2013 31-Oct-2022 IBRD/IDA (USD) Grants (USD) Original Commitment 100,000,000.00 0.00 Revised Commitment 125,000,000.00 0.00 Actual 114,130,060.75 0.00 Prepared by Reviewed by ICR Review Coordinator Group Katharina Ferl Fernando Manibog Avjeet Singh IEGSD (Unit 4) P145932_TBL Project ID Project Name P145932 Niger DRM and Urban Development Project ( P145932 ) L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) 6644005.23 Bank Approval Date Closing Date (Actual) 11-Dec-2013 Page 1 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 6,649,315.00 Revised Commitment 0.00 6,644,005.23 Actual 0.00 6,644,005.23 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Project Appraisal Document (PAD) (p.vi) and the Financing Agreement of December 16, 2013 (p. 5) the objective of the project was “to improve Niger’s resilience to natural hazards through i) selected disaster risk management interventions in targeted project sites and ii) strengthening of Government’s capacity to respond promptly and effectively to an eligible crisis or an emergency”. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 25-Apr-2019 c. Will a split evaluation be undertaken? Yes d. Components The project had four components: Component 1: Flood Risk Management Investments (appraisal estimate US$76.65 million, actual US$77.7 million): This component included three sub-components: Sub-component 1.1: Drainage, irrigation and priority socio-economic infrastructures: This sub-component was to finance the following activities: i) construction of drainage canals and collectors in Niamey, Dosso, Kollo, Say, Tera and Tillabéri; ii) rehabilitation of drainage canals, waste management and rehabilitation of Page 2 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) drinking water supplies and social infrastructures in Niamey; and iii) rehabilitation of irrigated perimeters damaged by 2012 floods along the Niger and Komadougou Rivers. Sub-component 1.2: Flood protection infrastructure: This sub-component was to finance the following activities: i) river bank protection using plants and technical measures; ii) stabilization of ‘koris’ (sandy intermittent streams) to reduce runoff intensity; and iii) rehabilitation/upgrading of dikes to protect urban areas and irrigated perimeters along the Niger and Komadougou Rivers. Sub-component 1.3: Rehabilitation of watersheds: This sub-component was to finance the following activities: i) development of sustainable land and water management practices; ii) reshaping or re-profiling of natural drainage canals (former arms of the Niger River and Gounty Yena) in Niamey; and iii) pond control structures along the Komadougou River in Diffa region. Component 2: Capacity Building for Urban Development and Disaster Risk Management (appraisal estimate US$22.0 million, actual US$21.1 million): This component included the following sub- components: Sub-component 2.1: Support to elected officials, municipal services and civil society: This sub-component was to finance the following activities: i) strengthening of local governments’ fiduciary and technical capacity; ii) developing information management systems and master plans for sewerage and development at municipal and regional levels. Sub-component 2.2: Support to central government: This sub-component was to finance i) the development of national capacity with a crosscutting approach of institutional collaboration through the development of a national policy on storm water, wastewater, basic sewerage, and solid waste; ii) capacity development for planning in urban and rural areas; iii) equipment and training to monitor river water levels and flows; and iv) supporting maintenance of irrigated perimeters. Sub-component 2.3: Strengthen disaster risk management capacities: This sub-component was to strengthen national and local DRM capacities, including risk evaluation, risk reduction, preparedness and emergency response capacity. Activities were to include: i) developing a risk atlas (hazard, exposure, vulnerability and loss probability information); ii) developing real-time multi-hazard information systems for proper monitoring of risk and available resources; iii) setting up standard operating procedures for early warning and response; iv) developing civil protection capacities for emergency response; and v) developing guidelines for safe construction practices and mainstreaming disaster risk reduction and climate adaptation into development processes. When the project received Additional Financing (AF) in 2019 the following sub-component was added: Sub-component 2.4: Building information and communication technology (ICT) capacities for urban development: This sub-component was to finance acquisition of high-resolution imagery for the six major cities; ii) training youth and community members on digital cartography to gather information for urban planning share as open maps and data; and iii) renovation of an existing innovation center plus equipment. Component 3: Project Management (appraisal estimate US$5.0 million, actual US$9.2 million: This component was i) to finance the coordination of all project activities including monitoring and evaluation and cover all eligible expenditures including strengthening capacity; ii) procurement of office furniture and equipment; hiring of essential staff, including technical and financial audits; and iii) all recurrent costs. Page 3 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) Component 4: Contingency Component (appraisal estimate zero, actual US$4.6 million): This component was to draw resources from unallocated expenditure categories and/or allow the government to request the Bank to re-categorize and reallocate financing from other components to partially cover emergency response and recovery costs. This component could also be used to channel additional funds should they become available as a result of an eligible emergency. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project was estimated to cost US$131.65 million. Actual cost was US$120.77 million. Financing: The project was financed by two IDA credits in the amounts of US$100 million (of which US$90.77 million was disbursed) and US$25 million (of which US$23.35 million was disbursed), as well by a Trust Fund grant in the amount of US$6.65 million (of which US$6.64 million was disbursed). Borrower Contribution: The project had no plan for the Borrower to make any contribution. Dates: The project was restructured four times:  On March 28, 2016, the project was restructured to: i) transfer project oversight from the Ministry of Planning, Land Development and Community Development to the Prime Minister’s Office; ii) update the disbursement schedule; and iii) update two targets in the Results Framework.  On March 23, 2017, the project was restructured to: i) reallocate US$3.4 million to component 4 as part of the IDA Immediate Response Mechanism (IRM) to address the impact of the 2016 flooding. Also, two additional regions affected by the floods (Agadez and Tahoua) were added to the project.  On April 25, 2019, the project received Additional Financing (AF) in the amount of US$25 million to: i) scale up project activities in two additional regions (Agadez and Tahoua), add new ICT activities (sub-component 2.4) and expand these new activities to Maradi and Zinder. The restructuring also made the following changes: i) extend the project closing date by 12 months to June 30, 2021 to allow for the completion of newly financed project activities; and ii) modify the Results Framework (increase of PDO indicator 1 target to reflect IRM and AF activities, decrease of PDO indicator 2 target due to a reduction of activities, reformulation of PDO Indicator 3 to capture activity impact and changes of units of measurement, plus a new target, and cancellation of two intermediate outcome indicators of non-budgeted activities).  On June 29, 2021, the project was restructured to: i) extend the closing date by 16 months to October 31, 2022, to allow for the completion of activities, which had been delayed; ii) reduce scope of certain flood protection infrastructure and watershed rehabilitation activities under component 1; iii) add two regions, Maradi and Zinder to the high resolution imagery acquisition activity under component 2; iv) cancel sub-indicator “Pond control infrastructure (Hectare)”as the rehabilitation was postponed and was not be completed as part of the project; and v) decrease target for the Koris drainage infrastructure rehabilitation from 21.5 km to 5 km, as some infrastructure rehabilitation in Tahoua was postponed, and was not to be completed as part of the project. In 2019, while the PDO was not revised, PDO indicators were modified and the scope of the project was reduced, especially of the first objective of the project ("improve resilience") with the reduction of activities to be accounted for under the interventions contributing towards improved resilience. Therefore, this review conducts a split evaluation. Page 4 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) 3. Relevance of Objectives Rationale Country and sector context. According to the PAD (p. 1) at the time of appraisal, in 2013, Niger was one of the poorest countries in sub-Saharan Africa with a per capita Gross Domestic Product (GDP) of about US$383 and a life expectancy of 57 years. Niger also suffered from extremely low human development indicators in health and education and faced political instability, an influx of refugees due to armed conflicts in neighboring countries, as well as the emergence of political and religious groups as alternative service providers. During the 30 years before project appraisal ten major droughts and nine flooding events took place in Niger. As a result of the droughts, Niger experienced uncontrolled demographic growth in urban areas. Agriculture was the most important sector of Niger’s economy and accounted for over 40 percent of national GDP. Also, agriculture was the principal source of livelihoods for over 80 percent of the country’s population, including the one living in urban areas. However, the performance of the agricultural sector was very volatile due to its high exposure to risks. A number of factors contributed to the rising disaster risks in Niger including population growth, deforestation, increasing soil erosion and land degradation in watersheds and upper catchment areas of major river basins, as well as climate variability and change. Furthermore, the risk of disasters was exacerbated by inadequate planning, especially regarding population settlements along the banks of the Niger and Komadougou rivers, poor building standards, obsolete or inadequate infrastructure such as vulnerable protective dikes in inhabited areas, lack of interconnectivity among information systems, as well as limited emergency response and recovery capacity. Alignment with the Government Strategy. The objective of the project was in line with several government policies including the following: i) Pillar 3 (“Accelerate Economic Growth”) and Pillar 5 (“Sustainable Management of the Environment”) of Niger’s 2017-2021 Economic and Social Development Plan (PDES); ii) the newly elected government 2021 policy declaration of “strengthening of urban planning capacities and investment, the improvement of public services, and the strengthening of DRM, as key priority areas”; and iii) the 2020 Prevention and Resilience Allocation Objectives and the Prevention and Resilience Action Plan (objective 2 “to decrease multidimensional insecurity through participatory security management and a stronger presence in the border and at-risk areas”). Alignment with the World Bank Strategy. The objective of the project was in line with the World Bank’s most recent Country Partnership Framework (FY18-22) and its Pillar three (“strengthening governance by promoting important sectoral reforms that improve service delivery, expand economic opportunity, bolster economic growth, and augment the resilience of the population to fragility and climate change”. The objective of the project was pitched at an appropriate level to address a critically important development problem. Overall, the relevance of the objective is rated High. Page 5 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Improve Niger’s resilience to natural hazards through selected disaster risk management interventions in targeted project sites Rationale Theory of change: The project’s theory of change postulated that project inputs/activities such as investments into drainage, irrigation, and priority socio-economic infrastructures, as well as flood protection infrastructure and rehabilitation of watersheds were to result in outputs such as sources of drinking water rehabilitated or developed, targeted irrigable land rehabilitated, watershed protection and land restoration implemented, as well as drainage rehabilitated. These outputs were to result in the outcome of Niger’s resilience to natural hazards being improved. The theory of change was sound and logical. Outputs:  492 sources of drinking water were rehabilitated or developed, exceeding the original target of 260 sources.  1,647.27 hectares of targeted irrigable land were rehabilitated, not achieving the original target of 2,000 hectares.  6,000 hectares of sand dunes were fixated exceeding the original target of 5,500 hectares. This target was not revised during project restructuring.  11,020 hectares of degraded land were restored, significantly exceeding the target of 6,500 hectares. This target was not revised during project restructuring. The overachievement of this target was due to including works under the Immediate Response Mechanism (IRM) 2016 for which resources were taken from this project and reimbursed through the AF without revising the target of this indicator.  80.59 kilometers of stonewalls were constructed, not achieving the original target of 200 kilometers. The ICR (p. 36) stated that the target was not achieved since the municipalities of Agadez, Tahoua, Loube, Bogon, and Azzem requested support for the protection of Koris and school enclosures, as well as for the rehabilitation of collapsed classrooms in Loube.  47.57 kilometers of dikes were protected, close to achieving the target of 50 kilometers. This target was not revised during project restructuring.  120.95 kilometers of drainage infrastructure were rehabilitated, significantly exceeding the target of 23.50 kilometers. This target was not revised during project restructuring. The large overachievement was a result of not only rehabilitating primary drainage networks but also secondary and tertiary drainage networks that were originally not planned but then requested by the government.  8.82 kilometers of drainage were built, close to achieving the target of 10 kilometers. This target was not revised during project restructuring. Page 6 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268)  9.30 kilometers of drainage pavement were built, not achieving the target of 20 kilometers. This target was not revised during project restructuring.  6.40 kilometers of Koris (sandy intermittent waterways) were constructed, exceeding the target of 5 kilometers. This target was not revised during project restructuring. The project involved the strategic enhancement of these Koris, including reinforcing banks, erecting stone barriers, and installing thresholds. These interventions aimed to reduce water runoff intensity, thus reducing flood risk. Koris are natural ravines formed by the erosive action of runoff water. Intermediate Outcomes:  Targeted flood protection and sustainable land and water management interventions contributing to increased resilience were completed. According to the ICR (p. 34) the original target of nine activities being completed for the achievement of this indicator target was reduced to five activities due to substantial underestimation of costs at project preparation. The nine activities originally part of this indicator were: 1. Protection/stabilization of koris banks through a combination of mechanical and biological measures; 2. Construction/ rehabilitation of protective dikes for towns/villages and Irrigated Perimeters 3. Construction/ rehabilitation of road dikes; 4. Construction of stone barriers to reduce the intensity of water flow; 5. Restoration of the watersheds of the tributaries of the Niger (Sirba, Gouroubi, Dargol, Gorouol) and the Komadougou rivers in order to increase infiltration, reduce erosion and fix sand dunes; 6. Regulation of flood management structures along the tributaries (Sirba and Goroubi) combining positive effects for agriculture and integrated flood management; 7. Rehabilitation/ development of pond control structures along the Komadougou River in the Diffa region in order to increase flood water storage capacity and optimize water use; 8. Rehabilitation of old natural drains that have collapsed and 9. Rehabilitation of drainage gutters/collectors/sewers. Per ICR, five activities were completed. Subsequently, the Bank team clarified (email dated October 16, 2023) that seven out of nine activities were completed. Activities 6 and 7 could not be completed due to growing insecurity, particularly in the Diffa Region due to Boko Haram, and the cost underestimations. The two non- completed activities (6 and 7) will be financed under the ongoing Niger Integrated Urban Development and Multisectoral Resilience Project (P175857) approved in April 2022.  The project benefitted 4,375,633 beneficiaries, exceeding the original target of 4.0 million beneficiaries. 49.16 percent of beneficiaries were female, close to achieving the target of 50 percent. Given that the project implemented seven out of nine activities per the original definition of the outcome indicator to enhance resilience, the achievement of this objective is rated as Substantial. Rating Substantial OBJECTIVE 1 REVISION 1 Revised Objective The objective remained the same but the targets were revised downwards Page 7 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) Revised Rationale The theory of change did not change when the targets were revised. Outputs:  492 sources of drinking water were rehabilitated or developed, exceeding the revised target of 450 sources.  1,647.27 hectares of targeted irrigable land were rehabilitated, almost achieving the revised target of 1,700 hectares. The ICR (p. 35) stated that the revised target was not achieved due to security reasons and the government declaring state of emergency in Diffa starting in 2016.  80.59 kilometers of stonewalls were constructed, exceeding the revised target of 10 kilometers. Outcome:  Targeted flood protection and sustainable land and water management interventions contributing to increased resilience were completed. The original target of 100 percent was achieved.  The project benefitted 4,375,633 beneficiaries, exceeding the revised target of 4.3 million beneficiaries. 49.16 percent of beneficiaries were female, close to achieving the target of 50 percent. The project was able to achieve the majority of output and outcome targets under this objective. In fact, it implemented seven out of nine activities, that exceeded the revised target of five activities to measure the resilience indicator. Therefore, the achievement under this objective with revised targets was Substantial. Revised Rating Substantial OBJECTIVE 2 Objective Improve Niger’s resilience to natural hazards through strengthening of government’s capacity to respond promptly and effectively to an eligible crisis or an emergency Rationale Theory of change: The project’s theory of change stated that project inputs/activities such strengthening the fiduciary and technical capacities of local governments, developing information management systems and master plans for sewerage and development at municipal and regional levels as well as setting up standard operating procedures for early warning and response were to result in several project outputs. These project outputs were to include the fiduciary and technical capacities of local governments being strengthened, information management systems, master plans for sewerage and development at municipal and regional levels being developed as well as standards operating procedures for early warning and response being set up. These outputs were to result in the objective of Niger's resilience to natural hazards being improved. The theory of change was sound and logical. Page 8 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) Outputs:  26 urban master plans and local development plans were developed and/or updated, not achieving the original target of 39 plans.  Five percent of municipal budget was allocated to solid waste collection and processing/transformation, achieving the original target of five percent.  The database of risk in Niger was fully functional, achieving the target of being fully functional.  Support was provided to civil protection to strengthening response capacity (facilities, equipment, training), hence the target of providing support was achieved.  Support was provided to 10 national DRM agencies (equipment, training), achieving the target of 10 agencies.  780 kilometers of area was detailed in maps in digital and paper formats, exceeding the target of 400 kilometers.  76 people were trained on digital cartography, not achieving the target of 200 people. Reasons for not achieving the target were related to delays in the start of ICT and academic activities and security related postponements.  Five local innovation projects were supported, not achieving the target of eight projects. Reasons for not achieving the target were related to security issues and rigorous selection criteria. Outcome:  Five institutions involved in early warning systems (EWS) functioned properly, achieving the target of five institutions. According to the Bank team (August 23, 2023) “functioning properly” entailed that these institutions possessed the capability and necessary resources to promptly deliver information, allocate resources, generate warning notifications for public dissemination, and respond efficiently to natural disasters. The project was able to achieve most output targets and the outcome target. Therefore, the achievement of the second objective with the original targets was Substantial. Rating Substantial OBJECTIVE 2 REVISION 1 Revised Objective The objective remained the same but the targets were revised downwards Revised Rationale The theory of change did not change when the targets were revised. Outputs:  26 urban master plans and local development plans were developed and/or updated, achieving the revised target of 25 plans. Page 9 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268)  The percentage of female participation in decision committees for development and/or updating urban master plans and municipal development plans was 23.27 percent, almost achieving the target of 25 percent. The target was not revised during project restructurings.  Five percent of municipal budget was allocated to solid waste collection and processing/transformation, exceeding the revised target of four percent. Outcome:  Five institutions involved in early warning systems (EWS) functioned properly, achieving the target of five institutions. According to the Bank team (August 23, 2023) “functioning properly” entailed that these institutions possessed the capability and necessary resources to promptly deliver information, allocate resources, generate warning notifications for public dissemination, and respond efficiently to natural disasters. Additional results indicators that were measured included:  The target of quality and timely submission of procurement and financial management reporting was achieved.  90.45 percent of planned project activities were implemented, not achieving the target of 100 percent.  90 percent of grievances related to the delivery of project benefits were addressed, exceeding the target of 80 percent.  It took the government eight weeks to prepare/submit the activation package for an eligible crisis or emergency triggering an Immediate Response Mechanism (IRM), not achieving the target of four weeks. The project was able to achieve several output targets and the outcome target. However, the project was not able to achieve the target of responding promptly to an eligible crisis or emergency. Overall, the achievement under this objective was Substantial with moderate shortcomings. Revised Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Achievement of the first objective with original targets was Substantial given that the project implemented seven out of the nine activities per the original definition of the indicator to enhance resilience. Achievement of the second objective with original targets was Substantial. The overall efficacy is Substantial. Page 10 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) Overall Efficacy Rating Substantial OBJR1_TBL OVERALL EFFICACY REVISION 1 Overall Efficacy Revision 1 Rationale Achievement of the first objective with revised targets and definition of the outcome indicator was Substantial. Achievement of the second objective with revised targets was Substantial with moderate shortcomings due to the delays in responding promptly to the emergency, which was triggered post-restructuring. The overall efficacy is Substantial. Overall Efficacy Revision 1 Rating Substantial 5. Efficiency Economic efficiency: The PAD (p. 14) conducted an economic analysis in which the avoidance of the 2012 damages were considered as benefits to accrue every three years as the cycle of serious floods was reported every three years based on the analysis assumptions. The PAD (p. 49) stated that due to the nature of the project, to quantify the potential project benefits and estimate the economic rate of return to project investments was challenging. First, the project had only a few revenue generating activities to quantify benefits. Second, the project had adopted a demand driven framework approach in which the specific investments for individual activities were not pre- identified. Third, the project allocated almost 25 percent of the project cost to strengthen the disaster risk management capacity and social accountability, which were difficult to quantify. Fourth, the first two components alone dealt with a large number of different and diverse activities making it difficult to estimate project benefits. Fifth, the project included a contingency component that could be activated during the lifetime of the project and for which it was difficult to determine the likely cost and benefits. The PAD (p. 50) assumed that through proper and timely operations and maintenance (O&M), the infrastructure rehabilitated and/or built under this project was to continue to provide flood control benefits for at least 25 years, depending on the type of infrastructure. The analysis calculated a Net Present Value (NPV) of US$189 million (applying a discount rate of 5 percent) and an Internal Rate of Return (IRR) of 33 percent. Furthermore, the benefit-cost ratio ranged between 2.33 and 3.10 when discounted between 5 percent and 20 percent and a sensitivity analysis considered a reduction in benefits and increase in cost of 20 percent. This analysis indicated that the project was a worthwhile investment. When the project received AF in 2019, the economic analysis for the original project remained the same. The AF paper (para. 28) stated that the impact of the additional activities was positive. However, no data was included in Page 11 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) the AF paper to provide details on the economic efficiency., making an assessment of the NPV and IRR for AF challenging. The ex-post economic analysis covered the project’s entire actual financing amount. The analysis used a 6 percent discount rate and calculated a NPV of US$158 million, an IRR of 16 percent, and a benefit-cost ratio of 2.6. This analysis indicated that the project was a worthwhile investment. Operational efficiency: The project experienced several implementation challenges because of weak safeguards and fiduciary capacity, which resulted in implementation delays. Also, the project experienced significant delays after effectiveness since the feasibility studies for the rehabilitation and construction activities took longer than planned. Furthermore, the COVID-19 pandemic resulted in implementation delays. Overall, the project’s implementation period had to be extended twice by a total of 16 months resulting in 99.15 percent of financing being disbursed. The ICR (para. 44) stated that while the project preparation costs were 0.4 percent of the disbursed funds and in line with the 0.4 percent regional benchmark, project supervision costs were high at 2.8 percent of the disbursed funds compared to the 1 percent regional benchmark as a result of four project restructurings and close supervision. Even though the project’s implementation period had to be extended by 16 months, the results of the economic analysis were positive. Therefore, efficiency is rated Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal  33.00  Not Applicable 100.00 ICR Estimate  16.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The Relevance of objectives is rated high. The efficacy rating is substantial in both pre-and post-restructuring. Efficiency is rated substantial. Although the split evaluation is applicable since the overall outcome rating is Satisfactory both before and after restructuring, the spliet evaluation will not make a difference. Page 12 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) The overall outcome is rated as Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The risks to development outcomes can be classified into the following categories: Government commitment/Financing: According to the Bank team (August 23, 2023) the infrastructure established through the project was formally handed over to the relevant authorities at both local and national levels, ensuring ongoing service provision, and operation and maintenance (O&M). The Government has reiterated its dedication to maintaining the operation of these infrastructure investments. This will be accomplished by furnishing suitable technical and financial resources in a manner that ensures long-term sustainability. A significant portion of these investments is managed by municipalities that are less susceptible to the political fluctuations occurring at the national level. Additionally, municipalities have access to their own sources of revenue, albeit limited, which mitigates the financial pressures originating from the national level. Nonetheless, given the recent political shifts in Niger and the cessation of all aid to the country, there is a potential risk that the municipalities' financial health could deteriorate, potentially leading to repercussions on the O&M of the investments. Technical capacity: According to the ICR (para. 91) investments made under the project require adequate operation & maintenance for which existing management committees were responsible. However, it is not clear to what extend these committees will be available given the limited technical and financial capacity of beneficiary municipalities. Also, the available capacity of service providers will be critical for ensuring the sustainability of project outcomes. 8. Assessment of Bank Performance a. Quality-at-Entry According to the PAD (p. 9) the project was built on lessons learned from other projects implemented in Niger as well as international projects, especially in the areas of rural and urban development, disaster risk management, and sustainable land and water management. For example, one lesson was that local government authorities, in consultation with community leaders and other stakeholders, were better positioned than the central government to prioritize urban development and interventions aimed at increasing resilience to disasters. According to the ICR (para. 83) even though the project design was complex and covered a large area of the country, it was sufficiently flexible to allow for adjustments during implementation such as adjusting indicators and activities to meet the challenges resulting from the 2016 floods and deteriorating Fragility Page 13 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) Conflict and Violence (FCV) conditions. However, the project was prepared in only 7.7 months following regular procedures, and project readiness was fully achieved when it became effective (ICR, para 83).. The Bank team identified relevant risks to project implementation including instability of the operating environment, the implementation complexity of the project design and weak capacity of the implementing institutions, especially in regard to fiduciary aspects. According to the PAD (p. 13) the Bank team tried to mitigate these risks by drawing on lessons learned from several Bank projects in Niger and building as much as possible on the experience and structure of existing government institutions. Also, procurement and financial management arrangements were designed to mitigate fiduciary risks through regular financial and procurement planning and reporting, following Bank and government guidelines and practices, and a qualified fiduciary team at the level of the PCU. However, the ICR notes that implementation experienced a slow start due to the lack of adequate management and staffing as well as delays in finalizing the feasibility studies of several infrastructure interventions. The PDOs were focused and realistic; however, as the project was prepared in less than a year in a fragile operational environment, it was a challenge to set targets for flood protection and sustainable land and water management interventions. Quality-at-Entry Rating Satisfactory b. Quality of supervision According to the ICR (para. 66), the Bank team conducted 22 supervision support missions, which allowed the team to identify implementation bottlenecks adequately. The Bank team members had appropriate expertise from relevant sectors, with some team members based in the country, allowing for close supervision support. The ICR (para. 84) stated that between November 2014 and February 2018, the project’s performance was rated Moderately Satisfactory due to delays in the recruitment of the regional PIU teams, a slow implementation start, capacity issues, and the poor quality of technical and procurement documentations. The Bank team addressed these issues by providing close technical assistance and building capacity in key areas such as safeguards, procurement, and financial management. The Bank team was also proactive in consistently improving project implementation through four restructurings in 2016, 2017, 2019, and 2021, particularly for Components 1 and 2 by the end of 2018 and for the CERC trigger. These restructurings provided further clarity to the RF and reallocated funds across components for greater effectiveness. The project experienced some delays in the Bank’s “no objection” due to Task Team Leader changes between 2018 and 2020. Quality of Supervision Rating Page 14 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The objective of the project was clearly specified and realistic. The project’s theory of change and how key activities and outputs were to result in intended outcomes was sound and reflected in the Results Framework. The intermediate outcome indicators were adequate to capture the project’s outputs as well as sufficiently specific, and measurable and had baselines (when available), and targets. However, some of the PDO indicators at design lacked clarity. For example, PDO indicator 3 (“performance of the early warning and response system or natural rapid onset hazards”) with a target of “five institutions” lacked clarity in terms of how the improved performance would be actually defined and measured. The ICR also notes as the project was prepared in less than a year in a fragile operational environment, it was a challenge to set targets for flood protection and sustainable land and water management interventions. b. M&E Implementation The Results Framework was revised four times during project restructurings to reformulate PDO Indicator 3 to capture activity impact and changes of units of measurement, as well as refine more realistic targets based on implementation experience and on the United Nations Office for the Coordination of Humanitarian Affairs (UNOCHA) post-2016 flood assessment. According to the ICR (para. 72) the project’s M&E performance was mostly rated Satisfactory. However, between December 2015 and June 2016, the rating was decreased to Moderately Satisfactory and Moderately Unsatisfactory due to the lack of an objective methodology to measure indicators in the first M&E report, as well as delayed delivery of the first M&E report (only 16 months after project effectiveness). This issue was addressed by the project adequately, and an integrated database including information from procurement, financial management, safeguards, and M&E was established, which allowed for a detailed overview of implementation by municipality, sector, sub-component, and cluster of beneficiaries. From April 2018 until the project closing, the project’s M&E rating was Satisfactory. According to the Bank team (August 23, 2023), the PIU was fully committed to ensuring the quality of M&E implementation. The PIU was responsible for the overall M&E oversight of the project and for reporting on results and progress. The PIU's M&E specialist gathered and presented data according to standardized reporting formats aligned with the World Bank's M&E requirements. This monitoring process effectively facilitated progress tracking and enhanced the project's implementation quality. The Bank team stated (August 23, 2023) that data was found to be reliable and of good quality. All data underwent thorough review, control, and validation by experts from the Bank (including Task-team members, financial management specialists, procurement specialists, environmental and social Page 15 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) specialists, etc.), as well as through the Geo-Enabling initiative for Monitoring and Supervision (GEMS) and/or via field visits conducted during implementation support missions or technical missions. c. M&E Utilization According to the ICR (para. 73), the project’s M&E data was used to track implementation progress toward achieving the objective, identify implementation bottlenecks, and inform decision-making. The project used remote sensing tools, such as the Geo-Enabling Initiative for Monitoring and Supervision (GEMS) in volatile and conflict-affected settings, which helped the PIU and M&E team to monitor and assess the progress of infrastructure works. The M&E was used to inform/ensure the presence of the project on the Internet through the creation of two websites, www.pgrcdu-niger.org and http://pgrcdu- niger.com, as well as a Facebook account. Overall, even though there were some design shortcomings, the project restructurings tried to address these during implementation, and the project effectively used innovative tools such as remote sensing GEMS to monitor the progress in the FCV context. The overall M&E rating is Substantial with some minor shortcomings. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as category B and triggered the Bank’s safeguard policy OP/BP 4.01 (Environmental Assessment), OP/BP 4.12 (Involuntary Resettlement), OP/BP 4.09 (Pest Management), OP/BP 4.11 (Physical Cultural Resources), and OP/BP 7.50 (International Waterways). According to the ICR (para. 76) the project prepared an Environmental and Social Management Framework (ESMF), Resettlement Policy Framework (RPF), and Pest Management Plan. When the project received AF in 2019, the safeguard instruments were updated to reflect the addition of the two regions, Tahoua and Agadez. The ICR (para. 26) stated that the project also prepared a Resettlement Action Plan (RAP) for the drainage construction works in Tahoua and Konni in compliance with OP/BP 4.12. According to the ICR (para. 85) the project experienced significant safeguard related issues such as: i) environmental and social screening of sub-projects not being performed in a systematic manner and civil works being started before screenings were conducted; ii) on some occasions civil works being started before safeguard documents had been prepared and implemented; iii) insufficient environmental and social staffing at PIU level; iv) low involvement of staff in the preparation and analysis of relevant bidding documents in the field; and v) insufficient communication with local communities. The project conducted a social audit on draining pavements works. The audit found that there was no economic or physical displacement, only very limited temporary restriction of access indicating a lack of compliance with safeguard requirements. Page 16 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) According to the ICR (para. 78) the project developed a Grievance Redress Mechanism (GRM) at the village/project activity location community and prefect levels. The project received 19 complaints, of which most were related to the lack of communication on how GRM functions and how beneficiaries can access it. b. Fiduciary Compliance Financial Management: According to the ICR (para. 80) the project complied with is FM reporting requirements and submitted Interim Financial Reports and audits reports in a timely manner. However, the project experienced several financial management related issues. The independent auditor provided five qualified opinions spanning from 2016 to 2020 due to unjustified or insufficiently justified expenses. The PIU addressed these issues and in 2021, the qualifications on unjustified advances were lifted based on acceptable evidence and the annual accounts received unqualified opinions. Also, according to the ICR (para. 86) the project used the project interest account to meet unplanned requests from sector ministries. The Bank addressed these issues by providing close technical support. When the project closed, there were no ineligible expenditures on IDA funds and the project’s FM rating was Moderately Satisfactory. Procurement: According to the ICR (para. 81) the project experienced procurement related bottlenecks due to long delays in awarding contracts, particularly in the evaluation stage, and implementation of activities due to the failure of companies. According to the Bank team (August 23, 2023) certain consulting firms and construction companies contracted by the government with project funds to carry out project activities were unable to meet their contractual obligations, leading to multiple delays in the execution of contracts. These delays were primarily due to technical shortcomings and the limited capabilities of these companies resulting in the cancellation of contracts. Also, the project experienced delays in the Bank’s “no objection” due to Task Team Leader changes between 2018-2020. The ICR (para. 30) stated that to address these issues, the Bank provided close technical support to strengthen technical capacity. According to the Bank team (August 23, 2023) the project followed the Bank’s procurement guidelines were followed, and procurement was rated either “Satisfactory” or “Moderately Satisfactory” in all Implementation Status and Results Reports (ISRs). Furthermore, the Bank team stated that the Bank Procurement Specialists maintained a close collaboration with their counterparts in the PIU to ensure a shared understanding of arising issues. These issues were subsequently managed in accordance with the Bank's guidelines. The Bank provided procurement capacity-building support throughout implementation, including training. When the project closed, the procurement rating was Moderately Satisfactory. c. Unintended impacts (Positive or Negative) Page 17 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) NA d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR (p. 30-32) included the following lessons learned, which were adapted by IEG:  Improving urban resilience may require a multi-sectoral approach. This project implemented activities related to flood risk management, drainage, land restoration, water management, urban development, and early warning, preparedness, and response. Also, combining structural investments with nonstructural activities such as flood risk mapping and communication, as well as land use planning and early warning positively may positively impact the sustainability of project outcomes.  Including mitigation measures that allow for project supervision in FCV environments may be beneficial for achieving positive project outcomes. This project used digital solutions such as GEMS for stronger territorial presence and remote supervision when there was a security risk.  Hiring project staff, also for regional PIUs, during project preparation allows for a swift project implementation start. In this project, implementation experienced several delays due to the lack of regional PIUs. Once the regional PIUs were staffed, implementation took off. 13. Assessment Recommended? No 14. Comments on Quality of ICR Page 18 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger DRM and Urban Development Project (P145268) The ICR was clear, candid, and consistent with OPCS guidelines. The ICR provided an adequate overview of project preparation and implementation and an adequate economic analysis. The lessons learned were useful and can be applied to other projects in this area. The ICR did not conduct a split rating even though the project's scope was reduced. Also, the team clarified that the ICR did not correctly present the information regarding the activities implemented to measure the resilience indicator. The ICR stated five out of nine activities were implemented. However, the team subsequently clarified seven activities were implemented. Overall, the ICR quality rating is Substantial. a. Quality of ICR Rating Substantial Page 19 of 19