Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) Report Number: ICRR0022204 1. Project Data Project ID Project Name P149714 Niger River Basin Management Project Country Practice Area(Lead) Western Africa Water L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-18539 31-Dec-2019 4,198,202.93 Bank Approval Date Closing Date (Actual) 28-Jan-2015 31-Dec-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 7,500,000.00 7,500,000.00 Revised Commitment 4,198,202.93 4,198,202.93 Actual 4,198,202.93 4,198,202.93 Prepared by Reviewed by ICR Review Coordinator Group Dileep M. Wagle Vibecke Dixon Ramachandra Jammi IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The objective of the Project, as stated in the Grant Agreement (p.5), was “to strengthen the institutional framework for regional cooperation in the Niger River Basin”. The project objective, as stated in the PAD (p.6), was identical to the above. Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? Yes d. Components Component A: Strengthening the Niger Basin Authority (NBA) for Sustainably Delivering its Mandate (cost at appraisal: US$3.75 million; actual at completion US$2.40 million) This component supported the institutional, organizational and financial strengthening of the NBA to enhance its capacity to implement the its mandate. There were three sub-components: A1. Further analysis and operationalization of selected financing mechanisms for the NBA (cost at appraisal: US$1.2 million), inter alia via (a) development of a business case for hydropower levy, (b) development of a related feasibility study, (c) regional and national stakeholder consultations, (d) design of am implementation plan, and (e) provision of support to the NBA to implement said plan. A2. Implementation of the Water Charter (cost at appraisal: US$0.55 million) through inter alia (a) revisions to the draft Annex 2, based on stakeholder consultations at national and regional levels, improvement and operationalization of the relevant hydrological information system tools for implementation of Annex 2. This would be supported through provision of consulting services, technical assistance (TA) and goods. A3. Strengthening the NBA’s institutional and organizational systems (US$2.0 million) by providing financial support to (i) the Dam Safety and Safeguards Panel of Experts, (ii) NBA operational expenses related to project implementation, (iii) project operational expenses including Project Management and Coordination Unit (PMCU) staffing, and technical/organizational support to the implementation of priority recommendations from the institutional and organizational audit prepared under the Niger Basin Water Resources Development and Sustainable Ecosystems Management (WRD-SEM) Program. Component B: Facilitating Evidence-Based Decision-Making in the Fomi multipurpose project (FMP) (US$3.75 million; actual at completion US$ 1.8 million). This component provided process support to the design and implementation of a clear roadmap for project development, decision-making points, and engagement and sensitization of concerned stakeholders, plus complementary studies on environmental and social impacts. There were four sub-components: B1. Process of engagement and decision-making around Fomi (US$1.65 million) through (a) Support to the NBA for development of a summary of the state of knowledge on the FMP, (b) development of a strategy and Action Plan for the decision-making process on the FMP, (c) undertaking a process of engagement and Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) sensitization of stakeholders, plus workshops to familiarize decision-makers with technical details of key studies. B2. Institutional frameworks for FMP (US$1.1 million). This sub-component supported studies and TA to explore institutional arrangements for the FMP. B3. Complementary environmental and social assessments (US$1.1 million) through complementary studies related to environmental and social risks arising from the FMP, including: (a) an advanced modelling of the ecosystem services in the Niger Inner Delta, and (b) further studies in response to issues emerging from the technical reports and Environmental & Social Impact Analysis (ESIA). B4. Donor and investor roundtables (US$0.35 million) via a program of activities to enable the NBA to mobilize resources from a range of potential sources, including investor conferences organized, and communications materials developed to introduce the FMP to investors. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The original estimated project cost was US$7.5 million. Actual cost at project completion was US$4.2 million. Financing: The project’s cost was financed through an IDA grant of US$7.5 million. Dates: The closing date for the original loan was December 31, 2019. There were no extensions to the closing date, and the project closed on schedule. Restructurings: The project went through a restructuring on June 27, 2018, to update the results framework, change some component and their costs, propose a partial cancellation of the grant (reducing the amount from US$7.5 million to US$4.2 million), and reallocate between disbursement categories. The PDO remained unchanged. 3. Relevance of Objectives Rationale Regional Context: The Niger River Basin is mostly rural, poverty-stricken, agriculture-dependent and vulnerable to food insecurity and conflict. At the time of appraisal in 2015, significant changes were taking place on the Niger River and its main western tributaries, impacting its transboundary water flows, which called for a strong river basin organization that would be able to ensure coordinated management and development, so as to avoid furthering poverty in already fragile areas. These changes included the development of three large dams in the upper basin in Guinea (Fomi), Mali (Taoussa) and Niger (Kandadji), which would significantly alter water flows downstream and potentially affect hydropower generation from downstream dams being constructed at the time in Nigeria. Given the heavy concentration of people living along the banks of the Niger River in the Sahel, including the cities of Bamako and Niamey and the vast NID wetland in Mali, it was critical to ensure that decisions potentially affecting the livelihoods of so many (over a million) people Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) be taken based on evidence, and taking into account their local and transboundary impacts. Towards this end, it was critically important to ensure that the Niger Basin Authority (NBA), which – despite the commitment of the member states - was struggling at the time to fill its regulatory role, be strengthened through upgrading of its institutional, financial and technical capacity, so that it could effectively fulfil its mandate of managing water resources in the Basin. This was strongly supported by Member States themselves, whose Council of Ministers validated and approved several steps for improving NBA as an organization, including adoption in November 2019 of a legally binding Annex to the Water Charter, to which they would have to abide when developing new infrastructure in the river basin; to be implemented by NBA. This support reflected the commitment to regional cooperation by the nine riparian states, under the Paris Declaration in 2004, which reaffirmed the central role of the NBA in implementing a Basin-wide Sustainable Development Action Plan (SDAP). Implementation of the SDAP, provided the basis for a US$8 billion 20-year investment program in 2008, to construct irrigation and cascade storage dams, ecosystem conservation and natural resource protection, which made support to the institutional strengthening and capacity building of NBA key to the growth and development of the Basin area. By strengthening cooperative management of water resources infrastructure around the Basin, leading to an improved understanding of environmental flows essential to sustain wetlands and fisheries, the project would help transform the livelihoods of vulnerable and poor communities in rural parts of the Basin, and serve as a catalyst for boosting shared prosperity. The broader benefits from enhanced cooperation would help catalyze benefits beyond the river, including development of agribusiness growth poles, enhanced regional trade (e.g. West Africa Power Pool), prosperity and security in the Basin. Alignment with Strategy: The project’s objectives were substantially aligned with Objective 4.2 of the Africa Regional Integration and Cooperation Assistance Strategy (FY18-23), which aimed to “support sustainable management and financing of transboundary water, coastal and marine resources”. A key expected outcome for this objective (p.58) was improved coordination systems for identified river basins. The project was closely coordinated with the World Bank’s wider engagement and projects in the Niger Basin, including the Water Resources Development and Sustainable Ecosystems Management program adaptable loan (APL) [WRD- SEM], whose second phase was in part dedicated to the Kandadji program, including financing of the hydropower plant and irrigation. Finally, the project’s objectives were consistent with the CIWA (Cooperation in International Waters in Africa) objective (Annual Report, FY19, Annex C) of strengthening cooperative water resources management and development to enable sustainable climate-resilient economic growth in the region. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) “To strengthen the institutional framework for regional cooperation in water resources in the Niger River Basin” Rationale Theory of Change: The ICR presents (p.7) a diagrammatic visualization of the theory of change (TOC), depicting the causal links between the various activities supported by the project, the intermediate results and ultimate outcomes. The activities included: (a) implementation of the Water Charter, and strengthening of NBA’s institutional and organizational systems, (b) analysis/operationalization of selected financing mechanisms for the NBA, (c) donor and investor roundtables, (d) engagement/decision-making processes and institutional frameworks around Fomi, and (e) studies of environmental and social risk. These would lead to intermediate outcomes such as: Clarification of coordination mandates for regulating infrastructure, Development of mechanisms for financial sustainability, Mobilization of resources by NBA from a range of sources, and Creation of a roadmap for decision-making among Member-States, which would lead to ESAs completed and discussed by Member- States. These intermediate outcomes could be expected in turn to lead to the key outcomes of institutional enhancements for coordination and management of shared waters between Member-States, a complementary financing mechanism endorsed by Member States and a joint decision-making process for Fomi, approved by the Inter-Ministerial Committee, and implemented. Together, these would help achieve the project development objective of strengthening the institutional framework for cooperation in water resources in the Niger River Basin. While the activities were appropriate to achieving the desired outcomes, it should be noted that the theory of change (TOC) discussion in the ICR does not specifically analyze whether they were of adequate scale to create a critical mass for change. Also, the TOC’s visualization of the long-term (higher-level) outcomes resulting from achievement of the developmental objectives does not adequately clarify the extent to which a properly functioning institutional framework for regional cooperation, supporting an efficient utilization of resources, could have a transformative impact on the basin region. However, the results framework reflected the results chain clarification of coordination mandates for regulating infrastructure and was appropriate to measure the outcomes being sought. After restructuring, the TOC was slightly revised to take account of a slight change in focus from the specific case of Fomi towards support for large, transboundary infrastructure in general within NBA’s mandate. As a result, activities relating to the institutional management arrangements for the Fomi dam, the donor roundtable and environmental and social studies were removed from the project and replaced with more generic methods for evidence-based decision-making for large transboundary infrastructure in the Niger Basin. Progress towards achievement of PDO targets was to be achieved by strengthening the primary institution responsible for facilitating regional cooperation in water resources in the Niger River Basin (NBA). Outputs: The project achieved the following outputs: 1. The clarification of coordination mandates for regulating infrastructure was partially achieved. (The indicator measured revision of Annex 2 and clarification of FOMI institutional framework, on a Yes/No basis).: - Annex 2 to the Water Charter on coordinated management of water infrastructure and its implementation modalities was endorsed by the NBA Council of Ministers, as per the target, making it legally binding. This Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) could be considered a groundbreaking achievement, as this was one of the few RBOs around the world to have a legally binding mechanism on coordinated management of dams. - The indicator that the Fomi institutional framework would be clarified was not achieved per se. However, the evidence-based decision-making process financed by the project did lead to a consensual decision in favor of an alternative solution. - Strengthening of NBA institutional and organizational systems was partially achieved. Priority measures were identified and some were implemented. 2. A new mechanism for financial sustainability was developed and discussed. - A business case for a hydroelectric levy, via a feasibility study and an action plan, was approved by the Technical Committee. (target, that it would be endorsed by member states). However, the levy itself was not approved by member states, on account of objections from Nigeria and Guinea. Revisions to the proposed financing mechanism are currently on-going. 3. Strategy and action plan for the decision-making process on the Fomi project was completed in November 2017, leading to the unanimous decision by key stakeholders to abandon the original site (target, that the decision-making process would be approved by the Fomi Inter-Ministerial Committee). 4. Design and discussion among relevant members of a roadmap for decision-making around Fomi, which was the indicator target, was not achieved. In part, this was because the sound decision-making process financed by the project led to a consensual choice of an alternative solution. Members decided to abandon the Fomi site due to undue estimated social and economic risks arising. The project was put on hold until new studies could be completed, which meant that actions related to the indicators lost relevance. 5. Complementary assessments of environmental and social impacts, which were to be completed and discussed by member states, as per the target, were partially achieved. Some of the studies were dropped, as being no longer relevant. 6. The holding of an Investment Forum for the Fomi project did not happen, as per target, as it became no longer relevant and the activity was dropped. Outcomes: The project partially achieved its objective of strengthening the institutional framework for regional cooperation in the Niger River Basin. There were three outcome-level indicators: (i) Institutional enhancements for coordination of development and management of shared water resources endorsed by member states (Target: that Annex 2 to the Water Charter would be endorsed by the NBA Council of Ministers): This was partially achieved. The key achievement was the development of a legally- binding tool on how to regulate the coordinated management of dams in the basin. Annex 2 was validated by the Technical Committee of the NBA and endorsed by Member States. A strategic institutional audit, leading to the identification of priority measures, endorsed by Member States, on how to strengthen the NBAs capacity was also financed by the project. These measures were now being implemented. Activities related to the setting up of an institutional framework for the FMP however lost relevance as a result of the evidence- Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) based decision-making process, supported by the project, as a result of which the original site was discarded in favor of an alternative solution. (ii) The target of a complementary financing mechanism endorsed by Member States was not achieved. Although both the Technical Committee and decision-makers did endorse the feasibility study (supported by the project) on the complementary financing mechanism based on a hydropower levy and proposed operational and financial frameworks, they did not endorse a legal component, as Guinea and Nigeria rejected the hydropower levy. The Committee did agree on an approach to continue the process, and the financing mechanism was being revised, but it remains unclear whether it will prove politically acceptable and be adopted in practice. (iii) Joint decision-making process for multipurpose project approved by the Fomi Inter-Ministerial Committee (FIC) and followed (Target was that this would be approved and followed): This was achieved. A methodology for joint decision-making around the Fomi dam was tested and validated by the FIC. A major outcome of this was the decision to halt the project on account of studies which showed major problems associated with the original site, plus knowledge gaps for a possible alternative site. Based on this, efficacy, pre-restructuring, is rated Modest. Rating Modest OBJECTIVE 1 REVISION 1 Revised Objective “To strengthen the institutional framework for regional cooperation in water resources in the Niger River Basin” Revised Rationale Though the PDO remained unchanged, the restructuring introduced changes in the results framework to take into account changes in scope of the project. Though the restructured project continued to support an operationally sustainable financing mechanism, it no longer planned to provide assistance to set it up, as this was deemed unrealistic within the project’s timeframe, and could be supported under future engagements. Endorsement of the financing mechanism was scaled down from ministerial to technical level, for the same reason. A (new) strategic institutional analysis of the NBA was included at the latter’s request. The scope of Component 2 was broadened from the specific case of the Fomi multipurpose project towards support for large transboundary infrastructure in general within the NBA’s mandate. Activities related to the review of institutional management arrangements for the Fomi dam, financing of the donor round table, and additional environmental studies were all removed. The restructured project focused on assisting NBA to develop methods for evidence-based decision-making relating to more generic transboundary infrastructure projects in the Niger Basin. The Theory of Change was modified accordingly in the ICR (p.15), to reflect the elimination of these activities, but otherwise retained its inherent logic. Outputs: Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) - Revised and finalized text of Annex 2 to the Charter was achieved, based on national and regional consultations and an operational hydrological too.l It was endorsed by the Council of Ministers in November 2019 (target achieved). - Priority actions to improve the organizational performance of the NBA to be approved by the Summit of Heads of State and implemented, were partially achieved. The council of Ministers had endorsed for implementation six important measures contained in a strategic institutional audit, supported by the project, on how to strengthen the capacity of the NBA. Two of these measures were implemented and a third (revision of the NBA’s organizational chart) was delayed but implemented subsequently, close to project closing. - A business case for the hydroelectric levy, including a technical justification, was prepared (as mentioned earlier). Feasibility study and roadmap were endorsed by an NBA Regional Workshop in October 2019, and subsequently by a technical committee of experts. - The decision-making methodology financed by the project was successfully tested via updated ESIA and feasibility studies of the Fomi dam, in December 2017 (target achieved). - Modelling report from the operation of the Niger Inner-Delta model was generated and published (target achieved) in December 2019, and staff were trained. Outcomes: The project partially achieved its objective of strengthening the institutional framework for regional cooperation in the Niger River Basin. There were three outcome-level indicators: (i) The target of institutional enhancements for coordination of development and management of shared water resources being endorsed by member states was achieved. The indicator remained unchanged from the pre-restructuring period, though some of the intermediate indicators were changed to adjust for abandonment of Fomi dam activities, while broadening the scope to more generic transboundary infrastructure. In addition to the priority actions that were endorsed and adopted, the revision of NBA’s organization chart, which has been delayed on account of the COVID crisis, is likely to be implemented once the crisis is over. It is worth bearing in mind that the forthcoming World Bank Kandadji project, focuses inter alia on strengthening of NBA, particularly through these priority actions – which makes it more than likely that they will all be implemented. (ii) The target of a sustainable financing mechanism, designed and endorsed by a Technical Committee (TC) and/or Regional NBA workshop was not achieved. Member states and the TC asked for revision of the legal framework of the financing mechanism based on a hydropower levy. (iii) The target of evidence-based decision-making process for transboundary infrastructure projects, endorsed by the TC and presented to decision-makers was achieved. The methodological note for such decision-making was validated by a TC on November 5 and 7, in Niamey. The NBA had already organized a regional workshop to validate the note, but had to postpone it on account of the COVID crisis. This can be expected to be held as soon as the crisis is over. Since the indicator did not require political endorsement and the methodology in question is not controversial, there is little risk that it will not be discussed and Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) endorsed when the workshop takes place. Overall, as a result of the operation, the institutional and technical capacity of the NBA improved substantially relative to pre-project levels. The NBA is now equipped with a legally-binding agreement, signed by Member States, on how to regulate the coordinated management of transboundary dams. It has tested a framework for evidence-based decision-making around this infrastructure. It has a technical tool to model the impact of operational decisions on dams on river flows, ecosystems, and livelihoods. All this would enhance NBA’s capacity to carry out its mandate vis-à-vis Member States. Due to the project’s support, NBA was also instrumental in influencing the choice of the site for the Fomi dam, relying on technical data and analysis. The endorsement of the financing mechanism at the highest political level is however the one thing that has yet to happen. Revised Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The project modestly achieved its objectives prior to restructuring, and substantially achieved them after restructuring. Disbursements were 27 percent pre-restructuring and 73 percent post-restructuring. Overall Efficacy Rating Substantial 5. Efficiency Economic and Financial Efficiency Measuring the economic impact of a technical assistance project is usually difficult. In the PAD, an analysis of the economic impact was attempted in terms of the interventions around the Fomi dam. Economic benefits were estimated on the assumption that this TA operation would expedite the implementation of the Fomi dam in scenarios of 2, 5, 10 and 15 years respectively, in comparison to the ‘business as usual’ scenario. Estimated net present value (NPV) of benefits varied from US$472.3 million in the ‘no delay scenario’ to US$114.4 million in the event that implementation was delayed by as much as 15 years. However, the fact that the original site of the Fomi dam was abandoned during the course of the project meant that a comparison with estimated benefits at project closing could not be made, severely limiting the usefulness of economic analysis to measure the efficiency of the project. Administrative and Operational Efficiency Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) In terms of administrative efficiency, while the project did complete all of its planned activities, and disbursed all of its funds (significantly reduced at restructuring), significant implementation delays did take place, including a near 18-month pause in project implementation. This arose on account of governance issues within the NBA, when the NBA Executive Secretary unilaterally dismissed members of the Project Management and Coordination unit (PCMU) and appointed its own members, without a formal No Objection from the World Bank – leading to a de facto suspension of the project. Other delays that took place included long internal administrative procedures for approval of the procurement process relating to key contracts. As a result of the delays, most major activities that were completed were completed only during the last three months before the closing date. Based on the above, the project’s efficiency is rated Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The project’s objectives were Substantially relevant to the World Bank Group’s Africa Regional Integration and Cooperation Assistance Strategy (FY18-23). Efficiency was rated Modest. Overall efficacy was Modest prior to restructuring and Substantial after restructuring, with disbursement being 27 percent pre-restructuring and 73 percent, post-restructuring. Rating the achievement of outcome as 3 (on a six-point scale) pre-restructuring, and 4 in the subsequent period, overall efficacy works out to (3 x 0.27 + 4 x 0.73 = 3.73, rounded out to 4, or Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) On account of the institutional enhancements put in place for coordination and management of shared waters, endorsed by member states, the sustainable financing mechanism endorsed by the Technical Committee, and the evidence-based decision-making process for transboundary infrastructure projects that were endorsed by the Technical Committee, the sustainability of the project’s development outcomes should be relatively assured. At a technical level, the adoption of Annex 2 of the Water Charter by NBA’s Council of Ministers is a key achievement in regional cooperation and coordination for the management of water resources in the Niger River Basin. It is a legally binding document to which all member states have committed to comply with – certain provisions of the Annex even taking precedence over national legislation. With the help of a computerized tool and a simulation model, this legal instrument allows for the management of current and future dams on the basis of interests that go beyond national boundaries. The NBA, aided by its “permanent technical committee” (in which all member states are represented) is well placed to be able to discharge its function. The risk that Annex 2 will not be implemented is considered low, though this something that is clearly difficult to predict. The risk that the NBA will not go through with a planned reorganization based on the recommendations of the audit is moderate. At a financial sustainability level, however, though the Technical Committee endorsed a sustainable financing mechanism, the NBA still needs to articulate a concrete and persuasive offer of services, as consistently recommended by development partners in recent years. As regards the near future, the risk that the proposed financing mechanism will not be endorsed by member states is high. At a political level, the risk of the NBA losing support and mandate for the evidence-based decision-making process for transboundary infrastructure projects designed and endorsed by the Technical Committee will depend upon whether any shared regional projects that can benefit from the process were not to be developed and presented. In the current context of water mobilization in the Basin (e.g. the Kandadji, Taoussa and Fomi dams), it is expected that the NBA will be able to continue providing its coordinating role for transboundary and other initiatives. However, the NBA will have to demonstrate its ability to implement the process in conjunction with project financing, with the support of member states. The risk that some countries might carry out some investments on their own or on a bilateral basis cannot be ruled out, as a result of which the risk to development outcome here remains Substantial. 8. Assessment of Bank Performance a. Quality-at-Entry The design and preparation of the project was informed by over a decade of engagement by the Bank and other donors in the Niger River Basin to strengthen the NBA, promote cooperation and plan the development of new transboundary infrastructure. The design of the project was further informed by the World Bank’s experience in transboundary river issues in other African river basins, including the Nile, Zambesi and Senegal River Basins. This experience determined the priority, for instance, of ensuring that the NBA’s funding needed to become independent of national budgets of member countries, and for developing and implementing a road-map for decision-making on Fomi. During appraisal, as mentioned in the ICR (p.32), the team assessed the Governance risk as Moderate. In hindsight, this probably underestimated the true risk, arising from weak governance at Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) regional and country-level institutions, and the team should have been less optimistic on the transparency and accountability of NBA in the early years. The fiduciary risk was however appropriately assessed as Substantial, on account of weak internal controls, and mitigation measures were planned to reduce this risk, including through the recruitment of an experienced accountant, and a qualified procurement specialist. The choice of lending instrument was sound. Other instruments were being explored at the time (ICR, p.33), including additional financing through the existing regional IDA WRM-SEM (Water Resources Development and Sustainable Ecosystems Management) Adaptable Lending Program (APL). The APL was however due to close in 2014, and extension was only possible for the national Guinean part of the loan, which was not felt to be the best vehicle to support the inherently regional activities that would be facilitated by NBA. Also, the scope of the second phase of the APL was relatively narrow, focusing on the implementation and supervision of the Kandadji dam in Niger, and activities relating to the Fomi dam would have been beyond its scope. Implementation arrangements were appropriately kept simple, with NBA being the grant recipient and the project using the existing project management structure already in place and available for implementation of the Niger Basin WRD-SEM Program. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision According to the ICR (p.33), the Bank team provided strong supervision throughout project implementation, providing regular guidance when needed and helping design appropriate solutions. The three and-a-half years since effectiveness in May 2016 and project closing in December 2019 saw 10 supervision missions being undertaken. Because of slow progress and governance issues during the initial 2 years, the Bank intensified monitoring and implementation support to make sure the project would achieve its targets by closing. Regular, bi-weekly discussions were held with the PMCU to monitor progress in procurement, and achievement of targets, with fairly intense support being provided by the task team during the final year of implementation. The ICR reports that agreements entered into during supervision missions were clearly recorded in Aide Memoires signed by the Executive Secretary NBA and the Bank team. Bank supervision teams were appropriately staffed with multidisciplinary specialists, supplemented by international experts hired with Trust Funds. The team also provided support to the PMCU on fiduciary issues. Task team leaders, as well as the PMCU, were provided with procurement training, and the team designed a practical tool for the NBA to report on procurement and disbursements. Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) The team also undertook a timely restructuring that changed some activities and clarified some of the indicators. According to the ICR, the team remained still too ambitious with some indicators (such as implementation of priority actions identified by the strategic institutional audit), and should have defined more clearly the level of endorsement of certain deliverables. Though no doubt driven by the need to have PDO indicators that could be realistically achieved by close of project, the revisions meant that in the RF the new wording merely differentiated between an output and an outcome in terms of endorsement. On the other hand, the team could usefully have revised legal covenants on the installation of a computerized accounting system and hiring of an accountant assistant, which were overdue and deemed no longer valid. The project had a high turnover of task team leaders (TTLs), with three TTLs managing the project in the course of 3.5 years. However, this did not have a negative impact on project implementation, as the second TTL had been on the team during preparation and the third had been managing another project involving NBA, and was therefore familiar with the project team. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The design of the M&E system and the results framework was adequate to monitor and evaluate the performance of the project’s activities. As indicated in the Theory of Change discussion in Section 4, the project’s objectives and results chain were fairly clear, and indicators not difficult to measure, although some indicators may have been somewhat ambitious, given the Bank’s ongoing relationship with NBA. Intermediate in indicators were generally well distributed along the project’s implementation timeline, with achievements expected from Year 2. b. M&E Implementation A full-time M&E specialist worked closely with the PMCU technical team, which prepared and submitted semi-annual progress reports to the Bank and the NBA Executive Secretariat. Between January 2017 and June 2019, M&E was rated Moderately Satisfactory. The rating was raised in June 2019, when the PCMU came up with a schedule to achieve all indicators, and identified the steps that needed to be taken to achieve them, along with associated dates. The ICR states (p.30) that despite these steps, not all Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) indicators in the results framework were measured and reported in the ISRs. However, a perusal of ISRs does not indicate that that was actually the case. c. M&E Utilization Data collected for M&E informed the restructuring of the project in 2018. Subsequently, the M&E expert and the technical teams developed a plan to ensure achievement of the targets by project closing, identifying necessary steps, roles and responsibilities. The delay in project implementation on account of the suspension of all activities for more than a year meant that the revised RF was used more intensively in the last two years of implementation (2018-19). M&E data findings are informing another operation (the Kandadji project), which will continue to support the NBA until 2026. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as Environmental Category B. As this was a technical assistance project, there were no planned works or physical footprints associated with activities under this project. The project was not planning on financing any of the core preparatory studies in relation to the Fomi Multipurpose Project (FMP). An Environmental Assessment (OP/BP 4.01) was triggered by the project to cover upstream analytical work, to enable the counterpart make an informed decision regarding the technical design of the FMP with adequate knowledge of environmental and social risks and opportunities. No other safeguards policies were triggered. The ICR does not clarify whether the policy relating to the environmental assessment was complied with or not. However, it does mention (ICR, p.31) that environmental and social risks were classified as low to moderate in all Implementation Status and Results Reports (ISRs) and compliance was rated moderate to satisfactory. b. Fiduciary Compliance Financial Management (FM): The project’s financial management was rated Moderately Satisfactory in ISRs, at project closing, despite the fact that two legal covenants remained overdue (recruitment of an assistant accountant and installation of a computerized accounting system). Earlier, during implementation, the rating had been lowered to Moderately Unsatisfactory in 2017, on account of the absence of an accountant acceptable to the Bank in the PCMU. Once the accountant was hired, NBA complied with the timely submission of the Intermediate Financial Reports and annual financial audit reports. The ICR does not specify whether external auditors’ opinions were qualified or not. However, the Project team clarified, separately (September 17, 2020), that they were indeed unqualified. The last financial audit (period January 1 to December 31, 2019) was expected to be submitted by the PCMU by the end of June, 2020. The Project team clarified (as above) that on account of the COVID situation, the due date for Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) submission was moved to December 31, 2020, but the report was expected to be received well before that date. Procurement: The ICR reports (p. 32) that the rating for procurement management had been lowered to Moderately Unsatisfactory in December 2016, because no progress had been made at the time in procurement of the two main contracts, despite appointment of a procurement specialist. It was raised to Moderately Satisfactory in December 2017, and remained so until project closing. c. Unintended impacts (Positive or Negative) According to the ICR (p.26), one unintended impact of the Bank’s 18-month suspension of project implementation (see Section 5), was that the Bank was not at the table when important matters were being discussed, such as the temporary withdrawal of Guinea from the NBA to carry out Fomi studies unilaterally – which threatened the legitimacy and raison d’être of the NBA itself. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory M&E Design, Implementation Quality of M&E Modest Substantial and Utilization were adequate to meet the needs of the project. Quality of ICR --- Substantial 12. Lessons IEG derives the following lessons drawn from the ICR: 1. An autonomous financial mechanism for a River Basin Organization (RBO) has a better chance of being approved if all Member States perceive tangible benefits: The case of the hydropower levy and the concerns expressed by some states (Guinea and Nigeria) about the potential for negative impacts on the business environment and the cost of electricity illustrates that the likelihood of endorsement by member states is dependent upon their perceived assessment of relative costs and benefits. The likelihood of member states providing regular financing contributions is higher if they perceive that being part of an RBO provides them with tangible benefits, such as being able to mobilize financing for investments; the fact that an RBO merely carries out a regulatory function may not be enough, Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) 2. Transboundary programs need to be cognizant and realistic about the political economy of the transboundary space: Such programs face challenges in achieving political buy-in, since several countries need to collaborate. This calls for long-term engagement to achieve results. Project durations also need to be commensurate with the basin structure and complexity of the project – which could mean being open to extensions of closing date. This also calls for a clear and coherent World Bank strategy, aimed at building trust with the client and facilitating project implementation. 3. It is essential for RBOs to have strong organizational and institutional capacity, to be able to carry out their mandate in a timely and effective manner: A strong Project Management and Coordination Unit, with well-trained staff – especially one that is familiar with implementation of World Bank projects - is key for the success of such Bank-funded operations. Regional programs involving RBOs must include a strong component on continuous learning and capacity building, capitalizing on previous training programs. Evidence-based, multicriteria decision-making needs to become embedded in the modus operandi of NBOs, based on the decision-making experience around Fomi. 4. A clear and coherent World Bank Group engagement strategy with an RBO builds trust with clients, which facilitates successful project implementation: Towards this end, it is essential for Bank management to give a clear signal of support, once it is engaged in the project, and to provide a consistent set of priorities to Bank teams working with the same RBO, and to ensure prompt and transparent communication in case changes are necessary. The engagement of Bank management is essential also to ensure that governance challenges are properly addressed. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is clearly written, concise and consistent with guidelines. It provides adequate details of the project’s activities and results. and is internally consistent. Its analysis is broadly evidence-based, and the ICR presents a number of annexes with details relevant to its conclusions. The ICR has identified a number of appropriate lessons and recommendations arising from the operation. There were one or two shortcoming, including that the ICR did not provide clarification on whether safeguards policies were complied with, as well as on whether all financial audits received unqualified opinions or not (this information was subsequently derived from the team). a. Quality of ICR Rating Substantial Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Niger River Basin Management Project (P149714) Page 17 of 17