ICRR 13163 Report Number : ICRR13163 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 11/09/2009 PROJ ID : P070252 Appraisal Actual Project Name : Reversal Of Land And Project Costs (US$M): US$M ): 19.6 11.0 Water Degradation Trends In The Lake Chad Basin Ecosystem Country : Africa Loan /Credit (US$M Loan/ ): US$M): 10.3 10.1 Sector Board : WAT US$M ): Cofinancing (US$M): 7.2 0.5 Sector (s): Central government administration (50%) Irrigation and drainage (25%) General water sanitation and flood protection sector (25%) Theme (s): Land administration and management (50% - P) Water resource management (50% - P) L/C Number : Board Approval Date : 01/21/2003 Partners involved : UNDP, DFID (UK), Closing Date : 01/21/2008 12/20/2008 DGIS (Netherlands), World Wildlife Fund, UNOPS Evaluator : Panel Reviewer : Group Manager : Group : John R. Heath George T. K. Pitman Soniya Carvalho IEGSG 2. Project Objectives and Components: a. Objectives: The Grant Agreement (GA) provides a more detailed and systematic statement of the Global Environment Objective than the Project Appraisal Document . This review rates the project against the statement in the Trust Fund Grant Agreement (p. 15). "The objectives of the project are : (a) To strengthen the capacity of the Recipient to promote and improve sustainable water management in the [Lake Chad] Basin; (b) To strengthen institutional mechanisms in Lake Chad Basin Commission Countries for management of transboundary water issues; (c) To develop a Strategic Action Plan to improve the conservation and management of land and water resources in the Basin; and (d) to assist the Recipient in coordinating donor support of the Strategic Action Program and transboundary management" (Global Environment Facility, Trust Fund Grant Agreement, p . 15). Note: The countries covered by this project are Cameroon, Central African Republic, Chad, Niger and Nigeria . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): (1) Establishment and operation of Project Management Unit and coordinating mechanisms (Expected cost at US$ 2.3 million; Actual cost, US$ 2.7 million, of which UNDP US$ 2.7 million ) appraisal, US$2 Aimed at establishing: the Project Management Unit (PMU) for the Lake Chad Basin Commission (LCBC); national lead agencies; and a Technical Advisory Committee and Project Steering Committee (PSC) to drive and coordinate transboundary diagnostic analysis, the Strategic Action Program (SAP), pilot projects, policy initiatives and institutional linkages. (2) Regional policy initiatives and institutional mechanisms to address trans -boundary issues (Expected cost at appraisal, US$ 1.1 million; Actual cost, US$ 1.3 million, of which World Bank US$ 1.3 million ) Aimed to strengthen the institutional and environmental management capacity at regional and national /local levels through: (i) review of current functions and responsibilities of LCBC, creating awareness of national policy makers on transboundary issues and building regional capacity; (ii) integration of transboundary water and environmental management policies into national action plans (NAP); (iii) review of legal, institutional, financial and socio -economic frameworks in member countries; and (iv) the harmonization of legal frameworks, regulations and approaches for transboundary management of shared resources . (3) Community level participation and education US$ 1.1 million, of which UNDP US$ 1.1 million ) (Expected cost at appraisal, US$ 1.2 million; Actual cost, US$1 Aimed at strengthening community level participation through : (i) supporting local planning initiatives to establish their sustainable development plans in line with sub -basin planning frameworks; (ii) workshops for the prioritization of local, national and regional initiatives to arrive at a coherent and integrated approach; and (iii) development of a regionally based methodology and mechanism for multi -level stakeholder participation, with provisions for environmental impact studies. (4) Transboundary diagnostic analysis and modeling framework for basin management (Expected cost at appraisal, US$ 2.1 million; Actual cost, US$ 1.9 million, of which UNDP US$ 1.9 million ) Aimed to examine basin-wide conditions and issues and to determine the linkages between environmental and socio-economic systems and their transboundary impacts . These would serve as an input to the Strategic Action Program (SAP) design (Component 6) and implementation. This component was also designed to collect data, improve data collection systems and capacity, develop water resources models and develop risk analysis capacity . (5) Demonstration pilot projects (Expected cost at appraisal, US$ 2.5 million; Actual cost, US$ 2.1 million, of which World Bank US$ 1.3 million and UNDP US$ 0.8 million ) Aimed to test and validate methodologies, secure stakeholder involvement and develop implementation modalities for national and local SAP and NAP interventions . Pilot projects implemented through the World Bank were (i) the Lake Fitri pilot in Chad, (ii) the Komadugu-Yobe (KY) pilot in Nigeria, and (iii) the Waza-Logone (WL) flood plains pilot in Cameroon. Pilot projects funded through the UNDP included : (iv) the Lake Chad Shorelines and Northern Diagnostic Basin pilot, and (v) the Upper Chari Basin land use systems pilot in the Central African Republic . (6 ) Strategic Action Program development and donor coordination (Expected cost at appraisal, US$ 0.4 million; Actual cost, US$ 0.2 million, of which World Bank US$ 0.2 million ) Aimed to develop a prioritized Strategic Action Program (SAP) for basin management (with an Investment Plan) and to mobilize donors support. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Components 2 and 6 were funded by the Bank; components 1,3, and 4 were funded by UNDP; component 5 was jointly funded by Bank and UNDP. DFID and DGIS were expected to contribute US$ 6.7 million but it is not clear from the ICR (p. 23) if this funding did not materialize or if the final amount was unknown . At project effectiveness (January 21, 2004) the closing date was extended till January 20, 2008, following which it was extended twice, ultimately to December 20, 2008. In 2006 there was : (i) a reallocation of funds; (ii) the merging of funds for goods and works with the funds for “micro grants� to allow full community participation in the implementation of local initiatives under the pilot projects; (iii) an adjustment of the financing parameters for all expenditures to 100 percent; and (iv) an amendment of the Procurement Schedule of the Grant Agreement (ICR, pp. 4-5). 3. Relevance of Objectives & Design: High ) Relevance of Objectives (Rating : High) The project objective remains consistent with the priorities of Poverty Reduction Strategy Papers of the member countries, including the most recent strategies prepared by Niger (2008) and Nigeria (2004). These papers give a central place to the sustainable management of water and other natural resources, the protection of the environment and reduction of environmental degradation . The project also responded to objectives declared by NEPAD (Agriculture and Environment programs ) and the MDGs, as well as to LCBC’s Vision 2025 (2003). Moreover, the project contributed to the CAS objectives of LCBC member countries through its support of community-based natural resource management and the reversal of land and water degradation trends in the Basin . Furthermore, the project development objective is fully consistent with the updated strategic priorities of the Global Environment Facility, specifically: “Balancing overuse and conflicting uses of water resources in surface and groundwater basins that are of transboundary nature.� Relevance of Design (Rating : Modest ) The Quality Assurance Group identified five flaws in the design of the project : "(i) the project’s scope, design and Global Environment Objective were too ambitious given the implementation capabilities of the Recipient; (ii) the institutional context was not properly appraised at entry, (iii) the project design did not sufficiently address the development of SAP implementation capacity in LCBC; and (v) the safeguard framework overly focused on a set of micro-level pilot sites, while failing to look at the region's problems more strategically " (ICR, p. 5). Design was overoptimistic in assuming that members states would be willing and able to support LCBC financially and politically . Also, it was a mistake to rely on an external implementing agency (UNOPS) to compensate for the weakness of LCBC; the challenge was rather to first build the capacity of LCBC . The ambitious objectives reflect GEF's mandate to "push the envelope" on transboundary issues . This consideration was weighed against the manifest design limitations . Thus, overall, relevance is rated substantial . 4. Achievement of Objectives (Efficacy): (a) To strengthen the capacity of the Recipient to promote and improve sustainable water management in the Basin (Rating : Modest ). A first step toward this objective was completing, at national and regional levels, a comprehensive Transboundary Diagnostic Analysis . This analysis identified three problems : (i) the absence of sustainable development from the political agendas of the riparian countries; (ii) low standards of environmental education and awareness; and (iii) population pressure. The analysis made it clear that reform of LCBC was the only way to address these problems . LCBC lacks the power to arbitrate water conflicts in the basin . Although the analysis was an indispensable first step there were no significant follow up initiatives to strengthen capacity at the transnational level. Some important local initiatives were sponsored : 56 micro-grants were delivered to local communities to demonstrate sustainable ways of reversing land and water degradation . However, the grants were disbursed with delays. The QAG supervision assessment noted that the focus on micro -projects was at the expense of strategic planning. (b) To strengthen institutional mechanisms in Lake Chad Basin Commission Countries for management of transboundary water issues (Rating : Modest ). The Institutional Assessment of LCBC was only completed and endorsed by the Council of Ministers in June 2008 so it was not possible to carry out the reform of LCBC in the span of project implementation. In particular, the fiduciary capacity of the LCBC was not improved and the agency's accountability to member states and other contributors was not enhanced . Some progress was made towards harmonizing the legal frameworks of the participating countries . Each country has developed a list of objectives, targets and proposed interventions and has prepared a strategy for mobilizing resources . SAP ) to improve the conservation and management of land and water (c) To develop a Strategic Action Plan (SAP) resources in the Basin (Rating : Modest ). The SAP was developed through a process of regional consultation and was endorsed by the Council of Ministers of the riparian countries in June 2008. The ICR describes this as a significant achievement: "few Basins in Africa have been able to complete a SAP in five years ". However, the project did not prepare the Action Plan for SAP implementation, failing either to specify costs or to develop a financing mechanism. (d) To assist the Recipient in coordinating donor support of the Strategic Action Program SAP ) and (SAP) transboundary management (Rating : Negligible ). Owing to the late endorsement of the SAP and the failure to develop an investment plan, the Chairman of the Council of Ministers postponed the donors' conference . Therefore, a coordinated program of donor support was not worked out during project implementation . Overall, efficacy is rated modest . 5. Efficiency (not applicable to DPLs): At appraisal, there was a compelling case that this project would make efficient use of resources . GEF funding covered the transaction and other costs of cooperation between the various countries in the Lake Chad basin . Given the transboundary nature of the issues and the public good aspect of the environmental benefits, it is unlikely that the riparian countries would or could have financed these activities . The estimated costs were genuinely incremental; but within this incremental cost envelope the project only partially achieved its Global Environment Objective . The project succeeded in leveraging additional funding (e.g. from the African Development Bank and from Nigeria ) but several factors reduced the efficiency of resource use : "prevailing security conditions in the region, time consuming land based transport, limited services of airlines, non -availability of up-to-date logistical and technical facilities in N’Djamena, and high costs of workshops " (ICR, p. 15). Added to these failings there were large implementation delays, including lags in procurement and recruiting consultants . Although the project was extended by two years the (ambitious) output targets were not delivered . Efficiency is rated modest . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re -estimated value at evaluation : re- Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The rating of substantial relevance, modest efficacy and modest efficiency adds up to an outcome rating of moderately unsatisfactory. a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: The project was only marginally successful in creating the framework needed to ensure the long -term, environmentally sustainable development of the basin ’s transboundary land and water resources . The biggest risk derives from the failure to carry out the Institutional Assessment of LCBC and to make the necessary reforms . The weak financial position of LCBC, the delayed recruitment of technical staff, the lack of staff performance incentives, and the poor management record bode ill for sustainability of the project's limited positive results . The ICR states that: "It is uncertain when the Investment Plan of the Strategic Action Plan (SAP) will be completed, when the envisaged donor conference will be held, and to what extent donors will be willing to support the SAP implementation" (p. 17). On a more positive note, the African Development Bank recently approved a program that will help fund the implementation of SAP and the Council of Ministers has now endorsed implementation of the Institutional Assessment. a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: at -Entry Quality -at- This was an ambitious and pioneering project in a difficult country environment and the objectives were highly worthwhile. But although the objectives were consistent with the various country CASs and with GEF strategy and the O.P. 9 guidelines, the PAD did not clearly describe what the project intended to do . The Quality Assurance Group did not assess the project's quality -at-entry but it did identify various design flaws in its 2004 Quality of Supervision Assessment (see Section 3 above), not least inadequate attention to institutional and management weaknesses. Supervision The project had 4 TTLs and the ICR notes that "supervision was of unequal quality ". However, the difficulties facing the Bank team should not be underestimated : the supervision budget was disproportionately small in relation to the need to cover five large countries in a complex region facing civil unrest (pp. 17-18). The circumstances would have challenged any supervision team . Supervision reports and ratings were realistic . But supervision of fiduciary matters could have been tighter . Also, the ICR comments (p. 9) that supervision missions and the mid-term review "did not pay adequate attention to safeguard issues " (p. 9). The Bank was too sanguine about impact of additional supervision efforts before the mid -term review, sending a signal that the institutional weaknesses had been remedied when this was not in fact the case . at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Moderately Unsatisfactory c. Overall Bank Performance :Moderately Unsatisfactory 9. Assessment of Borrower Performance: Government The governments of the member countries made only a limited commitment to project objectives (although there was some pick up in the final year of implementation ). The Council of Ministers which represented the various countries failed to address the poor performance of the LCBC or to resolve the problem of LCBC arrears . It was also partly responsible for delays with the Institutional Assessment (called for in 2002 but only endorsed by the Council in 2008). There were shortfalls in staffing and counterpart funding attributable to limited government support. But commitment picked up in the final phase of implementation . Implementing Agency LCBC lacked ownership of the project and did not adequately support the Project Management Unit (PMU). Project management was jointly handled by UNOPS and the PMU . The PMU was understaffed and its performance was hampered by repeated changes of Project Manager . In 2004 the Quality Assurance Group expressed concern about the weak performance of UNOPS /PMU in project execution (ICR, p. 5). Procurement was subject to long delays . There were also delays in implementing the Institutional Assessment . But performance improved in the last year of the project and despite difficulties posed by civil strife LCBC pushed ahead with two meetings of the Council of Ministers and a Heads of States Summit . a. Government Performance :Moderately Unsatisfactory b. Implementing Agency Performance :Moderately Unsatisfactory c. Overall Borrower Performance :Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: Design The PAD included a logframe, specifying objectives, outputs, and outcomes . There were lengthy lists of broadly defined output/process indicators. There were no quantified baselines and end -of-project outcome/output targets were not clearly specified. A Results Framework was introduced in 2005 but never formalized. Implementation Based on what is contained in the Bank and UNDP supervision reports, indicators appear to have evolved over time (ICR, p. 8). No indication is given in the ICR about the regularity with which monitoring data were collected . Utilization In 2008, a ‘Protocol of Agreement on Data Exchange between Member States of LCBC ’ was signed, with a view to promoting the exchange of water resources data for the Lake Chad Basin . These data include: water resources (quality and quantity), water use, land use (soil, vegetation and erosion ), climate conditions and meteorological observations, environmental aspects, navigation, and socio -economic conditions. This regional database is based in the Observatory at LCBC. a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Safeguards . The Project was classified as Environmental Category B due to potential impacts from the implementation of the pilot projects. Four safeguards policies were triggered for the pilot projects : OP 4.01 Environmental Assessment, OP 4.04 Natural Habitats, OP 4.12 Involuntary Resettlement, and OP 4.37 Safety of Dams. The safeguard ratings in the Environmental Safeguard Audit reports varied across pilot projects, but were overall moderately satisfactory for the OPs 4.01, 4.04 and 4.12, and unsatisfactory for OP 4.37. "The dam safety policy was triggered because it was felt that some pilot project activities might rely on the performance of existing dams, which constituted a reputational risk " (ICR, p. 8). "No severe negative impacts arose due to the small scale and thorough selection process of individual micro -interventions under each pilot project " (ICR, p. 9). Financial Management . The mid-term review found that the flow of funds through UNDP ’s country offices to pilot project activities (micro-grants) was not satisfactory. Supervision reports rated financial management performance as moderately satisfactory. UNOPS failed to provide the mandatory annual audit reports on time : those for years 2005 – 2007 were only submitted in October 2008. The fiduciary capacity of LCBC remained weak and consultant -provided technical assistance was inadequate . This contributed to the lack of donor confidence in LCBC . Procurement .. All supervision reports rated procurement satisfactory but the mid-term review identified a weakness in the filing of procurement documents and observed undue delays in the procurement process . Other . The ICR notes that "The protection of almost 100 percent of the lakefront of Lake Chad at Ramsar wetlands sites of international importance was a great achievement of this project . This is one of the few examples of fully transboundary lakefront in terms of Ramsar jurisdiction " (p. 16). 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Unsatisfactory Unsatisfactory Risk to Development Significant Significant Outcome : Bank Performance : Moderately Moderately Unsatisfactory Unsatisfactory Borrower Performance : Moderately Moderately Unsatisfactory Unsatisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: The ICR proposes several well-chosen lessons: In complex regional projects, it may be more difficult for the implementation /supervision process to fix design flaws. To increase the likelihood of achieving project objectives, design flaws need to be fixed early in project implementation, at minimum during the mid-term review. The full endorsement and commitment of all the riparian countries of a shared basin is desirable for successful implementation, but not necessary if adequate action plans are incorporated in the project design . The countries' compliance with regional commitments to basin management can be incorporated into the Country Management Unit's regular policy dialogue and care can be taken to ensure special oversight of project supervision. In the absence of a strong regional institution, development assistance should focus on capacity building and institutional strengthening of the Recipient, rather than attempting to resolve or bypass the institution ’s shortcomings by introducing outside agencies for project management and execution . Joint project implementation through two GEF implementation agencies (i.e. World Bank and UNDP) requires that roles and responsibilities during project implementation are agreed upon in advance . Key reporting and supervision processes need to be harmonized . The Bank should be realistic regarding project implementation planning, the time it takes to complete a project of this challenging nature, and the outcome targets to be achieved . 14. Assessment Recommended? Yes No Why? There is a good case for comparing the performance of this project with the recently completed GEF Senegal River Basin Project, which had an almost identical design (and also involved UNDP). A cluster PPAR of the two projects would shed light on the effectiveness of donor partnerships for regional water resources development and the capacity of the Bank and other agencies such as UNDP in tackling transboundary issues . 15. Comments on Quality of ICR: The ICR gives a candid and thorough account of project performance, and several important lessons are drawn . a.Quality of ICR Rating : Satisfactory