ICRR 13255 Report Number : ICRR13255 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 05/21/2010 PROJ ID : P064916 Appraisal Actual Project Name : Natural Disaster US$M ): Project Costs (US$M): 16.05 15.3 Vulnerability Reduction Country : Nicaragua Loan/ US$M): Loan /Credit (US$M): 16.05 15.3 Sector Board : UD Cofinancing (US$M): US$M ): Sector (s): Central government administration (67%) Sub-national government administration (26%) Other social services (7%) Theme (s): Natural disaster management (67% - P) Municipal governance and institution building (33% - S) L/C Number : C3487 Board Approval Date : 04/03/2001 Partners involved : Closing Date : 03/31/2005 02/28/2009 Evaluator : Panel Reviewer : Group Manager : Group : Anna Amato Robert Mark Lacey IEGSE ICR Reviews IEGSE 2. Project Objectives and Components: a. Objectives: The objective as written in the PAD is to improve Nicaragua's disaster management capacity by : (a) strengthening institutional capability in disaster management at the national level; (b) building institutional capacity for disaster mitigation at the national level; (c) promoting disaster awareness and "preventive thinking" through public sector education and awareness programs; (d) building local capacity to manage disaster emergencies, to assess risk and identify mitigation measures; and (e) implementing vulnerability reduction and mitigation measures at the local level . The Development Credit Agreement has small differences in three of the sub -objectives, as follows (differences are in italics): (a) strengthening institutional capacity in disaster prevention, preparedness and response at the national level; (b) building institutional capacity to program disaster mitigation at the national level; (c) promoting disaster awareness in society. The PAD version of the objectives are evaluated in this review since they are more specific . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components (or Key Conditions in the case of DPLs, as appropriate): A. Strengthening of National System Capacity for Institutional Disaster Management (US$4.6 million appraisal estimate; US$4.82 million actual) B. Development of National Mitigation Program and Management (US$1.54 million appraisal estimate; US$1.65 million actual) C. Building Public Awareness of Disaster Prevention (US$0.94 million appraisal estimate; US$0.94 million actual) D. Strengthening Local Capacity for Disaster Risk Management (US$3.5 million appraisal estimate; US$3.69 million actual) E. Implementation of Local Vulnerability Reduction (US$4.07 million appraisal estimate; US$4.16 million actual) F. Project Financial Management (US$0.73 million appraisal estimate; US$0.76 million actual) A contingency amount of US$0.67 was added to the total at appraisal . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: OTE : It is unclear from the ICR whether the final actual amount was US$M 15. NOTE: 15 .3 or US$M 16. 05 . The ICR author 16 .05. did not know and as of this writing, had not responded with the correct amount . The Bank Operations Portal disbursement records show US$ 15. 15 .3. The project was extended 5 times and was completed 4 years late.There were numerous reasons for the delays : Lack of knowledge about how long a project like this should take, since it was the first of its kind . Substantial staff turnover Political change followed by a lack of support for this approach by the new administration . Disruption by Hurricane Felix in August 2007, which forced the agency to turn their focus on responding to this disaster. Site change for new headquarters slated to be built for the new disaster agency . Procurement disagreements between the implementing agency and oversight agenc y. 3. Relevance of Objectives & Design: Objectives : The relevance of the project's objectives is rated substantial . The 1998-2002 Nicaragua CAS focused on private sector development, rural development and poverty alleviation . The regular occurrence of natural disasters has greatly hampered the development of these areas, and has set back advancements that had occurred in the country. In Central America, Nicaragua stands out in both the frequency and severity of natural catastrophes, being prone to hurricanes, severe earthquakes, droughts, fires, volcanic eruptions and tsunamis . Following Hurricane Mitch in 1998 the Nicaraguan Government decided to develop a comprehensive national disaster risk management system. With help from the United Nations Development Program and the Governments of Spain and Colombia, a task force designed a plan which led to the passage of Law 337 of 2000, creating the Sistema Nacional para la Prevención, Mitigación y Atención a los Desastres (hereafter "SINAPRED"). This project's objectives were formulated to develop the institutional capacity to implement this system . Design :The relevance of project design was modest . Design underestimated the initial challenges of dealing with a new agency, and therefore had an over -ambitious time frame and scope. The PAD mentions the desire to strengthen SE (Executive Secretariat -- SE-SINAPRED) while it is still a new institution, while leaving open the possibility of a follow-on operation. But the scope of the project was not designed as such . Instead, SE-SINAPRED was put in charge of implementation at the same time as it was being strengthened, which slowed implementation and created significant delays. Establishing a Project Coordination Unit from the beginning would have been a better mechanism to minimize project risks. The PDO indicators from the PAD were formulated by component and did not have quantitative targets . A second set with some targets were formulated halfway through project implementation, but were much more modest . The project was not formally restructured. 4. Achievement of Objectives (Efficacy): Achievement of the original PDO – to improve Nicaragua’s disaster management capacity – has been substantial , despite delays and some shortcomings . SINAPRED and SE-SINAPRED have been given their own dedicated budget line at the national level since 2008. This is an indication that the disaster management system has gained considerable political support and is now seen as an integral part of the country ’s development priorities, Thanks to this and other internationally supported projects, a number of new initiatives in disaster risk management (DRM) have been undertaken, including creation of an online disaster data base, development of a risk assessment platform for Nicaragua, a regional plan for disaster reduction, and a study to update DRM indicators. The operation has also successfully contributed to improved coordination between national institutions involved in DRM, and the main-streaming of DRM in key ministries, which was absent from public policy before the project. SE-SINAPRED successfully led the response to Hurricane Felix, an effort recognized by the international community. While Felix was not in the same category as the much more severe Hurricane Mitch, the official response showed substantial improvement compared to earlier emergencies . The vulnerability studies allowed for the elaboration of a DRM Plan which is now in place and was used successfully during Felix. A resettlement policy for populations located in vulnerable areas has been prepared and is awaiting approval by the National Assembly. DRM was introduced in educational curricula in 2008, teachers trained and educational material developed for schools. However, there has been no subsequent follow -up. Similarly, SE-SINAPRED is elaborating a communication strategy to promote a culture of disaster prevention, but no media awareness campaign has yet been launched. There has been considerable initial success in building local capacity to manage emergencies, assess risks and identify mitigation methods. 30 municipalities are producing preventive land use plans instead of the 25 targeted at appraisal, 87% of municipalities are now using DRM planning tools as opposed to 43% prior to the project. Sustainability of these achievements is, nonetheless, vulnerable to both human and financial resource constraints. Based on local observation of recurrent events rather than on systematic risk assessments, 15 selected municipalities have fully implemented disaster mitigation measures . On the negative side, there was little involvement of the local community – an important factor in promoting sustainability of the mitigation works – since there were no public consultations . This explains the presence of waste deposits and poor maintenance of the works. 5. Efficiency (not applicable to DPLs): No information is provided in the ICR with which to assess the project's efficiency, since no ex -ante or ex-post economic analysis was carried out . Nor does the ICR contain any cost effectiveness or other comparative evaluation which would enable a judgment of the project's relative efficiency . In the absence of evidence on efficiency, it is rated modest. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Outcome is rated as "moderately satisfactory � reflecting the high relevance of the project's objectives and its substantial design relevance and efficacy, but modest efficiency . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The inclusion of a line for SE-SINAPRED in the national budget shows that the system has gained considerable political credibility, and that SE-SINAPRED is seen as a regular ministry. Another positive manifestation of sustainability is the evidence that coordination between institutions has improved substantially, with new initiatives in DRM stemming from the pilot. Hurricane Felix demonstrated this improved coordination between institutions, regional governments, international organizations and civil society . Another example is how SE-SINAPRED led the coordination during the Managua floods and floods resulting from Tropical Storm Alma, in May 2008. However, the political context in Nicaragua is a major factor influencing the sustainability of this project and has manifested itself in two ways. First there has been substantial staff turnover : From one administration to another, entire institutions were drained of their staff . Second, there is discontinuity in policy priorities, depending on the new administration’s preferences.While disaster risk management has clear benefits and should be a continuous effort, sustainability depends on the commitment in the country to continue DRM main -streaming and promotion of prevention activities. SE-SINAPRED has remained active through the last three administrations with varying levels of support. But the agency has coordinated and executed projects and activities with other sources of financing that consolidate the results yielded by this project .These on-going activities show the agency ’s and the government’s commitment to maintain DRM efforts in the country . a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: Quality at Entry: The Bank correctly identified the country ’s needs and the operation is highly relevant to Nicaragua. However, the overestimation of the capacity and leadership of SE -SINAPRED was at the root of several implementation problems . Also, the creation of a national system for disaster emergency response and prevention was a difficult task to achieve in the original three year time frame . The Bank underestimated the challenges of supporting a nascent institution, and tasking it with the implementation of a new type of project . The initial set of performance indicators set up at appraisal was also incomplete. Supervision: Slow disbursement began to threaten the continuity of the project, and faced with the risk of closing the operation and cancelling 70% of the funding, the Bank assisted with the creation of an external Project Coordination Unit (PCU) in August of 2004. This decision was also based on the fact that SE-SINAPRED had a high staff turnover at managerial levels, which was hampering implementation progress. Additionally, in March 2005, the Bank team chose to grant (when requested by the Borrower) project extensions limited to six months at a time, and increased its supervision within the country, to maintain pressure on the Borrower. These actions greatly improved disbursements between 2005 and February 2007. at -Entry :Moderately Unsatisfactory a. Ensuring Quality -at- b. Quality of Supervision :Satisfactory c. Overall Bank Performance :Moderately Satisfactory 9. Assessment of Borrower Performance: SE-SINAPRED suffered from inherent institutional weaknesses, such as vertical decision -making and a slowing of the flow of information, actions, and resolution of issues . An additional complication was the significant staff turnover in essential staff such as the Executive Secretary (3 changes), due to the changing political context . Also there were tensions around the higher salary levels between those hired through the Bank project and those hired by the government, which also led to staff retention problems . After some faltering steps and delays, the PCU was set up and was eventually absorbed in the SE -SINAPRED. Financial management and procurement assistance from the UCRESEP( Coordination Unit for the Public Sector Reform and Modernization Program ), an agency with experience with the Bank, also helped the agency get on its feet . By the time of project completion SE-SINAPRED was a viable and self-standing agency. a. Government Performance :Moderately Satisfactory b. Implementing Agency Performance :Moderately Satisfactory c. Overall Borrower Performance :Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: The design of indicators is too focused on outputs, and not enough on outcomes, did not consider baseline data and did not have many measurable targets . Furthermore, the original indicators were not monitored closely (there was no monitoring arrangements table in the PAD ). A change in the internal reporting system of the Bank, which reflected a decision to increase supervision and reporting in projects, led to the introduction of a set of indicators in the Implementation Status Reports (ISRs) from March 2005. a. M&E Quality Rating : Modest 11. Other Issues (Safeguards, Fiduciary, Unintended Positive and Negative Impacts): Safeguards: No safeguard issues arose according to the ICR . Fiduciary: Procurement lags during the first phase of project implementation were attributable to delays in SE-SINAPRED’s completion of technical terms of reference and evaluations,and to a prolonged dialogue ending with the Bank approving an exception to its guidelines to encourage local offers . Difficulties re-emerged when SE-SINAPRED assumed all executing responsibilities and staffing became an issue, as the Ortega Government set a limit on civil servants ’ salaries. The introduction of the National Project Financial Management System - SIGFAPRO – resulted in additional delays as the SE -SINAPRED learned to operate in the new system toward the end of project implementation .Quarterly Financial Management Reports to the Bank were often late . Project Audit Reports concluded that internal control issues had not been well addressed in the execution period . The ICR does not provide further information . 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately In the absence of evidence on Satisfactory efficiency, it is rated modest. Risk to Development Moderate Moderate Outcome : Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate . 13. Lessons: 1. Disaster risk management is an important part of development strategy, particularly in highly disaster prone countries with high poverty rates . 2. Economic measurement of benefits from avoiding the damages from natural disasters should be a regular part of the economic accounting of disaster -prone countries. 3. A monitoring and evaluation framework that includes outcome indicators and specific targets is necessary to measure results. 4. Low ownership and awareness in the community leads to poor maintenance, as the presence of trash and debris on the works sites showed in this case .Frequently, the community was not introduced to the works, nor invited to any process of public consultation . 5. Establishing a culture of prevention takes time, and requires dedication and resources over multiple years and activities. The project shows that a combination of analytical work (risk mapping) followed by a concrete action (mitigation works) is a good way to keep actors involved through a full project cycle. 6. Clear roles and responsibilities should be assigned to the coordinating agency staff (in this case SE-SINAPRED) both in regular times and in times of crisis, to ensure there is enough capacity at headquarters at all times for prevention and mitigation purposes . 7.The Country Management Unit (CMU) at the Bank should monitor standards for PCU salaries according to the national levels and based on other institutions across Bank projects . This will diminish tensions that can emerge from internal and external equity problems among institutions funded by the Bank. 14. Assessment Recommended? Yes No Why? Since this was one of the first Bank projects to focus on the prevention and mitigation of natural disasters, it should be studied in more detail for its lessons . To assess efficiency and financial management issues . 15. Comments on Quality of ICR: The ICR contains most of the information necessary to evaluate the project . Its analysis is generally thorough, and the lessons are well drawn from the analysis . There are, however, a number of drawbacks . Organization of the report is confusing. An economic analysis to determine efficiency should have been attempted at completion . The differences between the IDA credit amount and the full project cost were not clearly reported and had to be requested subsequently. The ICR should have discussed how financial management issues were resolved given that Project Audit Reports concluded that internal control issues had not been well addressed in the execution period . a.Quality of ICR Rating : Satisfactory