IEG Report Number: ICRR14751 ICR Review Independent Evaluation Group 1. Project Data: Date Posted: 05/07/2015 Country: El Salvador Is this Review for a Programmatic Series ? Yes No Series ID: First Project ID : P122640 Appraisal Actual Project Name: Disaster Risk Project Costs (US$M): 50 50 Management Development Policy Loan With A Cat L/C Number: Loan/Credit (US$M): 50 50 Sector Board: Urban Development Cofinancing (US$M): Cofinanciers: Board Approval Date : 02/01/2011 Closing Date: 08/31/2014 08/31/2014 Sector(s): General water; sanitation and flood protection sector (63%); Other social services (33%); Health (4%) Theme(s): Natural disaster management (100%) Evaluator: Panel Reviewer: ICR Review Group: Coordinator: Victoria Alexeeva Peter Nigel Freeman Christopher David IEGPS1 Nelson 2. Project Objectives and Components: a. Objectives: The overall development objective of the Disaster Risk Management Development Policy Loan (DPL) with a Catastrophe Deferred Drawdown Option (CAT DDO)* was "to enhance the Government of El Salvador’s capacity to implement its Disaster Risk Management Program for natural disasters." (Project Document, page 1) * A Development Policy Loan with a Catastrophe Deferred Drawdown Option (CAT DDO) is a contingent credit line that provides immediate liquidity to IBRD member countries in the aftermath of a natural disaster. A deferred drawdown option for catastrophic risks allows a borrower to postpone drawing down a development policy loan for a defined drawdown period after the Loan Agreement has been declared effective. For the purpose of this ICR review, IEG uses the overall objective set out in the Program Document and assesses relevance and efficacy in relation to the program's policy areas. b. If this is a single DPL operation (not part of a series), were the project objectives/ key associated outcome targets revised during implementation? No c. Policy Areas: The program included two policy areas, with six prior actions taken by the Government. (i) Strengthening of the Institutional and Legal Framework for Disaster Management . This policy area included four prior actions: (1) Updating of the Borrower’s Civil Protection and Disaster Prevention and Mitigation National Plan in July 2009; (2) approval by the National Commission of the creation of seven Sectoral Emergency Response Commissions in October 2009, and submission for approval by the National Commission of seven Sectoral Emergency Response Plans (SERPs) in November 2010; (3) Approval of the National Earthquake Contingency Plan ( Plan Nacional de Contingencia para Terremotos) by the National Commission in October 2009; and (4) Approval of the Rainy Season Plan (Plan Invernal) by the National Commission in May 2010. (ii) Mainstreaming Disaster Risk in the National Development Plan and Investment Programs . This policy area included two prior actions: (5) Approval by the Borrower’s National Cabinet (Consejo de Ministros) of the 2010-2014 Development Plan (Plan Quinquenal de Desarrollo), incorporating a disaster risk management component; and (6) Development of the National Program for Risk Reduction 2010-2012 (Programa Nacional de Reducción de Riesgos 2010-2012). d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: The IBRD loan of US$50 million was approved on February 1, 2011, and became effective on May 24, 2011. The Government of El Salvador (GoES) first withdrew the amount of US$25 million on October 17, 2011, and US$24.75 million on October 27, 2011, in response to the impact caused by Tropical Depression 12E that hit the country from October 10 to 20, 2011. The remaining amount of US$0.25 million corresponded to the loan's front-end-fee of 0.5% of the total loan amount, and was paid to the Bank on May 24, 2011 using loan proceeds. The program was closed as scheduled on August 31, 2014. 3. Relevance of Objectives & Design: a. Relevance of Objectives: High. El Salvador is exposed to a variety of natural hazards, including hydro-meteorological and geophysical hazards, and has a history of destructive earthquakes, volcanic eruptions, tropical storms, and droughts. Recognizing the country's high vulnerability to natural hazards, the Government of El Salvador (GoES) has taken important steps towards strengthening its policy and institutional framework for disaster risk management (DRM), gradually moving from emergency response to a more comprehensive approach for the proactive reduction of disaster risks. The GoES created the National System for Civil Protection and Disaster Prevention and Mitigation in 2005 to enhance the country’s disaster preparedness and emergency response capacity. The DPL policy areas were well aligned with the main themes of the government priorities in disaster risk management. The objective was also relevant to the World Bank Group's Country Partnership Strategy (CPS) for FY2010-2012 for El Salvador, which recognized the country’s high vulnerability to natural hazards and included DRM under the first strategic objective to strengthen fundamentals for economic recovery by addressing macro and institutional vulnerabilities. The CPS explicitly mentioned the possibility of a DPL with a Catastrophe Deferred Drawdown Option (CAT DDO) to provide a source of immediately available liquidity during a state of emergency caused by a natural disaster. b. Relevance of Design: Modest. The program was built around two policy areas to support the implementation of the Government's Disaster Risk Management Program, in particular (i) strengthening the institutional and legal framework, and (ii) mainstreaming disaster risk in the national plan and investment programs. Four prior actions under the first policy area, in particular updates to selected governmental plans, were meant to better prepare the GoES to respond to adverse natural events. Two prior actions under the second policy area, in particular approval of the National Development Plan 2010-2014, and development of the National Program for Risk Reduction 2010-2012, were intended to ensure that risk prevention and mitigation would be included in public investment programs and other line ministries' development plans. This review finds that some prior actions were rather updates of the existing Government's framework and did not add much value. This, in particular, relates to prior action 1 that required the Civil Protection, Disaster Prevention and Mitigation National Plan (adopted in 2006) to be updated in July 2009, some 13 months prior to the DPL's concept review in September 2010. Two prior actions are considered to lack criticality for insufficient institutional depth, in particular, the submission of SERPs for approval (prior action 2) and the development of the National Program for Risk Reduction 2010-2012 (prior action 6). Besides, the outcomes defined in the results framework were rather reinforcing the same prior actions. For example, the objective of enhancing the Government's capacity to implement its disaster risk management program was to be achieved through prior actions that were expected to lead to better preparedness of the Government. 4. Achievement of Objectives (Efficacy): Substantial. Efficacy of the overall objective -- " to enhance the Government of El Salvador’s capacity to implement its Disaster Risk Management Program for natural disasters" -- is assessed through the achievement of the following policy areas: (i) Strengthening of the Institutional and Legal Framework for Disaster Management . Partially achieved. There were two outcome indicators: (i) the Government is better prepared to respond to adverse natural events, and (ii) Sectoral Emergency Response Plans are implemented in relevant situations.  The Civil Protection, Disaster Prevention and Mitigation National Plan, which was updated in 2009 (prior action 1), was further revised in 2011 and 2012 but the 2012 version had not yet been approved by DPL closure in 2014. The ICR p.10 reports that it was under review by the National Commission to avoid overlap with the Ministry of Environment and National Resources' National Program for Risk Reduction.  The Civil Protection Directorate increased its staff from 33 to 233 between 2011 and 2014 at different territorial levels. To complement territorial capabilities, training activities to guide the formulation of Civil Protection Plans and communication equipment have been provided to Municipal and Community Commissions, delivering 40 Municipal Plans, 1650 emergency kits, and increasing communication radios from 34 to 464.  Seven Sectoral Emergency Response Plans (SERP) - for Emergency Services, Infrastructure and Basic Services, Health, Logistics, Security, Shelters, and Techno- Scientific - were approved in 2010 (their submission for approval by the Sectoral Emergency Response Commissions was prior action 3).  Among contingency plans that include the National Earthquake Contingency Plan approved in 2009 (prior action 3) and the National Rainy Season Plan approved in 2010 (prior action 4), the Rainy Season Plan was being updated once a year (between 2010 and 2014). As reported after the simulation drill of the National Earthquake Contingency Plan in 2013, the evacuation time of the population in tsunami-prone areas of La Libertad reduced from 20 minutes to 15 minutes (ICR, p.11). During the rainy season in May 2014, the Departmental Commission of Civil Protection of San Miguel and the Municipal Commissions of Chinameca, San Jorge, San Rafael and San Miguel activated and implemented their contingency plans. Considering the high risk conditions, the Directorate of Civil Protection evacuated 259 families to shelters. Families received lease subsidies for six months and a commission was created to initiate a relocation process, in coordination with the National Housing Fund (FONAVIPO) (ICR, p.12). (ii) Mainstreaming Disaster Risk in the National Development Plan and Investment Programs . Achieved. There were two outcome indicators: (i) Public investment programs include specific activities to enhance risk prevention and mitigation as a result of the implementation of the 2010-2014 Development Plan (target for all investments was not established); and (ii) at least three ministries have included risk analysis initiatives in their development plans to guide investment programs to reduce vulnerability.  During 2009-2014, public investments to enhance risk prevention and mitigation in key sectors amounted to approximately US$303 million (approximately 13/1000 of the total 2010-2014 national budget). This surpassed budgetary allocations included in the 2010-2014 NDP in the amount of the US$ 173 million for priority investments in environmental and risk management as a whole (comprising prevention and mitigation activities).  As part of the National Program for Risk Reduction 2010-2014 adopted by the Ministry of Environment and Natural Resources (prior action 6 on development of this program for 2010-2012), four line Ministries incorporated risk analysis in their work programs, including monitoring natural events and implementing activities related to risk assessment (Target: 3). These were (i) Ministry of Environment and Natural Resources (MARN); (ii) Ministry of Public Works, Transport, Housing, and Urban Development (MOPTVDU), (iii) Ministry of Health (MINSAL), and (iv) Ministry of Education (MINED). (i) MARN: Between 2010 and 2014, vulnerability studies and hazard maps for flooding, landslides, and coastal erosion were undertaken for 50% of El Salvador’s territory, including 60 municipal vulnerability studies for specific localities. Environmental zoning and land use guidelines were approved in 2013. Technical information and data generated by MARN are used by various ministries and agencies, including MOPTVDU, to guide investments. (ii) MOPTVDU incorporated risk management considerations in the core agenda in 2011. Its Directorate for Adaptation to Climate Change and Strategic Risk Management (DACGER) implemented the following activities: (i) standardizing the evaluation of landslide risk through the development of guidelines (used during 2012-2014 in the development of the inventory of areas with stability problems that affect roads); (ii) developing guidelines for the assessment of the vulnerability of bridges to hydro-meteorological events (validated through the inspection of 126 bridges between 2013 and 2014); and (iii) formulating guidelines for the prioritization of works in highways and for the design of bridges taking into account climate change adaptation measures. (iii) MINSAL incorporated the concept of Safe Hospitals designed by the Pan- American Health Organization to retrofit six, and build two new hospitals in addition to the implementation of the Hospital Safety Index to evaluate 98% of the 30 national hospitals and 223 Health Community Centers. Through the Hospital Safety Index, MINSAL determined that 3% of hospitals needed immediate and 36% needed medium term interventions. Health facilities have established maintenance and monitoring processes for buildings lacking safety interventions in 30 years and follow-up actions are identified to improve safety in the medium term. (iv) MINED evaluated the safety of school buildings using the School Safety Index in the Metropolitan Area of San Salvador, establishing that 548 schools are exposed to landslides, and 289 to flooding. Complementarily, and in coordination with MARN, 1,050 school buildings were evaluated using the CAPRA methodology, finding that 40% of them do not meet seismic standards. MINED finalized the geo-referencing of the total portfolio of schools in the country (6,035 in 2014, including 923 private schools). 5. Efficiency (not applicable to DPLs): 6. Outcome: Although the relevance of the DPL objective was substantial, the relevance of design was only modest, since some prior actions were generally weak. Efficacy was overall substantial. a. Outcome Rating: Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating:  Political and institutional capacity risks . Increased efforts are still required for an effective institutional articulation and implementation of disaster resilience and climate change adaptation strategies.  Fiscal risk . Considering the country’s sustained large fiscal deficit, the macroeconomic situation continues to present a factor of vulnerability for El Salvador. A risk remains that resources could be diverted away from preventive DRM activities and that the GoES would need to redirect resources from existing development programs towards emergency response in case of a severe natural event. The ICR p.17 adds that this risk is being mitigated through the efforts of the GoES towards establishing a comprehensive Disaster Risk Financing and Insurance (DRFI) strategy with the support of the Bank’s technical assistance. a. Risk to Development Outcome Rating : Moderate 8. Assessment of Bank Performance: a. Quality at entry: The ICR p. 18 reports that following a close dialogue between the Bank and the GoES on the country’s DRM needs, the Bank addressed the formal request from the GoES for a DPL with CAT DDO in July 2010 and formulated the operation in close coordination with the Government for Board approval in February 2011. The DPL became effective in May 2011. The Bank’s multi-sectoral team (including DRM specialists and Treasury staff) worked with the GoES at entry to explain the nature of the DPL with CAT DDO product, drawdown triggers, and how to request withdrawal of funds. The ICR p.18 adds that this support early on in the operation was recognized by the GoES, and was instrumental in the timely withdrawal of funds when Tropical Depression 12E hit the country in October 2011, a few months after effectiveness. The ICR p.18, however, assesses that Quality at entry would have been enhanced with a stronger results framework to measure the achievement of the PDO. While the two areas of action supported by the operation and prior actions were adequately identified and consistent with the Government’s program, the Bank could have improved quality at entry by designing a more specific results framework including clear outcome indicators with baselines and measurable targets to track progress towards achieving the PDO. In addition, as noted by the ICR p.8, lack of clarity of expectations on reporting from the outset created a challenge for the consolidation of information provided by the GoES for program evaluation. Quality-at-Entry Rating: Moderately Satisfactory b. Quality of supervision: Supervision missions were carried out once or twice a year by a multi-disciplinary team including DRM and Treasury specialists. The ICR p.18-19 reports that Bank acted promptly following the GoES’ requests for disbursement of CAT DDO funds, and worked closely with the GoES throughout implementation in relation to the activation of the revolving feature, related costs, and renewal options of the CAT DDO. Shortcomings in the results framework were partially addressed through periodic implementation review workshops that the Bank team initiated in 2013, which brought together counterparts to monitor progress. However, as noted by the ICR p.18, the team could have set targets for monitoring of progress towards the achievement of the objective during implementation, and set clear expectations with the GoES early on regarding the periodicity and content of consolidated progress reports to be submitted. Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The ICR p.20 assesses that GoES showed satisfactory commitment to the operation, and to the implementation of its DRM program during the design phase and implementation. Following the declaration of a state of emergency because of Tropical Depression 12E, the GoES acted promptly to access CAT DDO funds through two separate withdrawal requests and provided necessary documentation, which in turn ensured a timely transfer of funds to the GoES to respond to the emergency. While the funds were fully disbursed five months after effectiveness, the GoES remained committed to the DRM program and engaged with the Bank through various implementation review workshops, which proved to be an effective platform for progress monitoring, with active participation of line ministries. The ICR p. 6 notes that the leadership of the Ministry of Finance and the Technical Secretariat of the Presidency was instrumental in the convening of key agencies involved in DRM and the openness of the dialogue. In addition, stability in terms of key counterparts in the Ministry of Finance, MARN, the Civil Protection Directorate, and other agencies both at the Ministerial, Director and technical levels throughout implementation helped ensure the continuity of the dialogue. Government Performance Rating : Satisfactory b. Implementing Agency Performance: Not applicable to this DPL where the Government and Implementing Agency are the same. Implementing Agency Performance Rating : Not Applicable Overall Borrower Performance Rating : Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The M&E design was weak. The outcomes were not well defined and measurable, and there were no baselines and targets, except for one indicator. As the ICR p. 7 assesses, while the policy areas and prior actions in Annex 2 of the Project Document (PD) were well described, the PD did not clearly outline the expected outcomes and how these would be measured. Only one indicator related to the number of ministries included a baseline and a target. A baseline of '0' included under the indicator related to public investments was not accurate since there were ongoing investments at the moment of the formulation of the PD (ICR, p.13, footnote 20). The ICR p. 7 further notes that there was no clear agreement between the Bank and the GoES from the outset on the submission and content of consolidated progress reports to monitor implementation of the DPL. The ICR adds while the Loan Agreement states that “the Borrower shall furnish to the Bank for its review and comment a report on the progress achieved, in such details as the Bank shall reasonable request” (Article III – Program), there was no agreement from the beginning on which Government agency was responsible for the submission of these reports, neither on their periodicity and content. b. M&E Implementation: The Bank obtained detailed progress reports during implementation from each institution, which included overall achievements in relation to DRM and did not always focus on the program indicators. To overcome these shortcomings, in 2013 the Bank started carrying out “implementation review workshops” as part of supervision to monitor progress and record achievements in implementation status reports (ISRs). These supervision missions and workshops were held twice a year, led by the Ministry of Finance, and included the participation of up to five representatives of each of the relevant agencies including the Civil Protection Directorate, STPP, and key line Ministries, namely MARN, the Ministry of Public Works, Transport, Housing, and Urban Development (MOPTVDU), the Ministry of Health (MINSAL) and the Ministry of Education (MINED). c. M&E Utilization: The ICR p.8 reports these workshops created a platform for these institutions to share detailed information and progress on DRM with one another through the end of the operation. M&E Quality Rating: Modest 11. Other Issues a. Safeguards: None. b. Fiduciary Compliance: The ICR does not report fiduciary issues. c. Unintended Impacts (positive or negative): None. d. Other: 12. Ratings: ICR IEG Review Reason for Disagreement/Comments Outcome: Satisfactory Moderately Due to modest design, in particular Satisfactory weaknesses in the prior actions. Risk to Development Negligible to Low Moderate The identified risks are of a moderate Outcome: nature. Bank Performance: Moderately Moderately Satisfactory Satisfactory Borrower Performance : Satisfactory Satisfactory Quality of ICR: Satisfactory NOTES: - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: Lessons from the ICR:  The policy dialogue on DRM could be made more efficient by improving the indicators of the policy matrix . DPLs support governments in implementing reforms to improve their policy and institutional framework. The impact of the reform program is closely related with the quality of the policy matrix, which should include specific, measurable and attributable outcome indicators with baselines and targets, given their importance for the quality of the policy dialogue.  Reporting responsibilities and monitoring arrangements should be agreed with the Borrower in the early stages of the operation to improve the effectiveness and monitoring of the policy measures supported by DPL. In the case of El Salvador, the submission of consolidated written progress reports would have helped ensure consistency in the quality and level of detail provided across the participating institutions. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR is concise and clear. The quality of analysis is adequate. The ICR offers a frank discussion of weaknesses in the project's M&E design. Lessons are evidence based. a.Quality of ICR Rating : Satisfactory