Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) Report Number: ICRR0023175 1. Project Data Project ID Project Name P117871 6O Regional Disaster Vuln Reduct. Projs Country Practice Area(Lead) OECS Countries Urban, Resilience and Land L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-49850,IDA-49860,IDA-54500,TF- 31-Dec-2016 94,385,359.82 10204,TF-10206,TF-11131,TF-11132,TF- 16733,TF-19232,TF-19396,TF-A3698 Bank Approval Date Closing Date (Actual) 23-Jun-2011 31-Jan-2022 IBRD/IDA (USD) Grants (USD) Original Commitment 32,120,000.00 45,191,511.54 Revised Commitment 100,519,366.90 45,181,185.83 Actual 94,385,359.82 47,013,628.93 Prepared by Reviewed by ICR Review Coordinator Group Wendy Schreiber Ayres Fernando Manibog Kavita Mathur IEGSD (Unit 4) P157918_TBL Project ID Project Name P157918 SVG RDVRP AF - EU ( P157918 ) Country Practice Area(Lead) St. Vincent and the Grenadines Urban, Resilience and Land L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) Page 1 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) 0 Bank Approval Date Closing Date (Actual) 02-Mar-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 0.00 Revised Commitment 0.00 0.00 Actual 0.00 0.00 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project was conceived as phase 1 of a three phase Program to be implemented through an Adaptable Program Loan. The first phase covered projects in Saint Vincent and the Grenadines (SVG) and in Grenada. The Project Development Objective (PDO) as stated in the Financing Agreement between SVG and the International Development Association (IDA) dated September 9, 2011 (page 5) was to measurably reduce vulnerability to natural hazards and climate change impacts in the Recipient’s territory and in the Eastern Caribbean Sub-region. The PDO as stated in the Financing Agreement between Grenada and IDA dated September 20, 2011 (page 5) was to measurably reduce vulnerability to natural hazards and climate change impacts in the Recipient’s territory and in the Eastern Caribbean Sub-region. The Project was conceived as phase 1 of a three phase Program to be implemented through an Adaptable Program Loan. The first phase covered projects in Saint Vincent and the Grenadines (SVG) and in Grenada. The Project Development Objective (PDO) as stated in the Financing Agreement between SVG and the International Development Association (IDA) dated September 9, 2011 (page 5) was to measurably reduce vulnerability to natural hazards and climate change impacts in the Recipient’s territory and in the Eastern Caribbean Sub-region. The PDO as stated in the Financing Agreement between Grenada and IDA dated September 20, 2011 (page 5) was to measurably reduce vulnerability to natural hazards and climate change impacts in the Recipient’s territory and in the Eastern Caribbean Sub-region. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Page 2 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) Date of Board Approval 09-May-2014 c. Will a split evaluation be undertaken? No d. Components Component 1: Prevention and Adaptation Investments (cost at appraisal US$19.1 million, actual cost US$54.23 million). This component focused on implementing and testing a broad spectrum of interventions aimed at reducing vulnerability in public buildings and infrastructure in the participating countries. Subprojects under this component supported: (i) developing infrastructure and carrying out related supporting studies, including: (a) developing community infrastructure, (b) rehabilitating and constructing bridges and reducing risk in public spaces, and (c) improving resilience to climate risks in the water supply system; and (ii) retrofitting and designing public buildings to improve disaster resilience, including: (a) rehabilitating selected schools’ buildings and (b) rehabilitating selected public accommodation for the elderly. Revised component 1: Under the first restructuring, funds were reallocated from component 1 to a new component 5 created to enable the project to finance the Caribbean Catastrophic Risk Insurance Facility (CCRIF) insurance premium. Through the Additional Financing (AF) of April 2014 for SVG, activities totaling US$23.56 million were added to component 1, bringing the total cost of the component to US$42.66 million. These included financing of river training, bridge rehabilitation and reconstruction, road realignment, coastal defenses, and replenishment of the emergency contingency component. It also included additional slope stabilization works, expansion of river defense works, and construction of additional satellite warehouses to strengthen community disaster preparedness. Through the AF of April 2015 for Grenada, activities totaling US$5.2 million were added to component 1, bringing the total cost of the component to US$47.86 million. These included the carrying out of selected infrastructure investments and related supporting studies, including: (a) pre-engineering studies and design for the Morne Rouge drainage system; (b) river training works for the St. Johns River flood mitigation sub-project; and (c) support for the preparation of sub- projects. Through the Additional Financing (AF) of May 2017 for SVG, activities totaling US$6.59 million were added to component 1, bringing the component cost to US$54.45. These included financing for select infrastructure investments. Under the restructuring of December 2018, errors in previous documentation were corrected, and funds in component 3, which had been earmarked for the contingency emergency response (CERC), were reallocated to component 1, bringing the cost of component 1 to US$54.23 million (Project Paper, December 2018, para. 13, footnote 8). Component 2: Regional Platforms for Hazard and Risk Evaluation, and Applications for Improved Decision Making (cost at appraisal US$22.2 million, actual cost US$40.82 million). This component focused on building the regional capacity for assessment of natural risks and integration of such assessment into policy and decision-making process for the development of investments, disaster risk mitigation, and disaster response across sectors by: (a) facilitating regional collaboration, including knowledge sharing to develop and apply construction standards and methods for critical public infrastructure and urban flood mitigation, and to strengthen regional collaboration for urban and flood risk reduction; and (b) reducing the risk for regional interconnectivity, including improving the international airport to maintain an adequate emergency response capability and to comply with the international operational standards. Page 3 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) Grenada took the lead on piloting integrated approaches to urban flood mitigation (St. John’s River in St. Georges). SVG took the lead on piloting integrated approaches to watershed management (Arnos Vale Watershed) and coastal protection (Georgetown). In collaboration with other countries from the region and the support of regional technical agencies, the respective ministries of works took the lead on organizing the Eastern Caribbean regional knowledge sharing and learning process to develop and apply construction standards and methods in the selected areas. Grenada also was to make necessary investments at its international airport to ensure its continued operations in accordance with international aviation regulations. Revised component 2. Under the first restructuring, funds were reallocated from component 1 to a new component 5 created to enable the project to finance the Caribbean Catastrophic Risk Insurance Facility (CCRIF) insurance premium. Through the AF of 2014 for SVG, activities totaling US$15.44 million were added to component 2, bringing the total cost of the component to US$37.64 million. These included financing for construction of bridges, river defense works, and emergency recovery coastal defense works. It also including funding for capacity building of the Ministry of Transport and Works to strengthen its disaster response. Through the AF of April 2015 for Grenada, activities totaling US$3 million were added to component 2, bringing the total cost of the component to US$40.64 million. These included financing for (a) hazard data development; (b) hydro-meteorological infrastructure and data management; (c) watershed analysis and training for flood and drought management; and (d) capacity building for forest management. Under the restructuring of December 2018, errors in previous documentation were corrected, and funds in component 3, were reallocated to component 2, bringing the component cost to US$40.82 million. Component 3: Natural Disaster Response Investments (cost at appraisal US$2.0 million, actual cost US$1.9 million). This component financed emergency recovery and reconstruction subprojects under an agreed action plan of activities (Agreed Action Plan of Activities) designed as a mechanism to implement the Recipient’s rapid response to an emergency. In the event of a request for reallocation of funds from other project components funded from the IDA regional pool, the Bank would first have to determine if the proposed subprojects would be eligible for IDA regional funding. Following an adverse natural event, and subject to: (a) the Bank’s satisfaction that a situation of national emergency exists; (b) a government’s declaration of emergency in accordance with national law; and (c) the submission of a recovery action plan satisfactory to the Bank that describes subprojects and activities to be financed, a government may request the Bank to re-categorize financing or provide additional financing to cover early recovery and rehabilitation costs. If no adverse natural event occurred during the lifetime of the project, financing under the component would not become active. This mechanism complemented the participating countries’ memberships of the Caribbean Catastrophe Risk Insurance Facility (CCRIF), since the trigger was a declaration of emergency following an adverse natural event, rather than the CCRIF’s parametric trigger. Revised component 3: Through the AF of 2014 to SVG, US$1 million was added to component 3 to replenish the Natural Disaster Response Investment Component of the original operation, which was utilized in response to the December 2013 disaster event. This brought the total cost of component 3 to US$3 million. Under the restructuring of December 2018, funds in component 3 were reallocated to component 2, bringing the component cost to US1 million. Component 4: Project Management and Implementation Support (cost at appraisal US$3.8 million, actual cost US$5.26). This component would support strengthening and developing the institutional capacity for project management, including: (a) preparation of designs and tender documents; (b) Page 4 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) preparation of project reports; (c) processing of contracts and tender evaluation; (d) coordination of participating line ministries; (e) supervision of the quality of works; (f) training of staff in project management and implementation support; (g) capacity building for accreditation to the United National Framework Convention on Climate Change (UNFCCC) Climate Adaptation Fund; and (h) related activities on project management and implementation. Revised component 4: Through the AF of 2014 to SVG, activities totaling US$0.6 million were added to component 4, bring the total cost of the component to US$4.4 million. These included financing for (a) a senior quantity surveyor and procurement and contract management specialist; (b) a monitoring and evaluation (M&E) specialist; and (c) provision of training to the Public Sector Investment Management Unit (PSIPMU) staff and the staff of key agencies involved in the implementation of the project. Through the AF of April 2015 to Grenada, activities totaling US$0.6 million were added to component 4, bringing the total cost of the component to US$5 million. These included support for strengthening and developing the institutional capacity for project management, and improving the capacity for disaster risk management and climate change monitoring. Through the AF of May 2017 to SVG, activities totaling US$0.26 million were added to component 4 for project management and implementation support. This brought the component cost to US$5.26 million. Component 5: Payment of CCRIF insurance premium. Under the first restructuring, a new component was added to finance the Recipient’s catastrophe risk insurance premiums for the period of May 30, 2013 to May 30, 2015 for a total not to exceed US$2 million. The Ministry of Finance was responsible for the management of this activity, although no Bank funds would flow directly into the government treasuries. Instead, a direct payment would be made from the World Bank to the CCRIF. A new intermediate indicator was added to the results framework to reflect this activity. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The original estimated project cost was US$47.1 million. Following the first AF of 2014 of US$40.6 million to SVG, the project cost rose to US$87.7 million. Following the second AF of 2015 of US$8.8 million to Grenada the project cost rose to US$96.5 million. Following the third AF of 2017 to SVG US$6.81, the project cost rose to US$103.31 million. The actual cost at closing date, January 31, 2022, was US$94.4 million. Financing: The project’s cost was financed through an IDA Credit of totaling US$10 million, with co- financing from the Climate Investment Funds in the amount of Pilot Program for Climate Resilience Grant of US$8.0 million, Strategic Climate Fund Loan of US$8.2 million, with an additional Strategic Climate Fund Loan of US$3.8 million and a Strategic Climate Fund Grant of US$5 million to Grenada; and a Credit for US$10.92 million co-financing from the Climate Investment Funds in the amount of Pilot Program for Climate Resilience Grant of US$7.0 million, Strategic Climate Fund Loan of US$3.0 million, with an Additional Financing Credit of US$35.6 million equivalent, including a Credit US$19.0 million equivalent from the Crisis Response Window resources and a Grant from the Strategic Climate Fund – Pilot Program For Climate Resilience of US$5.0 million and Additional Financing Grant from the European Union in the amount of US$6.81 million equivalent to Saint Vincent and the Grenadines and Grenada. A total of US$94.4 million was disbursed. Page 5 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) Recipients’ Contributions: The Recipients—Saint Vincent and the Grenadines, and Grenada—did not contribute to the project’s cost. Restructuring: First Restructuring. The project was restructured on May 31, 2013 to enable Grenada to finance the CCRIF insurance premium coverage from June 1, 2013, to May 30, 2015. A new disbursement category was introduced, activities in components 1 and 2 were reduced, and a results indicator was added to reflect the CCRIF insurance premium. Second Restructuring. The project restructuring was approved on May 9, 2014 to provide AF of US$40.6 to SVG to finance emergency recovery and reconstruction activities, scale up project activities, and cover a cost overrun. Targets in the results framework were revised to reflect the increase in the scope of activities and a new indicator was added. The closing date was extended by two years to December 31, 2018. Third Restructuring. The project restructuring was approved on June 8, 2015 to provide AF to Grenada US$8.8 million. Funds were to: (a) scale up investments for flood protection and improve drainage in urban areas; (b) acquire baseline data; (c) optimize and modernize the hydro-meteorological data collection network and data management system; (d) provide training in watershed modeling for flood and drought hazard mapping and calculation of hydraulic parameters for climate-proofing infrastructure design; and (e) build capacity in forest management. The AF comprised an additional Strategic Climate Fund Loan in the amount US$3.8 million, and an additional Strategic Climate Fund Grant in the amount of US$5.0 million. The loan, credit and grant closing dates were extended by two years to December 31, 2018, to accommodate the increased scope of the Project. The targets in the results framework were revised to reflect the increase in the scope of activities and new indicators were added. Fourth Restructuring. The project was restructured August 10, 2015 was to extend the premium payments for CCRIF coverage for an additional year in Grenada. In SVG, the government reallocated funds among disbursement categories after the December 2013 Christmas Eve Trough in SVG to access US$900,000 for emergency recovery and reconstruction subprojects, added to the US$1 million allocated to the CERC. In addition, a change in the disbursement process was made to simplify accounting procedures and improve efficiency so that expenditures were no longer split by the various sources of funding. Fifth Restructuring. The project was restructured on February 8, 2017 to trigger of the safeguards policy for Pest Management (OP/BP 4.09), to allow for incidental pesticide use under the project. Sixth Restructuring. The project restructuring was approved on May 16, 2017 to provide AF of US$6.81 million to SVG through a grant agreement between the European Union, through the European Development Fund, and the World Bank. The AF funded reconstruction of the Chateau Belair jetty and the rehabilitation of road infrastructure. In addition, overall project management was scaled up to provide adequate financial management and technical support for the supervision of civil works and to enhance the capacity of the Ministry of Transport and Works. Seventh Restructuring. The project was restructured on December 10, 2018 to extend the closing date by 24 months from December 31, 2018, to December 31, 2020, to allow for the completion of activities. This was the second extension of the closing date. The results framework was revised; funds in Category 2 that were earmarked for CERC were reallocated to component 1. Page 6 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) Eighth Restructuring. The project was restructured on December 3, 2020 to extend the closing date by six months from December 31, 2020 to June 30, 2021, due to the implementation delays caused by the impact of the COVID-19 pandemic. Ninth Restructuring. The project was restructured on June 28, 2021 to extend the closing date of SVG’s IDA credit 4986-VC (US$10.92 million equivalent) and IDA credit 5450-VC (US$35.6 million equivalent) by seven months from June 30, 2021 to January 31, 2022, to enable the completion of activities that suffered delays due to the eruption of La Soufriere volcano on April 9, 2021. (The Grenada portion of the Project closed on June 30, 2021.) Dates: The project was approved on June 23, 2011 and became effective on October 18, 2011. The project closed on January 31, 2022, five years and one month after the original closing date of December 31, 2016. Split Rating. As the project scope became more ambitious, a split rating is not applied. 3. Relevance of Objectives Rationale Country and Sector Context at Appraisal The impacts of climate change were being felt in the Caribbean at the time of appraisal. The small island economies were highly vulnerable to natural hazards because of their size, geography, and location. Climate shocks represented a dominant factor driving productivity fluctuations in the region, thus severely affecting development prospects. In the Eastern Caribbean, natural disasters accounted on average for almost 20 percent of the variance of real GDP growth and the evidence showed that an adverse natural event in one country has a statistically significant spillover effect on neighboring countries in the region. It was therefore critical for the Eastern Caribbean to find ways to achieve economic and social development that was more resilient to the climate they live in. The World Bank was supporting Caribbean-led efforts to increase climate resilience and decrease vulnerability to natural disasters. A concerted effort in this regard was taking place under the Caribbean Regional Pilot Program for Climate Resilience, which involved two closely linked and complementary tracks: (a) country-based investments in six states (Haiti, Jamaica, Dominica, Grenada, St. Lucia, and St. Vincent and the Grenadines); and (b) region-wide activities, including data management and monitoring for improving understanding of climate risks and potential impacts, necessary to prepare actions to enhance climate resilience, and to tackle risks and vulnerabilities common to all Caribbean countries. Alignment with Country Priorities The PDO was well-aligned with the priorities of Grenada, SVG, and the Eastern Caribbean subregion. The project was fully aligned with Grenada’s country priorities of coping with external shocks and laying the groundwork for broad-based economic growth.[1] The project was also aligned with Goal Four of the SVG’s country priorities—improving physical infrastructure, preserving the environment and building resilience to climate change—as presented in the National Social and Economic Development Plan 2013–2025, which was under preparation at the time of the project’s appraisal. This plan emphasizes the importance of Page 7 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) climate risk management in sustainable development. Objectives include enhancing the capability of SVG to prepare effectively for, respond to, and mitigate disasters and to reduce the adverse impacts of climate change. Finally, the PDO was aligned with the 2001 Comprehensive Disaster Management Strategy of the Caribbean Disaster Emergency Management Agency, the regional inter-governmental agency for disaster management, which focused on institutionalizing disaster management in the region. Alignment with Bank Strategy The project’s objectives are assessed as fully aligned with the World Bank’s strategy as laid out in its Regional Partnership Framework for the Eastern Caribbean States for the Period FY22-FY25 (Report 160349-LAC). The overarching objective of the framework is to support green, resilient, and inclusive development and competitiveness as the OECS countries recover from the Covid-19 crisis, address their medium-term development priorities, and build resilience to climate change and other external shocks. The framework supports the high-level outcome of strengthened resilience to climate change and other shocks and aims to help the countries recover from the pandemic while safeguarding their natural resources and building resilience to climate change. The Bank’s 2018 Systematic Regional Diagnostic for the OECS countries also highlights key constraints and opportunities for green, inclusive, and sustainable growth in the region, the national, and regional development strategies of the Eastern Caribbean. It also specifies the priorities for resilient recovery from the impacts of the Covid-19 pandemic crisis. Previous World Bank Experience The World Bank has a long history of partnership and collaboration with the states of the Eastern Caribbean, and the project was part of the World Bank’s ongoing support to the Eastern Caribbean to build resilience to natural disasters and climate change. As a key partner in disaster recovery and disaster risk reduction, it financed the Grenada Emergency Recovery and Disaster Management Project (P069922), for US$10.7 million (closed on October 31, 2005), which focused on disaster mitigation and institutional strengthening activities in Grenada and SVG. In addition, the Bank administered a trust fund in support of the Grenada Post-Hurricane Ivan School Rehabilitation Project for US$14.2 million (closed in June 2009), aimed at rehabilitating and reconstructing schools and health facilities damaged by the hurricane. The Bank also financed the Grenada Economic and Social Policy Development Credit (P117000) for US$8 million (closed on June 30, 2011), which supported policy reforms aimed at strengthening the countries resilience to external shocks. Other activities included the Caribbean Pilot Program for Climate Resilience technical assistance, the SVG Hurricane Tomas Emergency Recovery Loan (P124939), and the Saint Lucia Hurricane Tomas Emergency Recovery Loan (P125205). An evaluation carried out by the World Bank November 2010 following Hurricane Tomas in indicated that investments financed by the Bank to reduce risk over the past decade functioned well in the face of a 1-in-500-year rainfall event.[2] Similar conclusions were reached in the evaluation of school infrastructure in Grenada following Hurricane Ivan in 2004.[3] The World Bank’s involvement in reconstruction activities in both Grenada and SVG in the 10 years before appraisal also led to a shared understanding of the vulnerability and importance of various sectors and asset types proposed for construction, rehabilitation, or retrofitting under the project. Finally, a trust fund administered by the World Bank financed studies on the links between poverty and environment in four small island developing states and the potential of OECS catastrophe insurance to speed reconstruction and recovery following a natural disaster. Page 8 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) As the objective remained relevant throughout the project implementation period and was a necessary response to a development gap in Grenada and SVG, relevance is rated High. [1] These priorities were laid out 2009 Budget Speech, the Prime Minister’s speech at the Townhall Meeting of March 31, 2009, and the Poverty Reduction and Growth Facility Letter of Intent for the Third Review of the International Monetary Fund. [2] World Bank, Saint Lucia Hurricane Tomas Rapid Damage Assessment (2010), [3] World Bank, Grenada: A Nation Rebuilding (2005). Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Objective: Measurable reduction in vulnerability to natural hazards and climate change impacts in the Recipient’s territory and in the Eastern Caribbean Sub-region. Rationale Theory of Change The project’s theory of change indicated that the project’s inputs—financial and technical assistance support from IDA and other financiers—would directly lead to the achievement of project outputs, which in turn would lead to the project’s outcomes. The activities related to SVG consisted of (a) studies, and (b) retrofitting, constructing, and rehabilitating infrastructure. The key outputs were (a) schools, emergency shelters retrofitted; (b) roads and bridges rehabilitated, (b) satellite warehouses constructed, (c) retrofitting, constructing, and rehabilitating roads and bridges, (d) feasibility study and detailed design for a new hospital prepared, and (e) drainage systems improved. The activities related to Grenada consisted of (a) studies, and (b) retrofitting, constructing, and rehabilitating infrastructure. The key outputs were (a) schools, emergency shelters, and homes for the elderly retrofitted and rehabilitated; (b) roads and bridges rehabilitated, (c) La Sagesse and Beausejour community infrastructure rehabilitated or constructed, and (d) water tanks and associated structures installed. In turn, the outputs would be expected to result in the outcome of measurable reduction of vulnerability to natural hazards and climate change impacts in the SVG and in Grenada. Overall, the causal pathways from inputs to outcomes were valid and direct, and the outcomes achieved could be mostly attributed to the Page 9 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) project’s interventions. The activities, if completed, would be sufficient to provide a critical mass for the expected change. The key assumptions for achieving this objective were: (a) The political will and commitment of the governments remain high during project implementation; (b) natural hazard impacts will not derail project implementation, but rather serve to inform a responsive project design. The assumptions were reasonable. Outputs Infrastructure constructed or rehabilitated  100 percent of Grenada's infrastructure was made less vulnerable to natural hazard and climate change impacts in Project areas. This matched the end project target presented in the 2015 Project Paper for the 2015 AF. The activities contributing to strengthening of infrastructure included rehabilitating and retrofitting infrastructure and public buildings, river training, bridge rehabilitation and reconstruction, road realignment, and construction of coastal defenses.  Water storage capacity increased by 550,000 gallons in Grenada as a result of the project. This is 22 percent higher than the target of 450,000 gallons set out in the PAD.  13,369 gabion baskets were used in construction of flood mitigation works in SVG. This is 60 percent higher than the target of 8,400 set out in the PAD.  2.22 kilometers of rural roads were constructed. This is 47 percent below the target of 4.15 kilometers set out in the 2018 Project Paper.  Seedling holding capacity within rehabilitated nurseries in Grenada increased by 50,000. This matched the target presented in the 2015 Project Paper for the AF. Institutions and administrative capacity strengthened  15 government officials in Grenada were trained in spatial data analysis under the project. This matches the target presented in the 2015 Project Paper for the AF.  23 government officials in SVG have completed training on producing location specific exposure maps. This matches the target of presented in the 2021 Restructuring Paper, up from eight in the PAD.  An Operations Manual has been prepared and an action plan of activities is being updated annually to ensure preparation to facilitate disbursement in the event of an emergency. This matches the target presented in the 2015 Project Paper for the AF.  36 functional hydromet stations in Grenada are providing data to a shared platform. This exceeds the target of 35 presented in the 2018 Project Paper.  Light Detection and Ranging (LiDAR) mapping for the entire territory of Grenada has been completed and is available on a shared platform. This matches the target presented in the 2015 Project Paper for the AF.  Public building geo spatial information has been collected in SVG. This matches the target presented in the 2015 Project Paper for the AF.  100 percent of public buildings in SVG are geo referenced in a national exposure database. This matches the target presented in the PAD.  A country-wide emergency communication network is in place. This matches the target presented in the 2015 Project Paper for the AF. Page 10 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871)  Capacity of six communities to respond to disaster events has increased. This matches the target presented in the 2015 Project Paper for the AF.  21 location specific exposure maps have been completed in SVG by staff trained under the project. This exceeds the target of 20 presented in the 2015 Project Paper for the AF.  Six designs and pre-engineering/geotechnical studies were completed for road protection and bridges rehabilitated under the project. This matched the end project target set out in the PAD.  54 designs and pre- engineering/geotechnical studies completed for roads and bridges rehabilitated under the project in SVG. This is double the end project target of 27 set out in the 2018 Project Paper, up from six in the PAD.  No government officials in Grenada were trained in forest management under the Project. Output not achieved - the end project target set in the 2015 Project Paper for the AF was eight. Intermediate outcomes  425 beneficiaries in SVG experienced reduced risk of flooding in areas with flood mitigation works financed by the project. This matched the end project target set in the 2015 Project Paper for the AF, up from 60 people in the PAD. The increase is due to the increased scope of flood mitigation works in SVG following the AF.  27,297 (13,327 female) people benefitted from reduced risk to failure of roads and bridges due to natural hazards or climate change impacts in SVG. This is 15 percent lower than the target of 32,156 (15,660 female) set out in the 2018 Project Paper, up from 10,500 in the PAD.  216 agriculture workers in SVG are benefitting from upgraded feeder roads. This is 52 percent higher than the target of 142 set out in the 2018 Project Paper.  1,002.3 farm acres in SVG are benefitting from upgraded access roads. This is 41 times the target set out in the 2018 Project Paper.  100 percent of activities in SVG have incorporated a beneficiary feedback system. This is double the target set in the 2018 Project Paper.  Country is eligible for insurance payment (and has received payment) in case of an insured event. This matches the target set in the 2013 Project Paper. The PDO indicator "direct project beneficiaries, of which female" was fully achieved. As targeted (in the 2015 Project Paper for the AF to Grenada), 110,000 people directly benefited from the project, 49 percent of whom were female. The PDO indicator "beneficiaries with reduced risk to failure of public buildings and infrastructure due to natural hazards or climate change impacts in SVG and Grenada" was exceeded by 43 percent. 1,093 people in SVG and Grenada experienced reduced risk to failure of public buildings and infrastructure due to natural hazards or climate change impacts. The target set out in the PAD was 764. The PDO indicator "beneficiaries with reduced risk to failure of public buildings due to natural hazards or climate change impacts in SVG" was exceeded by 39 percent. 445 people in SVG experienced reduced risk to failure of public buildings due to natural hazards or climate change impacts. The target presented in the PAD was 320. Page 11 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) The PDO indicator "beneficiaries with reduced risk to failure of public buildings due to natural hazards or climate change impacts in Grenada" was fully achieved. As targeted in the PAD, 444 people in Grenada experienced reduced risk to failure of public buildings due to natural hazards or climate change impacts. The PDO indicator "relocated low-income households with access to safe infrastructure addressed under the project" was fully achieved. As targeted (in the 2015 Project Paper for the AF), 41 low-income households were relocated to areas with safe infrastructure under the project. The PDO indicator "government officials from Public Works, Agriculture, Forestry, and NAWASA able to set up and run watershed analysis software" was fully achieved. 15 government officials from Public Works, Agriculture, Forestry, and NAWASA were able to set up and run watershed analysis software. This matches the target presented in the 2015 Project Paper for the AF. Moreover, the training activities and the installation are completed, and the network is fully operational. The PDO indicator targets were fully achieved or exceeded. Nearly all the project outputs, which contributed to the achievement of the outcomes were achieved or exceeded. As discussed in the theory of change above, the outputs are expected to result in the outcome of measurable reduction of vulnerability to natural hazards and climate change impacts in the SVG and in Grenada. The efficacy of the project in achieving this objective is Substantial. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Overall efficacy is rated Substantial. The objective of measurable reduction in vulnerability to natural hazards and climate change impacts in the Recipient’s territory and in the Eastern Caribbean Sub-region is rated Substantial. Overall efficacy is rated Substantial. Overall Efficacy Rating Substantial 5. Efficiency Economic efficiency. An economic analysis of the project was conducted at appraisal (PAD, page 12). The benefits were calculated using the estimated damage expenditures averted from a Category 1 hurricane, as expressed as a percentage of GDP, due to the mitigating investments of the disaster management project. The costs included the investment and the operations and maintenance costs of constructing, rehabilitating, and Page 12 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) retrofitting infrastructure and of strengthening institutions to better prepare for and respond to disasters. The assumptions and methodology used in the cost-benefit analysis were appropriate. The net present value (NPV) of the investments in SVG at appraisal was estimated to be US$37.5 million using a discount rate of 12 percent, and the economic rate of return (ERR) was estimated to be 92 percent. The NPV of the investments in Grenada at completion was estimated to be US$17.5 million using a discount rate of 12 percent, and the ERR was estimated to be 26 percent. The economic assessment at completion followed a methodology similar to the assessment done at appraisal (ICR, paragraph 59). The net present value (NPV) of the investments in SVG at completion was estimated to be US$22.6 million using a discount rate of 12 percent, and the economic rate of return (ERR) was estimated to be 44 percent. The lower NPV and ERR for the investments in SVG than estimated at appraisal are due to higher costs of investments than originally anticipated. The NPV of the investments in Grenada at completion was estimated to be US$18.6 million using a discount rate of 12 percent, and the ERR was estimated to be 24 percent. The results indicate that the project interventions are economically justified. Neither the PAD, nor the ICR provided an overall ERR for the project. Implementation efficiency. The project closing date was extended by three years and one month following the approval of the 2015 AF—required in part to cover a cost overrun—to allow adequate time to complete ongoing contracts, and to absorb the AF. Time overruns detracted from efficiency, given the opportunity cost of capital and the service fee that borrowers continued to pay on Bank loans. Overall, efficiency was modest, and well below the estimates at appraisal due to higher than originally foreseen investment costs. There were shortcomings in planning and design of some investments, procurement, and contract management, that delayed project implementation. The efficiency of the project is rated Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  0  Not Applicable 0 ICR Estimate  0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Page 13 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) With high relevance of objectives, substantial efficacy, and modest efficiency, the overall rating is Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The risk to development outcome is considered to be moderate. Two risks exist, Financial Risk. The project strengthened institutional capacities to manage infrastructure. However, financing must be secured for the continued operation and maintenance of the infrastructure constructed, rehabilitated, and retrofitted under the project. Institutional risk. The continued relevance of and use of the data sharing platforms requires focused attention on their upkeep and maintenance. In addition, national data policies are needed to that include access, sharing, and privacy considerations. 8. Assessment of Bank Performance a. Quality-at-Entry At project entry, the objective of achieving a measurable reduction in vulnerability to natural hazards and climate change impacts in the Recipient’s territory and in the Eastern Caribbean Sub-region was of high strategic priority. In addition, the approach of developing a regional project allowed for sharing of experiences and regional capacity building. However, the project was ambitious given weak capacity of the government agencies. Extensive supervision by the task team was planned at entry to manage the risks. The project design was based on considerable understanding of the risks and vulnerabilities of Caribbean infrastructure by the Bank’s preparation team. The project design was also informed by lessons learned from other Bank-financed projects working on similar issues. The key lessons adopted in the project design included (a) retrofitting, rehabilitation, and disaster risk mitigation investments contribute to resilience in the face of adverse natural events; (b) integration of hazard risk information in territorial planning and investment decision making is institutionally complex and information intensive, requiring simultaneous capacity building at various institutions and collaboration with academic and professional communities; (c) combining modeling with technical assistance is required to strengthen policy makers’ understanding of risk and the decisions needed to mitigate risk; (d) having access to insurance to cover the costs of emergency response and reconstruction can speed recovery from a natural disaster (PAD, paragraphs 42–45). The project’s approach was straightforward, with support for strengthening the resilience of infrastructure, coupled with support for building capacity of government institutions. The project activities were sufficient Page 14 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) to achieve the project objectives. The technical aspects of the project were sound. Overall project management, and coordination of implementation was appropriate, with the ministries of finance of SVG and Grenada having overall implementation responsibility, with line ministries (education, health, public works, and the like) provided advice and technical support (PAD, page 10). Mitigation measures for project risks, including provisions for extensive staff training, were appropriately built into project design. The results framework was generally well designed, including indicators that were aligned with operational objectives and that had baselines and appropriate targets. Safeguards and fiduciary arrangements were adequately designed to ensure compliance with Bank requirements. Safeguards instruments acceptable to the Bank were prepared and disclosed and capacity building was carried out. The project was designed efficiently as a regional project, given the planned Adaptable Program Loan trajectory at the time of appraisal, with both regional and country-level objectives. This design was a test of the concept that building regional linkages through data coordination and management would provide regional improvements as a positive externality and reduce the transaction costs of each country developing its own data management system. The PDO provided flexibility to SVG and Grenada to define their own priorities in risk reduction. The design was also intended to lower transaction costs for the government on project management and supervision by consolidating interventions in multiple sectors under a single project managed by a single agency in each country. However, the quality at entry did have shortcomings. Estimated costs proved too low, leading to some activities being under-budgeted. In addition, the teams did not initially budget for relocation of utility poles and municipal water lines. The under-budgeting led to cancellation of some activities, and delays in implementation of others as AF was mobilized. Finally, the assessment of the risks during implementation as medium-low overestimated the country’s implementation capacity. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Five task team leaders led project preparation and implementation over a period of ten years. However, each TTL was first involved as a team member prior to assuming the TTL role, so the handovers involved minimal disruption. Some 20 implementation support missions were held during project implementation, one every six months or so. Aide Memoires and Implementation Status and Result Reports were of high quality and candid, thoroughly covering all key issues and providing practical recommendations on how to address challenges, such as proposing options for river remodeling options in Grenada to lessen the private land to be acquired and therefore the costs of relocating people. The Bank team proactively revised the project’s performance ratings in response to changes in implementation progress. Supervision of safeguard and fiduciary aspects of the project was adequate and appropriate. The Bank team paid close attention to the fiduciary and safeguard and environmental and social aspects during project implementation, holding workshops and trainings, and addressing challenges. Issues were Page 15 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) identified and reported on and were addressed in a timely and constructive manner. The team provided good technical inputs and arranged technical experts to support the project. The Bank team proactively restructured the project multiple times, to respond to Grenada’s need for support with financial protection through CCRIF, bring onboard AF, adjust activities to reflect implementation challenges and emerging country priorities, amend the results framework in response to the AF, and to extend the project closing date. Recognizing the weak capacity of the governments, the Bank focused considerable resources on supervision. Specifically, in Grenada, the Bank team followed up closely with the government teams to ensure access to land was secured, and budgeting and other issues were resolved with technical assistance and training. The Bank team responded to the challenges and worked consistently with government counterparts to adjust activities to ensure the project met its objectives. In SVG, the Bank team agreed with the government on an action plan to ensure that communication was robust, and challenges were quickly identified, and solutions identified and implemented. Actions included technical missions every two months or so, conference calls every two weeks, and regular updates to an implementation plan providing timely information implementation progress. The Bank proactively identified and resolved challenges to the achievement of the project development outcomes. The team paid close attention to the fiduciary and safeguard and environmental and social aspects during implementation, holding various workshops and trainings, and addressing challenges. Issues were identified and reported on and were addressed in a timely and constructive manner. The team provided good technical inputs and arranged technical experts to support the project. Overall, the quality of supervision is rated satisfactory notwithstanding minor shortcomings in the proactive identification of and resolution of challenges. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The theory of change was generally sound, specifying how the key activities and outputs led to the outcomes as reflected in the results framework. The project development objectives of achieving a measurable reduction in vulnerability to natural hazards and climate change impacts in the Recipient’s territory and in the Eastern Caribbean Sub-region were clearly specified. Although the PDO indicators were measurable, they were nearly all at the output rather than the outcome level, and did not indicate how people’s lives changed as a result of the project. Page 16 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) The results framework was revised during the 2013 restructuring to introduce the CCRIF insurance payment, and again during the 2014 AF, the 2015 AF, the 2017 AF, and the 2018 restructuring to reflect changes in the project’s activities and the additional results expected due to the AF and specify some indicators more clearly. However, the revisions in the results framework did not address the weaknesses noted above. The intermediate results indicators, tracking outputs, such as kilometers of roads rehabilitated, were adequate to capture the contribution of the operation’s activities and outputs toward achieving the project’s objectives. The proposed data collection methods were adequate for all indicators. Overall, the project implementing agencies had the capacity to implement the M&E arrangements. b. M&E Implementation The implementing agencies of Grenada and SVG collected the data related to the activities, outputs, and outcomes for which they were responsible and reported them in quarterly progress reports. They also prepared project scorecards that formed part of the project progress reports, which provided clear and concise information of the project's implementation status and progress towards the PDOs. The implementing agencies prepared an annual M&E report which it presented to the Pilot Program for Climate Resilience. Reported results were discussed and verified during each mission. However, no attempts were made to document the outcomes that were not included in the results framework. For example, the project teams could have explained what changed as a result of agricultural workers and farms gaining access to upgraded roads, or of increasing water storage capacity. c. M&E Utilization The quarterly progress reports prepared by the Grenada and SVG Project Management Units provided detailed status updates on progress towards outputs and disbursements. The M&E system as designed and implemented was sufficient to assess the achievements of the project, producing outputs that plausibly contributed to the PDO of achieving a measurable reduction in vulnerability to natural hazards and climate change impacts in the Recipient’s territory and in the Eastern Caribbean Sub-region. However, there were shortcomings in capturing other aspects of the results chain, such as how the outputs strengthened capacity of the government to prepare for and address natural disasters and climate events. Additional data are required to monitor the changes that took place as a result of training, upgrading rural roads, increasing water storage capacity, and the like. These data could have been collected through surveys and other data collection methods. M&E Quality Rating Modest 10. Other Issues Page 17 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) a. Safeguards At appraisal the project was assigned Environmental category “B,” because works proposed under the project were largely rehabilitation and retrofitting of selected infrastructure and public buildings, and impacts were generally associated with the actual construction phase of the works activities. Four safeguard policies were triggered during preparation of the project: Environmental Assessment (OP/BP 4.01), Involuntary Resettlement (OP/BP 4.12), Natural Habitats (OP/BP 4.04), and Physical Cultural Resources (OP/BP 4.11). In January 2017, the project was restructured to trigger the safeguards policy for Pest Management (OP/BP 4.09), as incidental pesticide use was envisaged for some sub-projects. This need for the incidental use of pesticides was identified in early 2016, when the designs were finalized for the rehabilitation of public buildings. Environmental Assessment (OP/BP 4.01). The ISR dated June 21, 2021 stated that the overall safeguards compliance of the project was moderately satisfactory, because compliance with Involuntary Resettlement was moderately satisfactory. Compliance with the other three safeguard policies was satisfactory. Neither the ICR nor the final ISR of the project dated December 22, 2021 reported on safeguard compliance at project closing. The Governments of Grenada and SVF prepared and disclosed in- country and through the Bank's website a project level Environmental Assessment, including an Environmental Management Framework. They include reference to two groups of projects: those which would require a stand-alone Environmental Assessment and those comprising uncomplicated rehabilitation works where the impacts are limited to the construction phase (for example, repair and retrofitting). Involuntary Resettlement (OP/BP 4.12). The governments prepared and disclosed in-country and through the Bank's website on April 1, 2011 resettlement planning frameworks (RPFs). In Grenada, only a small number of people were affected in any significant way. In SVG, a Resettlement Policy Framework for the project was developed as well as a Grievance Redress Mechanism. There were several project activities which resulted in relocation, land acquisition, and involuntary resettlement. All the relocation plans, and Resettlement Action Plans were disclosed on the project’s page of the government’s websites. Natural Habitats (OP/BP 4.04). In SVG, the performance rating for OP4.04 Natural Habitats was rated Highly Satisfactory due to the execution of a rapid avian assessment for road works in a sensitive/complex area (Fenton Road—Green Hill). Physical Cultural Resources (OP/BP 4.11). The ICR does not report on compliance with this safeguard policy. Pest Management (OP/BP 4.09). The ICR does not report on compliance with this safeguard policy. b. Fiduciary Compliance Financial management. The Project Management Units in the respective countries were responsible for the financial management of the project in their respective countries. In Grenada and SVG, the units had substantial experience with World Bank fiduciary guidelines and were managing World Bank-financed projects. The two units had an adequate financial system which provided accurate and timely information, with reasonable assurance, that funds were being used for intended purposes. At project close, the units Page 18 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) were assessed by the Bank financial management specialists to have adequate financial management systems deemed to be functioning. They also assessed that they had moderate shortcomings in financial management, which did not prevent the timely and reliable provision of information required to manage and monitor the implementation of the project. The project closed with a moderately satisfactory rating for financial management (ISR, dated June 21, 2021). The ICR does not report on compliance with the Bank’s financial management policies. Procurement. The country’s Project Management Units were responsible for procurement. Both has experience with procurement under previous Bank-funded projects. The project closed with a moderately satisfactory rating for procurement (ISR, dated June 21, 2021). The ICR does not report on compliance with the Bank’s procurement policies. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons  Strengthening resilience across the Eastern Caribbean Region requires identification of activities, outputs, and outcomes at the regional level, not just at the country level. A clear vision and strategy for strengthening resilience across the region is required to ensure that activities at this level, for example for enhancing regional connectivity and data sharing across countries, are identified and financed.  Including instruments and dialogue on financing reconstruction and recovery in addition to investments in infrastructure can strengthen resilience to natural disasters. The Bank has been assisting the countries of the Eastern Caribbean Region to learn about and adopt innovative risk reduction and risk financing instruments, such as the CCRIF, and the Catastrophe Deferred Drawdown (CAT DDO), a contingent credit line that provides immediate liquidity to countries in the aftermath of a natural disaster. In addition, in both Grenada and SVG, a comprehensive disaster risk management dialogue with the Page 19 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) governments allowed for further strengthening of financial protection mechanisms and strategies through the regional Disaster Risk Financing Technical Assistance.  Having more accurate estimates of the costs of compensating project-affected people at the start of the project can help government’s budget for the expenses in a timely manner. The Government of Grenada faced challenges in budgeting for the costs of compensating people affected by various infrastructure projects. Having a clear understanding of the number of people likely to be affected and the costs of compensating them at the beginning of the project can help the government budget for those costs at the outset of the project implementation period.  Making use of a dedicated portfolio manager can result in more accurate cost estimates of project activities. Actual project costs were much higher than estimated at appraisal, leading to some activities being canceled while others were delayed while AF was mobilized to cover cost over-runs. Having a dedicated portfolio manager assess costs of various activities during appraisal in collaboration with all participating ministries and institutions may have led to more accurate cost estimates from the beginning. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provides a detailed overview of the project. It is clearly written and is largely consistent with the guidelines. It provides adequate details of the project’s activities, including a detailed annex summarizing the efficiency analysis. The ICR also provides a good theory of change analysis in regard to the causal links and the full results chain. However, the ICR did not report on compliance with several safeguards issues triggered for the project. Nor does it report on compliance with the Bank’s financial management policies and procedures, including the timeliness of submission of interim financial reports, the opinions of the external audits, and whether all outstanding funds had been returned to the Bank at project closure. Further, the ICR did not report on compliance with the Bank’s procurement policies and procedures. While cognizant of the shortcomings, the ICR is rated Substantial. a. Quality of ICR Rating Substantial Page 20 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 6O Regional Disaster Vuln Reduct. Projs (P117871) Page 21 of 21