Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004558 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF13694/TF13810/TF13869 ON GRANTS FROM THE GLOBAL ENVIRONMENT FACILITY LEAST DEVELOPED COUNTRY FUND EUROPEAN UNION’S GLOBAL CLIMATE CHANGE ALLIANCE GLOBAL FACILITY FOR DISASTER REDUCTION AND RECOVERY IN THE AMOUNT OF US$ 11.52 MILLION TO THE REPUBLIC OF VANUATU FOR THE INCREASING RESILIENCE TO CLIMATE CHANGE AND NATURAL HAZARDS PROJECT (P112611) December 27, 2019 Environment, Natural Resources, and Blue Economy Global Practice East Asia And Pacific Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective June 28, 2019) Currency Unit = Vanuatu Vatu 115.19 VUV = US$1 FISCAL YEAR January 1 - December 31 Regional Vice President: Victoria Kwakwa Country Director: Michel Kerf Regional Director: Benoit Bosquet Practice Manager: Ann Jeanette Glauber Task Team Leader(s): Habiba Gitay & Tevi Obed ICR Main Contributor: Sofia Bettencourt and Cindy Robles ABBREVIATIONS AND ACRONYMS ACP Africa, Caribbean and Pacific LAF Land Acquisition Framework ADRA Adventist Development and Relief Agency LDCF Least Developed Country Fund AGRIC Agriculture Micro-Project M&E Monitoring and Evaluation BRACED Building Resilience to Climate Extremes and Disasters MPC Multi-purpose Centers Micro-Project CBA Community Based Adaptation MTR Mid-term Review CCA Climate Change Adaptation N/A Not Available CCDRR Climate Change and Disaster Risk Reduction NAB National Advisory Board CDD Community Driven Development NACCC National Advisory Committee on Climate Change DARD Department of Agriculture and Rural Development NAP National Adaptation Plan DEPC Department of Environmental Protection and NAPA National Adaptation Programme of Action Conservation DFID Department for International Development NDMO National Disaster Management Office DGMWR Department of Geology, Mines and Water Resources NDSP National Sustainable Development Plan DLA Department of Local Authorities NGO Non-Governmental Organization DoWR Department of Water Resources NPV Net Present Value DRM Disaster Risk Management ODI Overseas Development Institute DRR Disaster Risk Reduction OECD Organization for Economic Cooperation and Development EA Environmental Assessment OP/BP Operational Policy/Bank Procedure EIA Environmental Impact Assessment PAA Priorities and Action Agenda EIRR Economic Internal Rate of Return PAD Project Appraisal Document ESMF Environmental and Social Management Framework PCRAFI Pacific Catastrophe Risk Assessment and Financing Initiative ESMP Environmental and Social Management Plan PDO Project Development Objective EU European Union PMU Project Management Unit EWS Early Warning System PO PDO Level Indicator FR All-weather Access/Feeder Roads Micro-Project PREP Pacific Resilience Program FY Fiscal Year PWD Public Works Department GCCA Global Climate Change Alliance RDP Rural Development Program GDP Gross Domestic Product RPF Resettlement Policy Framework GEF Global Environment Facility RWC Rainwater Catchment Micro-Project GEO Global Environment Objective Sq km Square kilometers GFDRR Global Facility for Disaster Reduction and Recovery TC Tropical Cyclone GFS Gravity-Fed Systems Micro-Project TF Trust Fund GoV Government of Vanuatu UNFCCC United Nations Framework Convention on Climate Change GNS Institute of Geological & Nuclear Sciences UNICEF United Nations Children’s Fund GRM Grievance Redress Mechanism US$ United States Dollar ICR Implementation Completion and Results (Report) VARTC Vanuatu Agricultural Research and Technical Centre ID Identification (Number) VMGD Vanuatu Meteorology and Geohazards Department IFR Interim (Unaudited) Financial Report VT Vanuatu Vatu IRI Intermediate Results Indicator WASH Water, Sanitation and Hygiene ISR Implementation Status and Results Report WHO World Health Organization IRCCNH Increasing Resilience to Climate Change and Natural € Euro Hazards TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL ......................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ........................................................... 10 II. OUTCOME .................................................................................................................... 13 A. RELEVANCE OF PDOs ......................................................................................................... 13 B. ACHIEVEMENT OF PDOs (EFFICACY) ................................................................................... 14 C. EFFICIENCY ........................................................................................................................ 21 D. JUSTIFICATION OF OVERALL OUTCOME RATING ................................................................. 23 E. OTHER OUTCOMES AND IMPACTS ..................................................................................... 24 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 25 A. KEY FACTORS DURING PREPARATION ................................................................................ 25 B. KEY FACTORS DURING IMPLEMENTATION.......................................................................... 26 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 27 A. QUALITY OF MONITORING AND EVALUATION (M&E) ......................................................... 27 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE.................................................. 29 C. BANK PERFORMANCE ........................................................................................................ 32 D. RISK TO DEVELOPMENT OUTCOME .................................................................................... 34 V. LESSONS AND RECOMMENDATIONS ............................................................................. 35 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 37 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 47 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 50 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 51 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 66 ANNEX 6. RESULTS FROM BENEFICIARY SURVEY .................................................................. 70 ANNEX 7. DETAILS OF MICRO-PROJECTS SUPPORTED ........................................................... 81 ANNEX 8. REVISED THEORY OF CHANGE .............................................................................. 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) *The correct project effectiveness date is April 24, 2013. The datasheet shows the effectiveness date of the preparation grant. DATA SHEET BASIC INFORMATION Product Information Project ID Project Name Increasing Resilience to Climate Change and Natural P112611 Hazards Project Country Financing Instrument Vanuatu Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Vanuatu Meteorology and Geohazards Department, Republic of Vanuatu Vanuatu Agricultural Research and Technical Centre (VARTC) Project Development Objective (PDO) Original PDO The objective of the Project is to help increase the resilience of communities in Vanuatu to the impacts of climate variability and change and natural hazards on food and water security as well as livelihoods. Revised PDO The PDO is to strengthen disaster risk management systems and pilot investments in select villages in the Recipient's territory to increase resilience to the impacts of natural hazards and climate variability and change. Page 1 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 150,000 139,645 139,645 TF-95486 2,730,000 2,730,000 2,730,000 TF-13694 5,580,000 5,580,000 5,337,043 TF-13810 2,035,583 1,372,310 1,624,432 TF-13869 Total 10,495,583 9,821,955 9,831,120 Non-World Bank Financing 0 0 0 Borrower/Recipient 435,895 0 0 EC: EUROPEAID Agency 2,774,105 0 0 Total 3,210,000 0 0 Total Project Cost 13,705,583 9,821,955 9,831,120 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 04-Dec-2012 22-Jun-2009 16-Nov-2015 31-Dec-2018 28-Jun-2019 Page 2 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 31-Mar-2016 2.29 Change in Implementing Agency Change in Project Development Objectives Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Safeguard Policies Triggered Change in Institutional Arrangements 23-Jun-2017 2.88 Change in Implementing Agency Change in Project Development Objectives Change in Results Framework Change in Components and Cost Reallocation between Disbursement Categories Change in Safeguard Policies Triggered Change in Legal Covenants 20-Dec-2018 4.97 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Modest RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 13-Apr-2013 Satisfactory Satisfactory .14 02 11-Nov-2013 Satisfactory Satisfactory 1.22 03 28-Jun-2014 Moderately Satisfactory Moderately Satisfactory 2.24 04 23-Jan-2015 Moderately Satisfactory Moderately Satisfactory 2.70 05 22-Jun-2015 Moderately Satisfactory Moderately Unsatisfactory 3.52 Moderately 06 31-Dec-2015 Moderately Unsatisfactory 4.78 Unsatisfactory Moderately 07 24-Apr-2016 Moderately Unsatisfactory 5.43 Unsatisfactory Page 3 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Moderately 08 11-Nov-2016 Moderately Unsatisfactory 5.91 Unsatisfactory Moderately 09 24-May-2017 Moderately Unsatisfactory 6.73 Unsatisfactory 10 20-Dec-2017 Moderately Satisfactory Moderately Satisfactory 7.66 11 27-Jun-2018 Moderately Satisfactory Moderately Satisfactory 8.30 12 12-Feb-2019 Moderately Satisfactory Moderately Satisfactory 9.36 SECTORS AND THEMES Sectors Major Sector/Sector (%) Agriculture, Fishing and Forestry 33 Fisheries 3 Crops 27 Livestock 3 Public Administration 1 Other Public Administration 1 Transportation 1 Other Transportation 1 Water, Sanitation and Waste Management 65 Water Supply 19 Public Administration - Water, Sanitation and Waste 42 Management Other Water Supply, Sanitation and Waste 4 Management Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Page 4 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Private Sector Development 100 Jobs 100 Finance 13 Finance for Development 13 Disaster Risk Finance 13 Urban and Rural Development 39 Disaster Risk Management 39 Disaster Response and Recovery 13 Disaster Risk Reduction 13 Disaster Preparedness 13 Environment and Natural Resource Management 50 Climate change 50 Mitigation 50 ADM STAFF Role At Approval At ICR Vice President: Pamela Cox Victoria Kwakwa Country Director: Ulrich Zachau Michel Kerf Director: John A. Roome Benoit Bosquet Practice Manager/Manager: Michel Kerf Ann Jeannette Glauber Project Team Leader: Samuel G. Wedderburn Habiba Gitay, Tevi Maltali Obed Teresa Sofia Urbano M De ICR Co Author: Bettencourt Page 5 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) . I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL 1. Country Context. Vanuatu, with a population of 240,000 at appraisal, comprises about 80 islands with a total land area of 12,336 square kilometers (sq. km), spread over 1,300 km from north to south in the Western Pacific Ocean. Geographically, it is in the “ring of fire” and at the center of the Pacific “cyclone belt”, and is characterized by high frequency of volcanic eruptions, cyclones, earthquakes, tsunamis, storm surges, coastal and river flooding, and landslides. At appraisal, the economic base was narrow, with some 65% of Gross Domestic Product (GDP) coming from small-scale agriculture. Around 80% of the population was rural and dependent on agriculture, with relatively low productivity. These characteristics, together with limited financial and technical capacity in relevant government agencies, made Vanuatu one of the most vulnerable countries in the world. 2. Sectoral Context. At appraisal, Vanuatu was already affected by climate variability and change, and these were likely to increase in the future. They included sea level rise, increased intensity of extreme events, changes to the dry season, and wet season rainfall, with some parts becoming drier and others experiencing increasing rainfall. Most sectors were deemed likely to be affected, particularly agriculture (which is mostly rainfed and could exacerbate the already declining land productivity due to decreased fallow periods) and water resources (which were already under stress from growing demand and inefficiencies in use). Since the wet season rainfall provided the majority of water supplies to the smaller islands on Vanuatu, floods and droughts had particularly devastating impacts on both food and water security. Droughts were especially damaging in the more remote islands that lacked sufficient rainwater harvesting and storage capacity to withstand dry periods. Other sectors like coastal and marine resources, infrastructure, and tourism were also likely to be affected by changes in temperature and precipitation as well as by sea-level rise, coastal storms, and saltwater intrusion. 3. Key stakeholders in Vanuatu had a high level of awareness of recent and anticipated variations and changes in the climate. At appraisal, Vanuatu was the only Pacific Island Country to have completed both a National Adaptation Plan of Action (NAPA) and a National Action Plan (NAP) under the United Nations Framework Convention on Climate Change (UNFCCC). The Government of Vanuatu’s (GoV) main development plan, the Priorities and Action Agenda (PAA) (2006- 15) was revised in late 2011, to include a section on managing climate change and disaster risks. These frameworks and plans paved the way for fundamental institutional reforms and new operational structures with mandates to mainstream disaster risk reduction (DRR) and climate adaptation into the government, and to ensure effective and efficient coordination between agencies through a multi-hazard approach. 4. Institutional context. Vanuatu already had a strong enabling environment for undertaking disaster risk management (DRM) and climate change adaptation (CCA) initiatives, though it needed further strengthening. The GoV had recognized that a closer integration of DRM and CCA was essential. It therefore decided in 2012 to combine the National Advisory Committee on Climate Change (NACCC) and the National Task Force for Disaster Risk Reduction and Disaster Management into one joint committee for DRM and CCA, called the National Advisory Board for Disaster Risk Reduction and Climate Change (NAB). The NAB was to be supported by a secretariat/project management unit located in the Vanuatu Meteorological and Geohazards Department (VMGD). This provided a unique opportunity for institutional strengthening under the project. Whilst no Country Partnership Strategy existed at appraisal, the project design was Page 6 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) consistent with the Bank’s 2012 Policy and Practice Note for Climate and Disaster Resilient Development for the Pacific Islands, which emphasized an integrated DRM and CCA approach. Theory of Change (Results Chain) 5. The Project Appraisal Document (PAD) did not present an explicit theory of change – for the purposes of this report, the latter has been derived from the project description and results framework in the PAD. The theory of change that underpinned the project development objective was that by strengthening the ability of national, provincial, and community-level stakeholders to promote the integration of DRR and CCA, resilience to disaster and climate change impacts would increase in rural communities. Figure 1 conceptualizes the ways in which the key activities, outputs, and outcomes contributed to the overall development goal. Project Development Objectives (PDOs)1 6. The original project development objective as per the Financing Agreement was to: help increase the resilience of communities in the Recipient’s territory to the impacts of climate variability and change and natural hazards on food and water security as well as livelihoods. Key Expected Outcomes and Outcome Indicators 7. The PDO can be broken into three outcomes to enable a separate assessment of achievements in line with the theory of change (see Figure 1): • Outcome 1: Increase the resilience of communities to the impacts of climate variability and change and natural hazards on food security • Outcome 2: Increase the resilience of communities to the impacts of climate variability and change and natural hazards on water security • Outcome 3: Increase the resilience of communities to the impacts of climate variability and change and natural hazards on livelihoods 8. The project’s performance against these outcomes was to be measured through four PDO indicators: • Farmers in areas targeted under the project have adopted climate resilient food crop production practices (target: 10,000); • Percentage of households in areas targeted under the project satisfied with the quality and security of water supply (target: 75 percent); • Households in areas targeted under the project incorporating climate considerations in their decision-making (target: 600) and; • National and sub-national institutions targeted by the project have functioning DRM arrangements (target:10). 1Note that the PDO of the project is the same as the GEF’s Global Development Objective and the wording is the same in both PA D and Financing Agreement. Page 7 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Figure 1: The Original Theory of Change of the Project Critical assumptions were as follows: (1) NAB is providing adequate oversight and guidance to the participating agencies; (2) Government of Vanuatu agencies and partners have staff who are trained and willing to support communities to improve their resilience; (3) disaster warnings are effectively communicated to the public and increasing level of disaster preparedness in communities practiced; (4) community level investments are completed and properly maintained (by communities and/or local government); and (5) community sub-projects are adequate to the magnitude of climate variability and change and disaster risks. Page 8 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Components 9. The project had four components. The original cost (in United States Dollars - US$) of US$ 11.52 million was financed by a US$5.58 million grant from the Least Developed Countries Fund (LDCF) through the Global Environment Facility (GEF) – designed to support the implementation of the NAPA priorities including on food security and rainwater harvesting. It was co-financed by a grant of 2.11 million Euros (US$2.77 million equivalent) from the European Union (EU)’s Global Climate Change Alliance (GCCA), a grant of US$2.73 million from the African, Caribbean and Pacific (ACP)- EU Natural Disaster Risk Reduction Program managed by the Global Facility for Disaster Reduction and Recovery (GFDRR), and in-kind contribution of US$0.44 equivalent provided by GoV and beneficiaries mostly as in-kind labor and local material (Annex 3). 10. Component 1: Institutional Strengthening for Climate Change Adaptation and Disaster Risk Management (US$4.37 million, actual US$4.84 million) supported: (1) strengthening of the Project Management Unit (PMU) by building capacity in project management, procurement, financial management, and monitoring and evaluation; (2) strengthening the National Disaster Management Office (NDMO) including review of the National Disaster Management Act, facilitating learning and twinning with national disaster management offices of other Pacific Island Countries, establishing provincial disaster coordination centers in Tafea and Torba provinces, and developing volcano contingency plans for some volcanic islands; and (3) strengthening early warning systems through a real-time data communication network linking volcano, seismic, and weather observing stations with the national data and warning center, and strengthening multi-hazard standard operating procedures. Fiduciary Implementing Agency: VMGD, through NAB’s Secretariat/PMU, with implementation of Component 1.2 through NDMO. 11. Component 2: Increasing Community Resilience on Active Volcanic Islands and in Coastal Areas (US$2.43 million; revised US$2.61 million; actual US$2.62 million) supported increasing the ability of stakeholders at national, regional, and community levels to work together to enhance disaster and climate resilience in rural communities by: (1) strengthening methods, processes, and protocols for climate change adaptation and disaster risk management; and (2) enhancing community and ecosystem-based adaptation and disaster risk management in coastal areas and on active volcanic islands by implementing said methods and plans through small grants to communities for the financing of pilot community-based activities; these activities were designed to reduce vulnerability to climate and weather-related risks, as well as geohazards. Fiduciary Implementing Agency: VMGD, with implementation of Component 2.2 through Department of Local Authorities (DLA). 12. Component 3: Promotion of Improved Technologies for Food Crop Production and Resilience to Climate Change (US$2.48 million; revised US$1.54 million; actual US$1.42 million), supported development of approaches, strategies and technologies for CCA and responding to food security and market demand by: (1) producing and distributing improved plant genetic material; and (2) distributing improved agricultural technologies to support on-farm production and resilience to climate change. Fiduciary Implementing Agency: Vanuatu Agricultural Technology Centre (VARTC), with Agriculture and Rural Development (DARD) as implementing agency for component 3.2. 13. Component 4: Rural water security (US$2.24 million; revised US$1.06 million, actual US$1.05 million) supported increased access to secure water supply through: (1) completing the national inventory of rural water systems, updating rural water supply standards and guidelines for rural water supply, and improving outreach to partners working in the sector; and (2) installing rain water catchment and storage systems to supply safe water to rural communities. Fiduciary Implementing Agency: VMGD, with Department of Geology, Mines and Water Resources (DGMWR) as implementing agency. Page 9 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) B. SIGNIFICANT CHANGES DURING IMPLEMENTATION 14. In 2017, the project underwent a level 1 restructuring, changing the PDO, scope of activities, components, component costs, implementation arrangements, results framework, safeguards triggered, legal covenants, categories of expenditures, and closing date. Changes to the PDO and components are summarized in Table 1 below. Table 1: Summary of Key Changes in the Level 1 Restructuring Original New/Revised PDO Original: To help increase the resilience of communities in the Revised: Strengthen disaster risk management systems and pilot Recipient's territory to the impacts of climate variability and investments in select villages in the Recipient’s territory to increase change and natural hazards on food and water security as well as the resilience to the impacts of natural hazards and climate livelihoods. variability and change Components and Implementation Arrangements 1. Institutional Strengthening for Climate Change Adaptation and Added 1.3. Strengthen capacity of national multi-hazard warning Disaster Risk Management and emergency operations center to respond to disasters 2. Increasing Community Resilience on Active Volcanic Islands and Revised to read: 2. Increasing Community Resilience in Areas Coastal Areas (implemented through small grants) Affected by Tropical Cyclone Pam (as part of micro-projects) 3. Promotion of Improved Technologies for Food Crop Production No change to description, but change to implementation agency and Resilience to Climate Change (implemented by VARTC/DARD) from VARTC to VMGD and activities integrated in micro-projects 4. Rural Water Security (implemented by DGMWR) No change to description but activities integrated into micro- projects Revised PDOs and Outcome Targets 15. Following the level 1 restructuring, the revised PDO became as follows: Strengthen disaster risk management systems and pilot investments in select villages in the Recipient’s territory to increase the resilience to the impacts of natural hazards and climate variability and change. 16. The revised PDO was broken down to the two outcomes below to enable a separate assessment of achievements: • Post-Restructuring Outcome 1: Strengthen disaster risk management systems to increase the resilience to the impacts of natural hazards and climate variability and change • Post-Restructuring Outcome 2: Pilot investments in select villages to increase their resilience to the impacts of natural hazards and climate variability and change Revised PDO Indicators 17. The results framework was revised, resulting in the deletion of two original PDO-level indicators, revisions to the first PDO-level indicator to include all direct beneficiaries from the micro-projects, and revisions or deletion of most intermediary indicators (IRI) as summarized in table 2. The PDO indicator “% of households satisfied with the quality and security of water supply (70)” was revised to “people provided with access to improved water sources (1,500)” to reflect people that benefitted from repairs to their water systems after TC Pam (and different from the beneficiaries of new water-related micro-projects reflected in the first revised PDO indicator). Page 10 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Table 2: Overview of Revised Indicators at Restructuring2 Original Revised Original Indicators New/Revised Indicators Target Target PO-1: Farmers have adopted climate resilient food crop production Revised: No. of beneficiaries directly 10,000 3,000 practices supported by the project, disaggregated by: 1,400 (1) gender 1,800 (2) water-related investments3 1,000 (3) agriculture-related investments PO-2: % of households satisfied with quality and security of water Revised: People provided with access to 75% 1,500 supply improved water sources4 PO-3: # households incorporating climate considerations in decision Deleted 600 N/A making PO-4: # national and sub-national institutions targeted with Deleted 5 N/A functioning DRM arrangements IRI-1: Capacity of Secretariat/PMU to provide effective oversight Revised: Strengthened institutional structures Satisfactory High and guidance for CCA/DRM and capacity to deliver climate-resilient investments at provincial and local levels IRI-2: Availability of Operational Manuals providing No change Available 2 procedures/protocols for CCA/DRM IRI-3: Guidelines/manuals/tools based on good local and Revised: Same except ….”used at national and Guidelines 3 international practices developed and widely used provincial level” used IRI-4: No. of communities with climate resilience activities Revised: No. of villages receiving climate and 4 65 disaster resilient investments IRI-5: Effective collaborative research and extension between Deleted 5 N/A VARTC and DARD (no. of multiplication and delivery mechanisms) IRI-6: Farmers/farmer leaders trained and/or contacted to improve Deleted 1,500 N/A cropping practices IRI-7: No. Technologies demonstrated in project areas Deleted 50 N/A IRI-8: No. of people in rural areas provided with access to improved No change5 6,000 6,000 water sources IRI-9: % of NGOs/churches involved with rural water supply adopt Deleted 75 N/A RWS standards IRI-10: No. of national and sub-national DRM strategies, plans and Deleted 10 N/A standard operating procedures developed and tested IRI-11: Provincial Disaster Centers fully equipped and operational No change 2 2 IRI-12: % data communication links established to volcano sites No change 70 100 Not in original New: Investments in Post TC-Pam recovery N/A 8 Not in original New: No. of water systems repaired N/A 8 Not in original New: No. of early warning system stations N/A 8 established and/or repaired Revised Components 18. As summarized in Table 1, the major changes were to Component 2 in support of climate-resilient micro-projects. Activities under Component 3 and Component 4 were to be similarly delivered through the micro-project mechanism. 2 Abbreviations Used: PO – PDO-level indicator; IRI- Intermediary Results Indicator; No/# - number; N/A – Not applicable. 3 This indicator only measures beneficiaries that benefitted from the micro-projects. 4 This indicator only measures beneficiaries that benefitted from repaired water systems after TC Pam. 5 This indicator should have been deleted at restructuring as the two other indicators measured beneficiaries from new water-related micro- projects and water systems repaired after TC Pam (as explained above). Page 11 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Disaster affected locations (Tanna Island, Shepherds Islands, and later Ambrym) were prioritized. Changes to component costs are shown in paragraphs 10-13. Other Changes 19. In addition to changes in the PDO, indicators, components and component costs, the level 1 restructuring included changes to the implementation arrangements, safeguards triggered, legal covenants, categories of expenditures, and closing date (from the original December 31, 2017 to December 31, 2018). 20. Implementation arrangements were simplified. Whereas at appraisal, there were two separate fiduciary executing agencies (VMGD and VARTC) and multiple implementing agencies, after restructuring, VMGD became the only executing agency, responsible for the implementation of all the components with support from the responsible technical agencies (according to their mandates). 21. A new safeguard - Operational Procedure (OP)/Bank Procedure (BP) 4.12 on Involuntary Resettlement - was triggered at restructuring (see Safeguards Compliance section). 22. The legal covenants and categories of expenditures were amended to remove the terms “small grants to communities” and “sub-projects”, to reflect the change in delivery mechanism to micro-projects. A new dated covenant was also added to reflect the decreased funds provided to VARTC. 23. In addition, the project had two level 2 restructurings: in 2016,6 to extend the GFDRR trust fund closing date from March 21, 2016 to December 31, 2017, and in 2018, to extend the GEF Grant from December 31, 2018 to June 28, 2019. Rationale for Changes and Their Implications on the Original Theory of Change 24. The level 1 restructuring took place as a response to a request from the GoV to Tropical Cyclone (TC) Pam – a Category 5 storm, which hit Vanuatu in 2015 – severely affecting the central and southern islands and devastating one of the most densely populated and poorest islands, Tanna. Eleven people died, and damages to infrastructure, food security, and livelihoods were estimated at about US$300 million, equivalent to nearly two-thirds of the country’s GDP. 25. The rationale for the changes was based on GoV’s request to simplify the project and deliver support to the communities affected by TC Pam through resilient micro-projects. TC Pam led to a shift in GoV’s priorities away from communities in active volcanic islands and coastal areas, to supporting communities affected by TC Pam with resilient activities. As the small grant mechanism was yet to be implemented due to slow implementation and logistical challenges (see Section III.B), GoV requested that it be replaced by community-level micro-projects7 to accelerate support to the communities affected by the cyclone. This changed the project’s geographical and sectoral scope. The revisions of the PDOs reflected the new priorities more explicitly: to strengthen DRM systems (not reflected well in the original PDO) and 6 The datasheet incorrectly repeats the changes made in 2017 onto the March 31, 2016 restructuring. In reality, the latter was only an extension of closing date (level 2). 7 The designation “micro-projects” was adopted by the project to distinguish it from the community-driven development (CDD) mechanism originally considered for Component 2.2. Instead of small grants provided directly to communities, the PMU adopted bulk procurement for goods and services needed for the micro-projects, whilst communities provided local materials and in-kind labor as needed. Priority areas - in Shefa Province and North, East and South East Tanna (Tafea Province) - were selected by respective provincial governments and community leaders based on TC Pam rapid damage assessments. A participatory baseline survey then collected information on community vulnerability and hazard risks and engaged communities in a consultative process to prioritize key resilient actions /solutions. These were developed into micro-project proposals and submitted to the Provincial Government’s Technical Advisory Committees for selection and project funding. Page 12 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) strengthen the resilience of communities (affected by TC Pam) through pilot investments. The revised theory of change resulting from the Level 1 restructuring is summarized in Annex 8. 26. The closing date extensions were justified based on the initial delays in recruiting key staff for the PMU as well as TC Pam disruptions. The other changes in the level 1 restructuring reflected the changes in the activities that were introduced. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 27. The relevance of the PDO is rated as High, given the country’s extreme vulnerability to natural hazards and the impacts of climate change. Vanuatu has the second highest estimated average annual loss from disasters (6.6% of GDP) of all Small Island Developing States.8 TC Pam, for example, was one of the strongest tropical cyclones ever recorded in the Pacific, and its impacts were further exacerbated by a seasonal drought later in 2015. In 2016-17, the country also experienced a series of volcanic eruptions in Ambae resulting in evacuation of the 11,000 people on two occasions within eight months. Finally, Category 4 cyclone TC Hola hit Vanuatu in 2018, causing significant damage to infrastructure and livelihoods, especially in Ambrym. These events highlight Vanuatu’s vulnerability and the relevance of the PDO. 28. The project was highly relevant to GoV’s national plans and strategies. These include the revised PAA (2006-2015), as well as Vanuatu’s National Sustainable Development Plan (NDSP, 2016-2030) – specifically, Objective 3 of the Environment Pillar, to achieve “a strong and resilient nation in the face of climate change and disaster risks posed by natural and man-made hazards”. The project is directly aligned with four of the five NDSP policy areas: (1) Institutionalize climate change and disaster risk governance and build institutional capacity and awareness; (2) Improve monitoring and early warning systems; (3) Strengthen post-disaster systems in planning, preparedness, response and recovery; and (4) Promote and ensure strengthened resilience and adaptive capacity to climate related, natural and man-made hazards. The project is further aligned with the strategic goal of the Vanuatu Climate Change and Disaster Risk Reduction Policy 2016-2030 to make “Vanuatu a resilient community, environment and economy”. 29. The project was highly relevant to GEF’s global LDCF objectives. The project directly supported two of the highest priorities in Vanuatu’s NAPA, namely: (1) agriculture and food security, and (2) water management policies and programs (including rainwater harvesting). It also contributed to all three objectives of the LDCF adaptation strategy9, as documented in the Efficacy section. 30. The PDO is also consistent with the current World Bank’s Regional Partnership Framework and its focus on climate and disaster resilient development for the Pacific. The alignment with the Regional Partnership Framework for Fiscal Years (FY) 2017-21 falls under Focus Area 3: “Protecting Incomes and Livelihoods”, which emphasizes strengthened 8 OECD, World Bank (2016). Climate and Disaster Resilience Financing in Small Island Developing States. Organization for Economic Co-operation and Development (OECD) and World Bank. Washington DC. 9 Global Environmental Facility (2010). Revised Programming Strategy on Adaptation to Climate Change for the LDCF and the Special Climate Change Fund. GEF Secretariat, Washington DC. Page 13 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) preparedness and resilience to natural disasters and climate change as one of three fundamental constraints to ending absolute poverty and increasing shared prosperity. Finally, the PDO is consistent with the Bank’s Policy and Practice Note for Climate and Disaster Resilient Development for the Pacific Islands Regions (2012). B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 31. Despite the challenges, the project achieved notable results. It substantially achieved the revised outcome indicators and intermediate results targets, with several exceeded. The project also measured and achieved one of the original PDO indicators (beneficiary satisfaction with the quality and security of water supply). Given that the PDO was revised during the 2017 level 1 restructuring, a split evaluation of efficacy was applied to assess the performance of the project. The original and revised outcomes (post-restructuring) were unpacked and assessed separately. Some of the post TC Pam activities (such as the water and agriculture micro-projects) continued to contribute to the original PDO and are therefore also discussed as part of the original outcome statements. Outcome 1: Increase the resilience of communities to the impacts of climate variability and change and natural hazards on food security 32. The project contributed to increasing food security in Vanuatu by supporting farmers to adopt climate resilient food crops and production practices. According to project records, an estimated 5,200 farmers (52% of the original outcome target) received planting materials and support to crop production practices that were more resilient to droughts (such as mulching and multi-cropping) and floods. This included 1,800 beneficiaries in the immediate aftermath of TC Pam (from distribution led by VARTC), and 3,394 through the demonstration plots included as micro- projects (Table 3). Improved food crop varieties such as kumala (sweet potato), taro, manioc and yams were tested, and kumala and manioc, in particular, were multiplied in the demonstration plots and distributed to households/farmers. In addition to direct beneficiaries, it is estimated that an additional 5,000-10,000 farmers benefited indirectly10 through distribution of material from households to households (and not directly from the project). 33. The results for agriculture-related activities are also a good indication on the effective collaboration between research (VARTC) and government agencies (DARD) in achieving the relevant IRI target.11 Before the restructuring, eleven demonstration plots had been established in the island of Espiritu Santo to represent the climate of the north of Vanuatu, as well as two on Efate for the south of Vanuatu. Once the micro-project approach was adopted after the level 1 restructuring, 14 agriculture demonstration plots were established on Tanna Island with communities responsible for maintenance and distribution of the cuttings to other farmers. Four additional demonstrations plots were established in Espiritu Santo and Efate, increasing the total of agriculture-related micro-projects to 18, according to project records. 10 No ascertained data, information based on village and markets observation by PMU and Non-Government Organization (NGO) partners. 11 IRI-5: Effective collaborative research and extension between VARTC and DARD as measured by set up of multiplication and delivery mechanisms in main food areas. Page 14 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) 34. Twelve workshops/trainings were provided to farmers, field workers, extension officers, and NGO partners to maximize the benefits associated with the improved crop varieties.12 Several of the training participants received kits of planting materials. In addition, the project conducted three interviews on national radio promoting new improved varieties of kumala and manioc and raising awareness of the benefits of adopting climate resilient food production practices. 35. As part of the TC Pam recovery support, about US$300,000 worth of agriculture tools and trialed selected crops were distributed. The tools were distributed through the DARD agriculture extension network. According to the GoV’s completion report, approximately 176,000 people accessed the tools which helped them build back their gardens after the cyclone. Outcome 2: Increase the resilience of communities to the impacts of climate variability and change and natural hazards on water security 36. An estimated 21,843 people accessed improved water sources,13 through both repairs of existing water systems after TC Pam (9,456 beneficiaries) as well as through new systems supported through micro-projects (12,387 beneficiaries). The latter is double the original IRI target of 6,000. In the aftermath of TC Pam, the project helped repair 5 schemes in Tongoa and 5 in Tanna Island (including upgrades and extensions to the original scheme size) and distribute 20 water tanks in Efate. In addition, a total of 37 new water-related micro-projects (22 rainwater harvesting systems and 15 gravity-fed systems) were supported through the project (see Table 3). 37. The beneficiary survey conducted at the end of the project indicated that 89% of the respondents were satisfied/very satisfied with water quality, and 77% were satisfied/very satisfied with water quantity - exceeding the original outcome indicator target of 75% (see Annex 6). All the water systems are still in use and have reportedly been operating satisfactorily since their completion in 2018-early 2019. They were being maintained by community water committees and using the training received on operations and maintenance supported by the project. 38. Several outputs contributed to enhancing resilience and water security including the completion of water inventories for Torba, Shefa and Penama provinces to determine current coverage and future priorities; the establishment of a water resource database; and the development of manuals and guidelines for rainwater harvesting systems. The database also includes the number of direct and indirect gravity fed systems and hand pumps, to facilitate the future feasibility studies and delivery of water investments, thereby contributing to Outcome 3.2 of the LDCF Strategy “enhanced enabling environment to support adaptation-related technology transfer”. 39. Water-related investments – including rehabilitated systems after TC Pam - were implemented with resilience and safety standards. The rainwater harvesting systems were designed with adequate storage to operate during drought periods: for rainwater catchment systems, an average of 7 liters/person/day (for drinking and cooking). Gravity fed systems were designed to provide 50 liters/person/day for washing, bathing, drinking and cooking – representing a 2-3-fold increase in water consumption from traditional sources. The structures (buildings, storage tank towers, rainwater harvesting roofs, and water tanks) were designed to withstand earthquake and cyclone wind loads. In addition, the water tanks were elevated to ensure protection against floods and groundwater wells were built high to 12 IRI-6, farmers/farmers leaders trained and/or contacted on ways to improve cropping practices was not monitored. It was therefore only possible to document the number of farmers trained, or records of the number of farmers who received kits. 13 During appraisal, the project did not anticipate beneficiaries from repaired systems, only and new water supply systems. This was changed at restructuring where a differentiation was made due to rapid response after TC Pam and reflected in the revised results framework. Page 15 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) avoid flood water contamination and strapped down to minimize damage during cyclones. The gravity fed systems were also designed to minimize the risk and damage from landslides and with the involvement of the community representatives to ensure regular checks and maintenance. Communities were trained in monitoring water safety, and in regularly cleaning spring sources and water tanks to minimize contamination. By providing water that is accessible on premises, available when needed, and free from contamination, the water micro-projects met the United Nations Children’s Fund/World Health Organization (UNICEF/WHO) standards for safely managed water services. 40. Operation and maintenance plans for each of the micro-projects were prepared to ensure their sustainability. The DGMWR conducted community training on maintenance (including specialized plumber training), operating costs, roles and responsibilities of users, fees, and maintenance timelines. Outcome 3: Increase the resilience of communities to the impacts of climate variability and change and natural hazards on livelihoods14 41. The project contributions and expected results on climate and disaster resilient livelihoods were not well- reflected in the results framework.15 The results highlighted below are therefore based on the beneficiary surveys and government feedback which collectively show the project’s contribution to improved livelihoods. 42. The agriculture and water- related activities helped restore the livelihoods of the severely affected population (see beneficiary survey section). The repair and replacement of water systems after TC Pam supported a quicker recovery for the affected population and helped reduce the time that women spent in collecting water. The time saved was spent on gardening and preparing more items for sale, thus contributing to enhanced livelihoods.16 The crops that were tested and distributed through the project were mainly for local consumption and not for cash, but an NGO partner (World Vision) confirmed that the varieties of kumala and manioc improved through the project (identified by their distinctive color and shape) were being sold in large quantities in markets in Tanna and Port Vila, which indicates that the crops have been introduced to the local markets, thus contributing to the cash income of the population. It also provides an indication of the project’s contribution to Outcome 3.1 of the LDCF Strategy “successful demonstration, deployment and transfer of relevant adaptation technology in targeted areas”. 43. The all-weather access roads supported through the project contributed to enhanced livelihoods through improved access to markets. Prior to the project, trucks and buses had difficulties accessing remote upland villages because of poor roads worsened by bad weather conditions, which made it difficult for farmers to sell their products year-round. Several beneficiaries confirmed that the access roads had contributed to a more stable income and for many, increased income throughout the year. Based on interviews with beneficiaries of both access roads and water- related micro-projects, the incremental revenues derived from these market sales are in the order of 3-68% of the average monthly income for rural Tafea, the province where Tanna Island is located (see Section II.E). 44. There is some evidence that the project contributed to income diversification, and through it, more resilient livelihoods. Some beneficiaries interviewed had expanded their sources of income to include pen-reared pigs, tomatoes (high value crop) for sale to Port Vila, handicrafts, and food for sale at local bus stops – made possible by the time saved 14 The PDO indicator number of households in areas targeted under the project incorporating climate considerations in their decision-making was not monitored or measured. It was nevertheless not a good indicator to measure resilient livelihoods. Hence, the ICR team had to rely mostly on the beneficiary survey to produce some evidence on this outcome. 15 A resilient livelihood is “regarded as sustainable (resilient) when it can cope with, and recover from, external shocks and stresses, and maintain or expand its asset base”. Source: https://goodpracticereview.org/9/livelihoods-and-drr/sustainable-livelihoods-approaches/. 16 See more details in the efficiency analysis (See Annex 4). Page 16 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) in not having to collect water, by better access roads, and by new venues provided by the multi-purpose centers. These opportunities also contributed to improved savings and avoided depending on a single source of income that could be destroyed by disasters and variable weather. The complementary evidence above supports a positive achievement for the project under Outcome 3, and also illustrates its contribution to Outcome 1.3 of the LDCF Strategy “diversified and strengthened livelihoods and sources of income for vulnerable people in targeted areas”. 45. The remaining of this section discusses achievement of the revised PDO outcomes. Post-Restructuring Outcome 1: Strengthen disaster risk management systems to increase the resilience to the impacts of natural hazards and climate variability and change 46. The project contributed highly to strengthening GoV’s institutional structures and capacity to deliver climate- resilient investments, meeting the revised IRI-1 through various means. These included: (1) building the NAB’s Secretariat/PMU’s core capacity in project management, procurement, financial management, safeguards, and monitoring and evaluation, enabling the PMU to provide oversight and guidance (also consistent with the original IRI-1 target). Through this, the project contributed to the Secretariat’s establishment and strengthened NAB’s effectiveness. The NAB Secretariat’s mandate has now been formalized under the new Meteorology, Geological Hazards and Climate Change Act of 2016; (2) supporting the revision of the National Disaster Management Bill through the services of a legislation review specialist financed under the project. The new Bill, approved by Cabinet in October 2019 and now awaiting promulgation, supersedes the Disaster Act of 2000 which focused solely on emergency preparedness and response. The Bill also strengthens the regulatory framework and institutional capacities for multi-hazard, ex ante and ex post DRM, includes details on ensuring smooth transition from response to recovery, legalizes provincial and municipal climate change and disaster committees, establishes an emergency fund, and formalizes the new coordination and operational structures - particularly in decentralizing DRM services from national to provincial and local levels; (3) supporting the newly established Climate Change Department within the new Ministry of Climate Change through training, building construction, and equipment – thus helping the GoV move towards an integrated approach for climate resilience and DRM; and (4) mainstreaming DRR and CCA through a review of sector plans and policies in six sectors: agriculture, livestock, public works, energy, health and forestry which contributed to the NSDP. Through this, the project also contributed to Outcome 1.1 of the LDCF Strategy, to “mainstream adaptation in broader development frameworks at country level and in targeted vulnerable areas”. 47. To support GoV with decentralization of DRM, the project helped build and equip two provincial disaster centers, on Tanna Island in Tafea Province and on Sola in Torba Province, meeting the IRI-11 target of two. The provincial disasters centers became fully operational and actively provided support to the communities for micro-project implementation. They have also been used for awareness raising activities on climate and disaster resilience. The center on Tanna hosted most of the response and recovery coordination meetings following TC Pam in 2015-17. 48. Three operations manuals were produced in line with the revised IRI target of two. They included the project operations manual; a training of trainers’ manual for community development programs which the PMU reports has been used widely; and a micro-project appraisal manual which is being further developed by the DLA as a guide for implementation of community-level projects. 49. The project helped build Vanuatu’s volcanic/seismic monitoring network and link it automatically to the national multi-hazard warning center, meeting IRI-12. In total, eight new seismic stations were established on the islands of Gaua, North Ambae, Maewo, Santo, Malekula, Lopevi and Efate. In addition, one existing site on Tanna was repaired Page 17 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) after it was damaged by TC Pam, exceeding the revised IRI target of eight. The stations were fully operational and were automatically transmitting real-time data to the national multi-hazard warning center. As a result, Vanuatu’s seismic data has been recognized and used globally by international organizations such as the Hawaii Center and Meteo France. The support to the multi-hazard early warning system and seismic monitoring network also resulted in faster detection of geo-hazards, as evidenced by the critical role played in the ongoing Ambae volcano eruption, alerting the need for evacuation for the southern islands of Vanuatu immediately after the 7.6 magnitude earthquake struck New Caledonia on December 4, 2018. 50. The project co-financed the establishment of the national multi-hazard warning center, including refurbishment and equipment for the Forecast Division, leading to a significant improvement in operations through real-time and more accurate data. Through this project contribution, Vanuatu now has an internationally recognized warning center and emergency operations center operated by trained ni-Vanuatu nationals. Post-Restructuring Outcome 2: Pilot investments in select villages to increase the resilience to the impacts of natural hazards and climate variability and change 51. The resilient micro-projects reached about 28,800 beneficiaries, substantially exceeding the revised PDO target of 3,000. Of these, 13,800 were women (against a revised PDO target of 1,400); about 12,370 beneficiaries benefitted from water-related investments (against a target of 1,800); and 3,930 beneficiaries received support through agriculture-related micro-projects, exceeding the revised PDO indicator of 1,000. An additional 3,550 beneficiaries benefitted from multi-purpose centers and roughly 9,000 beneficiaries benefitted from all-weather access feeder roads (see Table 3). The significantly higher-than-expected number of beneficiaries was both due to the actual project achievements as well as an under-estimation of the population and thus potential beneficiary numbers at restructuring (see Section IV.A). 52. A total of 67 villages received climate and disaster resilient investments through the micro-projects, exceeding the revised PDO target of 65. In addition to the agriculture and water-related investments that continued and were expanded from the pre-restructuring phase, the project also supported micro-projects to build all-weather access roads (or feeder roads) and multi-purpose centers as they were identified as priorities by the provincial governments and communities. A summary of the 67 micro-projects is provided in the Table 3 and the full details in Annex 7. By implementing resilient micro-projects in multiple sectors, the project also contributed to Outcome 1.2 of the LDCF Strategy, “reduced vulnerability to climate change in development sectors”. Table 3: Summary of Community Micro-Projects Types of micro-projects Number of micro-projects Number of beneficiaries Total Cost (US$) Agriculture 18 3,934 112,014 Rainwater Catchments 22 2,931 189,876 Gravity Fed Systems 15 9,456 843,159 Multi-Purpose Centers 6 3,551 151,505 Feeder Roads 6 8,957 721,923 Total 67 28,829 2,018,478 Source: PMU project monitoring records Page 18 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) 53. To inform the selection17 and prioritization of the micro-projects, the project developed 230 village profiles in Tanna Island enabling micro-projects to be selected based on a participatory prioritization process informed by local knowledge. Prior to the project, Vanuatu did not have a selection mechanism to prioritize community climate and disaster resilient needs or investments. The participatory process (see footnote 7), involving both communities, village leaders and provincial governments, was critical to the delivery of the micro-projects and helped meet the revised IRI target of “strengthened institutional structures and capacity to deliver climate-resilient investments at the provincial to local levels”. Priority 54. The multi-purpose centers and feeder roads were built with resilient standards to address/minimize climate and disaster risks. The multi-purpose center buildings were designed to be cyclone-resistant and serve as shelters during disasters. The design was developed after TC Pam and has now been adopted by the Public Works Department (PWD) and the DGMWR. The access roads were built according to PWD standards used in steep and heavy rainfall terrain and constructed as cement pavements to facilitate replacement. In addition, on steep sections, 1-meter wide drains were included, and culverts placed to improve drainage and reduce the risk of undermining the pavement during heavy rains. 55. Overall beneficiary satisfaction with micro-projects was 78% - of which 62% were highly satisfied, and 16% satisfied. This was based on a beneficiary assessment of 108 key respondents from 49 beneficiary communities in Tanna Island (see Annex 6 and Figure 2 below). As the survey was conducted between December 2018 and March 2019 when some micro-projects were not yet finalized, this is likely an under-estimate. 56. Beneficiary satisfaction was Figure 2: Beneficiary Satisfaction with Micro-Projects highest for Rainwater Catchment (RWC) micro-projects, with 100% of respondents Very Satisfied (74%) or Satisfied (26%). The degree of satisfaction with Gravity-Fed Systems (GFS) and Multipurpose Centers (MPCs) was similar, with 73% Satisfied to Very Satisfied, but a larger proportion of respondents were Very Satisfied with GFS (61%). Satisfaction amongst Feeder Roads (FRs) beneficiaries was 72%, of which 61% were Very Satisfied and 11% Satisfied (Figure 2). Those dissatisfied cited their relative distance from the road or water tap, or the fact that the micro-projects were still incomplete at the time of the survey (Ienvitana, Lawinu). The number of 17 The criteria for selection of the micro-projects were as follows: (1) the sites are in TC Cyclone Pam affected areas; (2) the proposed activities would reduce the climate related risks and enhance resilience of the people in the area; (3) they would aim to target a large number of beneficiaries that are particularly poor, relative to the surrounding communities; and (4) they would not exceed the funding ceiling of VT5.0 million per micro-project. Page 19 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) beneficiaries of Agriculture Demonstration Plots (AGRIC) interviewed (4) was too small to derive meaningful conclusions. 57. The beneficiary survey also attempted to capture the beneficiaries’ perceptions of changes in multiple resilience dimensions. This input was considered important to the efficacy analysis as resilience is specifically mentioned in the PDO statement yet was not well captured by the project indicators. The survey used a set of questions derived from Jones and Tanner (2015) 18 to measure subjective resilience along seven dimensions (see Figure 3 below and Annex 6 for further details). It should be stated that measuring resilience in DRM and adaptation projects remains highly challenging due to the multifaceted nature of resilience, difficulty in attribution, and short project timescales. 19 58. In general, beneficiaries felt their coping and adaptive capacity, access to early warning, adaptive learning, and access to social capital were strong, whilst access to financial resources (for disaster recovery) remained weak.20 However, attribution to the project is difficult, as most beneficiaries had also received help from other aid programs after TC Pam. At the most, the project appears to have contributed to strengthened resilience, alongside other aid programs, with adaptive capacity and early warning particularly reinforced. Social capital through self-help mechanisms continues to be strong, whilst the project contributed only indirectly to financial resilience (by spurring agricultural sales). Figure 3: Multiple Dimensions of Community Resilience 59. In terms of adaptive capacity, 86% of the respondents felt confident they could adapt Coping successfully if a disaster was to occur in the next five Capacity years. Beneficiaries also felt positive about adaptive Access to learning, mentioning newly acquired skills such as Adaptive Early Warning Capacity operating disaster committees, building stronger houses and reinforcing roofs. Questions on transformative capacity were not well understood; however, several respondents stated they would be willing to change their future way of living if their community ceased to exist. Access to Social Transformative Capital 60. About 86% of the respondents agreed that if a Capacity major disaster occurred, they would have adequate access to early warning. Disaster alerts – transmitted by radio, mobile phones, internet, and government Access to Financial Adaptive officials and village leaders - were perceived to be Learning Resources more accurate and timelier (due to communication technology) than 5 years ago, when they relied on weak reception radio, traditional knowledge and person-to-person messages. They were in general familiar with 18 Jones, L. and T. Tanner (2015) “Measuring Subjective Resilience”. ODI Working Paper 423. Overseas Development Institute. London. 19 (Gregorowski, 2016 – internal paper produced for Department for International Development (DFID) on the Building Resilience for Climate Extremes and Disasters (BRACED) lesson learning. 20 This could also be a reflection of the explanations from the survey team who found it hard to explain the concept of financial resources after disaster to communities that rely on extended family support and/or are dependent on subsistence livelihoods. Page 20 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) standard preparedness procedures after alerts, and up to two thirds of the communities had active disaster committees. 61. Beneficiaries also articulated that their access to social capital (in terms of community self-help) remained strong. They felt able to rely on community or family support to recover from major disasters - such as helping each other rebuild damaged houses or replant gardens - and stated they too would help those affected. By contrast, only 59% of the respondents felt they would have access to sufficient financial resources to recover from an immediate disaster. Many respondents stated that it would take time to recover, particularly if their crops were destroyed, reflecting their semi-subsistence livelihoods. Justification of Overall Efficacy Rating 62. Based on the overall assessments of the PDO, the Pre-Restructuring Outcomes 1-3 were rated as Modest, mainly due to lack of enough evidence that the original outcome target of 10,000 farmers adopting climate resilient food crop production practices was reached and that the project contributed to enhanced livelihoods. The Post-Restructuring Outcomes 1-2 were rated as Substantial due to the significant strengthening of the country’s DRM systems, the number of beneficiaries reached through the micro-projects, and indications that the project contributed to several dimensions of community resilience. C. EFFICIENCY Assessment of Efficiency and Rating 63. Efficiency is rated as Substantial, based an ex-post cost benefit analysis covering about 58% of total costs, which showed results to be both significant and robust (see Table 4). In addition, the micro-projects proved in general to be cost-effective relative to the original engineering estimate or comparable technical standards. The full economic analysis and assumptions used are detailed in Annex 4. 64. Economic Analysis at Appraisal. The project did not carry out an ex-ante economic analysis, citing lack of household data - not uncommon amongst resilience and community-based operations, but nevertheless an appraisal shortcoming. The PAD qualified expected benefits as improved food and water security and reduced costs from disasters. Cleaner water was expected to improve water security and reduce public health costs, whilst better disaster monitoring and early warning would result in savings in evacuation costs and losses avoided. 65. Economic Analysis at Implementation Completion and Results (ICR) Stage. At completion, a cost-benefit analysis was performed on the early warning system (Component 1.3), and on community micro-projects (Components 1.1, 2, 3 and 4 after restructuring) covering 11.2% and 46.4% of the project costs respectively (see Table 4). The results show a total base case Economic Internal Rate of Return (EIRR) of 19.1%, and a Net Present Value (NPV) at a 10% discount rate21 of US$2.7 million equivalent. The EIRRs for various activities ranged from 20.0% for the early warning system, and between 11.3% and 42.0% for the community micro-projects (18.9% in aggregate), which is consistent with projects of comparable nature.22 The sensitivity analysis indicates that micro-projects economic benefits are fairly robust to a 25% 21 The 10% discount rate used is very conservative, but consistent with the rate at appraisal used by the Pacific Resilience Program. 22 The cost benefit analysis of the early warning systems in Samoa and Tonga, under the Pacific Resilience Program (PREP, P154839/40) resulted in an EIRR of 19% (see World Bank 2015. Project Appraisal Document for a Pacific Resilient Program (PREP). The sub-grants under the Solomon Page 21 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) decrease in benefits and increase in costs, as well as a structural replacement after 10 years (see Table 4). However, if benefits are assumed to decline progressively over a period of 10 years, the EIRRs drop significantly, highlighting the crucial importance of maintenance. Table 4: Summary of Ex-Post Economic Analysis Early Warning All Micro- Sampled Micro-Projects Results Total System Projects GFSs RWCs MPCs FRs Base Case: EIRR 19.1% 20.0% 18.9% 32.5% 11.3% 13.2% 42.0% NPV (10%) in US$ ‘000 2,700 500 2,200 169 14 28 1,043 % of Total Costs represented 57.6% 11.2% 46.4% 13%1 ---2 59%1 62%1 Sensitivity Analysis: EIRR -25% Benefits 13.3% 11.3% 13.7% 23.2% 5.0% 9.9% 30.4% EIRR +25% Costs 14.5% 13.2% 14.8% 25.1% 6.4% 10.6% 32.8% Replacement after 10 years 18.3% 20.0% 17.9% 30.7% 6.3% 7.8% 40.8% Net Benefits declining and stopping after 10 years 1.6% 7.6% 0.3% 17.5% -7.5% 7.4% 27.7% See Annex 4 for detailed assumptions. GFS – Gravity Fed Systems. RWC – Rainfed Centers. MPCs – Multipurpose Centers. FR – Feeder Roads 1 – Proportion of total activity costs covered by the analysis 2 – Not sampled; estimates were from secondary data 66. For the early warning system, benefits were computed conservatively23 and were based on probabilistic risk assessments for Vanuatu as well as international estimates (see Annex 4). Benefits of micro-projects were based on detailed beneficiary interviews for a sample of gravity fed systems, feeder roads and multi-purpose centers covering 13%, 62% and 59% of their respective costs. Rainwater catchment benefits were calculated through secondary data and beneficiary survey results. Agriculture micro-project benefits could not be estimated as it would have required tracing numerous farmers in remote locations. However, their costs were included in the micro-project aggregated analysis (EIRR 18.9%) as were all associated project management and technical assistance costs. The cost-benefit analysis also excluded institutional strengthening and policy support (which are difficult to quantify), as well as the VARTC agricultural technology activities and TC-Pam emergency supplies in 2015, which had the same logistical constraints as agriculture micro-projects. Adding these two cost items to the overall cost-benefit analysis results in an EIRR of 16.3% even without any computed benefits. 67. Cost Effectiveness Analysis. The micro-projects were generally cost-effective, with unit costs 10-63% below the engineering original estimates or, in the case of the gravity-fed systems, that of comparable schemes under the Department of Water Resources (see Table 4.6, Annex 4). The exception was feeder roads, where the actual cost per linear meter (VT 29,751) was slightly higher (3%) than the PWD engineering estimate (VT 28,880) but this is considered reasonable given that many of these roads were in remote and steep areas of Tanna Island. Islands Rural Development Program (RDP, P089297) which is similar in type to the evaluated operation’s micro-projects, yielded EIRRs of 21-39% for water structures, 11% for community centers, and 49% for footbridges. See Braun, Jennifer (2014). Financial and Economic Analysis for Solomon Islands Rural Development Programme (RDP 2)– Working Paper, and Excel worksheet (FEA_SI_RDP2_Updated_Nov2017.xls) with adjusted estimates at restructuring (2017). 23 It was assumed that the early warning system would only reduce average annual losses by 5%. If it achieved 10% savings in average annual losses, the EIRR would be much higher (46.5%) - See Annex 4 for detailed assumptions. Page 22 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) 68. Aspects of Design and Implementation that Influenced Efficiency. During its first 3 years, the project suffered from significant procurement delays due to limited experience in the PMU, and the impacts of TC Pam. Once it was restructured, micro-project implementation proceeded smoothly, with design to delivery averaging 4 months, and design to completion of 11 months. The restructuring also ensured that micro-project funds were used efficiently, reaching 28,800 beneficiaries at a cost of US$4.0 million, compared to the appraisal estimate of 20-22,000 beneficiaries at US$7.2 million. 69. Administrative Efficiency. Project management costs amounted to 31% of the total project costs, higher than estimated at appraisal (22%), as well as compared to similar projects.24 However, taking into consideration items that were incorrectly accounted against component 1.1, project management costs amounted to about 26% of the total project costs. The relatively high cost reflects: (1) the cost of international advisors and experts necessary during the initial stages of the project and for particular areas (monitoring and evaluation, project management, procurement and financial management) given the limited expertise in-country and within VMGD; (2) the large number of technical agencies in the original project design and the complex project design requiring extra management and financial management support; (3) adjustments due to TC Pam; and (4) significant travel and training costs in remote parts of Vanuatu (Tanna, Shepherds and Ambrym). In light of the special circumstances of island countries (remote locations, less human resources) these costs are deemed acceptable. D. JUSTIFICATION OF OVERALL OUTCOME RATING 70. The project’s overall outcome rating is Moderately Satisfactory. The split rating method used took into consideration the original and revised PDOs and indicators to derive the overall Outcome rating. Relevance was rated as High and efficiency rated Substantial at the end of the project. The calculations are summarized below. Table 5: Overall Outcome Rating Before Restructuring After Restructuring Relevance of PDO High Efficacy (PDO) Modest Substantial Efficiency Substantial 1 Outcome rating Moderately Satisfactory Unsatisfactory 2 Numerical value of the outcome 3 5 rating 3 Disbursement (US$ Million) 6.89 3.04 4 Share of disbursement 69% 31% 5 Weighted value of the outcome 3*0.69 = 2.07 5*0.31 = 1.55 rating 6 Final Outcome Rating Moderately Satisfactory (2.07 + 1.55 = 3.62 rounding it to 4) 24 For the Solomon Islands RDP (P089297), PMU costs were about 20% of the total at completion; for the São Tomé and Príncipe Adaptation Project (P111669), 15%. For the Second Kiribati Adaptation Project (P089326), administrative costs were 21% of the total. Page 23 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) E. OTHER OUTCOMES AND IMPACTS Gender 71. Women’s participation and the benefits they derived from the project were substantial. At restructuring, a target of 47% female beneficiaries was adopted, which was largely achieved (Annex 1). About 28% of the beneficiary survey respondents felt women and children had benefited the most from micro-projects (most stated that the community had benefited as a whole). Overall satisfaction was significantly higher for women than men: Up to 85% of the women professed being satisfied or highly satisfied with the sub-projects, compared to 72% for men25 (Annex 6). 72. Field interviews conducted during the ICR mission indicate that women benefited significantly from the time saved by accessing water closer to home. They used the extra time for gardening, feeding animals, doing household chores, and marketing (see Annexes 4 and 6). Female beneficiaries from gravity fed schemes also reported new-found feelings of cleanliness, dignity, and well-being from being able to wash more often. Feeder roads enhanced women’s safety (by no longer having to walk to markets through isolated footpaths) and allowed buses and trucks to access remote villages in Tanna on a year-round basis, saving time and promoting market opportunities for women. Women also benefited the most and determined the daily use of MPCs - as a childcare center (Latanu), community school (Lownolis), first aid clinic (Nalpinakeri), handicraft center (Ianatoka) or food market (Tongariki). Finally, there is some anecdotal evidence that the project may have enhanced women’s social roles: in the remote communities of Nalpinalevi in Northeast Tanna and Latanu in Central Tanna, women spoke highly of their MPCs, stating that while in Tannese kastom (custom) it is taboo for women to attend meetings at the Nakamai (the space under the Banyan tree where men gather to discuss community issues), the MPCs provided them with a comfortable space where women and children could also attend community meetings and have their voices heard. 73. Whilst – as per tradition – male chiefs played a major role in proposing the sub-projects, women actively participated in implementation, through provision of in-kind labor and local materials. Women helped carry sand and dig soil for installation of water pipes or tank bases, prepare food and water, and participated actively in implementation committees. The exception was feeder roads, where construction was mostly done by men. Women’s roles in maintenance is not yet known, given that most micro-projects were completed in early 2019. However, the beneficiary survey noted that half of the community water committees lacked women. This may require future government attention in light of a recent Department of Water Resources survey (the findings were presented to the ICR team in April 2019) which revealed that the presence of women in water user committees was correlated with better maintenance of the water schemes. Institutional Strengthening 74. The project provided substantial institutional strengthening benefits. It supported the NAB by developing Standard Operating Procedures, strengthening the capacity of the Secretariat, and coordinating the early stages of the Climate Change and Disaster Risk Reduction Policy. The project also strengthened the core capacity of the new Department of Climate Change, helping it finalize its 2019 business plan and secure an annual budget. The Department has now absorbed the project’s Communications and Safeguard Specialist as part of its core staff, while the Monitoring 25 This finding is noteworthy; Solomon Islands RDP II (P149282), women state level of satisfaction with sub-projects more conservatively than men. Page 24 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) and Evaluation specialist joined the Ministry of Health. Two Provincial District Officers supported originally through the project are also now funded by GoV. 75. Perhaps less visible, but as important, was the capacity built for inter-agency collaboration in support of the community micro-projects – between PMU and staff from the DGMWRD, PWD, the DARD, and the provincial governments, especially during the design and implementation of water-related micro-projects and MPCs. The GoV is now assessing this experience as part of a decentralization trend in Vanuatu – where it is expected that Provinces and Area Councils, and eventually community water committees will increasingly manage their own funds. Poverty Reduction and Shared Prosperity 76. By focusing 80% of the project beneficiaries on Tanna Island, the project contributed (albeit indirectly) to poverty alleviation. Tafea Province, where Tanna is located, had the lowest per capita income of all provinces in Vanuatu in the latest (2010) Household Income and Expenditure Survey: the average monthly income in rural Tafea was 10,026 VT/capita/month (about US$3/capita/day equivalent). Women producing goods for sale – which as a group benefited particularly from the project - are amongst the most vulnerable in rural Vanuatu: 8.8% were below the Basic Needs Poverty Level in 2010. 26 As mentioned in the Efficacy section, the revenues derived from time saved and incremental agricultural sales were in the order of 3-68% of the average monthly income for rural Tafea Province. It should be noted, however, that the rural population of Vanuatu as a whole has a lower incidence of basic needs poverty than the urban population, due to access to land, family support, and a largely subsistence-based economy. 77. In terms of shared prosperity, the majority of the beneficiary survey respondents felt that benefits were well distributed amongst the entire community (54%) or benefited women and children as a group (28%) – indicating that the project was perceived as having generated broad-based benefits (see Annex 6 for further details). III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 78. Delays in project preparation. A Project Preparation Grant Agreement was signed on June 2009 with VMGD as the main agency, and the expectation that the project would be prepared and delivered to the World Bank Board of Executive Directors by June 30, 2010. However, due primarily to lack of familiarity with Bank procedures, preparation took longer and the project was only approved by the Board on December 4, 2012. 79. Ambitious objectives and over-complex project design. Although limitations on capacity and human resources were mentioned throughout in the PAD, the project results, geographic influence, and implementation plans did not adequately account for these constraints. The design was complex, involving seven agencies and many sub-components. The original components were sectoral with few synergies or complementarities. The small grants mechanism was mentioned but there was little information on how it would be implemented. The selection process for beneficiary communities was also not outlined which created early challenges in delivering community investments (see also Bank Performance: Quality at entry for more details). 26VNSO and UNDP 2013. Vanuatu Hardship and Poverty Report. Analysis of the 2010 Household Income and Expenditure Survey . Vanuatu National Statistics Office and United Nations Development Programme. Suva, Fiji. Page 25 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) 80. Weak results framework and Monitoring and Evaluation (M&E) plan. As assessed in section IV.A, the original results framework was overly complex, given the GoV’s limited capacity. The PAD outlined the roles and responsibilities in terms of monitoring and evaluation, but greatly overestimated the PMU’s capacity to undertake M&E. 81. Financial complexity. The project had three funding sources, four separate ledgers, and two fiduciary agencies (VMGD and VARTC). The small grants mechanism added to its financial complexity. The risks were expected to be mitigated through strong technical assistance support for institutional strengthening. However, this design complexity proved to be difficult for the PMU to manage during the early stages of implementation. 82. Risk assessment and mitigation measures. The PAD included a risk assessment and relevant risk mitigation measures. However, the GoV’s capacity seems to have been greatly overestimated. More intensive support should have been provided from the Bank side during preparation given capacity constraints and the complex project design. B. KEY FACTORS DURING IMPLEMENTATION Factors within Government and World Bank control 83. Coordination and engagement problems due to complex institutional arrangements. The project was designed to promote inter-agency collaboration, but it proved difficult to effectively coordinate seven agencies. The early turnouts at the meetings organized by the PMU were poor and caused continuous delays to agree and make decisions. Mostly due to these delays, as well as the disruption caused by TC Pam in 2015, approximately US$0.66 million had to be returned to the EU.27 84. Limited human resources and organizational capacity. The limited human resources and capacity, as well as delays in the recruitment of key staff were a key factor for the slow implementation of the project, especially during the first two years. Even though procurement and financial management risks were rated high at appraisal, the design did not adequately consider Vanuatu’s lack of familiarity with World Bank procedures and the difficulties to find qualified fiduciary staff in a highly fragmented donor context (where numerous projects compete with the GoV for the few qualified staff). Recruitment of a qualified Financial Specialist proved difficult during the first year, and the original Procurement Officer resigned in September 2014, causing further delays. 85. Reconciliation problems between GoV financial management system and Bank requirements. At mid-term, it was agreed that the project’s accounting would be integrated with the GoV’s SmartStream system, linked to an accounting software to meet Bank-specific financial reporting requirements. It proved difficult due to transaction coding errors, delays in transaction entries, and the limited capacity of local accounting staff, leading to significant delays in financial reporting and multiple weaknesses in financial management (see Section IV.B). 86. Dropping the small-grant mechanism. To accelerate the support to communities affected by TC Pam, the GoV decided to drop the small-grants mechanism and pilot micro-projects to deliver investments to the communities. The proposed changes took an integrated approach to provide water, improved agriculture, feeder roads and multi-purpose centers to more efficiently build resilience and meet the needs of the most vulnerable communities. 87. Introduction of bulk procurement. Based on best practice from Solomon Islands and Kiribati, and given the smallness of the markets, limited number of reputable suppliers, and the communities’ remoteness, the PMU adopted 27 The EU grant could not be extended further due to the time limitations on the parent trust fund. Page 26 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) bulk procurement of materials to accelerate micro-project implementation (while communities remained responsible for local materials and in-kind labor). Without this mechanism, the project would most likely not have been able to support the number of communities that benefited in the end. Factors outside the control of Government 88. Natural disasters. As mentioned throughout, TC Pam hit Vanuatu in 2015 and forced the GoV to focus on disaster response and quick recovery. This affected the course of the project as well as its geographic focus. The volcanic eruptions in Ambae resulting in mass evacuation of 11,000 people, and the island of Ambrym was hit by a volcanic eruption, landslide, and tropical cyclone within months of each other. Consequently, most of the human and financial resources of the relevant GoV agencies were diverted to respond to these events. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 89. The project’s M&E design was overly complex relative to local capacity and provided weak guidance to measure outcomes. The PAD recognized severe constraints in the GoV’s M&E capacity, due to remoteness, lack of data, and limited presence outside the capital. To overcome this, it proposed using existing vulnerability and adaptation assessments to guide micro-project identification, and to carry out the baseline survey. Capacity constraints were to be addressed by a dedicated M&E specialist, who was to design reporting formats enabling the various implementing agencies (VARTC, Department of Agriculture, NDMO, NGOs, DHMWR) to provide timely reports to the Secretariat/PMU. In practice, this coordination proved difficult, resulting in early inconsistencies and delays in reporting key indicators. 90. The original results framework included 16 indicators (4 outcome and 12 intermediary), but for most cases did not describe how they would be measured. This was particularly problematic for 3 out of the 4 outcome indicators which would have been difficult to measure even in a higher capacity context: (1) “households incorporating climate considerations in decision making”; (2) “farmers targeted have adopted resilient food production practices”; and (3) “number of national/sub-national institutions with functioning DRM arrangements.” All three were later dropped at restructuring. The outcome indicators also did not capture well the PDO statement: community resilience in terms of livelihoods and food security, for example, was not measured, and the indicator “percentage of households satisfied with the quality and security of water supply” did not accurately measure whether communities were more resilient to impacts on water security. It is worth noting, however, that the project was prepared at a time (2012) when there was not yet any international guidance on how to measure disaster and climate resilience. Nonetheless, the choice of hard-to-measure indicators that did not fully capture the PDO was a shortcoming of project design. 91. Intermediate result indicators were generally adequate to capture the components and activities’ contributions towards the PDO, with some limitations: while the PAD described well how to capture “people in rural areas provided with access to improved water sources”, there was little specificity on how to measure other Page 27 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) indicators, such as “NGOs/churches involved with rural water supply adopt RWS standards”. Several of these were later dropped. 92. The project design envisaged a participatory baseline survey, as well as annual progress reports. This information was by and large collected, but late in the implementation (see next section). Except for the agriculture component, where the lack of baseline proved immediately constraining, the PAD correctly predicted that the early stages would focus on institutional strengthening, allowing the baseline survey and community pilots to follow later. However, it was over-ambitious in predicting that community level information could be collected during the first year, given the need to first establish the PMU. M&E Implementation 93. The PMU with World Bank team support was generally proactive in adapting the results framework to implementation challenges and in collecting and analyzing M&E data, despite initial delays and procedural weaknesses. The PMU contracted an M&E Coordinator and a part-time international M&E Advisor after a year’s delay. By the end- 2014 (1.5 years after effectiveness), the project team issued its first annual report with an updated results framework, but monitoring was severely disrupted by TC Pam, although a mid-term review (MTR) was carried out in November 2015. The 2017 restructuring simplified the results framework to 12 indicators, substantially addressing design weaknesses (see Section I.B). However, with the exception of number of beneficiaries, no indicator was included to measure “increased resilience to the impacts of natural hazards and climate variability and change” a key statement in the revised PDO. This may again have reflected persistent methodological difficulties in measuring climate resilience. In addition, it ended up duplicating a PDO level indicator with an intermediary indicator on benefits from water supply. 94. To prepare for community micro-projects, the PMU contracted an NGO consortium (Vatu Mauri) to carry out a baseline survey in 230 villages in East and West Tanna during early 2016, 3 years into implementation. The baseline, while comprehensive, did not cover communities in Shepherds and Ambrym, which were also later prioritized for project interventions. A second constraint was that whilst the village profiles were placed into a GIS-database (C2M), the system proved to be beyond the capacity of the PMU. As a result, a World Bank mission in December 2017 recommended that the PMU use simple Excel sheets to simplify monitoring, and link micro-project completion with the procurement plan and budget. From then on, the PMU started producing shorter progress reports and updating the results framework on a bi-annual basis, coinciding with Bank missions. The M&E Coordinator and Communication Specialist monitored the micro-project sites regularly, and the simplified project reporting proved for the most part adequate. However, unclear settlement boundaries and names, as well as new priorities arising from other disasters posed challenges in verifying the accuracy of micro-project data up until project closure. The final number of beneficiaries ended up significantly higher than the targets agreed at restructuring: at the time the targets were set (prior to the baseline survey) there was limited population data and capacity was overstretched due to the TC Pam response, indicating a relatively modest outreach. However, the last years of implementation brought about an acceleration of activities, and higher-than- expected number of beneficiaries due to the gravity fed systems and feeder roads, which were implemented in remote communities with high population densities. 95. In early 2019, the PMU carried out a comprehensive beneficiary survey of 108 key respondents from 49 communities in Tanna Island. The survey, designed with the help of the World Bank team, sought to make up for the lack of a resilience indicator by capturing beneficiaries’ perceptions of acquired resilience capacity in multiple dimensions (see Figure 3). Beneficiaries were also asked to rate their degree of satisfaction with the micro-projects, and Page 28 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) their opinion of implementation. The survey, discussed at a project closing workshop in June 2019, yielded good quality information which helped fill key gaps in the project’s monitoring framework - even if the results could not be directly compared to the baseline (see Annex 6). At the same time, it posed a heavy burden on the PMU at the eve of project closure. M&E Utilization 96. In many respects, the project’s M&E framework adequately reported project achievements and challenges and helped inform decision-making. As early as the 2014 annual report, the PMU remarked on the project’s financial complexity, and proposed concrete actions to simplify its structure, and concentrate activities geographically. The MTR and 2015 annual reports further proposed simplifying the design by better linking project components and refocusing on community-based activities. These recommendations helped inform the project Level 1 restructuring. 97. Site visits were used to both review and improve the micro-projects: a good example of this was the addition of solar panels to multi-purpose centers following the March-April 2019 technical mission. M&E findings were also discussed with national and provincial stakeholders, although local stakeholders’ involvement was generally restricted to their own micro-projects, in part due to the compressed implementation time frame following TC Pam. 98. Monitoring data was primarily used to report on intermediary indicators, but the final beneficiary survey included multiple questions aimed at better capturing the project’s outcomes (see M&E Implementation, above). The beneficiary survey and lessons learned from the project are being studied to inform subsequent GoV interventions in rural communities, as well as the design of new World Bank community resilience projects in the Pacific Island region. Justification of Overall Rating of Quality of M&E 99. The quality of the M&E is rated as Modest, primarily due to the significant weaknesses in M&E design and difficulties in capturing outcomes. These shortfalls were partially compensated by the beneficiary assessment and interviews with key respondents during the ICR mission, allowing for a more complete assessment of efficacy than what would have been possible through the results framework. Considering the challenges and capacity limitations in Vanuatu, M&E implementation and utilization was generally sufficient for project management, albeit with some shortcomings. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Safeguards Compliance 100. In general, the project complied well with its applicable environmental and social safeguards. The initial delay in field activities also allowed for extensive consultation and fine tuning of safeguard procedures before micro-projects started. 101. The project was rated as a Category B (partial assessment) at appraisal, triggering only OP/BP 4.01 on Environmental Assessment. This was justified by its focus on capacity building and limited piloting of community activities, which were expected to have minimal and/or localized environmental and social impacts. An Environmental and Social Management Framework (ESMF) and a Land Acquisition Framework (LAF) were prepared and disclosed on March 23, 2012, both in-country and at the Bank, and their requirements included in the Project Operation Manual. Page 29 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Bank safeguard staff trained PMU and community stakeholders at the 2013 launch as well as in the field in Tanna in 2014. 102. During the level 1 restructuring in 2017, the project’s environmental category (B) was retained, but Operational Procedure (OP)/Bank Procedure (BP) 4.12 on Involuntary Resettlement was also triggered due to the expanded nature, and more diverse types of micro-projects. A Resettlement Policy Framework was prepared and disclosed on June 6, 2017, and the ESMF and Project Operation Manual were revised to take the new priorities into account. Whilst the original project design did not envisage contracting a safeguards staff for the PMU28, a Community Outreach Support and Safeguards Officer was eventually hired in 2017. The safeguard procedures were also strengthened. Safeguard screening forms – signed by chiefs, landowners, community leaders, and the provincial authority - were applied to 113 sites, including all micro-projects, seismic stations and rainwater tanks. Mitigation plans were developed for each of the micro-project types, and the Safeguards Officer liaised with the Department of Environmental Protection and Conservation (DEPC) to carry out Environmental Preliminary Assessments and secure environmental permits. Through this process, it was determined that none of the micro-projects required a full Environmental Impact Assessment (EIA), Environmental and Social Management Plan (ESMP), or Resettlement Action Plan. 103. Bank mission reports confirm that the project’s environmental and social impacts have been for the most part small, localized, and adequately mitigated. Amongst water-related micro-projects, there were some problems of poor drainage around taps and standing water in barrels, requiring maintenance training and further awareness of mosquito- borne diseases. Construction of feeder roads started in October 2018 before the PMU had secured the necessary permits from DEPC but based on verbal assurances that they could proceed, and no EIA was required. The permits were eventually issued and shared with the Bank safeguards team in December 2018. Contractors for feeder roads in West Tanna were also found to have used sand and aggregate from a beach area, but upon PMU inspection, the amounts extracted were not found to be significant. Road contracts were amended to include an Environmental Code of Practice and three extra conditions requested by DEPC (including obligation to use authorized quarry materials). Field visits further showed compliance with workers safety procedures. 104. Originally the project did not have a Grievance Redress Mechanism (GRM), but this was later adopted and included in the revised Operational Manual. Reported grievances were resolved either on the spot with the Safeguards Officer, or with the supervising engineer during subsequent visits, and recorded on a GRM tracking system. During the seismic stations’ installation, for example, a dispute arose with landowners of the selected site at the Yasur Volcano (Tanna Island), leading to vandalism of the old station door. This was resolved in March 2018 through the mediation of provincial authorities and the Nikoletan Council of Chiefs. The landowners agreed to allow the land to be used voluntarily and in perpetuity for the seismic monitoring station, with the meeting minutes signed by both community leaders and VMGD. 105. As many micro-projects were completed on a tight schedule, safeguard issues may still arise after ICR completion. Some respondents from the beneficiary surveys, for example, felt excluded by not being sufficiently close to the feeder roads or not being provided with individual water taps. In this context, the GoV’s hiring of the Communications and Safeguards Officer should help ensure that grievances and mitigation actions continue to be addressed. Fiduciary Compliance 28 The original plan was that the Bank safeguard specialists would train project staff and beneficiaries on the ESMF and LAF, with each agency then responsible for mitigation, and the PMU responsible for monitoring compliance. This proved insufficient when the number of micro-projects expanded at restructuring. Page 30 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) 106. Procurement. Despite initial capacity constraints, project procurement was generally carried out in compliance with the procurement procedures in its Grant Agreements, the Procurement Plan, and the Procurement and Consultant Guidelines. There were no indications of fraud or corruption. 107. During a 2014 implementation support mission, the PMU Procurement Officer and Bank team noted an irregularity on the distribution of agricultural goods by VATRC, which had been awarded in 2014 to a single supplier for US$1,825 equivalent. VARTC was subsequently excluded as a fiduciary agency at restructuring. Procurement delays also affected project implementation during the first three years (see Section III.B), but after restructuring and the change to PMU bulk procurement, micro-project implementation accelerated with only minor delays on materials for feeder roads and gravity-fed systems. Bank post review missions (in December 2014, February 2017, and April 2018) further noted some procurement weaknesses such as incomplete documentation, weak consultant contract management (in terms of time claimed versus deliverables), and monitoring. There were also minor inconsistencies between the procurement plan, the budget, and the work plan which were rectified by the time of project closing. 108. The Bank team worked proactively with the PMU to strengthen procurement capacity and procedures, and by June 2017, procurement performance had substantially improved. In addition to procurement post-reviews, regular teleconferences with the PMU were held from June 2016 onward to resolve implementation issues. The PMU’s procurement capacity was reinforced with two local procurement officers, who received procurement training abroad, and were supported by a part-time senior procurement advisor. During the restructuring, procurement monitoring was also reinforced by linking the procurement plan with the project’s budget and work plan, developing an asset register, and adopting an improved contract monitoring system. The Bank implementation support team was also candid in its assessment, downgrading procurement ratings to Moderately Unsatisfactory from May 2015 to June 2017 and upgrading it to Moderately Satisfactory only after the most substantial capacity and procedural issues had been addressed. At project closure, procurement risk was rated moderate reflecting persistent weaknesses in document filing and contract monitoring, but all major outstanding issues had been resolved. 109. Financial Management and Disbursement. Financial covenants were largely complied with, but with significant delays in Interim Unaudited Financial Report (IFR) submission after 2017, as well as in most external audits - up to 9 months delay for the 2018 audit. There were also multiple weaknesses in financial management, described below. From 2013 to 2016, separate external audits were carried out for each of the two fiduciary agencies (VMGD and VARTC) but became a single annual audit as of 2017 when all project fiduciary functions came under the responsibility of VMGD. 110. The three VARTC audits (2013 to 2015) were qualified, as the auditors were unable to obtain adequate assurances regarding the quantities and condition of the inventory and biological assets. An EU verification mission in early 2017 determined ineligible expenditures of VT 362,036 (approx. US$3,400) consisting of a partial travel advance that was not returned and an overpayment of about US$800 to a consultant, both of which the Bank financial management review had found and advised the GoV to rectify.29 By contrast, all VMGD annual audits (from 2013 to 201830) were unqualified. The auditors noted several weaknesses, mostly administrative in nature, including lack of reconciliation between financial statements and the SmartStream ledger (see Section III.B); unretired Impress/advances and advances for non- 29 The Bank has reimbursed the amount to the EU and the GoV has been informed that as EU found ineligible expenses, a questionable expenses report and recommendation to declare ineligible expenditure will be prepared and sent by the Bank team to the Country Director. 30 As of the time of this ICR, the final audit for 2018 was yet to be officially submitted, but the draft report has been reviewed and is unqualified. Page 31 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) project activities, which had to be reimbursed; and expenses posted against the GEF/GFDRR grants, subsequently reposted against the EU grant. Most of the auditors’ recommendations had been addressed by project closure. 111. The Bank team noted some irregularities during the financial management reviews (conducted on November 2014, October 2016, October 2017, May 2018, and September 2019). The irregularities included consultant payments, including excessive per diem, travel advances not acquitted and/or repaid on time, and significant expenditures for non- project activities (VT 4 million, or US$35,600 equivalent). As a result of this and the lack of full-time and qualified accounting staff, from June 2018 to project closing, the project’s financial management was downgraded to Unsatisfactory. The Bank team also agreed with the PMU on a set of mitigating actions: The Financial Advisor was re- hired, the accounting software designer asked to come in to help with further training and in September 2019, VMGD and the Department of Climate Change also brought in a qualified accountant to help resolve outstanding acquittals and refunds. As of November 28, 2019, the government had shared a report to suggest that these issues are being resolved and are being supported and monitored until the disbursement closing date of December 28, 2019. C. BANK PERFORMANCE Quality at Entry 112. The project design considered GoV’s relative inexperience with project implementation, the lack of oversight arrangements, and the costs and burden of implementing large projects in remote islands. It drew on the experience of similar projects in other Pacific Islands, such as the Kiribati Adaptation Project (P112615). 113. The project was deemed technically viable due to demonstrated results obtained by ongoing and previous national programs at the time of appraisal. For example, the rural water supply activities were based on a national rural water supply program where the GoV showed technical capacity to design rainwater tank specifications taking into account current and projected precipitation patterns using climate change modelling software. The agriculture component was not new but was a scaling-up of pilot projects already under implementation in Vanuatu. In addition, the real-time data communication network was also a scale up of an initiative by VMGD, with the support of the Institute of Geological & Nuclear Sciences (GNS) New Zealand where volcanic monitoring networks were established and successfully piloted for the Yasur volcano on Tanna. The community-based activities built on existing experience with conducting Vulnerability and Adaptation Assessments in Vanuatu and community-based planning. Nevertheless, the inadequate human resources, capacity and limited experience with implementing World Bank projects seems to have been greatly underestimated at appraisal. 114. An assessment of the institutional and policy framework informed the project design and activities. At entry, Vanuatu’s institutional arrangements were in a state of transition with a proposal calling for the integration of functions of the NACCC, and the National Task Force for DRR and DRM. The new NAB Secretariat would be managed by VMGD and the Secretariat and PMU would get a key role in providing oversight, coordination between the implementing agencies, and guidance and support. The PMU was to be responsible for overall project management. The project design took this into account and was further designed to provide substantial capacity strengthening support through training and a review of the NDMO Act to clearly define the institutional set-up and roles and responsibilities of the agencies involved in disaster management. Page 32 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) 115. The implementation arrangements were complex, although the Bank acknowledged the very limited capacity of the GoV, in delivering services to rural and remote areas. At entry, therefore, the project was designed to involve NGOs, technical agencies, and other service providers working at the community level. The aim was to set up a system that would enhance inter-agency collaboration. However, the project components were not designed in a way that promoted this. 116. The project had a comprehensive risk assessment including proposed risk mitigation measures. The overall implementation risk was rated as substantial due to the weak GoV implementation capacity and the lack of experience in implementing projects of this size. The project implementation was phased to ensure that manuals and guidelines were tested and developed before some of the more complex activities were implemented. The limited capacity for project implementation was accounted for in the risk assessment, yet not sufficiently reflected and accounted for, given the complex project design. Quality of Supervision 117. Implementation missions were frequent and focused on accelerating implementation. The Bank team consisted of experienced DRM, procurement, financial management, and social and environmental specialists. Several of the missions focused on providing advice to the implementing agencies on specific financial, procurement, and safeguards issues (see section IV B) and, towards the end of the project, on ensuring ensure sustainable outcomes and hand-over to the government. In total, 12 Implementation Status and Review (ISR) reports were filed. For a period of almost a year (prior and post TC Pam), however, no ISR or Aide Memoire were filed as both the Bank team and the PMU was helping in recovery efforts which makes it difficult to assess the adequacy of supervision during this period. Implementation shortcomings were generally well-reflected in the Aide Memoire and in the ISRs. 118. Initially, the project focus was more on institutional strengthening than delivering services to the communities which created significant delays in field activities. The level 1 restructuring clearly aimed to rectify this by focusing on delivering services to the communities heavily affected by TC Pam. Investments in these areas therefore well targeted, by aiming to meet the communities’ basic development needs while simultaneously reducing disaster and climate risks. 119. According to the GoV, the Bank team provided good quality support during project implementation and especially during the last two years and the final phase of the project. The Bank Task Team Leader changed twice which provided some discontinuity, but the overall impression is that the handovers were managed well. The PMU expressed some dissatisfaction with lack of proper consultations, especially related to changes in implementation arrangements initially discussed. The PMU also felt that the Bank should allow more flexibility with training, such as procurement outside the region, to help build a pool of nationals competent with Bank rules and procedures. 120. The action from the Bank side which proved to significantly accelerate implementation was the 2017 level 1 restructuring which simplified the project design with a clearer focus on delivering support to the communities, as discussed in section I.B. However, as stated above, indicators were not adequate to fully capture the impact of the project and had to be complemented by a beneficiary survey prepared during the ICR process (see section on M&E for a more comprehensive assessment). Page 33 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Justification of Overall Rating of Bank Performance 121. Based on the above moderate shortcomings in ensuring the quality at entry and the quality of supervision, the overall rating for World Bank performance is Moderately Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 122. The risks that the project’s development outcomes will not be maintained are primarily political, institutional, environmental, and financial, as well as related to the GoV’s commitment. 123. For Outcomes 1-3 (Prior to Restructuring) and 2 (Post-Restructuring) the risks are deemed to be Substantial, both in likelihood as well as in potential impact on the PDOs, and they occur at both the community and national levels. The main risk is financial and related to maintenance funding - particularly for feeder roads and gravity fed systems where maintenance costs are significant. This risk is exacerbated by Vanuatu’s high disaster risk, particularly if the structures are not properly maintained. The project design (aligned with the traditional system in Vanuatu) of handing over maintenance responsibility to communities has been premised on the importance of preserving community self- help mechanisms. Despite limited finances from semi-subsistence communities, 64% of the respondents from the beneficiary survey were optimistic that the micro-projects would continue to operate in the near future. The provincial government has an allocation of VT 1 million annually and together with the Department of Water appears strongly committed to maintaining the gravity-fed systems. For the all-weather access roads, the design has taken into account the responsibility of PWD for maintenance (as for all roads). The main risk would be insufficient funds allocated to the PWD at national and thus provincial level as they require regular periodic maintenance (every two years) due to heavy rainfall and risks from landslide in steep areas. The GoV has ongoing projects in Tanna and the Tanna-based staff do regular site visits to the feeder roads, and this is expected to help ensure maintenance with the use of local contractors and/or communities. Communities, as part of the handover, have also committed to regularly clearing debris and weeding to prevent damages from heavy rainfall. Provided that regular maintenance is carried out, the incorporation of resilience standards into the design of the micro-projects should help minimize damages in the case of future disasters. Collectively, these measures reduce the likelihood and thus the potential impact on the PDO. 124. For Outcome 1 (Post-Restructuring) the main risks are institutional and financial. These risks are deemed to be Low in terms of the likelihood and High in terms of potential impact on the PDO. For the institutional risk, the approval of the draft Disaster Risk Management Bill by the Cabinet in October 2019 and the revised NDMO plan reduce the likelihood of such risk eventuating. In addition, GoV and the World Bank are preparing a Disaster Risk Management Development Policy Grant with Catastrophe Deferred Drawdown Option (Cat-DDO) under IDA18, providing a platform for continued dialogue on DRM, early warning systems and the importance of a functioning National Emergency Operations Center, as well as of the Provincial and Municipal Disaster and Climate Change Committees. In terms of financial risk, the timely issuance of early warning during the Ambae and Ambrym volcanic eruptions and recent cyclones illustrated the need for the maintenance of the early warning systems, which helped secure the necessary maintenance budget for VMGD (confirmed by the Government in June 2019). Prior to that, GoV had already committed funds to hiring the Provincial Disaster Officers which are key to functioning of the provincial centers. The significant improvements in early warning brought about by the expansion of mobile phones and radio communication to remote communities (as revealed by the beneficiary survey and independent of the project) has also help showcase its benefits Page 34 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) and act as a pressure on the government for continued institutional support and funding for a strengthened disaster risk management system. Collectively, these actions have decreased the likelihood of this financial and institutional risk. V. LESSONS AND RECOMMENDATIONS . 125. The project provides several lessons and recommendations for similar projects in the future: • Lesson 1. In countries such as Vanuatu, with limited capacity and frequent disasters, a project of this type needs to be simple, flexible, and longer. The project was designed with multiple implementing agencies and funding streams, and its complexity initially overwhelmed local capacity. Several aspects appear to have worked well in addressing these challenges: (1) a willingness from the GoV and Bank to be flexible, simplifying and adjusting project design as needed; (2) leaving a single agency in charge of coordination (VMGD), whilst ensuring the commitment and oversight from other sectoral agencies and provinces; (3) building capacity of local officers by pairing them with international advisors; (4) frequent Bank technical missions and videoconferences (at times monthly) to address bottlenecks; and (5) extending the project closing date to allow sufficient time for micro- projects to be completed. Recommendation: In contexts of limited capacity and high vulnerability to natural disasters, ensure project design matches local capacity, either by simplifying project design, or choosing a multi- phase program, with the first phase focused primarily on setting and testing procedures and building local capacity. • Lesson 2. Project management and monitoring tools need to be linked. The project faced difficulties in developing, reconciling, and updating its various systems, particularly in M&E and financial management. A key lesson learned is that these systems need to be linked in order to serve their main purpose of project planning, monitoring and reporting – if possible, in real time. This was partially done by developing an Excel template which linked project costs with procurement monitoring, but the detailed information on field activities and their monitoring remained separate. Recommendation: The Bank should consider developing simple and practical templates (including those on hand-held devices that allow for geo-referencing), for client countries with capacity constraints - linking cost tables with procurement monitoring, disbursement accounts, activity work plans, and performance monitoring. Reconciliation with country systems should be done at the national, rather than project basis. These tools should automatically help generate standard project reports such as updated performance monitoring tables, IFRs, and Withdrawal Applications. • Lesson 3. Bulk procurement, combined with community-led local inputs was effective and well accepted, but there is scope to pilot more community-driven approaches in the future. The beneficiary survey revealed generally a high degree of satisfaction with the process followed by the micro-projects and it did not seem to compromise community ownership – 89% of the beneficiaries felt the micro-projects were a high priority to their household, and 72% felt they had been carried out as expected. At the same time, 40% of the respondents felt that communities should play a stronger role in procurement, suggesting there is scope to pilot a more community- driven development approach in the future. Recommendation: Promote bulk procurement in rural areas with difficulty in accessing goods and services but allow for the gradual delegation of responsibilities to the communities, according to their capacity. • Lesson 4. Community micro-projects need clear exit strategies and follow-up. Largely due to the disruption caused by TC Pam, the project did not have sufficient time to follow up on micro-projects already completed and handed over to the communities, in order to verify whether they were adequately maintained. The agreed exit strategy at hand-over was that communities would be fully responsible for maintenance, except in the case of feeder roads. However, despite extensive training (e.g. for gravity fed systems), maintenance may become a challenge for Page 35 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) communities that may lack capacity, cash resources, or organizational skills to access province or national government support. Recommendation: The Bank should agree with governments on the maintenance strategy for community investments, preferably during project preparation, and articulate its plans and commitments in its final report. Community projects should continue to be monitored for a minimum period of 6-12 months after completion. • Lesson 5. Measuring community resilience is difficult and may require clear agreed guidance up front. A key lesson learned was the difficulty the project faced in formulating and measuring indicators aimed at capturing community resilience, and the need for a clear guidance on how to measure and report it. The PDO statement (both before and after restructuring) was also too broad and high level to be captured within a project normal lifetime. Recommendation: For projects of this type, the PDO and outcome indicators may need to focus on more specific aspects of resilience (e.g. community adaptive capacity). More analytical work may also be needed on how to measure indicators that capture multiple dimensions of resilience (drawing for example on resilience indices developed by other projects) while also contributing to measuring progress towards a longer-term resilience goal. . Page 36 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: : Pilot investments in select villages to increase the resilience to the impacts of natural hazards and climate variability and change Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of beneficiaries Number 0.00 15000.00 3000.00 28829.00 directly supported by the project (disaggregated by 01-Sep-2016 01-Jan-2013 23-Jun-2017 28-Jun-2019 gender, water- and agriculture-related investments) Beneficiaries disaggregated Number 0.00 0.00 1400.00 13800.00 by gender 01-Sep-2016 01-Jan-2013 23-Jun-2017 28-Jun-2019 Beneficiaries from water- Number 0.00 10000.00 1800.00 12378.00 related investments 01-Sep-2016 01-Jan-2013 23-Jun-2017 28-Jun-2019 Beneficiaries from Number 0.00 1500.00 1000.00 3934.00 Page 37 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) agriculture-related 01-Sep-2016 01-Jan-2013 23-Jun-2017 28-Jun-2019 investments Comments (achievements against targets): The beneficiaries counted here include those supported directly through the new micro-projects.The resilient micro-projects reached about 28,800 beneficiaries, substantially exceeding the revised PDO target of 3,000.The significantly higher-than-expected number of beneficiaries was both due to the actual achievements of the project as well as an under-estimation of the population and thus potential beneficiary numbers at restructuring. In addition, the gravity fed systems were more extensive than at restructuring and thus benefited a lot more people. Of these,13,800 were women (against a revised PDO target of 1,400); about 12,370 beneficiaries benefitted from water-related investments (against a target of 1,800); and 3,930 beneficiaries received support through agriculture-related micro-projects, exceeding the revised PDO indicator of 1,000. An additional 3,550 beneficiaries benefitted from multi-purpose centers and roughly 9,000 beneficiaries benefited from all-weather access feeder roads. Unlinked Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion People provided with access Number 0.00 0.00 1500.00 9456.00 to improved water sources 01-Jan-2013 01-Jan-2013 23-Jun-2017 28-Jun-2019 Comments (achievements against targets): 9,456 beneficiaries accessed improved water sources through repairs of existing water systems as part of the TC Pam recovery support. The project helped repair 5 schemes in Tongoa and 5 in Tanna Island (including upgrades and extensions to the original scheme size). Water-related investments – including rehabilitated systems after TC Pam - were implemented with resilience and safety standards. All the rainwater harvesting systems were designed with adequate storage to operate during drought periods. The structures (buildings, storage tank towers, rainwater harvesting roofs, and water tanks) were designed to withstand earthquake and cyclone wind loads. In addition, the water tanks were elevated to ensure protection against floods and groundwater wells were built high to avoid flood water contamination and strapped down to minimize damage during cyclones. Page 38 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) The beneficiary survey conducted at the end of the project indicated that 89% of the respondents were satisfied/very satisfied with water quality, and 77% were satisfied/very satisfied with water quantity - exceeding the original outcome indicator target of 75% (the survey did not differentiate between beneficiaries with access to repaired/improved water systems and new systems). A.2 Intermediate Results Indicators Component: Strengthen disaster risk management systems to increase the resilience to the impacts of natural hazards and climate variability and change Pre-restructuring Increase the resilience of communities t Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Strengthened institutional Text Low None as indicator was High High structures and capacity to revised deliver climate-resilient investments at the provincial 01-Jan-2013 01-Sep-2013 23-Jun-2017 28-Jun-2019 to local levels Comments (achievements against targets): Two Provincial Disaster Centers established and now operational on Tanna and Tafea Province and on Sola in Torba Province. Provincial governments and officers were involved in the micro-projects and are involved in climate and disaster resilience with the communities, using the processes and resource people housed in the provincial disaster centers for advice on project-related activities. Page 39 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of early warning Amount(USD) 0.00 8.00 8.00 9.00 system stations established and/or repaired 01-Sep-2016 23-Jun-2017 23-Jun-2017 25-Jun-2018 Comments (achievements against targets): Target exceeded: 8 new early warning stations were established and one existing site was repaired on Tanna after TC Pam. The new sites were established on islands of Tanna, Gaua, North Ambae, Maewo, Santo, Malekula, Ambrym, Lopevi and Efate. These stations have helped increase the coverage of VMGD who are now able to effectively monitor and act on seismic risks. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Availability of Operational Text None 2 2 2 Manuals providing procedures / protocols for 01-Jan-2013 01-Sep-2013 23-Jun-2017 28-Jun-2019 Climate Change adaptation & Disaster management projects/programs Comments (achievements against targets): Two manuals/protocols were produced which met the revised IRI target. They included the project operations manual and a micro-project appraisal manual which is being further developed by the DLA as a guide for implementation of climate and disaster resilience community-level projects. Page 40 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Guidelines/manuals/tools Number 1.00 3.00 3.00 3.00 based on good local and international practices 01-Jan-2013 01-Sep-2013 23-Jun-2017 28-Jun-2019 developed for guiding CBA in Vanuatu being used at national and provincial level Comments (achievements against targets): The indicator was revised from a text to a numerical target in level 1 restructuring. Three operations manuals were produced in line with the revised IRI target (2). They included the project operations manual; a trainer of trainers’ manual for community development programs which has reportedly been used widely; and a micro-project appraisal manual which is being further developed by the DLA as a guide for implementation of community-level projects. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Provincial Disaster Centers Number 0.00 2.00 2.00 2.00 Fully Equipped and Operational 01-Jan-2013 01-Sep-2013 23-Jun-2017 28-Jun-2019 Comments (achievements against targets): Target achieved: 2 Provincial Disaster Centers fully equipped and now operational. In addition the project supported the refurbishment of the national multi-hazards warning center which has improved the preparedness and response to disasters. Page 41 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Data communication links Percentage 0.00 100.00 100.00 100.00 established to volcano monitoring sites 01-Jan-2013 01-Jan-2013 23-Jun-2017 28-Jun-2019 Comments (achievements against targets): The eight early warning stations that were established by the project are all operational. Two are not yet on automatic transmission due to issues outside of the control of the project. Unlinked Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Investments in post TC-Pam Number 2.00 8.00 8.00 9.00 recovery needs 01-Sep-2016 23-Jun-2017 23-Jun-2017 28-Jun-2019 Number of water systems Number 2.00 8.00 8.00 9.00 repaired 01-Sep-2016 23-Jun-2017 23-Jun-2017 28-Jun-2019 Comments (achievements against targets): Page 42 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) This was a new indicator introduced at restructuring to capture the work on the support the TC Pam recovery. Because of the support from other development partners, the work ended up supporting gravity fed systems and the provision of 30 water tanks to islands affected by TC Pam. 4 water systems were repaired on the island of Tongoa and 5 on island of Tanna. Target was exceeded. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of villages receiving Number 0.00 0.00 65.00 67.00 climate and disaster resilient investments 01-Jan-2013 01-Jan-2013 23-Jun-2017 25-Jun-2018 Comments (achievements against targets): Target exceeded: The villages (or communities) are those affected by TC Pam as well as those in Ambrym affected by a cyclone, landslide and volcanic eruption. The project supported a total of 67 micro-investments exceeding the target of 65 (18 agriculture, 22 rainwater harvesting, 15 gravity fed systems, 6 multi-purpose centers and 6 feeder roads). Beneficiary satisfaction was assessed by the Bank in April 2019 in island of Tanna and the PMU also conducted beneficiary surveys in Ambrym and Shepherds. As per the main text, in Tanna, the beneficiaries reported high satisfaction with the micro-projects (78%). Beneficiary satisfaction was highest for Rainwater Catchment (RWC) micro-projects, with 100% of respondents Very Satisfied (74%) or Satisfied (26%). The degree of satisfaction with Gravity- Fed Systems (GFS) and Multipurpose Centers (MPCs) was similar, with 73% Satisfied to Very Satisfied, but a larger proportion of respondents were Very Satisfied with GFS (61%). Satisfaction amongst Feeder Roads (FRs) beneficiaries was 72%, of which 61% were Very Satisfied and 11% Satisfied. The number of beneficiaries surveyed for Agriculture Demonstration plots (AGRIC) interviewed (4) was too small to derive any meaningful conclusions. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 43 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Number of people in rural Number 0.00 0.00 6000.00 12387.00 areas provided with access to Improved Water Sources 01-Jan-2013 01-Jan-2013 23-Jun-2017 28-Jun-2019 under the project Comments (achievements against targets): This IRI target should have been deleted at restructuring as water-related beneficiaries were captured under other indicators. . Page 44 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) B. KEY OUTPUTS BY COMPONENT Strengthening Disaster risk management systems to increase the resilience to the impacts of natural hazards and climate variability and change Outcome Indicators 1. (No outcome indicators for this PDO) 1. Strengthened institutional structures and capacity to deliver climate-resilient investments at the provincial level to local levels Intermediate Results Indicators 2. Number of early warning system stations established and/or repaired 3. Data communications link established to volcano monitoring sites 4. Provincial disaster centers fully equipped and operational 1. Strengthening of the NAB & PMU (capacity building on project management, procurement, financial management and M&E) Key Outputs by Component 2. Strengthening of NDMO (NDMO Act and 2 provincial disasters centers (linked to the achievement of the Objective/Outcome 1) established) 3. Strengthening of EWS (establishment of 8 EWS and repair of one existing) Pilot investments in select villages to increase the resilience to the impacts of natural hazards and climate variability and change 1. Direct project beneficiaries (Disaggregated by gender, water- and agriculture- related investments (number) • Beneficiaries disaggregated by gender (number) Outcome Indicators • Beneficiaries from water-related investments (number) • Beneficiaries from agriculture-related investments (number) • People provided with access to improved water sources (number) 1. Investments in post TC-Pam needs Intermediate Results Indicators 2. Number of water systems repaired Page 45 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) 3. Availability of operational manuals providing procedures/protocols for climate change adaptation & disaster management 4. Guidelines/manuals/tools based on good local and international practices developed for guiding CBA in Vanuatu being used at national and provincial level 5. Number of villages receiving climate and disaster resilient investments 6. Number of people in rural areas provided with access to improved water sources under project 1. 67 micro-investments delivered Key Outputs by Component 2. 9 water systems repaired (linked to the achievement of the Objective/Outcome 2) 3. Project Operations Manual developed 4. Micro-projects manual developed Page 46 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Samuel Wedderburn Task Team Leader (from preparation to 2015) Michael Bonte-Grapentin Task Team Leader Jinan Shi Procurement Specialist David Bruce Whitehead Sr. Financial Management Specialist Kylie Coulson Financial Management Specialist Ademola Braimoh Environmental Safeguards Specialist Habiba Gitay Climate Change Specialist, Team member Tevi Maltali Obed Disaster Risk Management Specialist, Team Member Nathan Hale Team Member Isabella Micali Drossos Legal Counsel Wen Zhou Legal Jane Millicent Sprouster Operations Josefo Tuyor Social Specialist Supervision/ICR Habiba Gitay, Task Team Leader Tevi Maltali Obed Co-Task team Leader Samuel Wedderburn Task Team Leader (from preparation to 2015) Michael Bonte-Grapentin Task Team Leader (2015-May 2017) Zhentu Liu Senior Procurement Specialist David Bruce Whitehead Senior Financial Management Specialist Janet Gamarra Financial Management Specialist Peter Brandriss Operations Analyst Jane Millicent Sprouster Team Member Page 47 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Nathan Lee Engle Team Member Nicholas K.W. Jones Team Member Ross James Butler Social Safeguards Specialist Nathalie Staelens Senior Environmental Engineer Wolfhart Pohl Lead Environmental Safeguard Specialist Joyce Onguglo Social Development Specialist Cristiano Nunes Senior Procurement Specialist Jinan Shi Senior Procurement Specialist Duangrat Laohapakakul Senior Counsel Isabella Drossos Senior Counsel (at preparation) Laisa Boedoro Program Assistant Nathan Hale Program Assistant (till late 2017) Cynthia Dharmajaya Program Assistant (DC) Penelope Ferguson Senior Environment Safeguards Expert Felix Taaffe Environment Safeguards Expert Victoria Lazaro Social Safeguard Expert Richard Croad Water Resources & Coastal Engineering Expert Kylie Coluson Financial Management Specialist (former) Ademola Bairmoh Team member Josefo Tuyor Team member Nathan Engle Team member Nicholas Jones Team member Olivia Warwick Team member Wen Zhou Team member/legal Counsel B. STAFF TIME AND COST Page 48 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY09 4.460 37,215.18 FY10 5.156 27,785.39 FY11 3.490 18,289.37 FY12 8.749 123,356.53 FY13 4.347 22,984.81 Total 26.20 229,631.28 Supervision/ICR FY13 .550 17,562.64 FY14 13.062 116,426.52 FY15 13.097 115,653.66 FY16 27.283 114,876.22 FY17 19.276 79,581.23 FY18 13.576 58,403.94 FY19 25.796 174,437.39 FY20 4.238 38,270.45 Total 116.88 715,212.05 Page 49 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) ANNEX 3. PROJECT COST BY COMPONENT Table 3.1: Project Costs by Component Amount at Actual at Percentage Components Approval Project of actual to (US$ Million) Closing originally (US$ Million) approved Component 1 - Institutional Strengthening for Climate Change 4.37 4.84 111 Adaptation and Disaster Risk Management Component 2 - Increasing Community Resilience 2.43 2.62 108 Component 3 - Promotion of Improved Technologies for Food Crop Production and Resilience 2.48 1.42 57 Component 4 – Rural Water Security 2.24 1.05 47 Total 11.52 9.93 86 Table 3.2: Project Costs by Financing Source Amount at Actual at Project Percentage Financing Source Approval Closing of Approval (US$ Million) (US$ Million) Recipient (Government of Vanuatu) 0.44 N/A31 N/A Global Environmental Facility – TF13810 5.58 5.5833 100 EC EuropeAID Agency – TF 1386932 2.77 1.6230 58 Global Facility for Disaster Reduction and Recovery – TF13694 2.73 100 2.73 Total 11.52 9.93 86 31Government and community contributions were not formally tracked during implementation. However, the Government contribution included at a minimum a US$0.2 million co-financing for the national disaster center in Port Vila. Community contributions were estimated to be about 10-25%of the micro-project costs, contributed as in-kind labor and local materials and would have totaled an additional US$0.2- 0.525 million equivalent. Hence, the estimated total Recipient contribution would have at least equaled, and likely exceeded, the estimated US$0.44 million made at appraisal. 32 At appraisal and as reflected in the PAD, the EU commitment was estimated to be €2.11 million (US$2.77 million equivalent). In reality, it turned out to be lower, at € 1.897 million (US$2.04 million equivalent). This was disbursed in two tranches, the first (€ 0.923 million) released in September 2013, and the second in 2015. As the EU grant could not be extended further due to the parent Trust Fund constraints, it was closed on April 29, 2016, and the outstanding balance, equivalent to about € 0.627 million was canceled and returned to the donor, leaving a total of €1.27 million (US$1.62 million equivalent) disbursed. The final disbursement exceeds the total US$ estimate at restructuring due to the changes to the Euro and US$. Note that the datasheet financing table (page 2) contains several errors that are corrected in this Annex: it includes a project preparation grant (TF95486) which should not be accounted against this project; it also double counts the EU grant, by including both the original estimate at appraisal (US$2.77 million equivalent, reported as EC EuropeAID Agency) and the actual commitment (US$2.04 million, reported as TF13869). Table 3.2 gives the correct figures. 33Although US$0.26 million remained undisbursed as of December 23, 2019, withdrawal applications for the full undisbursed amount are expected before the application deadline of December 28, 2019. Page 50 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) ANNEX 4. EFFICIENCY ANALYSIS The efficiency analysis applied to the project at ICR stage relied on a cost-benefit analysis of: (1) the early warning system (Sub-Component 1.3), comprising 11.2% of the final project costs; and (2) the micro-projects implemented under Components 2-4 after TC Pam (comprising an additional 46.4% of the final project costs). In addition, cost effectiveness approaches were used to assess the efficiency of the micro-projects. This could not be applied to the early warning system as there was no relevant cost comparator available. A. COST BENEFIT ANALYSIS 1. Early Warning System The project provided two major contributions to Vanuatu’s Early Warning System (EWS): • A network of 8 seismic stations located near the major volcanos, allowing for automatic transmission of real-time seismic and volcanic data as well as images to Vanuatu’s Multi-Hazard Warning Center. This in turn enables VMGD to issue updated alert levels for all active volcanos (including the need for a possible evacuation). Previously, volcanic data was mostly dependent on observers’ visits. • It helped refurbish the Multi-Hazard Warning Center, integrate operations of the weather forecast and geo-hazards monitoring divisions, and link them to the National Emergency Operations Center. For the purposes of the economic analysis, it was assumed that the project contributed primarily to improving the early warning of volcanic hazards (including associated earthquakes). As no data existed to quantify the actual impact of EWS in Vanuatu, the economic analysis relied on Vanuatu’s risk profile as estimated by the Pacific Catastrophe Risk Assessment and Financing Initiative (PCRAFI)34, the methodology followed by the Pacific Resilience Program (P154839/40) to estimate EWS cost benefits in Tonga and Samoa35, as well as extrapolations from relevant international studies.36 The analysis is therefore a simulation and must be considered indicative, rather than representative of the actual project benefits. Costs were computed based on the actual investment costs of Component 1.3. (Strengthening early warning systems). Annual maintenance was estimated at 2% of investment costs, consistent with the rate used by the Pacific Resilience Program for EWS. The stations were assumed to last 10 years on average, requiring full replacement costs at the end of this period. Benefits were assumed to include (1) Avoided Asset Losses; (2) Value of Lives Saved; and (3) Value of Avoided Injuries, consistent with the methodology used by the Pacific Resilience Program. The analysis is conservative as it does not 34 PCRAFI (2011). Country Risk Profile: Vanuatu. Pacific Catastrophe Risk Assessment and Financing Initiative, Secretariat for the Pacific Community http://pcrafi.spc.int/documents/136/metadata_detail 35 PREP Project Appraisal Document. Pacific Resilience Program Phase I. The World Bank. 36 Hallegate, S. (2012). A Cost Effective Solution to Reduce Disaster Losses in Developing Countries. World Bank Sustainable Development Network. May 2012. Page 51 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) include savings from false alarms, which can be considerable (the PAD cites a false alarm in Ambae in 2005 as having costed some US$450,000 equivalent). It also does not include incremental benefits derived from preventive planning, such as adjustment of farming practices in accordance with volcano alert levels. Finally, it does not take into account the preventive value of project contributions to the entire multi-hazard warning system, or to offshore earthquakes, which some of the seismic stations also detect. Value of Avoided Asset Losses. Based on a simulation model of 7.6 million earthquakes, PCRAFI16 estimated in 2011 that the average annual loss due to earthquakes and tsunamis (the hazards most closely related to volcanos) in Vanuatu was US$ 11.2 million in direct losses, or 1.5% of GDP. As this analysis did not include volcanos, we used an earlier analysis of probabilistic risk for Vanuatu37 which separated the average annual loss for volcanos (US$3.2 million), from that of earthquakes (US$7.65 million) and tsunamis (US$0.06 million). This indicates that volcanos cause approximately 41.5% of earthquakes and tsunamis losses. Based on an analysis of early warning benefits in flood prevention in Europe, Hallegate (2012) estimated that EWS could help prevent 10-50% of asset losses. The lower value (10%) was assumed for the current analysis. To extrapolate this benefit to a country like Vanuatu requires making assumptions about the comparative development of the two regions’ early warning systems and meteorological services. It was assumed that, with the project interventions, Vanuatu is close to a Group 3 country (countries with well-functioning hydromet services but with gaps in the chain from data production to early warning systems), where the preventive value is about 50% of that of Europe (see Hallegate 2012). Thus, it was assumed that the EWS installed by the project helped prevent a conservative 5% of annual asset losses caused by volcanos. The value of annual avoided asset losses due to the project’s EWS was therefore estimated as follows: 1.5% Vanuatu GDP38 x 41.5% x 5% = VT 29 million/year (see Table 4.1 for further assumptions). This assumption, though conservative, is not unreasonable given that the EWS allows the Government to issue alert levels for individual volcanos and, if necessary, order an evacuation - as was done in 2018 in Ambae, when the entire population (11,000 people) was evacuated. At the same time, the relatively low preventive value used (5%) recognizes that many fixed assets – such as houses and fields – may still be destroyed by volcano ash and landslides despite the early warning, enabling people to save mostly household contents39. Values of Lives Saved. The PCRAFI model estimates 45 casualties a year from earthquake and tsunami risk but does not separate between fatalities and number of persons injured. Historical records kept by Vanuatu’s Geo-Hazards Department indicate that over the last 125 years (1894 to 2019) there were 41 reported fatalities from volcano eruptions, or an average of 0.33 fatalities a year40. Similar to asset losses, it was assumed that the project’s EWS would help save 37 https://www.preventionweb.net/countries/vut/data/. 38 The PCRAFI estimate of average annual loss (1.5% of GDP) was used here since it was considered more reliable than the estimates provided by Preventionweb. 39 Household contents were estimated at 5% of average house damage value in Government of Vanuatu 2015. Post Disaster Needs Assessment. Tropical Cyclone Pam, March 2015. 40 Sandrine Cevuard, Acting Manager of Geo-Hazards Department, personal communication, November 25, 2019. The reported fatalities occurred in Yassur (1994, 2 dead), Aoba (1914, 12 dead), Ambrym (1913, 21 dead) and Ambrym (1894, 6 dead). Volcano eruptions were also reported in 1958 (Kuwae), Yasur (1995) and Aoba (2018) but with no fatalities recorded. Page 52 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) 5% of the annual fatalities due to volcano risk, or 0.0164 lives a year on average, starting one year after the system’s installation. Deaths were assumed to occur at mid-life (35 years) and impact 37 years of life, based on an average life expectancy in Vanuatu of 72 (in 2016). To compute the Value of a Statistical Life, the analysis used the average monthly per capita income for rural Vanuatu from the 2010 Household Income and Expenditure Survey, adjusted to 2017 by the Consumer Price Index (2010=100), or VT 17,950/capita/month, which includes earnings, subsistence and remittance income. This yielded a Value of Statistical Life estimated at VT 8.0 million per capita, or US$71,200 equivalent, which is consistent with the Copenhagen Consensus international estimate (between US$35,000 and US$175,000) (Hallegate 2012). Thus, the value of lives saved was estimated as follows (see Table 4.1): Annual fatalities due to volcanos x 5% x Value of a Statistical Live = VT 130,703/year This is almost certainly an under-estimate, as the number of actual fatalities due to volcano eruptions is likely higher than what was reported internationally. The estimate also reflects an annualized benefit. The corresponding benefits during a given volcano eruption would be considerably higher. Value of Injuries Averted. Based on PCRAFI’s estimates of casualties and the same assumptions used above, the number of severely injured people averted by the early warning system is estimated to be: Number of Casualties x 41.5% x 5% - Lives saved = 0.93 injuries/year During a volcano eruption, affected people can be injured by projectiles, ash, or by resulting earthquakes and landslides. It was assumed that injured people would require 5 days of hospitalization and lose one month of income during convalescence. To estimate hospital costs, the analysis used the 2005 Cost per Bed Day for Tertiary Hospitals reported for Vanuatu by the World Health Organization41, adjusted to 2017 values (see Table 4.1), which is approximately VT 3,360 a day. To estimate potential income losses, the average monthly per capita income for rural Vanuatu was again used, yielding a value of injuries averted due to the project’s EWS as follows (see Table 4.1). # of Injuries Averted x [(Annual Health Costs Avoided) + (Annual Income Lost to Injuries)] = VT 32, 164/year Results. Taking all costs and estimated benefits into account, the resulting Economic Internal Rate of Return (EIRR) for the EWS sub-component was 20.0%, with a Net Present Value (at 10% discount rate) of VT 56.1 million (US$499,740) – see Table 4.4. This is comparable to the returns estimated by the Pacific Resilience Program for Samoa and Tonga (19%) although the latter analysis did not distinguish between ESW and risk reduction and resilient components and used less conservative assumptions (a reduction in mortality, injuries and losses of between 15-25%). The cost-benefit analysis of EWS was fairly robust to changes in assumptions, including a 25% increase in costs and 25% decrease in benefits. It remained above the social discount rate of 6%42 even when it is assumed that benefits decreased progressively and stop after 10 years (see Table 4.4). 41 Source: World Health Organization (2005), Choosing Interventions that are Cost Effective (WHO-CHOICE), 2005 https://www.who.int/choice/country/vut/cost/en/. 42 World Bank OPSPQ (2016). Discounting Costs and Benefits in Economic Analysis of World Bank Projects. Page 53 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Table 4.1: Summary of Assumptions in the Economic Analysis of the Early Warning System Page 54 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) 2. Micro-Projects The cost-benefit analysis followed established methodologies to evaluate benefits of small, community-based rural projects, including a mixture of market and non-market values (Table 4.2). The detailed assumptions are shown on Tables. 4.3-4.5 and are summarized below. Table 4.2: Micro-Projects Sampled for the Ex-Post Cost Benefit Analysis Type of Micro-Project Incremental Benefits Benefits Time Frame Costs Costs Time Frame Value of Time Saved Investment Costs Year of investment Gravity Fed Systems Value of Increased Agriculture 1 year after investment, Replacement Costs After 15 years Production1 then annually for 25 years Routine Maintenance Yearly Health Costs Avoided Rainwater Harvest Value of Time Saved Investment Costs Year of investment Value of Increased Agriculture Replacement Costs After 15 years Feeder Roads Production1 Periodic Maintenance Every 2 years Savings in Travel Costs Routine Maintenance 3 times a year Investment Costs Year of investment Value of Assets Saved Multipurpose Centers Replacement Costs After 25 years Value of Ancillary Use Routine Maintenance Yearly 1– Applied only for households reporting incremental sales, in lieu of value of time saved. For other households, the value of time saved was estimated at its opportunity cost. GFS – Gravity Fed Systems; RWC – Rainwater Catchment (Harvest); FR – Feeder Roads; MPC – Multipurpose Centers Costs were based on the actual micro-project costs as reported by the PMU, including transport, delivery and installation of materials, and cost of supervision (see Annex 6.B). The structures expected lifetime and routine and replacement costs were derived from interviews with project engineers and sectoral Ministries (see Tables 4.3 to 4.4). Incremental Benefits. The assumptions used to estimate incremental benefits were derived from in-depth interviews with key respondents of 1-3 sampled micro-projects per prototype, representing 7% of all GFS, 33% of FRs and 50% of all MPCs funded by the project (and 13-62% of their respective costs). The sampling was considered sufficient given that micro-projects were relatively standard and located primarily on an island (Tanna) with fairly homogenous socio- economic conditions. Major benefit assumptions (e.g. time spent collecting water, use of time savings, etc.) were compared and validated with answers from the beneficiary assessment and expert opinion from sectoral specialists, and complemented by national statistics. Benefits of RWCs were calculated through secondary data and the results of the beneficiary survey. The benefits of agricultural micro-projects could not be computed at ICR stage due to logistical constraints. However, their costs were included in the aggregated cost-benefit analysis. Benefits were assumed to accrue for 25 years, starting from the year immediately after completion of the micro-project. Financial prices of products sold, and all costs were assumed to be equivalent to economic prices given that (1) any imported materials were exempted from duties; and (2) beneficiary households operate well below the sales threshold (4 million VT) where they would register for, and use value added tax. Page 55 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Water Micro-Projects Gravity fed and Rainwater Harvest Catchment systems bring numerous benefits to beneficiaries, and particularly to women: they reduced time spent daily in collecting water, enabling them to dedicate more time to income generating activities, household chores, and children care. They also improve water quality and quantity, helping reduce the incidence of water-borne diseases, enhance the feeling of cleanliness, help them cope with volcano ashes, and prepare for potential droughts and cyclones. For the purposes of the cost-benefit analysis, the benefits of water micro-projects were estimated based on (1) the estimated value of time saved and (2) savings in health costs from the installation of safer water sources close to home. Value of Time Savings. According to Vanuatu’s Household and Water Inventory (conducted in 2014-2016) some 58% of the water is traditionally collected by women, and 31% by female children, although in Tafea, adult men also help with the collection.43 The time saved by the introduction of a gravity fed system was found to be significant: 50 minutes per day on average for all beneficiary assessment respondents. For rainwater harvest beneficiaries, time savings averaged 57.5 minutes a day per household. The value of time saved, in particular, was significant for GFS beneficiaries, as up to a third of the households reported using it to work in gardens, leading to incremental market sales. The analysis assumed conservatively that only one third of the households engaged in incremental agriculture sales, and 1 in 20 households in the sale of pigs, as reported by the beneficiary survey. For these households, the value of increased agricultural production was computed as follows: V - Value of increased agricultural production: V = (Q2-Q1) x P Q2 = Quantity produced without project Q1 = Quantity with project P = price per unit of product sold According to the women interviewed, they are now able to sell an incremental 5,000 VT/month from cooked food and mats. In addition, some respondents reported having time to now rear pen-fed pigs for local sales. Pigs are highly praised throughout Vanuatu and used extensively in traditional ceremonies. Key Respondents reported that as a result of the extra time saved, they were now able to sell approximately one pig a month, for 40-50,000 VT/unit. For all other houses that did not report incremental sales, as well as for all beneficiaries of RWC, the value of time saved was computed at its opportunity value, estimated according to Water Aid’s methodology44: V – Opportunity value of time saved: V= [(T1/Q1) – (T2/Q2)] x W x Q1 T1 = time spent before project T2 = time spent after project 43 Powerpoint summarizing the results of the 2014-16 Household and Water Inventory, Water Department. 44 https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=2ahUKEwi4xLil_dfgAhXC1uAKHeAUB60QFjAAegQIC RAC&url=https%3A%2F%2Fwashmatters.wateraid.org%2Fpublications%2Fevery-ones-a-winner-economic-valuation-of-water- projects-discussion-paper&usg=AOvVaw0gg9BX-Q6q7QkyhApQ1B75 Page 56 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Q1= quantity used before project Q2 = quantity used after project W = prevailing wage rate (daily or hourly as appropriate) This estimate takes into account the increased quantity of water provided by gravity fed systems, which are designed to yield a minimum standard of 50 l/person/day for washing, bathing, drinking and cooking – representing a 2-3 fold increase in water consumption from traditional sources. It was assumed that for RWC beneficiaries, the quantity of water consumed remained the same (as basic needs for drinking and cooking remain constant at about 7 l/person/day). To estimate the opportunity value of time saved, and given that there is no prevailing rural wage rate in Tanna Island, the analysis used the monthly per capita income in rural Tafea (where Tanna island is located) as reported by the 2010 Household Income and Expenditure Survey, (the latest available). This value, which includes subsistence income and remittances, was adjusted to current times by the crop production index (2010=100) as reported by World Bank Development Indicators45, yielding a value of VT 10,026 /month/capita (approximately US$3.0/capita/day). A workday was assumed to include 12 active hours, resulting in an opportunity value of time of 0.46 VT/minute/capita. Using the above formula, and assuming conservatively that only one person per household would benefit yields a value of time saved of VT 23.9/day/household for GFS and VT 26.7/day/household for RWCs. Value of Improved Health (illnesses avoided). The value of improved health was estimated as follows, according to the WaterAid methodology: V -Value of Improved Health (illnesses avoided): V = I x CL L = number of illnesses avoided Cl – treatment cost per household per illness Key respondents reported that before the project, their children suffered from 2-3 diarrheal episodes a year, requiring purchase of medication at a cost of 50-100 VT per episode. It was assumed conservatively, that access to cleaner water for both GFS and RWC beneficiaries would reduce the incidence of diarrheal diseases requiring a visit to the health clinic by half. This estimate did not take into account school days missed (about 2-3 days per episode), or the cost of health clinic’s visits, which is free to the community but not to the government, and it is therefore considered conservative. Results. Taking all costs and estimated benefits into account, the resulting EIRR was 32.5% for GFS and 11.3% for RWCs, in line with those reported for comparable gravity fed and rainwater catchment water schemes under the Solomon Islands Rural Development Program (21%)46. These estimates are fairly robust to changes in benefits and costs, even when it is assumed that benefits decline progressively over a period of 10 years due to lack of maintenance. However, for a 25% decrease in benefits, the EIRR for RWCs declines to 5%, slightly below the social discount rate of 6% (see Table 4.3). 45 WBDI CPI data are not available for Vanuatu. 46 Braun, Jennifer (2014). Financial and Economic Analysis for Solomon Islands Rural Development Programme (RDP 2) – Working Paper, and Excel worksheet (FEA_SI_RDP2_Updated_Nov2017.xls) with adjusted estimates at restructuring (2017). Page 57 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Table 4.3: Summary of Assumptions in the Economic Analysis of Water Micro-Projects Page 58 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Feeder Roads Benefits. Benefits of feeder roads were estimated based on (1) reduced costs of traveling to markets; (2) reported incremental vegetable and food sales; and, for households that did not report incremental sales, (3) the opportunity value of time saved. Increases in sales were estimated conservatively and did not include a reported boost in tomato sales in Iru-Iamsini (which can now be transported to Port Vila) or incremental kava sales. They also did not include the value of statistical lives potentially saved by easier access to health clinics – a significant benefit in Ienvitana where 5 elderly and expectant mothers reportedly lost their lives over the previous decade. Savings in Travel Costs. Based on local interviews, transportation costs declined by a fifth following improved access by buses and taxis, dropping to VT 200 per return trip (from 1,000/return trip without the project), or an estimated incremental benefit of VT 83,200 per household per year. As for gravity fed systems, it was assumed conservatively that only one tenth of the households would benefit from travel cost savings, primarily women who go down to local markets twice a week to sell agricultural produce. Incremental Sales. The boost in sales resulting from the feeder roads includes prepared food, vegetables and kava sold in local markets, as well as tomatoes, which in Iru-lamsini can now be supplied via a wholesaler to the Port Vila market. Sales of these products more than doubled following the construction of the feeder road, as women were now able to transport them by bus, rather than having to walk through trails to the previous local access point. The incremental value of agricultural production was estimated using the same method as for GFS. For the purposes of the baseline analysis, only increased sales of food and vegetables were accounted, yielding an incremental value of VT 338,000 /household a year (US$3,010/household/year). As for the gravity fed systems, it was assumed only one tenth of the households would benefit directly from these levels of incremental household sales. Value of Time Saved. For the remainder 90% of beneficiary households, the value of time saved was estimated as follows: V – Opportunity Value of Time Saved V = (T1 – T2) x W T1 = time spent without project (in minutes) T2 = time spent with project (in minutes) W = prevailing wage rate (per minute) The estimated time saved, based on key respondents interviewed, was 170 minutes twice a week, or 49 minutes a day, taking into account twice-a-week travel. Using the same proxy to value time saved as for the water micro-projects yielded an estimated value of time saved of VT 22.5/day/household. Results. Taking all costs and benefits into account yielded an EIRR of 42.0%, which is comparable with the values estimated for rural roads and footbridges in the Solomon Islands Rural Development Program (RDP, 49%). This estimate is fairly robust to changes of 25% in benefits and costs, as well as scenarios where benefits decline progressively over a period of 10 years (see Table 4.4). Page 59 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Table 4.4: Summary of Assumptions in the Economic Analysis of Feeder Roads Page 60 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Multi-Purpose Centers Benefits of multi-purpose centers (MPCs) were estimated based on (1) the value of household assets potentially saved at the centers during disasters; and (2) estimated benefits of other MPC uses. Benefits did not include health or time savings due to the installation of rainwater tanks and are therefore estimated conservatively. They also did not include the value of potential lives saved, a non-negligible benefit in a country where 6 people lost their lives during TC Pam, primarily by flying debris when they tried to take shelter. The detailed assumptions are outlined on Table 4.5. MPCs are designed to be used as safe houses in the advent of a disaster, as a multi-purpose community centers during normal times. This use varied by center and was generally chosen by women – some (like Latanu) chose to use it as a kindergarten; others as a market (Tongariki) or as a weaving/multiple sales center (Buninga). The maximum capacity of the centers is about 30 people when used as a shelter, and 10 people for market/kindergarten activities. Value of Household Assets Saved. Cyclone PAM resulted in total damages and losses of US$449 million according to the Post Disaster Needs Assessment.47 In Tanna Island and the Shepherds, some 80%of traditional houses were destroyed, with damages estimated at VT 300,000 per house. Lost household assets are estimated at about 5%of this value (VT 15,000/house).22 According to PCRAFI estimates, Vanuatu is estimated to incur an annual risk from cyclones and earthquakes of US$48 million/year – roughly 10%of the damages and losses inflicted by cyclone Pam every year. For the purposes of the cost-benefit analysis, it was therefore estimated that the value of household assets at risk would amount to VT 1,500/household per year. The standard guidance of NDMO for evacuations during disaster alerts is for people to go to safe houses taking only their children and food and leaving most of their household assets (packed) at home. Based on beneficiary interviews, however, it was estimated that people brought about 20%of the value of their household assets to the shelters (primarily as valuables and small items), or VT 300/household/year. Considering that each shelter can harbor 30 people, or 3.3 full households on average, the annual savings in household assets potentially provided by the MPC was estimated at VT 990 per center per year. Benefits of MPCs used as Kindergartens. The benefits of MPCs that are being used as kindergartens is deemed to be substantial. In the site visited in Tanna Island (Latanu) only one person had attended primary school, given the community remoteness from the main road where the school is located. The newly installed kindergarten plays a key function in teaching 5-6 year old children how to read and write, and preparing them to attend primary school. Once they complete kindergarten (usually 2 years) they are also old enough to walk safely to the primary school. The kindergarten hosted 10 children, of which 5 were girls. Based on the persistence rate to grade 5 in Vanuatu48 (76%), it was estimated that 8 children would finish primary school and would have the opportunity to earn at least a minimum wage salary, which is currently VT 200/hour or VT 34,667 per year. This was compared to a without project situation where, without having attended school, children would go on to earn a semi-subsistence livelihood, valued at the average monthly income for rural Tafea (VT 10,026 per year). It was assumed that the children would only start earning income 15 years after attending kindergarten (age 20). 47 Government of Vanuatu 2015. Post Disaster Needs Assessment. Tropical Cyclone Pam, March 2015. 48 WDI data, 2008. Page 61 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Table 4.5: Summary of Assumptions in the Economic Analysis of Multi-Purpose Centers Page 62 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) As kindergarten attendance lasts two years, it could be argued that MPCs would continue to yield benefits on future batches of students. However, it was assumed that given the importance of the kindergarten for this community, an alternative site would have been found, and thus only benefits of the current batch of students (10 children) were considered. This estimate also ignores the future benefits that educated girls bestow on their respective children and is therefore considered conservative. Benefits of MPCs used for Income Generation. For the two centers used for income generation, the cost-benefit analysis estimated the net value of sales with and without the center, based on interviews with key respondents. In Tongariki (Shepherds Island) for example, women reported being able to sell on average VT 1,000 per day at the center, 10 days a month, with no associated costs (asides from the maintenance of the center, considered below). About 10 women used the center as a market. Without the center, they had to take a boat to Port Vila, at a cost of VT 9,000 per week, selling on average VT 2,000-5,000 a day for 3 days. During their stay in Port Vila, they also incurred food and accommodation costs, returning with little profit. The incremental net benefits were estimated at VT 426,000 per year. A similar calculation was made to estimate the benefits of the MPC in Buninga (Shepherds). Table 4.5 above details the assumptions used. Results. Taking these benefits and the investment and maintenance costs of the centers yielded an EIRR for MPCs sampled of 13.2% in line with the values estimated for RDP community and education centers in the Solomon Islands (11 and 19%, respectively). The EIRR is robust to changes in benefits and costs, remaining above the social discount rate of 6% even for large deviations from the assumptions used (Table 4.6). Aggregated Costs and Benefits for all Micro-Projects Aggregated costs and benefits for the 67 micro-projects were estimated using extrapolated benefits from the sample sites, and all micro-project costs. The costs also included: (a) Costs of agriculture micro-projects (where the benefits could not be computed due to logistical constraints); (b) All associated project management costs, including technical assistance and operating costs. The resulting aggregated EIRR was 18.9% with a Net Present Value at 10% of US$2.2 million (Table 4.6). The aggregated analysis proved fairly robust to changes in assumptions, including a 25% increase in costs (EIRR 14.8%), a 25% decrease in benefits (EIRR 13.7%), and full replacement costs after 10 years (EIRR 17.9%). However, when incremental benefits are assumed to decline progressively and stop after 10 years, the EIRR becomes 0.3%, well below the social discount rate of 6 %. This highlights the crucial importance of post-project maintenance. 3. Final Results for Cost-Benefit Analysis Table 4.6 summarizes the final results of the cost-benefit analysis, covering 57.6% of the project costs. The EIRR of the base case was 19.1%, with a Net Present Value at 10% discount rate of US$2.7 million. As was the case for individual activities, the results are robust to changes in assumptions, except for the “without maintenance” scenario, where the EIRR drops to 1.6%. The ex-post cost benefit analysis did not cover the following project activities: 1) Component 1.2. Strengthening the NDMO, as it involved primarily technical assistance and institutional strengthening. However, the efficacy analysis indicated that benefits were considerable. Page 63 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) 2) Pre-restructuring agricultural technology activities carried out by VARTC and distribution of TC-Pam emergency supplies. These activities involved numerous farmers and recipients in outer islands which were too logistically scattered to be interviewed for the analysis. Adding these costs to the overall cost-benefit analysis results in an EIRR of 16.3% even without any associated benefits, covering 74.4% of the final project costs. Table 4.6: Final Results of Cost-Benefit Analysis Early Warning Total Micro- Sampled Micro-Projects Results Total System Projects GFSs RWCs MPCs FRs Base Case: EIRR 19.1% 20.0% 18.9% 32.5% 11.3% 13.2% 42.0% NPV (10%) in US$ ‘000 2,700 500 2,200 169 14 28 1,043 % of Total Costs represented 57.6% 11.2% 46.4% 13%1 ---2 59%1 62%1 Sensitivity Analysis: EIRR -25% Benefits 13.3% 11.3% 13.7% 23.2% 5.0% 9.9% 30.4% EIRR +25% Costs 14.5% 13.2% 14.8% 25.1% 6.4% 10.6% 32.8% Replacement after 10 years 18.3% 20.0% 17.9% 30.7% 6.3% 7.8% 40.8% Net Benefits declining and stopping after 10 years 1.6% 7.6% 0.3% 17.5% -7.5% 7.4% 27.7% GFS – Gravity Fed Systems. RWC – Rainfed Centers. MPCs – Multipurpose Centers. FR – Feeder Roads 1 – Proportion of total activity costs covered by the analysis 2 – Not sampled; estimates were from secondary data B. COST EFFECTIVENESS ANALYSIS The cost-effectiveness of the micro-projects was assessed by comparing actual unit costs with an engineering original estimate developed by Kramer Ausenco and in the case of gravity-fed systems, that of comparable sized schemes under the Department of Water Resources. The analysis showed that, with the exception of feeder roads, the micro-projects were generally cost-efficient relative to original engineering estimates and/or GoV comparative structures, with unit costs 10-63% below original estimates (see Table 4.7). For RWC structures, the original engineering estimate was VT 67,000/m2 for a 39 m2 open structure (with roof) and water tanks, or VT 2.6 million per unit. Comparable units implemented by the Water Resources Department cost about VT 4 million per unit. Actual project costs for RWC units averaged VT 1.6 million in Tanna Island, and VT 0.4 million in Ambrym, or an average of VT 0.97 million per unit for all 22 RWC micro-projects – a saving of 63% over the original engineering estimates, attributable largely to bulk procurement. For MPCs, original engineering estimates were VT 80,000/m2 for a 39 m2 closed building, including water tanks – or VT 3.1 million per unit. The average project costs of VT 2.8 million entailed savings of 10% relative to the original estimates. The average costs per m2 of multipurpose centers (VT 72,709 or US$647) are higher than for comparable buildings under the Solomon Islands RDP (US$402) but this could be related to transport and local cost differentials between the countries. For GFS, where no original engineering estimate was available, the average project costs of VT 6.3 million per micro- project were compared with similar systems implemented by the Water Resources Department, which cost about VT 10 million for new schemes, and VT 5 million for rehabilitated schemes. Given that most GFS were new schemes, the project Page 64 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) was relatively cost-efficient, although care must be taken in interpreting these results due to likely variations in scheme sizes. For Feeder Roads, the original engineering estimate was VT 6,140/m2 for a 4 m wide road, and VT 2,160/m2 for drainage (on both sides of the road), or VT 28,880 per linear meter (including drainage). The actual project cost averaged VT 29,751 per linear meter, 3% higher than the original estimate. However, this deviation is considered reasonable given that many roads were in remote and steep areas of Tanna Island, where contractors and materials are not readily found. Table 4.7: Cost-Effectiveness Analysis Average Unit Cost (per micro-project) Type of Micro-Project Deviation Engineer’s Estimate/ Project Department Standard Gravity Fed Systems VT 6.3 million VT 10 million -37% (US$56,000) (US$89,050) Rainwater Harvest VT 0.97 million VT 2.6 million (US$8,640) (US$23,150) -63% Feeder Roads VT 29.75 million/km VT 28.88 million/km (US$265,000/km) (U$$25,700/km) +3% Multipurpose Centers VT 2.8 million VT 3.1 million (US$$24,900) (US$27,600) -10% The cost per beneficiary for all micro-projects amounted to VT 7,863 (US$70), ranging from VT 3,198 (US$28) for agriculture activities to VT 10,013 (US$89) for gravity fed systems. The average cost per household was VT 70,418 (US$627). Since the micro-projects were a pilot for Vanuatu, there is no easy comparator in-country. These costs are in general higher than those recorded for the Solomon IslandsRDP (US$56/beneficiary and US$347/household), which could reflect price differentials between the two countries, a higher population concentration in RDP project sites, and RDP’s higher cost effectiveness gained through two phases of a large, community driven development program. Page 65 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS A. FEEDBACK FROM GOVERNMENT An early draft of the key findings of the ICR were discussed with the GoV (including Acting Director General of the Ministry of Climate Change, the Director for the Departments of Climate Change and the DMO) during a technical mission in September 2019 and the feedback received. The feedback is summarized below and has been incorporated in the ICR: − If possible, reference National Sustainable Development Plan and the important point that the plan incorporated climate and disaster risk and resilience. − The pilot micro-projects were good learning by doing process and led to the adoption of bulk procurement, efficiencies in the construction of the all-weather access roads − Efficacy –the importance of self-reliance and self-help and self-supporting/reliant communities as part of the cultural norm in Vanuatu and the understanding of communities would be responsible for maintenance needs to be explained. Diesters bring shocks where external help can be provided, but the importance of being self-reliant after recovery is important. The communities can also bring the need for maintenance to the attention of the provincial governments which can then help prioritize the funding for the major investments from the Provincial fund (Vatu1million annually). − Government commitment to maintenance of the all-weather access road (responsibility of PWD) and the gravity-fed systems (DGMWRD) should be noted. The handover ceremony and the decision-making within the village, the national-province-community decision making also testifies to the empowerment. The Provincial Disaster Centers are being used and maintained by the government and also funding the Disaster officers for both centers. This is also aligned with the summary from the President of Tafea that joined the March/April mission. − Elite capture risk was minimized – the micro-project decision is captured in the governance system (depicted in one of the Aide Memoire). In Tanna in particular, the chief is responsible for the welfare of his people and the decision- making is led through the chiefs. Having the various stakeholders involved along with the Provincial government and the Technical advisory group minimized the risk of elite capture. The diagram could be used to emphasize the message. − Ensure there is recognition (given their importance), of the support to the National Disaster Centre, operations center and the Multi-Hazard Early Warning System in Port Vila − Women on water committee – There should be a note on the cultural context of formal decision-making having to be by men, especially in Tanna. The size of the committee would also determine if women would be represented. Even if there is no formal representation by women, the results from the beneficiary survey should be used to explain the role of women. Water Resource Management Act– amended and gazeted in 2017. It requires the …”in order for the water committees to be registered, at least 40 percent have to be women”. − Access roads – ensure there is enough emphasis on access to basic services from these investments which also contributes to resilience. Bring in specific examples on the importance of the MPCs and demonstration plots for resilience of the wider communities. B. EXECUTIVE SUMMARY OF GOVERNMENT COMPLETION REPORT The project financing agreement was signed in December 2012 between the World Bank and the Government of the Republic of Vanuatu (GoV) followed by a World Bank (WB) mission support in early 2013 where the WB provided training Page 66 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) on their key processes to respective government agencies such as Department of Agriculture and Rural Development (DARD), Department of Local Authorities (DLA), Vanuatu Meteorology and Geo-Hazards Department (VMGD), National Disaster Management Office (NDMO), Department of Finance and the PMU- NAB. The project as stipulated in the objective statement above, focused mainly on institutional strengthening of the government services related to climate change adaptation and disaster risk management as well as focusing on enhancing capacities related to agricultural technologies, increased access and availability of fresh water and livelihood opportunities at the local community and village levels. This is clearly reflected in the project appraisals document through component two micro-investments at local village levels. In the beginning, all components were to be co-coordinated in collaboration with key sectors and the PMU however, two years after the signing agreement, implementations were still slow due to limited capacity in each sector to procure in line with both the GoV and WB procurement processes therefore the project was fully returned to the PMU in 2015 to manage and implement. After much discussions and planning at the national level, the project was finally lifted off the ground and field implementations started in late 2014. Years 2013 and part of 2014 were spent on recruitment and establishment of proper fiduciary processes to allow for project implementations to progress. This led to the development of the project operations manual which was finalized in 2014 with support from WB and technical advisors. The Increasing Resilience to Climate Change and Natural Hazards (IRCCNH) Project was instrumental in completing the establishment of the NAB and PMU and further supported NAB’s early discussions and meetings throughout 2012 – 2014 when Vanuatu started its journey into an integration of the two critical issues of climate change and disaster risk reduction (CCDRR). This early support also paved way forward for initial discussions on CCDRR policy development. The PMU under VMGD’s oversight then took lead in project implementations in collaboration with key sectors such as the DLA, DARD and DoWR in early coordination and planning of the project activities in 2013 – 2014. The project supported VMGD to procure and complete installations of seismic equipment for the seven islands of Vanuatu with active volcanoes. These volcanic activities are monitored closely on a day to day basis and overseen by the operations of the national warning center under VMGD. The project continued supporting the core PMU functions that were started by the EU-GCCA project in 2012 and ending on June 2014. These functions included; project manager, finance, procurement and M&E. In 2014, the project procured a total of fifteen consultants of which seven were international consultants along with a project advisor who supported the development of the Project operations manual for both IRCCNH and MDDRR projects and assisted PMU to form the project steering committee. Most of the international consultant contracts were output based therefore produced their outputs and then left the project in not more than a year span with exception to the Procurement Advisor, Financial Advisor and M&E Advisor whose contracts were longer. PMU managed IRCCNH commencing 2013 alongside two other World Bank projects namely; Mainstreaming Disaster Risk Reduction Project (closed in 2016) and Forest Carbon Partnership Facility Project still implementing in collaboration with department of Forests. The project co-financed the National Warning Centre and the complementary emergency operations centre at NDMO. It further built two Provincial Disaster Centres in Tafea and Torba provinces providing space for disaster and climate change initiatives at the provincial level. The project was initially housed at the VMGD but recently moved to the newly established Department of Climate Change in December 2018. After TC Pam hit Vanuatu in March 2015, the government requested for the project to support the government’s emergency and recovery efforts. IRCCNH then stepped in to support Water, Sanitation and Hygiene (WASH) cluster by providing a total of twenty (20) 10,000 liter water tanks that were distributed to badly hit areas of Efate, Epi and Tanna islands. It further supported the food and security cluster through a provision of agricultural tools worth VT 32 million that were distributed through the agriculture extension networks based in all the six provinces of Vanuatu. A total of Page 67 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) 18,000 farmers and community people received these farming tools from the Area council secretaries on the main islands of each province to support them with their agricultural recovery efforts post TC Pam. Vanuatu Agriculture Research Technical Centre (VARTC) then started by testing Sweet potato, Manioc, Taro and Yam in order to test their resilience to different climatic conditions. Taro and Yam took more time to produce results whilst manioc and kumala took six months to see the results and was able to replicate the cuttings to the islands of Tanna, Santo, Efate and Tongoa/Shepherds after TC Pam in 2015. Similarly, at this stage, the department of water resources (DoWR) procured and built thirty rainwater catchments on the islands of Ambae, Malo, and Epi. The project undertook a Mid-Term Review (MTR) at the end of 2015 involving both the WB and the GoV including all the implementing agencies (VMGD, DARD, DLA, DoWR, NDMO and VARTC). This review also involved important Technical Advisors (TAs) internal to the WB group and other independent technical advisors for finance, procurement, M&E and Agriculture. The MTR focused on reviewing the project achievements and re-planning. The MTR recommended for closure of components 3 and 4 and emphasized on community engagements through the implementation of component two (2) micro-investments (water and agriculture related investments) at community and local village levels. Year 2016 started with a direction to focus on component two (2) – the micro-investments component for the local community and villages levels. A national stakeholder workshop was held to stock-take all small grants initiatives within Vanuatu. It was discovered that there is no micro-project manual which is Vanuatu-owned or such documents that would indicate how to manage any small grants therefore the project started working on a micro-project manual. This document exists in draft and lies with the DLA to develop it further to ensure it is used by government and its stakeholders. Despite initial delays in getting the micro-projects off the ground and challenges faced after devastating TC Pam in 2015, the project field implementations are now 100% completed and while some micro-projects have been smoothly handed over to the local communities, some are still waiting hand over to the provincial government and the local communities. The micro- projects components have made massive impacts to the lives of people living at the community and villages within Tanna, Tongoa, Shepherds and Ambrym. This statement was confirmed through a community beneficiary survey that was conducted in March and April 2019 for Tongoa and Tanna project intervention areas. It emphasized community involvement (and participation by provincial authorities) through buying-in into the initiatives from different partners on the ground from village level to the provincial level which is the governing body. The strategy is to promote the integration of disaster risk reduction and climate change adaptation and to build on successful pilots and programs already tested by government and other agencies however at the time of the project commencement, all government agencies were just starting to integrate climate change and disaster risk reduction into their sector plans and policies. Overall, the focus of the project mainly focused on building resilience in rural communities to climate change, extreme weather events and geo-hazards. Page 68 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) C. EXECUTIVE SUMMARY OF GOVERNMENT ENVIRONMENTAL AND SOCIAL SAFEGUARDS FINAL REPORT It is a World Bank requirement to ensure there is prevention and mitigation of harm to people by protecting their communities and environment in the development process of any projects including the Increasing Resilience to Climate Change and Natural Hazards (IRCCNH) Project. Conducting the Social and Environmental Safeguards prior to implementation to proposed sites is an important part of the World Bank policy that ensures people and their environment are not harmed in any development activities of this project. To guide the implementation of the Project’s activities, the Government and the World Bank have agreed to establish two safeguards framework documents and one of which this project is using is the Environmental and Social Management Framework (ESMF). This Framework is the main document that guides and inform the project officers who are responsible on what actions and or activities to do regarding safeguards requirements. This is to set out the principles and processes that would apply if specified environmental or social issues emerged especially with the selected projects sites initiated under this Project. As a result, an Environmental and Social Safeguards Screening (ESSS) form was developed to help with screening of projects and their sites prior to implementations and also to manage any issues during implementation period. All micro-projects are categorized under the World Bank Category B according to the World Bank ESMF requirement. This means that all projects are geographically limited and have readily identified impacts that can be easily mitigated, therefore will not cause huge environment and social impacts to project sites and surrounding communities. The safeguards implementation process and activities used in this project are as follows: • Community consultation process • Environmental and Social Safeguards Screening • Mitigation measures developed • Environmental preliminary assessment • Safeguards monitoring and reporting • Grievances redress mechanisms There are challenges faced during the implementation of this process at all levels – government stakeholders, implementing partners, communities and internally within the project. Challenges are mainly due to the lack of understanding of the Safeguards requirement as it is a first ever World Bank project to be implemented with new staff who have limited knowledge on such issues. Despite the many challenges faced, a lot of lessons have been learnt over the years of implementation and collaboration with various stakeholders and partners in relation to safeguards policies. These lessons and reflections have informed few recommendations put forward by the project team with the aim of assisting the Government and the World Bank for planning purposes for similar projects requiring safeguards into the future. Page 69 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) ANNEX 6. RESULTS FROM BENEFICIARY SURVEY The IRCCNH Project Management Unit carried out a Beneficiary Survey between December 2018 and April 2019 which included 108 key respondents from 49 communities in Tanna Island– about 0.4% of the total project beneficiaries (28,829 people). Because of limited time and resources, the survey was not designed to cover a representative sample, but rather to provide an indication of the perceived project impact and processes followed. The Shepherds Islands and Ambrym, which comprised 19% of the project beneficiaries, were not covered. In each community, between 1 and 6 respondents were interviewed. As such, some communities were over-represented in individual answers, whilst some were under-represented. To account for this, both individual responses as well as weighted community responses were analyzed. In the latter approach, if a community had 6 respondents, their individual answers were assumed to count for only 1/6 and averaged across the community, so that each community counted equally to the final results. In practice, the results were similar to those where all individual answers were considered equally, and the latter is reported in the text herein. The analysis reported here is based on the March 2019 questionnaire coding by the project’s monitoring and evaluation team. A. CHARACTERISTICS OF RESPONDENTS Sex. Of the 108 respondents, 56 (53%) were male and 50 (47%) female. Age and Education. Respondents averaged 42 years in age, and were fairly well distributed by age groups, with 23% under 30, 45% between the ages of 31-50, and 32% above age 50. Slightly more than a third (36%) had received no education, 34% had at least some level of primary education (grades 1-6), 24% had some secondary education (grades 7-13) and 5% had tertiary education (including technical colleges). Income. Reported income levels were low, reflecting the semi-subsistence society prevalent in Tanna communities. About one quarter of all respondents (25%) reported no income, another 25% reported receiving less than 1,000 Vt a month, 25% reported receiving between 10-20,000 Vt a month, and about 25% reported more than 20,000 Vt a month. The average reported income was 9,530 Vt a month, slightly lower than the average income for rural Tafea reported by the 2010 Household and Income Expenditure Survey (10,700 Vt a month). This question must be interpreted with extreme care, given the inherent bias involved in income-related questions. Household Characteristics. Only 18% of the respondents came from female-headed households (the remainder were male- headed), but a relatively high proportion reported living with elderly (47%) or disabled members (33%). Micro-Project Type. Close to half of the respondents and communities (44% and 45% respectively) were beneficiaries from gravity fed systems, 21% were beneficiaries of rainwater catchment systems 14% were beneficiaries from multipurpose centers, and 17% from feeder roads. Only 4% (4 respondents from two agriculture demonstration plots) were beneficiaries from agriculture inputs. Vulnerability to Disasters. Respondents had experienced a range of impacts from extreme events in the past, including shortages of water during droughts; contamination of water sources from volcanic ash; damage to garden crops and roof iron sheets (rust) from volcanic ash; damage to houses and crops from cyclone winds and rainfall; and landslide damage to houses. Page 70 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) B. SATISFACTION WITH MICRO-PROJECTS Overall Satisfaction. In total, 101 respondents from all 49 communities rated their degree of satisfaction with Micro- Projects (SPs), from 1 (Very Dissatisfied) to 5 (Very Satisfied). About 78% of the respondents reported being Highly Satisfied (62%) or Satisfied (16%). Twenty-two respondents (22%) professed being Dissatisfied (13%) or Highly Dissatisfied (9%). These were mostly respondents who complained that the micro-projects had not yet been completed or that they did not benefit as much as expected. Satisfaction by Type of Micro-Project. Satisfaction with the SPs was highest for Rainwater Catchment (RWC), with 100% of respondents Very Satisfied (74%) or Satisfied (26%). The degree of satisfaction with Gravity-Fed Systems (GFS) and Multipurpose Centers (MPCs) was similar, with 73% Satisfied to Very Satisfied, but a larger proportion of respondents were Very Satisfied with GFS (61%). Respondent who were dissatisfied with GFS cited that the system was still incomplete, did not reach their household, or was still experiencing problems (e.g. shortage of pipes in Imaio, or incomplete access in Enima). MPCs dissatisfied beneficiaries cited insufficient consultation (resulting in vested interest from the chiefs on the choice of location), and the small size of the center/water tank relative to the community population. Satisfaction amongst Feeder Roads (FRs) beneficiaries was 72%, of which 61% were Very Satisfied and 11% Satisfied. Those dissatisfied cited their relative distance from the road, or the fact that the road was still incomplete at the time of the survey (Ienvitana, Lawinu). Amongst the four beneficiaries of Agriculture Demonstration plots (AGRIC), those from Lawaihau were Very Satisfied, while the 2 respondents from Louanathom were Very Dissatisfied but the reasons given were unclear (one of these respondents, however, mentioned that the agriculture inputs reflected their needs, and that they had learned new adaptation techniques). However, the sample size for agricultural SPs is likely too small to derive any meaningful conclusions. Page 71 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Satisfaction with Water Quality and Quantity (for GFS and RWC micro-projects). Respondents were in general satisfied with improvements in both water quality and quantity, although satisfaction was higher for water quality (89.1% satisfied or highly satisfied) than for quantity (77%). Those that were dissatisfied cited continuing interruptions in supply (e.g. many taps compared to tanks), unequal access amongst households, lack or insufficient taps, low pressure, and, amongst RWC beneficiaries, insufficient tanks relative to the population. C. RESILIENCE PERCEPTIONS Measuring resilience in DRM and adaptation projects is extremely challenging: in the absence of rigorous control groups, it is difficult to attribute resilience changes to a specific project; the short timescale of a project may be insufficient to reflect changes in assets and capacities; successful adaptation and resilience building are difficult concepts for beneficiaries to grasp; and there may not be a single indicator that can correctly capture the multifaceted dimensions of resilience.49 The survey therefore attempted to capture multiple dimensions of beneficiaries’ resilience: their ability to cope with, recover from, adapt and transform when confronted with climate and disaster events (see Figure below), as well as their access to financial and social capital, information and learning. It used a set of questions adapted from Jones and Tanner (2015) evaluation of subjective resilience (see Appendix 1).50 49 (Gregorowski, 2016 – internal paper produced for DFID on BRACED lesson learning. 50 Jones, L. and T. Tanner (2015) “Measuring Subjective Resilience”. ODI Working Paper 423. Page 72 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Source: The 3D Resilience Framework (Béné et al. 2012)51 Recovery from TC Pam. About 63% of the respondents stated that they had recovered fully from Tropical Cyclone Pam, with Coping 37% stating that they were now better off. About 34% had Capacity experienced only partial recovery but were worse off than Access to Adaptive before the cyclone, and 2% stated that they had not Early Warning Capacity recovered. Coping Capacity. Coping capacity was asked in two different The Multiple ways, one directly (how have the activities supported through Dimensions the project increased your household ability to cope with of Community Resilience extreme events) and one indirectly (if an extreme event was Access to Social Transformative to occur in my area tomorrow, would your household be able Capital Capacity to fully recover from the damage within 6 months?). For the first question, 81% of respondents responded affirmatively, and 13% negatively. Responses to the second question were more muted, with two thirds (67%) of the respondents stating Access to Financial Adaptive that they would be able to recover within 6 months, whilst Resources Learning 33% felt they would need more time or could recover only partially. Several stated that it would depend on the disaster (with recovery from a cyclone quicker than from a volcano eruption or a landslide), the building materials affected (with traditional grass houses quickly rebuilt in 2-8 weeks), and access to assistance and resources. Respondents of water micro-projects felt that it was now easier to access and store water during disasters; that the water helped them clean volcanic ash from their houses, and keep iron roofing from rusting; that it helped them do home chores, manage home gardens, and improve their health and hygiene (some communities previously relied on seawater). RWCs were also perceived to help beneficiaries cope with droughts and preserve water during cyclones, and the roof and structure was deemed important to shelter them from the sun, as well as from winds and rainfall during cyclones. MPCs were perceived to provide a venue for community meetings and workshops, a kindergarten (in one village) and facilitate fundraising and income generating activities, asides from providing shelter during disasters. FRs were perceived to increase tourism and income generating opportunities, as well as facilitate transportation of emergency items and evacuations during disasters, but some beneficiaries mentioned that the road was too far from their homes. Agriculture beneficiaries cited new planting techniques and disaster-resistant varieties (particularly of kumara) as helping their coping capacity. Adaptive Capacity. Respondents felt fairly confident that they would be able to adapt to future disasters if they received advance warnings, with 86% stating that they would be able to adapt successfully if a disaster, such as heavy flooding/strong cyclone/big drought, was to occur in their area in the next 5 years, and only 6% feeling unable to adapt. Perceived adaptive capacity was particularly strong for beneficiaries of water micro-projects. In general, many respondents 51 Christophe Béné, Rachel Godfrey Wood, Andrew Newsham and Mark Davies (2012) “Resilience: New Utopia or New Tyranny”. IDS Working Paper 2012 (45). Page 73 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) mentioned that they had gained experience with TC Pam and other disasters in the past. Amongst adaptation options cited were stronger housing, preserving water in preparation for a drought, and cultivating resilient (e.g. Fiji taro, coffee) and more diverse crops (e.g. vegetables). Some mentioned having access to more resources. Volcanos and landslides were again seen as the more difficult to adapt to. Transformative Capacity. This question - If [heavy flooding/strong cyclone/big drought] was to occur in my area in the next 5 years, and my community ceased to exist, my household would be able to cope/adapt successfully, even if it required us to completely change our ways of life – was not well understood by those surveyed. However, many respondents felt they had, and were able to, change their way of living (some answers given below): Yes, it has changed our way of thinking and we are able to prepare ourselves if there is another natural disaster Yes, we have seen plenty changes that are supporting us. The government and the World Bank has really helped Yes, we are men, we have the power to improve and change our way of living Yes, as a widow with 4 sons, I think we would change if we have too I/my family have the capacity to change my/our way of living People have built strong concrete houses (and changed the way of living in the process) We’ve learnt new ways of farming It will be very hard to recover quickly if community ceased to exist It depends entirely on our resources and income (standard of living) Access to Financial Resources. Respondents did not feel as strongly about their financial capacity to recover from disasters as their coping and adaptive capacities – possibly due to their semi-subsistence livelihoods. Only 59% felt they would have access to sufficient financial resources to recover from an immediate disaster, with 31% lacking those resources, and 10% mentioning that they could eventually access resources, but it would take time. Those who felt confident in accessing financial resources mentioned being able to rely on sales of pigs, chicken, kava, coconut leaves, vegetables and fish. Some earned income from tourism, some relied on family members who worked, and others engaged in cash-for-works such as grass cutting. Still others felt they could go abroad to earn income. However, many respondents stated that it would take time to recover, particularly if their crops were destroyed. This may argue in favor of cash-for-works programs after major disasters, to provide a rapid infusion of cash income to those affected. Access to Social Capital. Most beneficiaries felt they could rely on community, family or friends’ support to recover from major disasters, and stated that they too would help those affected. A few stated they did not need other peoples’ help, while one (a widow) complained that her community had not helped her at all. In general, however, most felt that the communal social mechanisms remained strong in helping each other rebuild houses and replant gardens, as well as being able to rely on family members in Port Vila for support. Interestingly, one respondent mentioned that it took only a person to help rebuild a grass house, but many people to rebuild permanent homes – suggesting that reinforcing houses as an adaptation option may have the unintended consequence of requiring higher social support if the house is damaged. Access to Early Warning Information. About 86% of the respondents agreed that if a major disaster was to occur in the near future, they would have access to early warning to ensure that they were fully prepared. Early warning alerts – transmitted by radio, mobile phones, internet, and key officials - were now perceived to be both more accurate and timelier (e.g. a 5 day lead time for cyclones) than five years ago, when they relied on weak reception radio, traditional knowledge (signs of nature, such as trees, weather, and birds), and person-to-person messages. Many respondents attributed the improvement to the faster communication technology. In general, women and the most vulnerable people in the Page 74 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) community were seen as having equal access to the alerts, although in some communities, men received the information first and passed it to women. In one community, people used a bell to disseminate alerts. Most respondents seemed familiar with standard preparedness procedures after a disaster alert: preserve food and water, prepare clothing, firewood, torches, prepare the house, hold community meetings, and look for a safe house such as a school, MPC, church, or a relatives’ strong house. Disaster committees existed in about two-thirds of the communities interviewed, and most included women. However, some communities such as Latanu and Leunola reported lacking such committees altogether and may need to be further targeted by NDMO. Adaptive Learning. In general, respondents felt they had learned considerably from the experience of previous disasters and had acquired new adaptation skills, such as setting up disaster committees to deal with future events; cutting branches close to buildings; building strong permanent houses or reinforcing the roof; planting resilient crops (like Fijian taro); and learn how to be safe in times of emergencies (e.g. prepare food and preserve enough water). One woman mentioned using awareness on “Woman I toktok” to help women in times of disaster. It was unclear, however, whether they had learned primarily from past experience, or from NDMO interventions. Attribution. Most respondents mentioned that, asides from IRCCNH, they had also received help from other aid programs - such as CARE, the Tafea Provincial Government, NDMO, Red Cross, World Vision, Samaritan Purse, Adventist Development and Relief Agency (DRA, in the RWC of Lamenaura), UNICEF, Australian Aid and Nasituan (for MPCs and FRs). Perhaps due to their smaller population size, recipients of rainwater catchment systems and multi-purpose centers had the least number of additional aid programs, but the fact that communities in Tanna were assisted by multiple programs in the aftermath of TC Pam makes attribution particularly challenging. At most, it can be said that IRCCNH appears to have contributed to strengthened resilience, alongside other aid programs, with adaptative capacity and early warning appearing particularly reinforced. Social capital continues to be strong, largely due to intact community self-help mechanisms, whilst access to financial resources may need to be strengthened in the future. How much of the Perceived Improved Resilience was due to IRCCNH Project? Micro-Project Beneficiaries Only/Mostly IRCCNH One other aid program 2 or more aid programs GFS 15% 14% 72% RWC 39% 35% 26% MPC 36% 45% 18% FR 0% 37% 63% Agriculture Dem Plot 0% 50% 50% D. IMPLEMENTATION ASPECTS Relevance. Close to 89% of the respondents stated that the micro-projects were a high priority to their household, reflecting their needs. Respondents who disagreed (11%) felt that only a few people had benefited (from water taps, or from their proximity to the road or multi-purpose centers), that they still experienced problems (e.g. water cuts), or that the project was not yet completed (feeder road). Page 75 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Proponents of Micro-Projects. According to the respondents, chiefs played a major role in proposing the Who Proposed the Micro-Projects? micro-projects for gravity-fed systems (48% of the respondents), multi-purpose centers (69%) and feeder roads (61%). By contrast, the majority of rainwater catchments had been proposed by the broader community (87%) or by chiefs and the community jointly (9%), which may have played a role in the higher degree of perceived satisfaction with RWCs noted in the survey. For GFS, respondents also mentioned that water committees, the Area Secretary, the Government, women, and pastors/elders had been involved in proposing the micro-projects (grouped as “others” in the adjacent graph). For MPCs, the category “others” includes joint proposals by chiefs and communities, as well as by named individuals (possibly chiefs). For FR, “others” includes males, and other “influential people” (chiefs, leaders, landowners, and pastors). Distribution of Benefits. Whilst, as per tradition, chiefs may have played a major role in proposing the micro-projects, benefits were seen generally as having been broadly distributed, with 53% of the respondents stating that the whole community had benefited, whilst 28% regarded women and children as the main beneficiaries. Women and bus drivers were viewed as key beneficiaries of feeder roads. Nevertheless, up to 15% of the respondents believed benefits accrued to only a few households (9%), primarily to the chiefs (4%) or to nobody at all (2%). GFS, MPCs and, to a lesser extent, FRs were the micro-projects perceived as having the most uneven benefits; by contrast, all RWC recipients believed they benefited either the whole community, or women/children as a group. Agriculture micro-projects were not considered in this analysis due to the small number of respondents (4). Women’s Involvement. With the exception of feeder roads, women seem to have been actively involved in micro-project implementation and maintenance. They carried sand and coral, cleaned and dug soil for the installation of water pipes or tanks, prepared food and provided water, and participated actively in discussion meetings and committees. Construction Page 76 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) of feeder roads seems to have been an activity performed mainly by men (with women’s role limited to cooking), but women seem to have become major beneficiaries of the new roads through improved trade and access to markets. Involvement of Vulnerable Groups. Many respondents perceived “vulnerable groups” to include children, in addition to the elderly, widows and disabled people, and stated that they had actively contributed to water micro-projects and MPCs. The elderly participated by helping to look after children when their parents were working on the micro-project’s construction. Children helped carry sand and materials to the workers, clean up around the sites, mix cement, and connect water pipes. By contrast, only a few helped out on the construction of feeder roads (e.g. children helped carry sand). Perceived Implementation Process. About 72% of the respondents felt that the micro-projects had been carried out as expected, with an additional 11% citing some problems, and 17% that it had not met their expectations – due primarily to the fact that the SPs had not yet been completed, they felt left out, or there were other problems with materials (e.g. shift to another location, damaged cement). In terms of timeliness and resources, some GFS respondents mentioned that materials could have arrived earlier, that more manpower could have sped up implementation, and that staff turnover had caused delays. Amongst RWC respondents, many felt the work had been done well and on time, but some stated that the remoteness of their community (Ienuak) and their other commitments had slowed down implementation. Many MPC respondents felt that they could have benefited from more resources (in particular, to adjust the center’s s ize to their population needs). The majority of feeder road beneficiaries interviewed, by contrast, felt that works had proceeded quickly and with sufficient resources (some even mentioned receiving extra funds). The partnership built between government, the province, and the communities was generally praised, particularly for water SPs, with some respondents stating that it had really improved their way of living. However, a few respondents – particularly those from communities where micro-projects had not yet been completed at the time of the survey – complained that the partnership needed strengthening to ensure completion of the works. Respondents were also asked what the project could do differently if implemented in other villages/countries. Specific suggestions include: GFS RWC MPC FR Agriculture Plots More consultation and awareness Provide (water) taps to each See other chiefs and no Other villagers to seek Come in time compared to household expectations for the help from the world other times paramount chief bank to have their roads fixed Always get permission first from The house has to be completed See the people's exact Advice other members Need good consultation landowners before any work is done (walls built) for the families to need and do the exact to continue with the and good approach in their areas stay safe (and activities to be project to help (i.e. size activity until it is fully done in the building) MPC to the community completed need) More committee members to fast Make sure the water system It's best for the track project activities and to be reaches all the household; all community to look for equal to everyone (this village village members need to have good workers to speed up committee had 6 members) access to water supply the work There is a high number of population Select the right person to built More water tanks are and more tanks are needed needed Check the water tanks daily Make sure the shelters are Extend the building completed and have enough materials We need a bathroom to have our Maintain it bath Project should visit the systems every 2-3 months Page 77 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) Sustainability. Respondents were mostly optimistic that micro-project activities would continue in the Will Activity Continue in the Future (2/5 years?) future, with the community or a committee providing maintenance support (see chart). However, about a quarter of the respondents stressed that sustainability would depend on their 63.8% fundraising and strengthening community maintenance responsibilities. Amongst the less optimistic, low water pressure (in a GFS), potential 25.7% cyclone/rainfall damage (in a RWC), and lack of, or poorly functioning committees were cited. This 6.7% 3.8% highlights the importance of continuing to accompany the micro-projects after their Yes Yes, if No Unsure N=105 maintained completion. Role of Government vs. Community. Most respondents felt that the Government should continue to provide external materials, equipment, and supervisory assistance, whilst the community would contribute labor, local materials and maintenance of the micro-projects. Government’s continuing involvement was seen as important as communities might take too long to complete the work and/or misuse project funds (one respondent stressed that “there is no one in the community that can look after a huge amount of cash”, probably reflecting their semi-subsistence economy). Another respondent felt that initial Government involvement helped ensure that they would receive spare parts and repair equipment in the future. At the same time, a significant proportion of GFS respondents (about 40%) felt that communities should take the lead role (in procurement) to strengthen their ownership, and only seek external help if needed; some also highlighted the importance of having a strong water committee and fundraising process so they would not depend on government’s help. This suggests that there is scope to pilot a more complete Community-Driven Development (CDD) approach in communities of higher capacity, as it was done in the Solomon Islands and Samoa. E. WATER MICRO-PROJECTS’ SPECIFIC QUESTIONS (GFS AND RWC) Time Savings. Prior to the installation of water systems (GFS or RWCs) beneficiaries mentioned having to walk between 30 minutes to “a whole day” to fetch water, most often from a creek or from the ocean. This time was reduced to 2-3 minutes with the new water systems, with the time saved now spent mostly on gardening, feeding animals such as pigs, studying, and doing household chores such as cooking, washing, and cleaning. Role in Disasters/Climate Change. Respondents felt that the new water systems helped keep the water cleaner and protect them from water borne diseases and contamination from volcanic ash. It also made water available during disasters, the dry season or in times of drought. They mentioned feeling safer during disasters as the system was closer to their homes. Several respondents further cited the benefits of the RWC’s roof in keeping them safe from the hot sun. Persisting Problems with Water Systems. Close to half of the GFS respondents (45%) and one third of the RWC beneficiaries reported continuing problems with their new water systems: for GFSs, it involved mostly pipe blockages Page 78 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) during heavy rainfall, irregular or insufficient water pressure, and uneven system distribution across households. For RWCs, respondents reported worm contamination in the tanks, and insufficient water during droughts. This highlights the importance of continuing Government technical support, even after micro-project completion. Water Committees. About two-thirds of the GFS and all of the RWC communities interviewed had village water committees, but as many as half of these had no women representation. The committees help maintain the system, check water connections to the households, ensure tanks are clean and working properly, and collect water fees. Some communities had dedicated water maintenance workers. F. MULTIPURPOSE CENTERS (MPCS)-SPECIFIC QUESTIONS Perceived Benefits of the Centers. In addition to being designated as shelters during emergencies, individual communities used MPCs according to their specific needs: one community used it for emergency trainings, and as a venue for generating business income (Iarkei); one as a kindy garden (Latanu) or school (Lownolis); another (Ianatoka) used it to produce handicrafts, as a food market, and to hold community meetings; and yet another (Nalpinakeri) was considering using it as a first aid clinic. Whilst the water tanks and shelter were seen as benefiting women and men equally, women generally were seen as benefiting more from the ancillary use of the centers. Role in Disasters/Climate Change. Most respondents (85%) viewed the MPCs as a safe/strong place to seek shelter, particularly during cyclones. Those who disagreed, however, mentioned its limited space relative to the community population, and uneven access to the center’s location (too far for some households). G. FEEDER ROADS (FRS)-SPECIFIC QUESTIONS Perceived Benefits of Feeder Roads. Most respondents felt that transportation had really improved and become safer following feeder road construction. Whilst before, access to and from the communities was difficult and very slow - as people had to walk and the poor condition of the roads made it accessible primarily to 4-wheel vehicles – the new feeder roads enabled bus and truck access to and from the sites, saving time and enhancing marketing opportunities (particularly for women), as well as facilitating the arrival of tourists. Many respondents mentioned that construction of the roads had generated quick money flow and a steadier income. The roads also improved access to schools and health centers, except in Ikity, where respondents mentioned that services were still far from the road. Role in Disasters/Climate Change. Respondents reported that prior to the construction of the feeder roads, it was difficult to reach a safe shelter because they had to walk under heavy rainfall (some mentioned that it took a day or more to reach a safe place). With the feeder roads, they felt they could more easily escape to their relatives (safer) homes and reach out to emergency and relief services52. However, some respondents (in Lawinu, West Tanna and Ienvitana, South Tanna), mentioned that they would continue to experience difficulties evacuating due to their households’ distance from the feeder roads. 52 During the ICr mission, beneficiaries reported that prior to the roads’ repair, disaster relief agencies had to leave emergency supplies at the main road and wait for people to collect them, which led to delays and sometimes losses. Page 79 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) H. AGRICULTURE-SPECIFIC QUESTIONS Perceived Benefits. The few beneficiaries (4) interviewed reported receiving agricultural tools (pig wire, nets, nails, axes, hoes, hammers, spade), training on new ways of farming, and seeds. New varieties adopted included peanut, kumala, carrots and cabbage, with one farmer reporting a higher income of about 5,000 Vt/month. Appendix 1: Set of Questions Used to Measure Subjective Resilience Measures Questions Recovery from PAM Which of the following best describes the extent to which your household has recovered from TC Pam? (1) Did not recover; (2) Recovered some, but worst off than before the cyclone; (3) Recovered same level as before cyclone; (4) Recovered and are better off; (5) Not affected by the cyclone Coping Capacity If [heavy flooding/strong cyclone/big drought] was to occur in my area tomorrow, my household would be able to fully recover from the damage caused by the floods within 6 months (Yes/No/Partially) Have the activities supported through the IRCCNH project increased your household’s ability to cope with cyclone and other extreme events? How? (Yes/No/Partially) Adaptive Capacity If [heavy flooding/strong cyclone/big drought] was to occur in my area in the next 5 years, my household would be able to cope/adapt successfully (Yes/No/Maybe If...) Transformative If [heavy flooding/strong cyclone/big drought] was to occur in my area in the next 5 years, and my community ceased to exist, my household would be able to cope/adapt successfully, even if it required us to completely change our ways of life Capacity (Yes/No/Partially) Financial Capital If [heavy flooding/strong cyclone/big drought] was to occur in my area tomorrow, my household would have access to sufficient financial resources to ensure that we fully recover from the threats (Yes/No/Maybe if...) Social Capital If [heavy flooding/strong cyclone/big drought] was to occur in my area tomorrow, my household would be able to draw on the support of family and friends to ensure that we fully recover from the threats (Yes/No/Maybe if...) Access to Early Warning If heavy flooding/major cyclone/major tsunami/drought/volcanic eruption] was to occur in my area tomorrow, my household would have access to early warning information to ensure that we are fully prepared for the threats posed by the [event]? (1= Information Strongly Disagree to 5=Strongly Agree) Adaptive learning My household has learned considerably from how we have dealt with past [heavy flooding/strong cyclone/big drought]. This knowledge is crucial in successfully dealing with future events (Yes/No/Other) Attribution How much of the above was due to the project? Did you also receive help from other sources (which)? Page 80 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) ANNEX 7. DETAILS OF MICRO-PROJECTS SUPPORTED # Area Council Micro-project site Micro Total Cost Total Cost Number Number of Time to Project (VT)* (US$) Benefi- households complete Type ciaries (months) Tafea Province (Tanna Island): 1 West Tanna Launelapen RWC 2,110,000 19,537 660 68 7 2 West Tanna Louiowanan RWC 2,110,000 19,537 155 15 7 3 West Tanna Lounapkamei RWC 1,465,936 13,573 319 29 6 4 West Tanna Iunier (Lawinu before) RWC 1,465,936 13,573 81 13 6 5 West Tanna Launikauk RWC 1,465,936 13,573 180 17 6 6 West Tanna Lepkit RWC 1,465,936 13,573 108 13 6 7 West Tanna Lamtawekel RWC 1,465,936 13,573 228 25 6 8 SE Tanna Enuak RWC 1,465,936 13,573 63 8 6 9 SE Tanna Enikahi RWC 1,465,936 13,573 87 14 10 SE Tanna Imaio GFS 1,506,803 13,952 179 25 7 11 SE Tanna Enimah GFS 1,126,069 10,427 251 31 8 12 SE Tanna Iatukwei GFS 1,362,670 12,617 117 19 7 13 West Tanna Iru scheme** GFS 15,582,052 144,278 1504 82 7 14 West Tanna Louinio system GFS 1,893,225 17,530 928 132 4 15 West Tanna Bethel GFS 1,893,225 17,530 675 83 16 West Tanna Letakren/Lamenaura GFS 1,893,225 17,530 1352 150 4 17 West Tanna Ikurup GFS 6,506,618 60,246 204 22 7 18 Southeast Tanna Nazareth GFS 6,506,618 60,246 358 40 7 19 Southeast Tanna Galilee/Isaka GFS 12,071,625 111,774 627 84 7 20 Central Tanna Latanu MPC 2,312,210 21,409 800 88 5 21 Northeast Tanna Iarkei MPC 2,312,210 21,409 218 27 3 22 NE Tanna Launawingin MPC 2,312,210 21,409 216 24 5 23 SE Tanna Ianatoka MPC 2,312,210 21,409 1662 180 3 24 West Tanna Iru-Iamsine (950m) FR 27,072,000 250,667 592 74 5 25 West Tanna Kouanumane (300m) FR 8,640,000 80,000 434 54 5 26 West Tanna Lowekau/ Lawinu (150m) FR 7,200,000 66,667 213 30 5 27 SE Tanna Yanawadeng (200m) FR 5,760,000 53,333 218 28 5 28 South Tanna Ienvitana FR 23,472,000 217,333 3500 380 5 (122m+475m+218m) 29 Southwest Tanna Ienakul (310m) FR 8,928,000 82,667 4000 440 5 30 Tanna Imaki Agric 312,746 2,896 360 40 31 Tanna Karimasanga Agric 312,746 2,896 280 32 32 Tanna Lapurmat (WVO) Agric 312,746 2,896 265 30 33 Tanna Lounelpen Agric 312,746 2,896 660 68 Page 81 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) # Area Council Micro-project site Micro Total Cost Total Cost Number Number of Time to Project (VT)* (US$) Benefi- households complete Type ciaries (months) 34 Tanna Fetukai Agric 312,746 2,896 46 5 35 Tanna Ianakul Agric 312,746 2,896 122 13 36 Tanna Iwel Agric 312,746 2,896 57 8 37 Tanna Loweau Agric 312,746 2,896 216 37 38 Tanna Ietapir Agric 312,746 2,896 36 4 39 Tanna Lounathom Agric 312,746 2,896 541 60 40 Tanna Louwerpian Agric 312,746 2,896 42 7 41 Tanna Imaio Agric 312,746 2,896 179 25 42 Tanna Laonialu Agric 312,746 2,896 416 41 43 Tanna Lopukas Agric 312,746 2,896 104 9 Shefa Province: 44 Tongoa Mangarisu/ Morua GFS 8,868,928 82,120 900 100 11 45 Tongoa Kurumambe (incl school) GFS 8,868,928 82,120 257 32 11 46 Tongoa Lupalea GFS 8,868,928 82,120 168 21 11 47 Tongoa Pele RWC 886,893 8,212 300 37 11 48 Tongoa Purau RWC 886,893 8,212 116 14 11 49 Tongoa Woraviu RWC 886,893 8,212 54 7 11 50 Tongoa Matangi, Itakoma GFS 8,868,928 82,120 1025 128 11 51 Tongoa Morua/Mangarisu GFS 8,868,928 82,120 911 101 11 52 Shepherd islands Buninga MPC 5,176,760 47,933 265 29 5 53 Shepherd islands Tongariki MPC 2,588,380 23,966 390 42 5 Malampa Province: 54 North Ambrym Linbul SDA Church RWC 418,090 3,871 100 11 5 55 West Ambrym Melvat RWC 418,090 3,871 70 9 5 56 North Ambrym Fangever RWC 418,090 3,871 60 7 5 57 North Ambrym Fonah RWC 418,090 3,871 50 6 5 58 North Ambrym Topanga RWC 418,090 3,871 50 6 5 59 North Ambrym Tobol RWC 418,090 3,871 50 6 5 60 North Ambrym Ranhor RWC 418,090 3,871 40 5 5 61 West Ambrym Leyley RWC 418,090 3,871 50 6 5 62 North Ambrym Tohu RWC 418,090 3,871 60 7 5 63 West Ambrym Poliktarper RWC 418,090 3,871 50 6 5 64 SE Ambrym Ranon Agric 2,050,179 18,983 250 27 5 65 West Ambrym Craig Cove Agric 2,050,179 18,983 140 15 5 66 North Ambrym Lalinda Agric 2,050,179 18,983 130 13 5 67 East Ambrym Assi Agric 2,050,179 18,983 90 10 5 67 TOTAL 226,675,042 2,098,843 28,829 3,219 * - Includes costs of goods, delivery, consultants/supervisors ** - Includes Laulepen, Lounapkiko, Letauapan, Tanayepa, Loukuera For 3 RWC in Tongoa (Pele, Purau, Woraviu) only gutters and pipes were provided, estimated at 10% of standard costs of other RWCs. Page 82 of 83 The World Bank Increasing Resilience to Climate Change and Natural Hazards in Vanuatu ( P112611 ) ANNEX 8. REVISED THEORY OF CHANGE Page 83 of 83