Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005440 IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-13665 and TF-13557) ON A GRANT IN THE AMOUNT OF US$16.0 MILLION AND A LOAN IN THE AMOUNT OF US$15.0 MILLION FROM THE PILOT PROGRAM FOR CLIMATE RESILIENCE OF THE STRATEGIC CLIMATE FUND TO NEPAL FOR THE BUILDING RESILIENCE TO CLIMATE-RELATED HAZARDS PROJECT October 28, 2021 Urban, Resilience, and Land Global Practice South Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective November 15, 2020) Currency Unit = Nepali Rupee (NPR) NPR 119.67 = US$1 FISCAL YEAR July 16 – July 15 Regional Vice President: Hartwig Schafer Country Director: Faris H. Hadad-Zervos Regional Director: John A. Roome Practice Manager: Abhas Kumar Jha Task Team Leader: Haris Khan ICR Main Contributor: Yunziyi Lang ABBREVIATIONS AND ACRONYMS AAB Agromet Advisory Bulletin ADB Asian Development Bank AITC Agriculture Information and Training Center AKC Agriculture Knowledge Center AMIS Agriculture Management Information System API Application Program Interface AWS Automated Weather Station BRCH Building Resilience to Climate Related Hazards CAAN Civil Aviation Authority of Nepal CARE Climate Adaptation and Resilience for South Asia CPF Country Partnership Framework CUSI Composite User Satisfaction Index DHM Department of Hydrology and Meteorology DMS Database Management System DRRNSPA Disaster Risk Reduction National Strategic Plan of Action EM-DAT Emergency Events Database ENPV Expected Net Present Value EOP End of Project ESMF Environmental and Social Management Framework ESMP Environment and Social Management Plan EWS Early Warning System FAQ Frequently Asked Question FAT Factory Acceptance Test GA Grant Agreement GDP Gross Domestic Product GIDC Government Integrated Data Center GIS Geographical Information System GON Government of Nepal GRM Grievance Redress Mechanism GTS Global Telecommunication System HKA Hamro Krishi App HPC High-Performance Computing ICR Implementation Completion and Results Report ICT Information and Communication Technology IFC International Finance Corporation IPR Implementation Progress Report IRI Intermediate Result Indicator ISR Implementation Status and Results Report JT/JTA Junior Technical Officer Program KCC Kisan Call Center M&E Monitoring and Evaluation MOALD Ministry of Agriculture and Livestock Development MOEWRI Ministry of Energy, Water Resources, and Irrigation MOFE Ministry of Forests and Environment MTR Midterm Review NARC Nepal Agriculture Research Council NCCP National Climate Change Policy NCS National Climate Services NDC Nationally Determined Contribution NFCS National Framework for Climate Services NPV Net Present Value NSDRM National Strategy for Disaster Risk Management NWP Numerical Weather Prediction O&M Operation and Maintenance PAD Project Appraisal Document PDO Project Development Objective PIM Project Implementation Manual PMU Project Management Unit PPA Public Procurement Act PPCR Pilot Program for Climate Resilience PPR Public Procurement Regulations SAHF South Asia Hydromet Forum SERF Shadow Exchange Rate Conversion Factor SI System Integrator SOP Standard Operating Procedure SPCR Strategic Program for Climate Resilience SYNOP Surface Synoptic Observations USI User Satisfaction Index VDC Village Development Committee VHLSEC Veterinary Hospital and Livestock Service Expert Center WAMIS World Agrometeorological Information Service WCRP World Climate Research Programme WIS WMO Information System WMO World Meteorological Organization WOGRAM The Working Group of Agricultural Meteorology WWRP World Weather Research Programme TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ................................................................9 II. OUTCOME .................................................................................................................... 10 A. RELEVANCE OF PDOs ............................................................................................................ 10 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 11 C. EFFICIENCY ........................................................................................................................... 15 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 16 E. OTHER OUTCOMES AND IMPACTS ......................................................................................... 17 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 18 A. KEY FACTORS DURING PREPARATION ................................................................................... 18 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 19 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 21 A. QUALITY OF MONITORING AND EVALUATON (M&E) ............................................................. 21 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 23 C. BANK PERFORMANCE ........................................................................................................... 25 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 26 V. LESSONS AND RECOMMENDATIONS ............................................................................. 27 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 28 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 48 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 50 ANNEX 4. ECONOMIC AND FINANCIAL ANALYSIS ................................................................. 52 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 73 ANNEX 6. SUPPORTING DOCUMENTS .................................................................................. 75 ANNEX 7. IILUSTRATIVE PHOTOS ......................................................................................... 77 The World Bank Building Resilience to Climate Related Hazards (P127508) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P127508 Building Resilience to Climate Related Hazards Country Financing Instrument Nepal Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Department of Hydrology and Meteorology (DHM), Nepal Ministry of Agriculture and Livestock Development (MoALD) Project Development Objective (PDO) Original PDO Project Development Objective: The main objective of the proposed project is to enhance government capacity to mitigate climate-related hazards by improving the accuracy and timeliness of weather and flood forecasts and warnings for climate-vulnerable communities, as well as developing agricultural management information system services to help farmers mitigate climate-related production risks. This would be achieved by establishing multi-hazard information and early warning systems, upgrading the existing hydrometeorological system and agricultural management information system, and enhancing capacity. Activities funded through the project would help improve decision-making and planning in key climate-vulnerable and water resources dependent sectors particularly agriculture, health, water and disaster management, and contribute to building climate resilience for communities at risk. Page 1 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) PDO as stated in the legal agreement Enhance government capacity to mitigate climate-related hazards by improving the accuracy and timeliness of weather and flood forecasts and warnings for climate-vulnerable communities, as well as developing agricultural management information system services to help farmers mitigate climate-related production risks. FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 16,000,000 12,956,460 12,956,460 TF-13665 15,000,000 11,962,496 11,962,496 TF-13557 Total 31,000,000 24,918,956 24,918,956 Non-World Bank Financing 0 0 0 Borrower/Recipient 300,000 0 2,340,000 Total 300,000 0 2,340,000 Total Project Cost 31,300,000 24,918,956 27,258,956 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 15-Jan-2013 20-Jun-2013 25-Jan-2016 30-Nov-2018 15-Nov-2020 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 14-Aug-2018 18.45 Change in Loan Closing Date(s) 10-Dec-2019 25.21 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Unsatisfactory Moderately Satisfactory Modest Page 2 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 19-Jun-2013 Satisfactory Satisfactory 0 02 31-Dec-2013 Moderately Satisfactory Moderately Satisfactory 6.00 03 29-Jun-2014 Moderately Satisfactory Moderately Unsatisfactory 6.07 04 31-Dec-2014 Moderately Satisfactory Moderately Satisfactory 6.12 05 24-Jun-2015 Moderately Satisfactory Moderately Unsatisfactory 6.68 06 29-Dec-2015 Moderately Satisfactory Moderately Unsatisfactory 7.64 07 05-May-2016 Moderately Satisfactory Moderately Satisfactory 7.90 08 21-Dec-2016 Moderately Satisfactory Moderately Satisfactory 9.71 09 22-Jun-2017 Moderately Satisfactory Moderately Satisfactory 11.12 10 24-Jan-2018 Moderately Satisfactory Moderately Satisfactory 13.40 11 24-Aug-2018 Moderately Satisfactory Moderately Satisfactory 18.57 12 08-Mar-2019 Moderately Satisfactory Moderately Satisfactory 20.62 13 30-Dec-2019 Moderately Satisfactory Moderately Satisfactory 25.98 14 25-Jun-2020 Moderately Satisfactory Moderately Satisfactory 26.98 SECTORS AND THEMES Sectors Major Sector/Sector (%) Agriculture, Fishing and Forestry 19 Public Administration - Agriculture, Fishing & Forestry 5 Other Agriculture, Fishing and Forestry 14 Page 3 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Water, Sanitation and Waste Management 81 Public Administration - Water, Sanitation and Waste 31 Management Other Water Supply, Sanitation and Waste 50 Management Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Finance 13 Finance for Development 13 Disaster Risk Finance 13 Public Sector Management 20 Public Administration 20 E-Government, incl. e-services 20 Urban and Rural Development 39 Disaster Risk Management 39 Disaster Response and Recovery 13 Disaster Risk Reduction 13 Disaster Preparedness 13 Environment and Natural Resource Management 27 Climate change 12 Mitigation 12 Water Resource Management 15 Water Institutions, Policies and Reform 15 ADM STAFF Role At Approval At ICR Regional Vice President: Isabel M. Guerrero Hartwig Schafer Country Director: Ellen A. Goldstein Faris H. Hadad-Zervos Director: John Henry Stein John A. Roome Practice Manager: Bernice K. Van Bronkhorst Abhas Kumar Jha Page 4 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Task Team Leader(s): Claudia W. Sadoff Haris Khan ICR Contributing Author: Yunziyi Lang Page 5 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Country Context 1. In 2011, the global risk analysis firm Maplecroft ranked Nepal the 4th most climate-vulnerable country in the world.1 Nepal was prone to a multitude of climate-induced hazards such as floods, glacial lake outburst floods, landslides, windstorms, droughts, and epidemics. Floods and landslides were the most recurrent natural hazards in Nepal. Climate projections for the country predicted increases in temperature and increases in the frequency of extreme events.2 Nepal is projected to suffer increased floods and landslides due to changes in patterns and intensity of rainfall.3 Agriculture was predominantly rain-fed and especially vulnerable to changes in weather patterns and extreme events. 2. Poor and rural populations were considered the most vulnerable to climate risks in Nepal due to their dependency upon subsistence agriculture for their livelihoods. Agriculture was, and continues to be, the mainstay of the economy, providing a livelihood for over 80 percent of the population.4 Climate change was expected to increase floods, droughts, and ecosystems degradation, directly affecting these livelihoods. More than any other group, poor subsistence farmers were unable to cope with such changes. Changes in precipitation and temperature patterns affected the availability of fuelwood, fodder, grass, and drinking water. These and other pressures forced large numbers of people in Nepal (mostly men) to move out of their rural communities and augment their livelihoods with income earned in large cities and abroad. Nearly 19 percent5 of the gross domestic product (GDP) was from remittances, which left women with the additional burden of managing their households, the elderly, the sick, and the young while continuing the farming operations needed to sustain their families. Sectoral Context 3. The need for well-functioning hydrometeorological (or hydromet) services, an early warning system (EWS), and a more effective agrometeorological (or agromet) information system had been identified as key priorities in the National Strategy for Disaster Risk Management (NSDRM, 2009)6. Much of hydromet system managed by the Department of Hydrology and Meteorology (DHM) relied on manual data collection with infrequent and unreliable reporting. From a service delivery standpoint, there was no legal basis in the Government of Nepal (GON) for issuing authoritative warnings on weather events to government authorities and key user groups, nor a sophisticated system for issuing timely and targeted warnings to communities at high risk. The strategy noted that a well-functioning modern hydromet system should allow for efficient data collection, transmission, storage, processing, use, and dissemination to the public, government agencies, and targeted user groups. Further, agromet information was insufficient to manage and mitigate climate risks in Nepal's large and highly climate-vulnerable agriculture sector and underpin the country’s food security efforts. A science-based agriculture information system was needed 1 Maplecroft. https://www.maplecroft.com/risk-indices/climate-change-vulnerability-index/. 2 Nepal Initial National Communication. 2004. https://unfccc.int/resource/docs/natc/nepnc1.pdf. 3 The Intergovernmental Panel on Climate Change's Fourth Assessment Report 4 Ibid. 5 Central Bureau of Statistics and Nepal Rastra Bank. 6 NSDRM. http://extwprlegs1.fao.org/docs/pdf/nep143046.pdf. Page 6 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) to deliver climate and agricultural information that would provide the agricultural sector with a decision support information tool to mitigate climate-related agricultural production risks. The strategy envisioned that such a system would provide weather parameters and index to farmers in support of crop management; pesticide management; and disaster risk transfer mechanism. 4. Transformation of Nepal’s hydromet and agromet services into modern, service-oriented systems was seen to strengthen the long-term adaptive capacity and resilience of the country and its inhabitants. Given its unique geographic position, modernization of the hydromet system and agromet service delivery in Nepal was expected to generate hydrological, meteorological, and agrometeorological information that could be of substantial regional and global value in terms of weather and climate forecasts, disaster risk management, and risk mitigation in the agriculture sector. In turn, links with regional and global climate forecasts could benefit Nepal’s capacity to cope with events. Rationale for World Bank Support and Higher-Level Objectives to which the Project Contributed 5. The Building Resilience to Climate Related Hazards (BRCH) Project was one of five Program components identified in Nepal's Strategic Program for Climate Resilience (SPCR), which was developed by the GON with support from the World Bank, the International Finance Corporation (IFC), and the Asian Development Bank (ADB) and approved in June 2011 by the Pilot Program for Climate Resilience (PPCR) Sub-Committee. The project was one of the key activities within the World Bank Group (WBG) FY12/13 Interim Strategy Note for Nepal (Report No. 63381-NP), particularly the second pillar, ‘Reducing vulnerability and building resilience’, by improving the country’s resilience to exogenous shocks such as climate change effects and natural disasters. Theory of Change (Results Chain) 6. The project’s Theory of Change (see figure 1) is derived from the project description in the Project Appraisal Document (PAD, Report No. 73158-NP). The project aimed to strengthen the institutional and technical capacity of the DHM to mitigate climate-related hazards by (a) delivering high- quality nationwide weather forecasts and flood early warnings to vulnerable communities prone to floods and (b) supporting the Ministry of Agriculture and Livestock Development (MOALD) in developing and institutionalizing an Agriculture Management Information System (AMIS) to provide essential agromet advisories to farmers in 26 pilot districts.7 To achieve the first short-term outcome, the project was to support the DHM in the development of legal and regulation frameworks for hydromet services; implementation of a capacity-building and training program on weather forecasting; modernization of the hydromet observation networks, and improvement of service delivery on hydromet, flood early warning and climate services. For the second short-term outcome, the project intended to support the MOALD in the development of an AMIS, development of agromet information products, capacity building for AMIS staff on weather agromet services provision, and dissemination of AMIS services to farmers. The project was to generate the long-term outcomes of increased resilience of the general public to climate-related hazards and mitigation of climate-related production risks faced by farmers. 7The districts covered were Dhankuta, Sunsari, Siraha, Bara, Kavre, Rupandehi, Banke, Jumla, Sankhwasabha, Jhapa, Morang, Saptari, Mahottari, Chitwan, Surkhet, Kailali, Dolakha, Dhading, Palpa, Kaski, Mustang, Dang, Rukum, Doti, Darchula, and Kapilvastu. Page 7 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Figure 1. Theory of Change Project Development Objectives (PDOs) 7. The objective of the project, as stated in the Loan Agreement (SCF Loan No. TF013557) and the PAD, was to enhance government capacity to mitigate climate-related hazards by improving the accuracy and timeliness of weather and flood forecasts and warnings for climate-vulnerable communities, as well as developing agricultural management information system services to help farmers mitigate climate- related production risks. Key Expected Outcomes and Outcome Indicators 8. The indicators linked to the PDO statement were the following: • Increased financial sustainability of DHM operations (measured as percentage of allocation of public funds in relation to essential operational needs of DHM) • Increased accuracy and timeliness of weather forecasts (verified using standard methods for forecast verification - WMO8 WWRP9/WCRP10) (measured as skill between 0 and 1 for weather forecasts of 24 hours lead time where 1 is a perfect score) • Increased satisfaction of users with DHM services (measured using a Composite User Satisfaction Index [CUSI] expressed as a percentage where 100 percent is completely satisfied). • Introduction of a sustainable AMIS that systematically streams and archives agriculture-relevant hydromet and agromet data and information (measured using a CUSI expressed as a percentage where 100 percent is completely satisfied). 8 WMO = World Meteorological Organization. 9 WWRP = World Weather Research Programme. 10 WRCP = World Climate Research Programme. Please refer to Annex 1: Results Framework for more details. Page 8 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Components 9. The project financed four components as follows: • Component A: Institutional Strengthening, Capacity Building and Implementation Support of DHM (original PPCR allocation: US$5.10 million; actual PPCR cost: US$3.35 million). This component aimed to develop and/or strengthen the DHM’s legal and regulatory frameworks; improve its institutional performance as the main provider of weather, climate, and hydrological information for the nation; build the capacity of department personnel and management; ensure operability of new networks; and support project implementation. • Component B: Modernization of the Observation Networks and Forecasting (original PPCR allocation: US$16.45 million; actual PPCR cost: US$13.99 million). This component aimed to modernize the DHM observation networks, communication, and information and communication technology (ICT) systems; improve hydrometeorological numerical prediction systems; and refurbish/construct the DHM offices and facilities. • Component C: Enhancement of the Service Delivery System of DHM (original PPCR allocation: US$3.45 million; actual PPCR cost: US$1.61 million). The objective of this component was to enhance the service delivery system of the DHM by creating a public weather service that provided nationwide weather and impact forecasts and information services for countrywide climate-vulnerable communities prone to floods and key weather dependent sectors; and a National Climate Services (NCS) that includes a National Framework for Climate Services (NFCS) with sectoral working groups, computer systems and software for climate downscaling, and a digital library of climate-relevant information from various sectors (e.g. agriculture, water resources, public health). • Component D: Creation of an AMIS at MOALD (original PPCR allocation: US$6.00 million; actual PPCR cost: US$5.97 million). The objective of this component was to provide critical and timely agro-climate and agro-weather information to farmers in the 26 pilot districts to increase their productivity and reduce losses from meteorological and hydrological hazards. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION 10. The project underwent two Level 2 restructurings, in August 2018 and in December 2019, where the closing date was extended. In the August 2018 restructuring, the closing date was extended by 13 months, from November 30, 2018, to December 31, 2019, and in the December 2019 restructuring, the closing date was extended by additional 10.5 months to November 15, 2020, for a total extension period of 23.5 months. The extensions were needed to complete critical activities under Components A, B, and C, which would directly contribute to achieving the PDO. These activities included the development of the forecast verification system, development of the NFCS, completion of specialized trainings, submission of a proposed Hydromet Bill to Parliament for endorsement, full adoption of the AMIS by subnational stakeholders, and the operationalization of the Kisan Call Centers (KCCs).11 These changes did not alter the project’s Theory of Change. 11 The KCC responded to farmers’ queries through telephone in their dialect. Page 9 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating Rating: High 11. The objective of the project remains highly relevant to Nepal’s need to build resilience to climate change, natural disasters, and other exogenous shocks. Nepal is ranked by Germanwatch as the as the 10th most affected country to the impacts of climate-related extreme weather events (storms, floods, heat waves, and so on) between 2000 and 2019.12 Its high-risk profile is due to the country’s fragile topography, the climate-sensitive livelihoods of the people, and their limited adaptive capacity. The negative effects of climate change continue to undermine economic growth, food security, and the livelihoods of communities due to climate-induced disasters such as flood, landslides, and windstorms and wildfires every year.13 Between 2000 and 2020, a total of 54 hydrological, meteorological, and climatological disasters occurred in Nepal, affecting over 5 million people and causing 4,314 deaths and close to US$1 billion in damages.14 The need to strengthen Nepal’s capacity to predict and cope with weather and hydrological extremes is therefore still highly pertinent to its needs. 12. The PDO is highly relevant to the WBG’s Country Partnership Framework (CPF) for Nepal for FY19–FY23 (Report No. 121029-NP), particularly on inclusion and resilience (Focus Area 3) and increasing resilience to natural disasters and climate change (Objective 3.3). The WBG endeavors to promote opportunities for greater inclusion and resilience to achieve greater equity in human capital development and preparedness against climate change impacts and natural disasters in Nepal, recognizing that the poor and marginalized, including farmers, face disproportionate economic impacts from such shocks. This project supported the enhancement of Nepal’s hydromet and agromet systems and services delivery, which contributed to boosting the resilience of vulnerable communities through better preparedness and recovery from the impacts of weather- and climate-induced natural disasters. 13. The PDO is strategically linked to the priority areas of Nepal’s Second Nationally Determined Contribution (NDC II) (2020) and the Disaster Risk Reduction National Strategic Plan of Action for 2018– 2030 (DRRNSPA, 2018). Nepal’s NDC II articulates the Government’s commitment to developing and implementing adaptation policies, plans, and actions. The GON has committed to establishing and operationalizing a climate information system by 2025 and a multi-hazard monitoring and early warning system covering all the provinces by 2030. The outcomes of the project formed the foundation for these commitments by strengthening the technical and institutional capacity of the DHM to provide hydromet services and piloting a flood EWS, which could be expanded to serve as a multi-hazard EWS. The outputs of this project directly contribute to the DRRNSPA’s strategy to expand and modernize the real-time weather observation systems and real-time water level and rainfall observation systems in major flood- prone river basins. 12 Germanwatch. Global Climate Risk Index 2021. https://germanwatch.org/sites/default/files/Global%20Climate%20Risk%20Index%202021_1.pdf. 13 National Climate Change Policy, 2076 (2019) of Nepal. http://mofe.gov.np/downloadfile/climatechange_policy_english_1580984322.pdf. 14 Emergency Events Database (EM-DAT). https://www.emdat.be/. Page 10 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 14. The evaluation of the achievement of the PDO is based on results achieved at project closure as measured by the PDO-level Indicators and intermediate result indicators (IRIs) based on the Results Framework (annex 1), and other outcomes that were measured beyond the Results Framework. The efficacy analysis is grouped in two parts according to the short-term outcomes stated in the PDO statement. Outcome 1: Improved accuracy and timeliness of weather and flood forecasts and warnings 15. The financial sustainability of the DHM operations (PDO Indicator 115) increased steadily and achieved the set target by the end of the project, except in FY18/19. The annual operational budgets of the DHM were US$2.7 million and US$8.9 million for FY19/20 and FY20/21, respectively, which sufficiently covered the O&M needs of US$1.44 million per year.16 The increase in the O&M budget for the DHM is likely a result of the GON’s recognition of the importance and value of the DHM’s services to the economy and the society given its project-supported services to the general public. It created a foundation for sustained O&M for the hydromet stations and capacity-building activities through the DHM budget to further enhance its technical capability for hydromet service delivery. 16. The accuracy and timeliness of weather forecasts provided by the DHM (PDO Indicator 2) measured as skill between 0 and 1 for weather forecasts of 24 hours lead time was 37 percent lower than the set target. This is mainly because the forecast verification system was not updated with a more advanced automated system by the end of the project as planned. Nonetheless, the DHM provided a range of hydromet services for the entire country, including climate-vulnerable communities, which included a forecasting service around the clock at a range of time scales, from nowcasting up to multiple days, as well as weekly outlook and seasonal forecasts. This is an improvement compared to the baseline which did not have a sophisticated system for issuing timely and targeted warnings to communities at high risk or for issuing authoritative warnings to government authorities and key user groups. 17. The overall composite users’ satisfaction with the DHM’s services (PDO Indicator 3) remained the same as the baseline (51 percent); while the user satisfaction index for the provision of early warnings for extreme weather events to climate-vulnerable communities (IRI 7) was 43 percent in 2019. User satisfaction surveys were conducted by DHM in August 2015 (baseline) and December 2019 (endline)17 among 2,232 households based on eight key parameters for its services. The CUSI averaged 51 percent in 2019 compared to the end target of 65 percent, indicating that the DHM needed to further improve the accessibility and quality of its services. However, there was a 5 percent increase compared to the baseline in terms of the timeliness, adequacy, and accuracy of the DHM’s services. The survey also measured the user satisfaction level of flood-prone communities in Sunsari, Saptari, Mahottari, Rupandehi, Banke, and Kailali districts with the DHM’s services in providing early warnings for extreme weather events, which resulted in a CUSI of 43 percent compared to the end target of 65 percent, a 15 Please refer to Annex 1: Results Framework for a unique identifier for each result indicator. 16 The desired annual O&M cost for the DHM modernized system was calculated by a hired system integrator (SI) using annual maintenance and reinvestment costs. Source: Biannual Progress Report I, SI, September 2017. 17 Final Report for Consulting Services to Design, Conduct, and Analyze and Endline Survey on Users Satisfaction with Services Delivery by DHM. September 2020. Page 11 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) decrease of 7 percent from the baseline survey, revealing a similar message. The survey also showed that about 74.5 percent of the Nepalese population directly or indirectly benefited from the DHM’s services through the BRCH Project. The endline survey was conducted in 2019, hence it does not fully capture the impacts of the project at closing. 18. The project supported the development of a draft Hydromet Bill and a Regulation, as well as the standard operating procedures (SOPs) for DHM on hydromet services (IRI 1). A bill for the ‘Development of Legal and Regulatory Framework for DHM Operations’ was drafted in 2018 and forwarded to the MOEWRI for further processing and comments. Comments have since been received and the DHM intends to submit the draft bill to the MOEWRI again after incorporating all the comments received, following which it will be circulated to the Law and Finance Ministries for clearance and submission to the Cabinet for approval. A legally binding Hydromet Regulation was also drafted, and it is expected that the bill and the regulation will be approved jointly. The Hydromet Bill and Regulation envisioned a cost recovery mechanism through fees levied on civil aviation for providing aviation weather services that will enable continued upgrading of the DHM aviation forecasting capability. The SOPs of hydromet services were developed and implemented to support the DHM’s mission. In addition, an assessment was funded by the project, proposing a forward-looking institutional framework and organogram of the DHM that will allow it to systematically operate and maintain the modernized hydromet service production systems in line with the proposed DHM Regulations and SOPs. 19. The project improved the DHM staff’s access to and use of training opportunities (IRI 2), which is evidenced by a steady increase in the percentage of DHM staff trained or retrained in O&M of the new hydromet system, even though all annual targets were lower than expected. A specialized capacity- building plan was prepared by a project-financed consulting company (or the System Integrator [SI]) – and submitted to the DHM at an early stage of the project, which could not be fully materialized due to procurement delays. At closing, 72 percent of the DHM staff benefited from technical and professional trainings compared to the original target of 100 percent. Some of the specialized trainings envisioned by the SI were incomplete (e.g., weather presentation system, hydromet workstation, aviation meteorology, quality management system, and nowcasting). Hence, the capacity of the DHM staff may still be insufficient for the smooth O&M of various modernized systems. 20. The project enhanced data transmission from the DHM’s central database to the WMO’s GTS/WIS (IRI 3) at a rate extremely close to the original target. At project closing, 16 synoptic stations of DHM were transmitting all of the surface synoptic observations (or SYNOP)18 from its central database to the WMO’s GTS/WIS19, resulting in a 94 percent data transmission rate, substantially achieving the original target of 95 percent. This result was accomplished owing to the modernization of the communications and ICT systems, which allow more efficient and timely collection of data from the observational network and receive and process information products from leading international meteorological centers. 21. The project improved the status of the DHM’s surface meteorological (IRI 4) and hydrological (IRI 5) observation networks at a rate higher than what was planned, which enabled provision of real- time hydromet data to users (IRI 6) as planned. The project improved the status of the surface 18 SYNOP is a numerical code (called FM-12 by the WMO) used for reporting weather observations made by manned and automated weather stations. 19 1 station generate SYNOP 8 times and the rest 5 times per day. DHM is in the final phase of transmitting the upper air data from its radiosonde station to WMO. This has been verified on Oct 19, 2021: https://wdqms.wmo.int/nwp/land_surface/six_hour/availability/temperature/all/2021-10-19/00 Page 12 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) meteorological observation network by installing and operationalizing 88 AWSs, of which 21 AWSs were installed in or near the airports, aiming to benefit the aviation sector. This represented a 100 percent improvement, surpassing the original target of 90 percent. The status of the hydrological observation network was improved by installing and operationalizing 70 hydrological stations, representing a 100 percent improvement result compared to an 80 percent target. Most of these stations were installed at the existing DHM’s surface hydrological and meteorological observation networks, managed by the Offices of Hydrology and Meteorology. In addition, the project financed the first weather radar and lightning detection system in Nepal along with three automated weather observation systems, an upper air sounding station, a high-performance computing (HPC) system in the Government Integrated Data Center (GIDC) located at Singha Durbar, a high-resolution numerical weather prediction (NWP) system with data assimilation, a calibration lab, and a Geographical Information System (GIS) lab. A 10-story headquarter office was constructed in the DHM's existing premise at Babarmahal, Kathmandu, to accommodate all the ICT systems financed by this project and the radar on its rooftop. Between December 2020 and June 2021, 80 percent of the AWSs provided reliable and uninterrupted real-time precipitation and temperature information to users (uptime20 above 80 percent), fully achieving the planned target. This is a substantial improvement compared to 2013 when much of Nepal’s hydromet system relied on manual data collection with infrequent and unreliable reporting. 22. An NFCS was drafted but was not established fully by project closing and the planned impact assessment by sector was not carried out (IRI 8). An NFCS was prepared by the Asian Disaster Preparedness Center with financing from the CARE Project, as noted in line with the WMO climate services guidelines21 and submitted to the DHM in November 2020. The NFCS provides a comprehensive overview of the existing state of climate services in Nepal; user needs for climate services; and current and future capacity to produce and deliver timely, accurate, and relevant climate products. The prioritized sectors identified were agriculture and food security, disaster risk reduction, water, and health. At project closing, the NFCS required further improvements and consultations to be finalized and adopted by the GON. 23. The project also financed the development of a flood EWS model for Koshi and West Rapti river basins, which was commissioned in 2019. During the 2019 and 2020 monsoons, the system presented moderate accuracy in terms of its prediction of water runoff and was further calibrated to improve its accuracy. The EWS model now provides water runoff forecasts, which are essential for issuing flood early warnings to vulnerable communities in the two river basins and is a model for further replication. Outcome 2: Availability of AMIS services for farmers to mitigate climate-related production risks 24. The project financed the establishment of an AMIS that systematically streams and archives agriculture-relevant hydromet and agromet data and information (PDO Indicator 4) with a lower CUSI rate than planned; however, the AMIS presents improved data accessibility as planned (IRI 9) and a higher-than-planned linkage to the WAMIS (IRI 10). The AMIS portal22 has improved farmers’ access to agromet information owing to monthly or more frequent updates by different relevant entities since it became live on May 23, 2018. The AMIS portal offers a variety of information products and services, such 20 The uptime of AWSs was calculated based on the delivery of hourly precipitation and temperature data to the central database within 15 minutes after nominal observation time. 21 This includes ‘Step-by-Step Guidelines for Establishing a National Framework for Climate Services’ and ‘WMO Integrated Strategic Planning Handbook’. 22 http://www.namis.gov.np/. Page 13 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) as weekly Agromet Advisory Bulletins (AABs); Hamro Krishi App (HKA) updates;23 integrated crop calendars; Nepal Agriculture Research Council (NARC) digital library, etc. As planned, the AMIS portal has been fully connected with the WAMIS24 since January 25, 2018, with 100 percent of the AMIS products linked to the WAMIS. The endline survey, completed in December 2019,25 revealed a 40.7 percent CUSI from the AMIS services26, which is lower than planned but shows a 12.4 percent increase from the baseline survey in 2015, nonetheless. It should be caveated that the endline survey was conducted in 2019; hence, it does not fully capture the impacts of the project in its last year of implementation. 25. The project financed all the planned infrastructure needed for farmers and cooperatives to interface with the AMIS (IRI 12). Data and web servers, computer, monitor, printer, digital color scanner, and plotter were installed in the GIDC and in the Disaster Recovery Center in Hetauda as part of the National Information Technology Center of the Ministry of Communication and Information Technology. A total of 1,263 sets of smart mobile phones, rain gauges, and thermometers were purchased and distributed to 1,263 selected farmer groups and cooperatives in the 26 pilot districts. One central-level and 52 district-level KCCs were established in the District Agricultural Development Offices and District Livestock Services Offices, to be later transferred to 51 Agriculture Knowledge Centers (AKCs) and 47 Veterinary Hospital and Livestock Service Expert Centers (VHLSECs) under the provincial government to enhance the interface with farmers and cooperatives using local dialect and facilitate information exchange with agromet experts. It should be noted, however, that the KCCs in all pilot district offices are not currently functioning due to an unclear administrative structure within these offices. Two sophisticated videoconferencing systems were installed at the MOALD and the Agriculture Information and Training Center (AITC) to support the provision of AMIS services. 26. The percentage of farmers who use ICT in 26 pilot districts receiving the AMIS information reached 100 percent by project closing compared to the 65 percent target (IRI 15). The AMIS disseminated, and continues to disseminate, agromet information through various AMIS media such as the AMIS portal, the HKA, an Online Audio Notice Board, and regular SMS alerts. By project closing, the AMIS information reached 100 percent of the targeted farmers (or 31,575 farmers) residing within the pilot districts through these media with the HKA downloaded more than 52,000 times by end users; a total of 276 weekly AABs issued through the AMIS portal; and over 40,000 farmers receiving approximately two to three SMS alerts every week, compared to the original target of 65 percent (or 20,524 farmers). 27. The project financed the development and dissemination of 12 agromet information and risk management products that have since been used by farming communities (IRI 13), exceeding the target of seven products. The products are listed in annex 1, which were disseminated through the AMIS web 23 The HKA was developed to connect farmers of Nepal. Farmers, farming communities, and other stakeholders all over the country can equally obtain the relevant agro-climatic and weather information disseminated through the ICT-enabled AMIS web portal. 24 http://www.wamis.org/countries/nepal.php. 25 MOALD. 2019. Final Report on Endline Survey to Assess User Satisfaction with Delivery of AMIS Services . 26 The CUSI was calculated as the average percentage of users highly satisfied and moderately satisfied with the main AMIS services (that is, AMIS web portal, HKA, PPCR focal person, KCC, SMS/mobile, and AAB). Page 14 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) portal, YouTube,27 and a Facebook page28 to farmers with information to mitigate weather- and climate- related production risks, as verified by the endline survey.29 28. The project supported an increase in the MOALD extension service and community training (IRI 11), with 69 percent of extension service staff and community leaders certified to train farmers on climate-resilient agriculture—a rate higher than the 25 percent target—and improved access and use of training opportunities by the AMIS staff as planned (IRI 14). By project closing, around 69 percent of the extension service personnel and community leaders (1,380 people out of the 2,000 targeted) were certified as master trainers in climate-smart agriculture by the MOALD. As planned, 420 AMIS staff (or 100 percent) received technical trainings on weather data analysis, agromet modeling, and climate services. Justification of Overall Efficacy Rating Rating: Modest 29. The overall efficacy rating is ‘Modest’ considering the ‘Modest’ achievement of the first outcome and the ‘Substantial’ achievement of the second outcome. The former was evident by the increased DHM O&M allocation target (although this was not a direct measure of the outcome but rather a measure of the sustainability of services); slight increase in the CUSI for the accuracy and timeliness of weather forecasts of DHM, largely improved hydromet networks; achievement of the provision of reliable real-time weather data to users; and underachievement or partial achievement of the remaining targets. The latter was evident by the overachievement of the adoption of the AMIS and its services, improved information dissemination efficiency, improved access to extension services and community training, strengthened linkage between the AMIS and WAMIS; full achievement of AMIS interface infrastructure and training to MOALD staff; and partial achievement of the AMIS satisfaction rates. C. EFFICIENCY Assessment of Efficiency and Rating Rating: Substantial Economic Analysis 30. An ex-post economic analysis30 of the project was carried out based on project documentation, consultations with the project team, and secondary data (see annex 4). The analysis focused on updating and improving ex-ante costs and benefits’ estimates, as well as assessing field-level evidence to verify that the expected benefits, in fact, took place. The benefits quantified and included in the economic analysis calculations at project closing were those arising from reduced asset losses from natural hazards due to improved hydromet and EWS services (project Components A–C), improved agricultural productivity due to improved agromet services (Component D), and improved productivity in other weather-sensitive sectors due to improved hydromet services (Components A–C). The project also provided a number of hard-to-quantify benefits, which were excluded from the economic analysis calculations, including avoided deaths, injuries, and illness from weather-related disasters; improved general convenience of the public; and improved long-term environmental monitoring. 27 https://www.youtube.com/channel/UC4XCXT6NAEA7nJQ0cL_Xebw. 28 https://www.facebook.com/NAMIS.GOV.NP. 29 MOALD. 2019. Final Report on Endline Survey to Assess User Satisfaction with Delivery of AMIS Services . 30 This analysis was conducted and authored by Nikolai (Nick) Beresnev (World Bank Consultant). Page 15 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) 31. The ex-ante and ex-post estimates of benefit-cost ratios are quite similar (5.0–14.6 and 2.8– 14.0, respectively). The project was economically highly viable even with the most conservative benefits’ estimates: expected net present value (ENPV) of US$42.2 million with a 9 percent discount rate and an economic internal rate of return (EIRR) of 40.4 percent (see table 4.5 in annex 4). These calculations also do not consider the non-quantified benefits described above, meaning that the project’s economic viability is likely to be underestimated. The project is also robust to changes in key parameters, such as project costs, estimated productivity gains, and discount rate used. In terms of project sustainability, the attainment of the economic benefits is contingent upon effective operation of the hydromet and agromet systems for the economic life of the project. It is therefore paramount that the DHM and MOALD have sufficient technical and financial resources for the O&M of these systems. Implementation Efficiency 32. The project demonstrated high implementation efficiency in terms of cost savings in high value procurement packages and effective mobilization of technical stakeholders; although at times the project experienced implementation delays, which negatively affected its efficiency. The project practiced competitive selection processes for most of the procurement packages in the domestic and international markets. Comparing the planned and actual costs of high-value procurement packages (60 percent of total packages) across the four components, reveals a cost saving of more than US$2.3 million (see table 3.2 in annex 3 for details). The project placed strong emphasis on the establishment of inter- ministerial steering and technical committees, which enabled strong partnership and efficient coordination among the PMUs and other relevant stakeholders. The project faced a slow start-up, mainly due to the late arrival of the SI, lengthy procurement process for high-value packages, frequent staff turnover in the PMUs and unforeseen exogenous factors. However, through World Bank team’s technical assistance and effective mobilization of relevant technical agencies (e.g., NARC) to support the PMUs in technical implementation, the project was able to achieve many of its set results by closing. D. JUSTIFICATION OF OVERALL OUTCOME RATING Rating: Moderately Unsatisfactory 33. The overall outcome rating is ‘Moderately Unsatisfactory’, reflecting the project’s ‘High’ relevance, ‘Modest’ efficacy, and ‘Substantial’ efficiency. The project and its objectives are strongly aligned with the priorities of the Nepal’s CPF (FY19–FY23), NDC II (2020), and DRRNSPA (2018–2030) and their strong focus on building resilience to climate-related natural disasters, which the project supported by strengthening of hydromet systems and services delivery, flood early warning, and agromet information sharing with vulnerable households. The project partially achieved the objective of enhancing the accuracy and timeliness of weather forecasts and flood early warnings, while it successfully implemented the agromet information system and services’ delivery used by many people in the targeted districts. The project presented efficacy shortcomings, for instance, the non-completion of the specialized training program for the DHM staff, the non-completion of the NCS, and the non-promulgation of the Hydromet Bill and Regulation. When these and further trainings are completed, the project’s results are expected to be further enhanced. The project was highly viable economically with an estimated ex-post benefit-cost ratio ranging from 2.8 (the most conservative estimate) to 14.0 and was able to purchase several high value items at a cost lower than planned. Nonetheless, it experiences implementation delays and high staff turnover which affected its efficiency. Page 16 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) E. OTHER OUTCOMES AND IMPACTS Gender 34. Women in Nepal are among the most vulnerable groups to climate risks because of their limited access to timely weather forecast information, limited available options for crop and livelihood diversification, and low decision-making power to apply adaptation measures.31 The project made weather and agromet information available to beneficiaries, including women, through provision of easily accessible data and information and strengthened women’s capacity to make weather- and climate- informed decisions for adaptation measures in the agriculture sector by targeting female farmers in the capacity-building programs. A total of 6,085 female farmers took part in the capacity-building program under Component D. The 2019 endline survey conducted by the MOALD confirmed that 59.3 percent of project area households were aware of the AMIS, and of these, 48.2 percent of household members were women. About 70 percent of households accessed information on climate and 75 percent of households accessed the problem-solving forum of HKA in relation to agriculture production and farm management decisions. The survey also revealed that over 90 percent of female respondents found the AAB and HKA highly satisfactory or satisfactory. This indicates that the project effectively facilitated female farmers’ access to agromet information and supported them in decision-making to manage weather- and climate- related risks in the agriculture sector. Institutional Strengthening 35. The project strengthened the interaction between key agencies in Nepal responsible for hydromet and agromet services. The institutional interactions in hydromet projects are often complex because there are many disparate uses for water, weather, and climate services. Unclear roles, weak capacities, and insufficient cooperation and coordination among agencies often diminish hydromet service effectiveness. A technical advisory committee and steering committee were formed under the DHM and MOALD to provide overall guidance on the project design and implementation. The project also fostered strong collaboration between these two agencies and other agencies pertinent to the hydromet services value chain, such as the NARC and Civil Aviation Authority of Nepal (CAAN), by forming an interagency initiative—the Working Group of Agricultural Meteorology (WOGRAM)—comprising officials from these agencies. During project implementation, the WOGRAM met regularly and enabled the three agencies to work closely on the generation and issuance of weekly AABs. 36. The project boosted the ownership and confidence of the DHM in providing hydromet services by facilitating an exchange of experiences with other countries in the region and initiating a dialogue on regional collaboration around hydromet, early warning, and climate services. In November 2019, the project financed participation of the GON at the Second South Asia Hydromet Forum (SAHF II).32 The Director General of the DHM presented the progress made through this project on hydromet modernization. Given the transboundary nature of weather and hydrological systems, such knowledge exchange workshops are instrumental in improving the understanding of national hydromet agencies on changing weather patterns, devising effective response strategies, lowering investment costs, and fostering regionally relevant technological innovation. 31 Food and Agriculture Organization. 2019. Country Gender Assessment of Agriculture and the Rural Sector in Nepal. http://www.fao.org/3/ca3128en/ca3128en.pdf. 32 https://www.worldbank.org/en/events/2019/11/19/south-asia-hydromet-forum-ii Page 17 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Private Sector Mobilization 37. The project supported the NARC in conducting a series of studies33 on agricultural insurance for Nepal to help the Government understand better the available financing for risk transfer mechanisms. While the project did not intend to mobilize private sector resources to support the development impact, these studies’ recommendations, particularly on the usefulness of establishing and implementing such market-based insurance schemes, provide a strong analytical basis for future government action in mobilizing private sector resources to promote sustained efforts in disaster risk management. Poverty Reduction and Shared Prosperity 38. Agriculture remains the primary employer for 86 percent of poor people in Nepal, making growth in the sector critical for poverty reduction. However, growth in agriculture has been low and volatile, driven by higher relative prices for agricultural commodities and favorable monsoons. The project likely contributed to poverty reduction in the country by improving early warning alerts and agromet advisories to farmers, thus increasing their productivity and reducing their socioeconomic losses from recurring floods and other hydromet-related hazards. The MOALD estimates that over 1.55 million households residing within Nepal’s 26 districts benefited from the AMIS and EWS for managing agricultural production risks due to natural disasters and climate change. The endline survey (2019) by MOALD revealed an increase of 35.18 percent in gross income saving, increase of 3.78 percent in improved occupation, decrease of 8.38 percent in hard-core poor,34 and decrease of 2.84 percent in migration in project-treated households compared to the baseline data from 2015.35 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 39. Soundness of project design. As this was one of the five key projects of the Nepal SPCR, the project design at appraisal rested on sound analysis and extensive stakeholder consultations and was part of the integrated approach to strengthening the resilience of the country to climate change. The modernization part of the project conformed to best practices established by the WMO, which was the intergovernmental body responsible for coordinating the international exchange of information needed to maintain and improve weather, water, and climate services. Consequentially, the project was designed to improve both the physical and management systems of the GON to provide reliable information to the public and climate-sensitive sectors. 40. Adequacy of risk assessment and mitigation. Overall, appropriate project risks and mitigation measures were identified at appraisal, with substantial risks identified with respect to procurement, governance, and implementing agency capacity. The project well recognized the challenges associated 33 These reports are ‘Agriculture Insurance in Nepal: Case of Banana and Livestock Insurance’, ‘Risk Management in Agriculture: Global experience and lessons for Nepal’, ‘Preparation for Implementing Weather Index Insurance in Nepal’, and ‘Suggested Polices for Improving Insurance System for Nepal’, which were published on the AMIS portal for public access. 34 ‘Hard-core poor’ is defined as households with food security of less than three months. 35 MOALD, 2019. Final Report on Survey to Assess User Satisfaction with Delivery of Agriculture Management Information System (AMIS) Services. The impact on social dimensions was taken from the difference between the baseline data and project- treated households’ endline data. The study did not run an isolation impact analysis, so the impact produced is not fully claimed by this project. Page 18 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) with procurement of sophisticated equipment for hydromet modernization, mitigating this through the hiring of full-time procurement specialists as support. Governance risks were to be mitigated by implementing a Governance and Peace Action Plan that ensured that resources were spent for the intended purposes. Weak implementation capacity and coordination risks due to the DHM’s lack of experience in implementing World Bank projects and the potential complexities of interactions between the MOALD/NARC PMUs and other stakeholders were to be mitigated through training of implementing agencies’ staff and the hiring of a SI with international experience. However, these mitigation measures in response to procurement and capacity limitation risks could have been further strengthened in view of existing weaknesses, as elaborated in section III.B. 41. Reflection of lessons learned. The project design relied on common lessons learned from other hydromet projects in developing countries that had illustrated poor records of sustainability and recognized the importance of designing a system with equipment well-suited to the geography, needs, and capacities of the country. Also learning from these lessons, the project design emphasized the need to transform the DHM into a modern service-oriented agency that delivers products of immediate value to other government agencies and the public at large that would ensure the sustainability of developed systems, and a PDO-level indicator was included in the Results Framework to monitor the allocation of government budget for the maintenance of the AMIS. Further, previous projects in Nepal showed that coordination among ministries had posed challenges for multi-sectoral projects. The first component of this project was therefore designed to clarify the mandate of DHM in relation to other government agencies through the Hydromet Bill and Regulation and the implementation arrangements of the project looked to enhance the coordination between stakeholder agencies as discussed. B. KEY FACTORS DURING IMPLEMENTATION 42. Lack of dedicated team and capacity in DHM. DHM lacks practical experience in implementing World Bank projects. During the project signing, GON agreed that an organization and management survey of DHM will be conducted to restructure/add manpower the department in order to prepare the necessary work force for project implementation and build its capacity through trainings. However, no new positions were created in DHM during project implementation, which overloaded existing DHM officials with additional responsibilities on top of their regular and full departmental responsibilities; and created challenges in the quality supervision of the internal consultants recruited as well as the effectiveness of the trainings conducted. 43. Procurement delays. This project required procurement of several activities concurrently, which poses risks of delays in managing large number of procurements, particularly in evaluation, timely contract award decisions, and contract management. Several rules were introduced by the recent three consecutive amendments of Nepal’s Public Procurement Regulations (PPR), 2007, making the decision- making process stringent: inclusion of additional layers of hierarchy for decision making authority, requirement of decision by respective ministry for inclusion in bid evaluation when cases are pending in court are causing delay and, in some cases, even hangover of contract implementation. The majority of the procurement for this project was of consultancies and goods. The procurement of some high-value and complex goods (e.g., weather radar and hydromet stations) were delayed because of the DHM and MOALD PMUs’ limited experience with such procurement packages. Even small-value consultancy selection processes failed repeatedly due to the unavailability of such expertise in the national market, which required the selection processes to be reinitiated at an international scale; and underestimated cost by the PMU compared to the prevailing price in the international market. Page 19 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) 44. Delayed and insufficient availability of key consultants. At appraisal, it was envisioned that the detailed implementation plan of the project would be designed by a group of international consultants— an SI team. However, recruitment of the SI was delayed by several months and completed only in June 2014, roughly 12 months after effectiveness because of a lengthy procurement process. Once the SI team was recruited, the terms of reference and technical specifications that it prepared for the assessment and design of a modernized hydromet observation network, communication, data management, forecasting systems, and service delivery lacked quality or the necessary details in many cases, which resulted in further delays in the preparation of priority procurement packages. In addition, the performance of the individual consultants (e.g., procurement and financial management) recruited by the DHM PMU to support project implementation was moderately unsatisfactory as the documents they prepared lacked quality and were not produced in timely manner. More importantly, the majority of these individual consultants demonstrated little commitment in fulfilling their contractual agreement with DHM by quitting or threating to quit in the midst of project implementation. Together, these factors negatively affected the DHM PMU’s capability to carry out project activities timely and effectively. 45. Procurement related audit observations. Despite the fact that the Public Procurement Act (PPA) states that the donor’s procurement guidelines will be used in donor-funded projects, many audit observations were made from the Office of the Auditor General (OAG) using the PPA and PPR as the basis for auditing while PMU follows WB’s procurement guideline despite the PMU’s efforts to clarify this with auditors each year. These audit observations often suggest that the decision makers of the PMU should be held accountable; make financial recovery from the various parties including decision makers; and reiterate audit observations from previous years. These issues led to risk aversion mentality of PMU resulting in delayed or non-procurement. For successful project implementation, it is essential that all the key agencies have minimum common understanding and use common standards. In March 2021, the Bank team has discussed and requested the OAG to address those procurement related audit issues. OAG has informed its staff to take note of the donor-funded procurement regulations. By this, the Bank team hope improvements will be seen in regard to those audit issues in the years to come. 46. Federalization. Starting in 2015, Nepal underwent a federalization process following promulgation of a new constitution, which led to a realignment of national and subnational structures. This required the World Bank team’s discussions with the GON to ensure that changes would not adversely affect institutional strengthening activities and that assets would be transferred to the new decentralized subnational entities. These discussions put several activities on hold and diverted the attention of the PMUs from their core implementation functions. 47. Gorkha Earthquakes in 2015. The mega earthquakes of April 25, 2015 (7.8 Richter Scale) and May 12, 2015 (6.8 Richter Scale) severely hindered in-country mobility, approval processes, and building codes, resulting in significant disturbances to several activities. The construction/refurbishment of the new DHM office building with its ICT and computing equipment and linked services were delayed by one-and-a-half years. These, in turn, delayed training activities, which relied on these facilities and equipment. The earthquakes and aftershocks also hampered field work of baseline survey activities, the delivery of capacity-building activities, and the response of international consulting firms to submit bids due to security concerns. 48. Terai unrest and fuel crisis in 2015–2016. The five-month long blockade, which began on September 23, 2015, severely delayed the land acquisition for radar, hydromet stations, radiosonde station, and snow, glacier, and weather monitoring stations; field visits for the preparation of an Initial Page 20 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Environmental Examination and other social and environmental safeguards documents; field-level environmental and social screening of proposed hydromet stations; and the willingness of international firms to submit bids given the deteriorated working conditions. 49. Inability to find alternative training facilities. The specialized training program was delayed because of the unavailability of training facilities in time. Attempts to have the training in the WMO training centers were not successful because of ambiguity in procurement method to follow, leading to indecision in the procurement of these services. As another alternative, the DHM, in consultation with the World Bank, agreed in February 2020, that several highly specialized training activities on hydromet capacity for the DHM staff would be provided by the WMO in its facilities; however, this was delayed and eventually canceled because of a failed procurement process and COVID-19-related social distancing requirements and travel restrictions. 50. Midterm review (MTR). The midterm evaluation carried out in February 2016 anticipated the need for the first closing date extension in view of experienced delays. The mission also agreed that the DHM would closely monitor the performance of the SI team and seek additional technical consultant support for the remaining project period if necessary. However, the MTR underestimated the implementation challenges and overlooked the shortcomings of the Results Framework (see section IV.A). This resulted in missed opportunities for adjusting some project activities to align with the existing capacities (for example, data transmission to the WMO) and improving the M&E system of the project. 51. COVID-19 pandemic. The pandemic erupted in Nepal on January 23, 2020, bringing the country to a nationwide lockdown between March 24, 2020, and July 21, 2020. Despite the good progress made at the latter stage of implementation, the restrictions posed on domestic and international travel and social gatherings created substantial challenge for the delivery of training activities (e.g., hydromet workstations, weather presentation system) and completion of other activities (e.g., development of a web portal for public web service, establishment of a quality management system, commissioning of the calibration laboratory). IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATON (M&E) Rating: Modest M&E Design 52. The M&E system design was overall solid, relying on close monitoring by the DHM and MOALD PMUs using the Results Framework (annex 1) in line with the M&E guidelines provided by the Project Implementation Manual (PIM) for their respective components. M&E was to be executed through four distinctive procedures: (a) trimester (four-monthly) monitoring and preparation of Implementation Progress Reports (IPRs), which were to include information on procurement and financial management; (b) biannual reporting on IRIs and annual reporting on PDO-level indicators; (c) biannual compliance monitoring based on the Environmental and Social Management Framework (ESMF) progress reports and/or field verifications; and (d) program-level (that is, PPCR) reporting executed and coordinated by the Ministry of Forests and Environment (MOFE) through technical working group meetings at least once a year. Page 21 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) 53. The design of the Results Framework demonstrated several shortcomings, which made the evaluation of project outcomes challenging. First, the PDO was lengthy and complex, comprising outcome statements at varying levels. Second, while the indicators were mostly aligned with the project components with realistic targets, their titles, definitions, units of measure, and end targets were sometimes unclear, inconsistent, and non-conducive to the measurement of project outcomes. For instance, PDO Indicator 4 (introduction of a sustainable AMIS) was measured by a USI instead of a set of mandatory system features, and this methodology was repeated in IRI 16 with the same target (increased satisfaction of users with AMIS services), where it was more appropriate. In addition, the indicators that directly measured the achievement of the second outcome were placed at the intermediate level: IRI 9 (improved access to AMIS data), IRI 13 (increased adoption of AMIS tools by farmers), and IRI 15 (improved efficiency to disseminate and obtain feedback of agro-weather and agro-climate information). These issues caused inconsistencies in the reporting on results between the implementing agencies and the World Bank team and were not rectified in the restructurings. M&E Implementation 54. With support from the M&E consultants, the PMUs prepared and implemented a comprehensive monitoring action plan, following the four prescribed M&E procedures. Both PMUs regularly produced IPRs using the monitoring templates and the Results Framework and actively participated in program-level monitoring by sharing information with the MOFE PMU. The World Bank team provided technical and advisory support to the PMUs to further enhance the contribution of project- related results to the PPCR-level M&E framework. The two agencies’ M&E focal points were trained on the methodology and processes for program and project reporting in 2017. End-term surveys were programmed into both the DHM and MOALD’s 2018 Procurement Plans, which contributed to reporting on the results indicators (user satisfaction indices) and formed part of the terminal evaluation (timeline and costing); however, the early completion of the endline surveys in 2019 did not allow the outcomes of the project to be fully captured. The above Results Framework weaknesses were mitigated by the M&E consultants who created a mutual understanding of the indicators between the PMUs and the World Bank task team. The project went an extra mile to strengthen program- and project-level coordination and monitoring by supporting the formation of two technical committees (chaired by the DHM and MOALD), which liaised with the PPCR and mainstreaming M&E into the regular DHM and MOALD processes. This was critical to ensure that the long-term impact of the program was monitored after project closure and knowledge management across the five SPCR program components. M&E Utilization 55. The M&E data collected by the PMUs were used to prepare the IPRs and biannual/annual reports, as required, and inform the PPCR annual reporting. The IPRs provided essential inputs for the World Bank’s Implementation Status and Results Reports (ISRs), detailing evaluation of progress using the Results Framework, safeguards compliance, procurement progress, financial management information and issues, and so on. Reports and other M&E data also informed World Bank missions and decisions on the extensions by monitoring implementation progress, tracking results, assessing impacts, and updating the project’s economic and financial analysis. Close coordination between the M&E functions of the BRCH Project and the PPCR enabled assessment of the overall impact of climate adaptation-related projects in Nepal, consolidation of lessons learned, and facilitation of information and data sharing among the climate change programs. Page 22 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Justification of Overall Rating of Quality M&E 56. Overall, the M&E system design was adequate and allowed for periodical monitoring of components and overall progress at the project and program levels with Results Framework weaknesses as noted. All planned M&E and reporting procedures were timely implemented, however with some data inconsistency due to the ambiguity of some indicators. The M&E data were well used to track project and program progress and for decision-making on project adjustments needed (extensions) and optimization of actions. The shortcomings of the Results Framework hindered the effective M&E of the outcomes until external support was provided to the PMUs, and the early completion of the endline surveys in 2019 did not allow the outcomes of the project to be fully captured. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental and Social 57. The project was classified as an Environmental Category B (Partial Assessment) and triggered three environmental safeguards policies: Environmental Assessment (OP/BP 4.01), Natural Habitats (OP/BP 4.04), and Forests (OP/BP 4.36).36 While the project was expected to benefit targeted communities, the implementation of the proposed subprojects (for example, hydrological stations, meteorological stations, and the DHM building) was expected to result in adverse impacts on people and the environment, including local land and water resources. During project preparation, the DHM prepared an ESMF (Report No. E4028)37 to address foreseen environmental and social impacts. The ESMF required each subproject to undergo environmental screening and biannual compliance monitoring, and the project was to undergo two audits/evaluations in the second year and last year of implementation. The ESMF also prescribed the recruitment of full-time environmental specialists to support the development and implementation of Environment and Social Management Plans (ESMPs). 58. The project was in compliance with World Bank environmental safeguards policies with audit shortcomings. An environmental safeguard consultant was recruited in 2014, who ensured that all hydromet stations and building sites’ constructions complied with the ESMPs and underwent environmental screening as planned; however, the audits in the second year and last year of implementation were not carried out as planned. In line with the World Bank’s recommendation, hazardous waste management training was delivered to field staff of the regional offices to ensure that hazardous waste materials from dismantled stations were managed, disposed of, or stored appropriately. The project was also in compliance with occupational health and safety procedures during construction of the DHM building. By the end of the project, no incidents or complaints on environmental risks and impacts were reported. 59. The project triggered two social safeguards policies: Involuntary Resettlement (OP/BP 4.12) and Indigenous Peoples (OP/BP 4.10). The project was expected to generate positive social impacts by delivering improved warnings of meteorological hazards and better agrometeorological data to farmers 36 The project received an exception to the riparian notification requirement under paragraph 7(a) of Projects on International Waterways OP/BP 7.50, because the project interventions related only to monitoring international rivers and were considered non-abstractive in nature. 37 The final ESMF was disclosed by the DHM on June 1, 2012, and by the World Bank on October 1, 2012. https://documents1.worldbank.org/curated/en/485841468062634945/pdf/E40280EA0P12750C0disclosed010010120.pdf; http://www.dhm.gov.np/uploads/getdown/476130118ESMF_PPCR_DHM.pdf. Page 23 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) throughout Nepal. However, potential adverse social impacts during the construction phase of investments were identified as loss of land or structures, loss of income or access to areas for livelihood support, and public safety issues. The ESMF was to be used to ensure that the activities selected and implemented were socially responsive, and it included a Resettlement Policy Framework, a Vulnerable Community Development Framework, and a Gender Development Framework. 60. The project was in compliance with the World Bank social safeguards policies. The 21 hydrological and meteorological stations that had been identified as requiring land acquisition were adequately addressed by using a ‘willing seller’ approach and by relocating the stations to government land. A log of land ownership and ownership transfer of hydromet stations was well maintained in the project’s office. The project satisfactorily implemented the key actions and milestones identified in the Gender Action Plan and regular meetings were conducted with field and basin office social specialists on project implementation. Project information was disseminated through the DHM website, biannual news bulletins, brochures, and the Nepali calendar, which were distributed among key stakeholders on a regular basis. A project-specific grievance redress mechanism (GRM) was established and was functional throughout implementation, comprising information officers at the PMU level and a Grievance Redress Cell at the district level. Station-level GRMs included a Village Development Committee (VDC) secretary, a local nongovernmental organization representative, and an opinion leader from the VDC. During project implementation, only one grievance was received and resolved,38 and no child labor was reported. The compliance to safeguards was monitored regularly on site. Procurement 61. The project was not in full compliance with the World Bank procurement guidelines and procedures. The PMUs had clear systems of accountability with well-defined procurement responsibilities and decision-making roles. Although procurement audit was not in practice in Nepal, the Office of the Auditor General looked over the procurement decisions during the regular accounts’ audits. The PMUs produced and maintained written records of all procurement and contract documents. Both the PMUs were supported by a procurement consultant each to help carry out procurement assignments, although with limited capacity as noted in section III.B. The PMUs did not have an effective complaints management system known to the bidding community to review and resolve complaints at appropriate stages of the procurement cycle. This was mitigated by post reviews of technically complex contracts and regular monitoring of high-value contracts by the PMUs and the World Bank team. The PMUs also did not satisfactorily practice publication of contract award details, maintenance of contract implementation documentations, and submission of complete evaluation reports. Further, the DHM demonstrated weak contract management capabilities, including in the Letter of Credit opening and management, and was responsible for delays in contract awards and payment.39 Financial Management 62. The project was not in full compliance with World Bank’s financial management procedures and demonstrated several shortcomings. Despite the World Bank training and a PIM with clear guidelines, the Interim unaudited financial reports and audit reports were not always submitted on time and 38 In 2017, a grievance notification was received with regard to the construction of the DHM building asking for the provision of an alternative access, and this was addressed. Source: World Bank Aide Memoire (April 2017). 39 Source: WBG BRCH Project Procurement Post Review Report in June 2020. Based on four contracts reviewed in June 2020, the total duration from the publication of invitation for bids to contract award was 205 days. Page 24 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) identified shortcomings (for example, weak internal controls and delayed payments to contractors) were not always addressed on time by the PMUs due to the lack of a project-specific and stable finance officer throughout the project period and the financial management consultant not being in place for some time. Expenditures were low at the early stage of the project due to delays in procurement and opening of Letters of Credit for importing equipment. Compliance with Legal Covenants 63. The project was in compliance with all the financing agreement’s legal covenants. Out of the nine covenants, three (#1, 2, and 440) were fully complied with throughout the implementation period, while the others were partially complied with during certain project periods and eventually complied with. The project was in compliance with the disbursement conditions, which were that a technical coordinator, a financial consultant, and a procurement consultant be hired for the DHM PMU; a procurement consultant be hired for the MOALD PMU; and a PIM be prepared and adopted by the PMUs, all acceptable to the World Bank. The placement of the SI team was delayed by a few months due to complexity in the bidding process and was completed in June 2014. The key consultancy for the MOALD was completed in February 2015. The procurement and financial management consultants for the MOALD were on board until November 2019 but worked intermittently. With technical support from the World Bank team, the GON developed the initial PIM to serve as the key guideline for project implementation by both the PMUs. C. BANK PERFORMANCE Rating: Modestly Satisfactory Quality at Entry 64. The World Bank supported the identification, preparation, and appraisal of the project through a team with an appropriate skill mix and extensive country experience, including in disaster risk management, procurement, financial management, and social development (see annex 2). The design of the project demonstrated strong strategic relevance to the World Bank and national strategies for building resilience to climate change and disaster risks as it rested on Nepal’s SPCR. During the design phase, intensive stakeholder consultations took place, which led to a strong sense of ownership within the Government. Lessons learned on sustainability of hydromet modernization activities and importance of inter-ministerial coordination were well reflected in the design. Risks associated with procurement, governance, and capacity of implementing agencies for the PDO achievement were properly identified. However, the risk and mitigation measures proposed at appraisal proved to be insufficient given the several challenges faced during project implementation, mainly delayed recruitment of the SI and full- time procurement consultants at the PMUs and insufficient capacity of the DHM PMU in managing the deliverables of SI. The Results Framework also demonstrated several weaknesses as discussed in the M&E section. The project design insufficiently considered options to address potential challenges due to the sequential design of the components. 40(1) entering into a contract agreement with NARC for clarifying roles under Component D; (2) establishing the GRM in DHM and thereafter maintaining it throughout project implementation; (4) maintaining PMUs in DHM and MOALD. Page 25 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Quality of Supervision 65. The task team presented an adequate mix of technical skills to properly supervise and support the PMUs during project implementation. Implementation support missions were carried out every six to twelve months and comprised field visits; high-level discussions with the Ministry of Finance, MOEWRI, CAAN, and Ministry of Science, Technology, and Environment; and detailed technical meetings with the DHM and MOALD PMUs and implementing partners (e.g., WMO, ADB, and IFC). The mission reports were comprehensive and provided an honest evaluation of performance. The core operational team members were engaged throughout project implementation and maintained a strong working relationship with the government counterparts. The team instituted monthly progress review meetings and technical missions every three months to provide support and monitor progress in between implementation support missions. The team also provided timely support in preparing terms of reference, expressions of interest, requests for proposals, technical documents, and project deliverables submitted by the consultants. Despite a change in task team leadership in 2017, issues reported by the PMUs were always addressed promptly with support from the safeguards and fiduciary specialists stationed in the country. Justification of Overall Rating of Bank Performance 66. Overall, the World Bank performance rating is ‘Moderately Satisfactory’, reflecting the adequate skill composition of the task team during both project preparation and supervision and diligence in project supervision and implementation support. However, mitigation measures could have been further strengthened in response to changes in the political environment and other implementation challenges. There were also at-entry weaknesses related to the sequential design of the project, the complex PDO statement, and the Results Framework indicators which were later not rectified through restructuring. D. RISK TO DEVELOPMENT OUTCOME 67. The Hydromet Bill and Regulation to establish the DHM as the legally authorized agency for hydromet services are yet to be promulgated by the Parliament. Though the DHM is a government department, it lacks any dedicated act or rule and regulation. Once promulgated, the Hydromet Bill will provide the essential legal and regulatory framework for its operations and enable a cost-recovery mechanism for the O&M of hydromet infrastructure, which is critical to ensure the sustainability of the development outcomes of this project. Due to frequent change of Irrigation Secretary in the MOEWRI in 2021, the DHM has been required to reinitiate its consultations with the ministry and resubmit the bill and regulation for review and approval. In addition, the DHM is yet to be restructured to meet the requirements of smooth operation of new equipment, systems, and infrastructures. 68. Adequate and continuous allocation of O&M budgets are critical to sustain the services provided by the DHM and MOALD. For FY20/21, the DHM has secured sufficient budget to carry out activities that could not be completed by the project’s end date. For instance, the DHM confirmed that trainings on the hydromet workstation were completed in Jan 2021; and the development of a web portal for public web service were completed in July 2021. The DHM has been making provisions in its annual budget starting since project closing for regular O&M of the equipment installed. However, it should be noted that the DHM’s recurring budget for O&M has not increased by much to allow this continuous O&M. The MOALD has handed the AMIS to the AITC, which has taken the lead in O&M of the system, upgrading of modules, and carrying out of capacity-building activities for operation of the AMIS portal. However, the approved annual budget for FY20/21 of the AITC was NPR 3.0 million (US$26,000)—an amount which may Page 26 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) not be sufficient for maintaining the full functions of the AMIS without compromising the quality of the agromet information and services. V. LESSONS AND RECOMMENDATIONS 69. Hydromet and early warning services projects are highly technical and complex in nature; hence, they require robust technical support and a multi-phased long-term engagement. The implementation process of the BRCH Project involved a steep learning curve of the implementing agencies, especially the DHM, resulting in slow progress in the initial years of implementation. Future hydromet and early warning operations should consider conducting a robust needs assessment during project preparation, potentially through linked/parallel analytics, and reflect the assessment’s findings in the scope of activities, project timeline, and technical support consultancies. Moreover, considering the cascading implications of the key elements of the hydromet value chain (observation -> forecasts -> early warning -> tailored sectoral services), a multi-phased program for a long-term engagement may be more suitable for such operations than a stand-alone project. 70. National hydromet modernization investments can contribute to regional knowledge sharing and establish the foundation for regional collaboration and public-private engagement. This project financed participation of the DHM in the SAHF annual workshops in 2018 and 2019, during which the GON benefited from peer-to-peer knowledge exchange with other national meteorological and hydrological services in the region on developing and delivering user-oriented hydromet services for key economic sectors; exchanging innovative approaches to improve forecasting skills; addressing common implementation challenges; prioritizing areas for stronger regional collaboration, such as training weather forecasters; and discussing institutional arrangements to sustain and scale up national and regional efforts to deliver hydromet and climate services. Recognizing the cross-border nature of hydrological and meteorological hazards in South Asia, it is crucial for national hydromet projects to engage with stakeholders in the region to ensure a cascading system on hydromet, early warning, and climate services for strong regional collaboration, integration of disruptive technology, and public-private engagement. 71. The application of an SI for hydromet programs should be thoroughly considered in future such projects. One of the unique characteristics of hydromet investments of this generation has been the use of an SI consultancy. An SI is intended to bring together a consortium of key technical providers to support system design, implementation planning, procurement, and capacity building. However, from the experience of this project and additional anecdotal evidence from similar projects, there is difficulty in securing the intended support from SIs due to the immaturity of the SI market, lack of SIs’ local presence and client engagement, lack of clients’ capacity to manage this big consultancy, the requirement for high- level approvals of the SI contract within the Government, and other reasons. All of these have cascading consequences on the quality of systems’ design and downstream activities, many of which often rely on the outputs of the SI. Alternative arrangements to boost technical support for similar projects include taking an agile approach to develop these complex systems with early prototypes that can be improved/scaled up gradually; and leveraging innovative technologies (e.g., open-source earth observation data, deep machine learning-based weather prediction) to improve operational forecasting capabilities where ground station data are limited. . Page 27 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: 1: Improved accuracy and timeliness of weather and flood forecasts and warnings Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased financial Percentage 40.00 100.00 100.00 sustainability of DHM operations (Percentage, 15-Apr-2013 30-Nov-2018 15-Nov-2020 Custom) Comments (achievements against targets): • Identifier: PDO Indicator 1 • Unit of Measure: Percentage of allocation of public funds in relation to essential operational needs of DHM • Achievement: Fully achieved • Explanation: The allocated total DHM operational budget for FY 2019-20 was US$2.7 million (DHM, HQ = US$1.6 million & Field Office = US$1.1 million); and for FY 2020-21 = US$8.9 million (DHM, HQ = US$6.7 million & Field Office = US$2.2 million). The annual maintenance and reinvestment costs were estimated at US$1.44 million. The annual O&M cost was calculated based on the O&M Policy & Practice of the DHM as per Expenditure Codes of both Capital and Recurrent Budget adopted since FY 2019-20. The budget allocations for FY19-20 and FY20-21 are sufficient to cover 100% of the O&M needs of DHM. • Information Source: O&M Budget Summary Tables for FY2019-20 and FY2020-21 submitted by DHM. • Attribution: The increase in the O&M budget was a strategic/political decision of the Ministry of Finance (MOF) in recognition of the improved DHM services supported by the project and their importance. Page 28 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased accuracy and Text No verification system 0.75 0.47 timeliness of weather in place - no skill forecasts (verified using standard methods for 15-Apr-2013 30-Nov-2018 15-Nov-2020 forecast verification - WMO WWRP/WCRP) Comments (achievements against targets): • Identifier: PDO Indicator 2 • Unit of Measure: Skill between 0 and 1 for weather forecasts of 24 hours lead time where 1 is a perfect score. • Achievement: Underachieved • Explanation: The Weather Forecast Verification Skill Score for 2019-2020 was computed by DHM for a 24h lead time forecast using the following formula: Skill score = [Hit Rate (Tmax) + Hit Rate (Tmin) + Skill (Tmax) + Skill (Tmin) + TS (RR) + KSS (RR) + ROC_A (RR)+ BSS (RR)]/8; where • Hit Rate (Tmax) is the Hit Rate of daily maximum temperature within ± 2˚C • Hit Rate (Tmin) is the Hit Rate of daily minimum temperature within ± 2˚C • Skill (Tmax) is the Skill Score against the reference forecast for daily maximum temperature • Skill (Tmin) is the Skill Score against the reference forecast for daily maximum temperature • TS (RR) is the Threat Score for the categorical precipitation forecast • KSS (RR) is the Hanssen-Kuiper Skill Score for the categorical precipitation forecast • ROC_A (RR) is the Relative Operating Characteristic Skill Score for the probability of precipitation forecast • BSS (RR) is the Brier Skill Score for the probability of precipitation forecast • Further details on this methodology can be found at https://www.swpc.noaa.gov/sites/default/files/images/u30/Verification%20of%20Categorical%20Predictands.pdf. • Information source: Review of the DHM calculation sheet. • Attribution: The project financed hydromet networks' upgrades and relevant capacity building of DHM staff. Page 29 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased satisfaction of Percentage 51.00 65.00 51.00 users with DHM services 31-Aug-2015 30-Nov-2018 31-Dec-2019 Comments (achievements against targets): • Identifier: PDO Indicator 3 • Unit of Measure: A Composite User Satisfaction Index (CUSI) expressed as a percentage where 100% is completely satisfied. • Achievement: Partially achieved • Explanation: DHM service users were classified into four categories viz. Household level (HH), Key Informants (KII), Informal Group Discussion (IGD), and real data users (UG). The CUSI was calculated giving different weights to the results of the User Satisfaction Index (USI) of each group, which was calculated to assess the state of satisfaction from the DHM services across eight parameters. The parameters were P1: Timeliness; P2: Adequacy; P3: Accuracy; P4: User-friendliness; P5: Modern and Innovation; P6: Needs or Requirements; P7: Accessibility; and P8: Preparedness. The CUSI averaged 51 percent in 2019 compared to the end target of 65 percent, indicating that the DHM needed to further improve the accessibility and quality of its services. However, there was a 5 percent increase compared to the baseline in the CUSI values for the timeliness (P1), adequacy (P2), and accuracy (P3) of the DHM’s services. Specifically, 75 percent of the respondents noted that they were satisfied with the DHM’s timely forecasts of rainfall and temperature. Around 83 percent and 72 percent of respondents said that the DHM’s forecasts of rainfall and temperature, respectively, were highly accurate, particularly from 2019 onward. The endline survey was conducted in 2019; hence, it does not fully capture the impacts of the project at closing, especially given that a significant portion of the hydromet stations (over 20 percent) and capacity-building activities on the modernized software and hardware were completed in 2020, which likely had improved the DHM’s capacity to provide more timely and accurate weather forecasts. • Information Source: DHM Baseline Survey Final Report (June 2015), DHM Endline Survey Final Report (September 2020) • Attribution: The project financed hydromet systems’ modernization, capacity building to DHM staff, and institutional strengthening of DHM. Objective/Outcome: 2: Availability of AMIS services for farmers to mitigate climate-related production risks Page 30 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Introduction of a sustainable Percentage 28.03 65.00 40.70 AMIS that systematically streams and archives 31-Dec-2015 30-Nov-2018 31-Dec-2019 agriculture relevant hydromet and agromet data and information Comments (achievements against targets): • Identifier: PDO Indicator 4 • Unit of Measure: A Composite User Satisfaction Index (CUSI) expressed as a percentage where 100% is completely satisfied. • Achievement: Partially achieved • Explanation: The CUSI was calculated as the average percentage of users highly satisfied and moderately satisfied with the main AMIS services (i.e., AMIS web portal, HKA, PPCR focal person, KCC, SMS/Mobile, and AAB). The Endline survey that was completed in December 2019 revealed a 40.7% CUSI for these AMIS services (29.3% of 411 respondents were highly satisfied and 51.7% of 411 respondents were moderately satisfied). The Baseline survey completed in June 2015 established that the CUSI was 28.3%. • Information Source: MOALD Baseline Survey Final Report (June 2015), MOALD Endline Survey Final Report (December 2019) • Attribution: the project financed the establishment of AMIS and its information products, and capacity building of relevant stakeholders on its application, maintenance and regular updates. A.2 Intermediate Results Indicators Component: Component A: Institutional Strengthening, Capacity Building and Implementation Support of DHM Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 31 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) New DHM regulations and Text Existing operating DHM Regulations and DHM Bill and business practices procedures are operational guidelines Regulation drafted but inadequate to meet fully functioning and promulgation DHM mission support DHM mission pending; SOPs developed and fully functioning to support DHM mission. 15-Apr-2013 30-Nov-2018 15-Nov-2020 Comments (achievements against targets): • Identifier: IRI 1 • Unit of Measure: Text • Achievement: Partially achieved • Explanation: A Hydromet Bill for the "Development of Legal and Regulatory Framework for DHM Operations" was drafted in 2018 and forwarded to the MOEWRI for further processing and comments. Comments have since been received and DHM intends to submit the draft Bill to the MOEWRI again after incorporating all the comments received, following which it will be circulated to the Law and Finance Ministries for clearance and submission to the Cabinet for approval. A legally binding Hydromet Regulation was also drafted and it is expected that the Bill and the Regulation will be approved jointly. The SOPs of hydromet services were developed and have been implemented to support DHM’s mission. • Information Source: Review of draft Hydromet Bill, Regulation, and SOPs. • Attribution: The project financed the consultancies to draft the Bill, Regulation, and SOPs. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improved access to and use Percentage 22.00 100.00 72.00 of training opportunities 30-Jun-2015 30-Nov-2018 15-Nov-2020 Page 32 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Comments (achievements against targets): • Identifier: IRI 2 • Unit of Measure: Percentage of DHM professional staff trained and/or retrained. • Achievement: Substantially achieved • Explanation: The total number of DHM professional staff as per DHM administration record is 173, including those transferred, retired and expired. The total number of DHM professional staff trained or retrained was 124. Considering these, the training rate was 72%. • Information Source: Data provided by DHM per request in July 2021 in line with the DHM's ICR (Feb 2021). • Attribution: The project financed the technical and professional trainings provided by the SI experts, PMU consultants (Social & Communication Specialist, Environment Specialist, and M&E Specialist), external consultants, seminars and visits abroad, and equipment associated with the operational training package. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improved transmission of Percentage 0.00 95.00 94.00 data to WMO GTS/WIS 15-Apr-2013 30-Nov-2018 15-Nov-2020 Comments (achievements against targets): • Identifier: IRI 3 • Unit of Measure: Percentage of data from synoptic stations and upper air stations reaching WMO via GTS/WIS • Achievement: Substantially achieved • Explanation: 16 synoptic stations of DHM were transmitting all of the surface synoptic observations (or SYNOP) from its central database to the WMO’s GTS/WIS, resulting in an 94 percent data transmission rate [16 synoptic stations / (16 synoptic stations + 1 radiosonde station)] which is quite close to the 95 percent original target. Specifically, 1 station generate SYNOP 8 times and the rest 5 times per day. DHM is in the final phase of transmitting the upper air data from its radiosonde station to WMO as of Oct 19, 2021. This percentage is a proxy of how much data are being shared with the regional and global WMO centers. • Information Source: WMO WIGOS Data Quality Monitoring System. https://wdqms.wmo.int/nwp/land_surface/six_hour/availability/temperature/all/2021-10-19/00 Page 33 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) • Attribution: The project financed the installation of the AWSs, upgrade of communication, ICT, and data management system, and capacity strengthening of DHM staff on data transmission. Component: Component B: Modernization of the Observation Infrastructure and Forecasting Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improved status of surface Percentage 10.00 90.00 100.00 meteorological observation network 15-Apr-2013 30-Nov-2018 15-Nov-2020 Comments (achievements against targets): • Identifier: IRI 4 • Unit of Measure: Percentage of Automated Weather Stations (AWSs) operating in line with the hydromet services’ SOPs of DHM • Percentage of Achievement: Overachieved • Explanation: The result relies on the operation of the 88 installed AWSs in line with the hydromet services’ SOPs of DHM. This represents a 100 percent improvement compared to the original 90 percent target (79 AWSs functional in line with the SOPs). • Information Source: DHM Data Management System • Attribution: The project financed the installation of meteorological stations and development of SOPs. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improved status of surface Percentage 10.00 80.00 100.00 hydrological observation network 15-Apr-2013 30-Nov-2018 15-Nov-2020 Page 34 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Comments (achievements against targets): • Identifier: IRI 5 • Unit of Measure: Percentage of hydrological stations operating in line with the hydromet services’ SOPs of DHM • Achievement: Overachieved • Explanation: All 70 installed hydrological stations are operational in line with the hydromet services’ SOPs of DHM. This represents a 100 percent improvement compared to an 80 percent target (56 hydrological stations functional in line with the SOPs). • Information Source: DHM Data Management System • Attribution: The project financed the installation of hydrological stations and development of SOPs. Component: Component C: Enhancement of the Service Delivery System of DHM Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Provision of real-time or near Percentage 10.00 80.00 80.00 real-time data to the users 15-Apr-2013 30-Nov-2018 15-Nov-2020 Comments (achievements against targets): • Identifier: IRI 6 • Unit of Measure: Percent of stations providing reliable and uninterrupted real-time information in an appropriate form to the users. • Achievement: Fully achieved • Explanation: The uptime of AWSs is calculated based on the delivery of hourly precipitation and temperature data to the central database within 15 minutes after the nominal observation time. From December 2020 to June 2021, the uptime of the 88 AWSs financed by the BRCH averaged around 90%. For the same time period, 80% of the stations provided reliable and uninterrupted real-time hourly precipitation and temperature information to the users at above 80% uptime. • Information Source: DHM Data Provision Report (July 2021). Page 35 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) • Attribution: The project financed the modernization of the hydrological and meteorological stations, and the upgrades of communications and ICT systems. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Development and reliable Text 49% 65% 43% operation of an authoritative public weather service 31-Aug-2015 30-Nov-2018 30-Nov-2020 providing meteorological and hydrological warnings for extreme and high impact events reaching vulnerable communities Comments (achievements against targets): • Identifier: IRI 7 • Unit of Measure: While the unit of measure was stated as Text, the target reflected a user satisfaction index stated in percentage. • Achievement: Unachieved • Explanation: The result is based on a User Satisfaction Index (USI) of 1,090 households in the flood affected districts of Sunsari, Saptari, Mahottari, Rupandehi, Banke, and Kailali. A baseline survey carried out in 2015 showed a baseline USI of 49%, which shows that the overall satisfaction level of the climate vulnerable communities of DHM’s public weather service has decreased. • Information Source: DHM Baseline Survey Final Report (August 2015), DHM Endline Survey Final Report (September 2020) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 36 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Establish National Text None Climate impact The NFCS was Framework for Climate Assessment by sector developed through a Services parallel project but remains to be adopted by GON (Year 1 target); The climate impact assessment by sector was not initiated. 15-Apr-2013 30-Nov-2018 15-Nov-2020 Comments (achievements against targets): • Identifier: IRI 8 • Unit of measure: Text • Achievement: Unachieved • Explanation: A National Framework for Climate Services (NFCS) was drafted in November 2020 by the Asian Disaster Preparedness Center (ADPC) and submitted to DHM through a parallel project - Climate Adaptation and Resilience for South Asia (CARE) (P171054). The NFCS provides an overview of existing state of climate services in Nepal, user needs for climate services, and current and future capacity to produce and deliver timely, accurate and relevant climate products. The prioritized sectors identified by the NFCS were agriculture and food security, disaster risk reduction, water, and health. It remains to be finalized and approved by DHM. An assessment of climate impact by sector was not initiated. • Information Source: Review of the Draft National Framework for Climate Services (November 2020) Component: Component D: Creation of an Agriculture Management Information System Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improved access to AMIS Text 0.00 Once every month Once every month Page 37 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) data 15-Apr-2013 30-Nov-2018 15-Nov-2020 Comments (achievements against targets): • Identifier: IRI 9 • Unit of Measure: Frequency of updates and improvements of the AMIS portal • Achievement: Fully achieved • Explanation: The AMIS Portal has been updated on a monthly or more frequent basis by different entities since it became live on May 23rd, 2018, which includes weekly AMIS Agromet Advisory Bulletins (AAB); Hamro Krishi App (HKA) updates; Frequently Asked Question and Answer (FAQs)’ on major crops; NARC digital library of books/reports on livestock, crop and soil; videos related to project activities, climate change, and the impact of ICT in agriculture; names of district focal persons as changes occur; and other information. • Information Source: The MOALD BRCH completion report (November 2020). AMIS Portal web analytics report not available. • Attribution: The project financed the establishment of the AMIS, its information products, and training of MOALD/NARC staff on its maintenance and updates. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improved linkage between Percentage 0.00 85.00 100.00 AMIS and WMO AMIS 15-Apr-2013 30-Nov-2018 15-Nov-2020 Comments (achievements against targets): • Identifier: IRI 10 • Unit of Measure: Percentage of AMIS products linked to the WMO AMIS (WAMIS). • Achievement: Overachieved • Explanation: The Nepal AMIS web portal was linked to the WAMIS on January 25, 2018, with all AMIS products accessible on the WAMIS. The following products are directly listed on the WAMIS for quick access: AAB, Weather Statistics (Charts), GIS Maps of Agricultural Production, Crop Calendars, and Reports on Agromet Topics and Agricultural Atlas of Nepal. Page 38 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) • Information Source: Review of the WAMIS (http://www.wamis.org/countries/nepal.php). • Attribution: The project financed the development of the AMIS portal and its associated information products; and consultancy to link the AMIS with WAMIS. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased extension service Number 0.00 25.00 69.00 and community training 01-Mar-2013 30-Nov-2018 15-Nov-2020 Comments (achievements against targets): • Identifier: IRI 11 • Unit of Measure: While the unit of measure was stated as Number, the target reflects the percentage of extension service personnel and community leaders certified to train farmers on climate resilient agriculture. • Achievement: Overachieved • Explanation: Total target was 2000 (50 focal person of pilot districts, 100 MoALD staffs, 5 PMU staffs, 1263 lead farmers and others) of which 1,380 (69%) were certified master trainers by MOALD till project closing. These include: Out Country Study Visit -157 people, Technical Training including ToT - 1,135 and Farmer's Exposure Visit - 88). • Information Source: Clarification on "certified trainer" provided by MOALD per request. Review the list of people trained from Table 13: Capacity Building Training by Type, Organizer and Sex of the MOALD BRCH ICR (November 2020). • Attribution: The project financed out of country study and exposure visits and technical training workshops which formed the basis for the certification. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 39 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Improved infrastructure Percentage 0.00 100.00 100.00 development 15-Apr-2013 30-Nov-2018 15-Nov-2020 Comments (achievements against targets): • Identifier: IRI 12 • Unit of Measure: Percentage of farmer groups and cooperatives in all 26 pilot districts provided with AMIS infrastructure, namely software and hardware to help interface with the AMIS. • Achievement: Fully achieved • Explanation: The target and result reflect AMIS infrastructures (i.e., Hamro Krishi app installed on smart mobile phones, Kisan SIM, rain gauge and thermometer) distributed to all targeted 1,263 farmer groups and cooperatives in the 26 pilot districts. • Information Source: MOALD BRCH ICR (November 2020), Annex 8: AMIS Infrastructure Distribution List. • Attribution: The project financed the hardware and software for the AMIS infrastructure. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased adoption of AMIS Number 0.00 7.00 12.00 tools by farmers 15-Apr-2013 30-Nov-2018 15-Nov-2020 Comments (achievements against targets): • Identifier: IRI 13 • Unit of Measure: Number of agro-weather and agro-climate information and risk management products developed and used in farming communities • Achievement: Overachieved • Explanation: The agro-weather and agro-climate information and risk management AMIS products are: (1) Agromet Advisory Bulletin (AAB), (2) Kisan Call Centre (KCC), (3) an SMS system, (4) Hamro Krishi App duly synchronized with AMIS web portal, (5) Package of Cultivation Practices Page 40 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) (POCP), (6) Climate Vulnerable Area Mapping (CVAM), (7) a Frequently Asked Questions feature (FAQ), (8) Irrigation Canal Data, (9) Crop calendars of various crops, (10) SMART Agriculture Manual, (11) Weather and Climate Change induced Disease/Pest Occurrence and relevant management practices in crops and livestock, and (12) Agriculture Handbook. • Information Source: AMIS web portal and WAMIS; MOALD BRCH completion report (November 2020), Table 4: Download/distribution of Hamro Krishi Mobile Application; and MOALD Endline Survey Final Report (December 2019) • Attribution: The project financed the establishment of the AMIS and agromet information products listed above. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improved access to and use Number 0.00 100.00 100.00 of training opportunities 15-Apr-2013 30-Nov-2018 15-Nov-2020 Comments (achievements against targets): • Identifier: IRI 14 • Unit of Measure: While the unit of measure was stated as Number, the target reflected the percentage of staff in MOALD receiving technical trainings and/or advanced degrees in weather data analysis, agromet modeling and climate services, etc. • Achievement: Fully achieved • Explanation: The target was based on 420 technical staff of MOALD (including AMIS experts, agriculture extension staff, and MOALD/NARC staff). By project closing, a total of 420 AMIS technical staff received technical training in GIS, crop modeling, and climate change modelling focused on agriculture and food security, including 100 from MOALD/NARC, 15 PMU staff, 5 additional staff members from MOALD/NARC with an advanced degree in the Agriculture and Forest University, 150 staff members from the MOALD Junior Technical Officer Program (JT/JTA), and 150 other staff members from MOALD. • Information Source: MOALD BRCH completion report (November 2020), Annex 1: Capacity Building Trainings Summary • Attribution: The project financed the technical trainings and enrollment of selected staff to advanced university degree as mentioned above. Page 41 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improved efficiency to Percentage 0.00 65.00 100.00 disseminate and obtain feedback of agro-weather 15-Apr-2013 30-Nov-2018 15-Nov-2020 and agro-climate information Comments (achievements against targets): • Identifier: IRI 15 • Unit of Measure: Percentage of farmers ICT users receiving AMIS information. The target reflected 65% of 31,575 farmers (i.e., 1263 farmer groups and 25 farmers per farmer group) residing within the pilot districts which were considered direct AMIS beneficiaries. • Achievement: Overachieved • Explanation: At project closing, more than 40,000 farmers are receiving approximately 2-3 SMSs every week from the AMIS, and there have been more than 52,000 ‘Hamro Krishi’ app downloads. • Information Source: MOALD BRCH completion report (November 2020) • Attribution: The project financed the establishment of the AMIS, and dissemination of agromet information through products such as the HKA and SMS early warning alerts. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increased satisfaction of Percentage 28.03 65.00 40.70 users with AMIS services 31-Dec-2015 30-Nov-2018 31-Dec-2019 Comments (achievements against targets): • Identifier: IRI 16 • Unit of Measure: Composite satisfaction index expressed as a percentage where 100% is completely satisfied. Page 42 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) • Achievement: Substantially achieved • Explanation: A CUSI was calculated as the average percentage of users highly satisfied and moderately satisfied with the main AMIS services (i.e., AMIS web portal, HKA, PPCR focal person, KCC, SMS/Mobile, and AAB). The Endline survey completed in December 2019 revealed a 40.7% CUSI for AMIS services (29.3% of 411 respondents were highly satisfied and 51.7% of 411 respondents were moderately satisfied). The Baseline survey completed in June 2015 established the CUSI as 28.3%. • Information Source: MOALD Baseline Survey Final Report (June 2015), MOALD Endline Survey Final Report (December 2019) • Attribution: The project financed the establishment the AMIS, its services, and relevant capacity building. Page 43 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) B. KEY OUTPUTS BY COMPONENT Outcome 1: Improved accuracy and timeliness of weather and flood forecasts and warnings for climate-vulnerable communities • Increased financial sustainability of DHM operations • Increased accuracy and timeliness of weather forecasts (verified using standard methods Outcome Indicators for forecast verification - WMO WWRP/WCRP) • Increased satisfaction of users with DHM services • New DHM regulations and business practices • Improved access to and use of training opportunities • Improved transmission of data to WMO GTS/WIS • Improved status of surface meteorological observation network • Improved status of surface hydrological observation network Intermediate Results Indicators • Provision of real-time or near real-time data to the users • Development and reliable operation of an authoritative public weather service providing meteorological and hydrological warnings for extreme and high impact events reaching vulnerable communities • Establish NFCS Component A: • Development of a legal and regulatory framework for DHM operations: A team of national legal experts prepared a draft DHM Bill and a Regulation in 2018, which was thoroughly discussed at various levels—the department itself, MOEWRI, and other relevant institutions. Key Outputs by Component • Trainings for Building a Roadmap of a Quality Management System in 2016. All training (linked to the achievement of the material was submitted to the DHM. Objective/Outcome 1) • Training in project management and planning for senior DHM staff. At the end of the training all the participants were able to build visionary aspects of a desired project, including strategic planning and development management. • Needs assessment of air quality monitoring networks. The study highlighted the need and outlined a strategic plan for a pilot project on an air quality monitoring network. Page 44 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Component B: • Technical re-equipment of the hydrological network and provision of special equipment for hydrological stations. A total of 70 automatic hydrological stations were installed and commissioned across the country. • Delivery and installation of a weather radar system. A C-band weather radar system was installed Guranse hill, Ratanangla, Surkhet of western Nepal in November 2018. The system became fully operational in January 2019. • Establishment of an upper air sounding station at the premises of Tribhuvan University, Kirtipur, and has been in regular operation since March 2019. • Surface observation meteorological network. 88 AWSs were installed in these sites and commissioned following successful Site Acceptance Test and Operational Test. • Training in SOPs was provided to the DHM. • A lightning detection system ‘LINET’ comprising 9 stations, mostly at airports across the country, was established and Site Acceptance Test and Operational Test were successfully completed in July 2017. • Establishment of a laboratory to calibrate meteorological equipment in November 2019. • Communication equipment for ICT infrastructure and a management system were installed in the GIDC in Singha Durbar in 2018. • A remote sensing and GIS laboratory were installed and commissioned in April 2016. • A high-performance computing system was installed and commissioned in the GIDC located at Singha Durbar. • A high-resolution local area NWP system was installed and commissioned at the DHM. A Weather Research and Forecasting model version 4.1 was installed in HPC at the GIDC. The model is running four times a day using Global Forecast System data and generates hourly forecasts of various weather parameters for the next three days. The resolution of the model is 9 km for the parent domain and 3 km for the nested domain. • A pilot station for air quality monitoring was established in Nagarkot in November 2020 to monitor the ambient air quality of the valley. • A 10-story DHM headquarters office was constructed/refurbished in the department's existing premise in Babarmahal, Kathmandu. Page 45 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Component C: • A hydromet workstation was established and commissioned at DHM headquarter office. • A weather presentation system was established at the DHM headquarter office. • A dynamic and integrated public web service for the DHM was established and commissioned. • Establishment of an ‘end-to-end’ flood EWS in Koshi and West Rapti river basins to support and strengthen disaster risk management in these basins. • Development of SOPs for the DHM for the end-to-end EWS of the two pilot river basins. • Development of a digital elevation map for Terai Region, used to develop the EWS for the Koshi and West Rapti river basins. Outcome 2: Availability of agricultural management information system services for farmers to mitigate climate-related production risks • Introduction of a sustainable AMIS that systematically streams and archives agriculture Outcome Indicators relevant hydromet and agromet data and information • Improved access to AMIS data • Improved links between the AMIS and WMO AMIS • Increased extension service and community training • Improved infrastructure development Intermediate Results Indicators • Increased adoption of the AMIS tools by farmers • Improved access to and use of training opportunities • Improved efficiency to disseminate and obtain feedback of agro-weather and agro- climate information • Increased satisfaction of users with the AMIS services Component D: • The AMIS web portal, including a data migration function, an SMS gateway, and a mobile Key Outputs by Component application (both iOS and android), went live on May 23, 2018. The system is functioning (linked to the achievement of the well except for the SMS gateway. Objective/Outcome 2) • The required hardware and software for both the data center and disaster recovery center were installed successfully. Altogether, 1,263 sets of mobile phones and rain Page 46 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) gauges and thermometers were distributed to 1,263 selected farmers’ groups/cooperatives in the 26 pilot districts. • One central-level and 52 district-level KCCs were installed at the 26 pilot districts and then transferred to the corresponding AKCs and VHLSECs. • Both NARC and PMU digitized and archived on the AMIS web portal more than 20,000 sets of various historical agriculture, livestock, market, and soil information and other relevant data, research documents, and books. • Several studies on agricultural insurance were prepared by NARC and published on the AMIS web portal for public access. • Web portal, HKA, SMS, and KCC were the major ICT-based systems to disseminate the information produced under this project. • Several technical and other capacity-building trainings in agromet were organized by the MOALD/NARC for 20,112 people (14,027 male and 6,085 female) from the PMU, NARC extension workers, senior officials from the MOALD, farmers from pilot districts, and other stakeholders. Page 47 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Claudia W. Sadoff Task Team Leader Vladimir V. Tsirkunov Senior Environmental Engineer Pai-Yei Whung Adviser Stephanie Borsboom Operations Officer Minneh Kane Lead Counsel Abedalrazq F. Khalil Water Resources Specialist Tenzin Dolma Norbhu ICT Policy Specialist Saurabh Suresh Dani, Anil Pokhrel Disaster Risk Management Specialists Bigyan B. Pradhan Financial Management Specialist Shambhu Prasad Uprety Procurement Specialist Hiramani Ghimire Governance Specialist Drona Raj Ghimire, Parthapriya Ghosh, Bandita Sijapati Social Development Specialists Purna Bahadur Chhetri Rural Management Specialist Sulochana Nepali Team Assistant Non Bank Staff David Rogers Hydromet Expert Harlan Shannon, Clyde Fraisse Agricultural Experts Aurora Bell Nowcasting Expert Spreafico Manfred Hydrologist Supervision/ICR Haris Khan Task Team Leader Vladimir Tsirkunov Lead Hydromet Specialist Bishwa Raj Basaula, Praksh Jung Thapa, Yogesh Bom Financial Management Specialists Malla Shambhu Prasad Uprety, Ramesh Raj Bista Procurement Specliasts Hiramani Ghimire Governance Specialist Jaya Sharma, Rekha Shreesh, Jun Zeng Social Development Specialists Annu Rajbhandari, Jie Li, Drona Ghimire Environmental Specialists Arati Belle, Kamran Akbar, Avani Dixit, Adam McAllister, Disaster Risk Management Specialists Dechen Tshering Mannava Sivakumar Agro-Meteorologist Shruti Mishra Agricultural Economist Mark Heggli Hydromet Specialist Dugkeun Park Ladsldie Specialist Page 48 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Name Role Jitendra Bothra Building Structural Engineer Shyam Ranjitkar Water Resoruces Engineer Stephanie Borsboom Operations Officer Sulochana Nepali, Najaf Khan Operations Analysts Neelesh Shrestha, Hari Bhattarai, Sujan Bahadur Thing, Consultants Basanta Paudel, Pratima Shrestha, Abhishek Yadav Lalita Rai, Marie Elvie Team Assistants B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 34.041 281,597.91 FY13 15.990 185,141.44 FY14 0 - 57.81 Total 50.03 466,681.54 Supervision/ICR FY13 6.600 35,529.08 FY14 8.652 239,760.51 FY15 18.338 206,452.56 FY16 16.869 206,231.22 FY17 5.085 53,966.92 FY18 16.886 77,105.86 FY19 6.376 16,500.34 FY20 5.335 24,498.78 Total 84.14 860,045.27 Page 49 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) ANNEX 3. PROJECT COST BY COMPONENT Table 3.1. Project Cost by Component Components Amount at Approval Actual at Project Percentage of (US$, millions) Closing (US$, millions) Approval PPCR GON PPCR GON PPCR GON Institutional Strengthening, Capacity Building and 5.10 0.00 3.35 — 65.68 — Implementation Support of DHM Modernization of the Observation Infrastructure and 16.45 0.30 13.99 2.34 85.04 780 Forecasting Enhancement of the Service 3.45 0.00 1.61 — 46.66 — Delivery System of DHM Creation of an Agriculture 6.00 0.00 5.97 — 99.50 — Management Information System Total 31.00 0.30 24.92 2.34 80.38 780 Note: The ICR datasheet shows a revised trust fund amount of US$24,918,956. It reflects the cancelation of unused trust fund proceeds by the World Bank at project completion. Table 3.2. Planned vs. Actual Cost of High Value Procurement Packages Planned Actual Market Cost Saving Component Activity Description Amount Amount Approach (US$) (US$) (US$) System Integrator - Consulting Service for Detailed Design of Open - A 2,150,380.00 2,627,441.05 (477,061.05) the DHM System, Procurement International and Implementation Support Construction and Open - B refurbishment 3,800,000.00 3,013,680.96 786,319.04 International of DHM office building Surface Hydrological Network and Special Equipment for Open - B Hydrological Stations (Supply, 2,300,000.00 1,938,600.54 361,399.46 International Delivery, Installation and Commissioning) Surface Meteorological Network (Supply, Delivery, Open - B 3,700,000.00 3,780,599.43 (80,599.43) Installation and International Commissioning) Delivery and installation of Open - B 2,750,000.00 1,560,349.39 1,189,650.61 Weather Radar National High-power Computers for Open - B Numerical Weather 400,000.00 327,327.00 72,673.00 International Prediction Open - B Lighting detection network 500,000.00 344,277.00 155,723.00 International Page 50 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Establishment of hydrometeorological Open - C workstations and automated 1,100,000.00 1,031,728.00 68,272.00 International production system Consultancy service for "Establishment of " End-To- Open - C End" Flood Early Warning 738,000.00 792,788.03 (54,788.03) International System in Koshi and West Rapti River Basins AMIS portal Open - D hardware requirements for 399,870.00 293,044.13 106,825.87 National Data Center (DC) Procurement of AMIS portal Open - D 122,400.00 122,400.00 - software requirements National Procurement of Hardware for Open - D 217,000.00 163,631.44 53,368.56 Diester Recovery (DR) National Procurement of ICT Open - D equipment for KCC in 51 128,000.00 114,999.30 13,000.70 National Knowledge Centers Consulting services for AMIS portal development, data Open - D migration, digitization, training 608,000.00 502,114.96 105,885.04 National and management Total 18,913,650.00 16,612,981.23 2,300,668.77 Page 51 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) ANNEX 4. Economic and Financial Analysis Section 1: Introduction 1. The BRCH Project pursued two objectives: (a) to reduce the impact of climate-related hazards in Nepal by enhancing hydromet and early warning systems and services delivery and (b) to improve the capacity of Nepalese farmers to mitigate climate-related production risks by installing an agro-weather and agro-climate advisory system (also known as AMIS). The key expected economic benefits were (a) reduced damage from climate-related hazards to assets such as land, houses, livestock, and infrastructure; (b) reduced deaths and injuries from weather-related events; and (c) improved productivity in weather- sensitive sectors, including agriculture, energy, and tourism. This report provides a summary of the economic performance of the project. 2. In terms of report structure, section 2 provides a brief assessment of the achievement of project outputs and outcomes. Section 3 outlines key methodology and design assumptions used in the economic analysis. Section 4 reviews financial and economic project costs, including comparison with costs envisaged during project design. Sections 5 and 6 present quantified and unquantified economic benefits delivered by the project, respectively. Section 7 presents key parameters of project economic performance and a sensitivity analysis of economic viability to different values of uncertain parameters. The MS Excel model upon which this analysis is based is available here; the model contains tables 4.7– 4.20 mentioned at the end of this annex. Section 2: Achievement of Project Outputs and Outcomes 3. The project sought to achieve its objectives by delivering four components: Component A - Institutional Strengthening, Capacity Building, and Implementation Support of the Department of Hydrology and Meteorology; Component B: Modernization of the Observation Networks and Forecasting; Component C: Institutional Strengthening, Capacity Building, and Implementation Support of DHM; and Component D: Creation of an Agriculture Management Information System (AMIS) at the Ministry of Agriculture and Livestock Development. Despite a number of shortcomings in the delivery of these components—discussed in section II B: Achievement of PDOs (Efficacy)—the project is considered to have delivered its main outputs: (a) a modern, well-functioning and sustainable hydromet and early warning systems and (b) the AMIS. The attainment of project benefits discussed in the following paragraphs, is contingent upon effective operation of these systems for the economic life of the project. Section 3: Methodology and Assumptions 4. The economic analysis is based on information sourced from project documents, consultations with the project team, and secondary data. The project was completed in November 2020, meaning that insufficient time has passed for quantitative data on project impacts to become available. Consequently, the analysis focuses on updating ex ante costs and benefit estimates using the latest information; improving the ex ante methodology, where feasible;41 and assessing field-level evidence to verify that the expected benefits are, in fact, taking place. 41The ex ante methodology appears to have used financial rather than economic prices. It did not use discounted future costs and benefits, and it presumed no changes in Nepal’s GDP structure during the economic life of the project. Page 52 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) 5. The key methodology and design assumptions used in assessing the economic performance of the project are as follows: (a) Project economic life is estimated at 15 years (2013–2027). (b) A domestic price numeraire is used for economic pricing, with a shadow exchange rate conversion factor (SERF) of 1.07 applied to the financial price of tradeable items (both costs and benefits).42 (c) The exchange rate of US$1.0 = NPR 85.9 (official exchange rate as on December 6, 2012)43 is used for conversion of financial prices between international and domestic currencies. (d) All prices are expressed in 2013 constant terms. (e) All labor used for equipment installation and O&M is presumed to be skilled, and hence the shadow wage rate factor was not applied to labor costs. (f) The average domestic taxes and duties on applicable items are estimated at 13 percent (the current VAT rate).44 (g) The estimated proportions of tradeable costs and domestic resource costs by major subproject costs categories are reflected in the estimates of subproject economic costs (see tables 4.10 and 4.11 of the economic model). (h) The project O&M costs for Components A, B, and C (US$1.44 million per year) are based on estimates developed by the project implementation consultants.45 For O&M costs for Component D, the PAD estimates (10 percent of investment costs, or US$600,000) are used in the absence of alternative information (table 4.12). Project expenditure on O&M is estimated to have commenced in Project Year (PY)6 (at 20 percent) and gradually increased to 100 percent by PY10 (table 4.13). (i) The benefits of improved hydromet and EWS services accrue to the whole country, while the benefits of AMIS accrue to the 26 targeted districts (which represent 44 percent of Nepal’s farming households).46 (j) The effectiveness of hydromet services in Nepal before the project are estimated at 20 percent, as sourced from secondary literature47 and confirmed by the project team (Tables A8a and A10). (k) The stream of project benefits is estimated to commence in PY3 (at 5 percent), gradually increasing to 100 percent by PY10 due to the timing of the capital works and lagged adoption of the hydromet, EWS, and AMIS services by the beneficiaries. 42 The SERF for Nepal was sourced from a recent ADB project: https://www.adb.org/sites/default/files/linked- documents/54107-001-ea.pdf. 43 PAD. 44 https://www.ecovis.com/nepal/tax-guide/. 45 DHM. 2021. Building Resilience to Climate Hazards Project - Implementation Completion and Results Report. 46 MOALD. 2020. Project Completion Report - Agriculture Management Information System. 47 World Bank. 2021. The Value of Surface-based Meteorological Observation Data. Page 53 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) (l) Annual discount rate of 9 percent is used in estimating NPV of future costs and benefits, as per the current practice of multilateral development banks.48 6. The key differences in the ex-post methodology compared to the evaluation at appraisal are as follows: (a) The actual costs instead of projected costs were considered. (b) The costs were presented in economic rather than financial prices. (c) An annual discount rate of 9 percent was used in estimating the net present value (NPV) of future costs and benefits according to the standard practice for multilateral development banks,49 though the ex-ante methodology at appraisal did not apply discounting. (d) A gradual, rather than a sudden, stream of project benefits was assumed (from 5 percent in Year 3 to 100 percent in Years 10–15), due to the lagged adoption of new services by the beneficiaries. (e) Changes in Nepal’s GDP and agricultural output during the 15-year economic life of the project (2013–2027) were incorporated into the analysis rather than presuming them as constant. Section 4: Project Costs 7. A summary of the project capital costs at project completion is provided in table 4.1. The overall financial cost (US$27.26 million) is 87.1 percent of the project budget (US$31.3 million). This underspending is due to the non-delivery of a number of project activities, including establishment of a National Climate Service and certain capacity-building initiatives. Project expenditure took place during PY1–9 (2013–2021). The economic capital cost of the project is US$27.86 million. The economic cost was obtained by multiplying the cost of tradeable goods and services by SERF (1.07) and subtracting taxes and duties (of average 13 percent) from domestic goods and services. Table 4.1. Project Capital Cost (Financial and Economic) (US$) Component Financial Cost Economic Cost Component A 3,353,437 3,394,571 Component B 13,985,857 14,493,850 Component C 1,610,706 1,669,210 Component D 5,968,956 6,043,229 Counterpart financing 2,340,000 2,261,318 Total capital cost 27,258,956 27,862,179 8. Annual project O&M costs are presented in table 4.2. The O&M costs are estimated to have commenced in PY6 and are expected to continue for the economic life of the project (that is, till PY15). 48 ADB. 2017. Guidelines for the Economic Analysis of Projects. Equivalent World Bank guidelines are not available, but multilateral development banks agree on the figures included in the ADB guidelines. 49 ADB. 2017. Guidelines for the Economic Analysis of Projects. Equivalent World Bank guidelines are not available, but multilateral development banks agree on the figures included in the ADB guidelines. Page 54 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) The O&M costs are significantly lower than those envisaged during project design (10 percent of the investment cost, or US$3.1 million per year).50 Table 4.2. Annual O&M Costs (Financial and Economic) (US$) Component Financial Cost Economic Cost Components A, B, and C 1,440,000 1,407,568 Component D 600,000 586,487 Total annual O&M cost 2,040,000 1,994,055 9. The NPV of the total project economic costs over the economic life of the project (15 years), at a discount rate of 9 percent, is US$23.66 million. Further detail on project costs, including expenditure by cost category and by year, is provided in tables A4.8–4.13. Section 5: Quantified Project Benefits 10. The benefits quantified and included in the economic analysis calculations are those arising from (a) reduced asset losses from natural hazards due to improved hydromet and EWS services (project Components A–C); (b) improved agricultural productivity due to improved agromet services provided by the AMIS (Component D), and (c) improved productivity in other weather-sensitive sectors due to improved hydromet services (Components A–C). Reduced Asset Losses from Natural Hazards 11. Improved hydromet and EWS services can significantly reduce asset losses from natural hazards such as floods and storms by increasing preparation time for authorities and the general public and preventing dangerous travel. Asset losses can be reduced by activities such as moving machines and equipment to safer areas and moving toxic materials and chemicals to safe places to prevent pollution. Before the project, much of Nepal's hydrological and meteorological system relied on manual data collection, with infrequent and unreliable reporting. From a service delivery standpoint, there was no system in Nepal for issuing authoritative warnings for weather and weather extremes to government authorities, key user groups, and communities at high risk. Among other activities, the project installed 70 automatic hydrological stations and 88 AWSs across the country and supported mass dissemination of flood and storm early warning to households through SMSs, emails, and social media. 12. The ex-ante economic analysis used two approaches to estimate reduced asset losses to be delivered by the project. The ‘sector-specific’ methodology used historical data to estimate Nepal’s average annual losses from weather-related events (US$18.75 million) and predicted that the presence of modern hydromet and EWS services would reduce these losses by 5–10 percent.51 Using these assumptions, the average annual benefits were estimated at US$0.9–US$1.8 million over the 15-year project life (in financial terms and with no discounting). This estimate has now been adjusted by (a) using 50PAD. 51The 5–10 percent estimate was sourced from the following: World Bank. 2003. Project Appraisal Document on a Proposed Loan in the Amount of US$80 Million to the Russian Federation for a National Hydromet Modernization Project . A review of relevant literature produced a range of 10 to 60 percent, indicating that this estimate is conservative (World Bank. 2008 Weather and Climate Services in Europe and Central Asia - A Regional Review; World Bank and WMO. 2015. Valuing Weather and Climate: Economic Assessment of Meteorological and Hydrological Services; Hallegatte, S. 2012. A Cost-Effective Solution to Reduce Disaster Losses in Developing Countries - Hydro-Meteorological Services, Early Warning, and Evacuation). Page 55 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) an updated estimate of Nepal’s average annual losses due to weather-related events (US$32.5 million);52 (b) incorporating the gradual nature of the take-up of EWS services; and (c) incorporating an estimate of Nepal’s hydromet and EWS services providing 20 percent of these benefits before the project (thus making the approach consistent with the benchmarking methodology described below). The revised estimate of average annual benefits (with no discounting) is US$0.72–US$1.44 million. When discounting is applied, the NPV of these benefits is US$4.26–US$8.53 million over the 15-year economic life of the project. For further details, see table 4.14. 13. The alternative, ‘benchmarking’ methodology followed Hallegatte (2012),53 a study which found that well-functioning, modern EWSs in Europe reduce disaster-related asset losses by between 0.003 and 0.017 percent of GDP. This range was increased to 0.017–0.1 percent of GDP for Nepal to consider the country’s high degree of climate vulnerability. Nepal’s hydromet and EWS services were assumed to provide 20 percent of these benefits before the project and 100 percent after the project. Using these assumptions, the ex ante economic analysis estimated an average annual benefit of US$2.9–US$17 million over the 15-year project life (with no discounting). This estimate has now been adjusted by (a) incorporating changes in Nepal’s GDP during the economic life of the project (2013–2027);54 (b) incorporating the gradual nature of the take-up of EWS services; and (c) updating the estimate of Nepal’s hydromet and early warning systems’ capacity before the project to 20 percent, according to the latest World Bank guidance for developing countries.55 The revised estimate of the average annual benefits (with no discounting) is US$2.3–US$13.7 million. When discounting is applied, the NPV of these benefits is US$13.4–US$78.6 million over the 15-year economic life of the project (for further details, see table 4.15). 14. In terms of evidence of benefits, by project completion a total of 88 AWSs and 70 automatic hydrological stations were installed throughout the country and fully operational, of which 21 AWSs were installed in or near the airports aiming to benefit the aviation sector. The 2019 Endline User Satisfaction Survey of DHM services found that 75 percent of respondents were satisfied with the accuracy of rainfall and temperature forecasts, while 67.5 percent reported that the improved weather forecasts were helpful in protecting their lives and property.56 Notably, the survey was conducted a year before the completion of the project, when a number of major activities were still under implementation; consequently, the level of user satisfaction at project completion is likely to be higher. Overall, it appears that reductions in asset losses as a result of improved hydromet and EWS services are taking place. Improved Agricultural Productivity 15. Agromet services can improve agricultural productivity by informing farming practices, for example, when to plant, harvest, or apply fertilizer. For example, a 1998 study on lettuce production in New York found a 10 percent productivity increase when farmers considered weather forecasts to 52 This figure was obtained from 1990–2017 EMDAT data (https://public.emdat.be/). 53 Hallegatte, S. 2012. A Cost-Effective Solution to Reduce Disaster Losses in Developing Countries - Hydro-Meteorological Services, Early Warning, and Evacuation. 54 2013–2019 GDP figures were sourced from the World Bank (https://data.worldbank.org/indicator/NY.GDP.MKTP.KD.ZG?locations=NP); 2020 and 2021 projections were sourced from the ADB (https://www.adb.org/countries/nepal/economy); 2022–2025 projections were sourced from statista.com(https://www.statista.com/statistics/425729/gross-domestic-product-gdp-growth-rate-in-nepal/); and for 2026– 2027 projections, real GDP growth was estimated at 5 percent per year. 55 World Bank. 2021. The Value of Surface-based Meteorological Observation Data. 56 DHM. 2021., Building Resilience to Climate Hazards Project - Implementation Completion Report. Page 56 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) schedule irrigation.57 Before the project, seasonal climate predictions and the AMIS services in Nepal were largely unavailable.58 Among other activities, the project developed an AMIS web portal; institutionalized a range of methods to disseminate agromet services to farmers (including HKA, SMS services, KCCs, and paper-based publications); and trained extension workers and farmers in the use of agromet services. 16. The ex-ante economic analysis estimated that agromet services installed under the project would increase Nepal’s agricultural output (US$6.6 billion in 2011, or 35 percent of GDP) by 0.5–1.0 percent.59 This resulted in an estimated annual benefit of US$33–US$66 million over the 15-year project life (with no discounting). This estimate has now been adjusted by incorporating (a) the changes in Nepal’s agricultural output during the economic life of the project;60 (b) the gradual nature of the take-up of agromet services; and (c) the fact that agromet services under the project reached only 26 out of Nepal’s 77 districts, thus affecting only 44 percent of Nepal’s households rather than the whole country. The revised estimate of average annual benefits (with no discounting) is US$7.7–US$15.4 million. When economic prices and discounting are applied, the NPV of these benefits is US$45.6–US$91.1 million over the 15-year economic life of the project. For further details, see table 4.16. 17. In terms of evidence of benefits, the HKA has been downloaded over 52,000 times to date and over 40,000 farmers are receiving two to three agromet SMSs every week; over 20,000 people have received agromet training under the project. The 2020 Endline User Satisfaction Survey reported an overall 40.7 percent user satisfaction for the AMIS services (12.4 percent higher than the baseline but 24.3 percent less than the intended target), as well as yield increases of 0.8–4 tons per hectare in grains, cauliflower, cabbage, and summer potato among users compared to the baseline study. 61 Lastly, the MOALD project completion report62 documents a number of case studies of farmers using agromet services to inform their crop selection and cultivation practices, resulting in greater yields and higher incomes.63 Overall, it appears that increases in agricultural productivity due to improved agromet services are taking place. Improved Productivity in Other Weather-Sensitive Sectors 18. Apart from agriculture, modern hydromet services can provide useful information to other weather-sensitive sectors, thus improving their productivity. In the energy sector, weather forecasts are used to anticipate electricity demand, thus informing the timing of power generation. Weather forecasts are also used to optimize air traffic and ship routes in the transport sector as well as the use of labor resources and avoid accidents in the construction sector. In the tourism sector (of particular importance 57 Wilks, D.S., and D. W. Wolfe. 1998. “Optimal Use and Economic Value of Weather Forecasts for Lettuce Irrigation in a Humid Climate.” Agricultural and Forest Meteorology 89: 115–130. 58 PAD. 59 While the PAD didn’t explicitly state so, this estimate is likely to have come from the following: Perrels. 2011. Social Economic Benefits of Enhanced Weather Services in Nepal. A Finnish Meteorological Institute report for the Finnish Nepalese Project. 60 Figures for 2013–2020 were sourced from https://cbs.gov.np/wp-content/upLoads/2021/01/National-Accounts-of-Nepal- 2019_20.pdf; figures for 2021–2027 were extrapolated (1 percent decline per year). 61 MOALD. 2020. Project Completion Report - Agriculture Management Information System. 62 MOALD. 2020. Project Completion Report - Agriculture Management Information System. 63 As an example, one case study documents Nirmala Sapkota (a female farmer from Manikapur village of Banke district) using SMS recommendations from the HKA to plant a high-yield, weather-resistant variety of potato, resulting in savings of NPR 150,000 (around US$1,280) per season. Page 57 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) to Nepal),64 weather forecasts are used to plan future activities such as accessing a particular trekking route.65 19. The ex-ante economic analysis estimated that modern forecasts add a value of 0.1 to 1.0 percent in weather-sensitive sectors.66 Agriculture and other weather-sensitive sectors were conservatively estimated to contribute 25–35 percent of Nepal’s GDP, resulting in an estimated annual benefit of US$42.5–US$59.5 million over the 15-year project life (with no discounting). However, separation of agriculture and other weather-sensitive sectors is recommended in this case because (a) agromet services provided under Component D are more detailed than hydromet services provided to other sectors, and are hence likely to have greater impact on productivity and (b) agromet services cover only 26 out of Nepal’s 77 districts, while hydromet services cover the whole country. 20. Weather-sensitive sectors included in the ex-post analysis are (a) electricity, gas, and water; (b) construction; (c) transport, storage, and communication; and (d) tourism. Collectively, they accounted for 21 percent of Nepal’s GDP in 2020. Economic contributions of these sectors during 2013–2020 were sourced from secondary literature and extrapolated to 2027.67 Using the 0.1–1.0 percent range for productivity improvements due to modern hydromet services and presuming a gradual take-up of the hydromet information, the revised estimate of average annual benefits (with no discounting) is US$2.8– US$27.7 million. When economic prices and discounting are applied, the NPV of these benefits is US$16.1–US$160.8 million over the 15-year economic life of the project. For further details, see table 4.17. 21. In terms of evidence of benefits, 21 AWSs installed under the project are located in or near airports, making the aviation sector a clear beneficiary. While no evidence of benefits to other sectors was obtained, this does not necessarily mean that these benefits are not taking place (particularly given the recent completion of the project). Nonetheless, this is judged to be the least certain of the sources of quantified economic benefits; consequently, the scenario of absence of such benefits is included in the sensitivity analysis. 22. The flow of quantified economic costs and benefits under the most conservative project scenario is presented in table 4.3. 64 The tourism industry accounted for an average of 7.9 percent of Nepal’s GDP during 2011–2020 (www. https://knoema.com). 65 Hallegatte, S. 2012. A Cost-Effective Solution to Reduce Disaster Losses in Developing Countries - Hydro-Meteorological Services, Early Warning, and Evacuation. 66 This estimate was sourced from Hallegatte (2012). 67 Figures for 2013–2020 were sourced from https://cbs.gov.np/wp-content/upLoads/2021/01/National-Accounts-of-Nepal- 2019_20.pdf; figures for 2021–2027 were extrapolated using growth averages for 2013–2020. Page 58 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Table 4.3. Economic Costs and Benefits under the Most Conservative Scenario Project Year PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY8 PY9 PY10 PY11 PY12 PY13 PY14 PY15 Calendar Year 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 Capital cost (US$, millions) 0.00 6.07 0.68 0.98 2.15 6.70 4.28 5.78 1.22 0.00 0.00 0.00 0.00 0.00 0.00 O&M cost (US$, millions) 0.00 0.00 0.00 0.00 0.00 0.40 0.80 1.20 1.60 1.99 1.99 1.99 1.99 1.99 1.99 Total project economic costs (US$, millions) 0.00 6.07 0.68 0.98 2.15 7.10 5.07 6.97 2.81 1.99 1.99 1.99 1.99 1.99 1.99 Avoided asset losses (US$, millions) 0.00 0.00 0.07 0.13 0.20 0.26 0.52 0.78 1.04 1.30 1.30 1.30 1.30 1.30 1.30 Productivity gains in agriculture (US$, millions) 0.00 0.00 0.67 1.34 2.01 2.72 5.48 8.50 11.03 13.96 14.02 14.02 13.98 13.90 13.78 Productivity gains in other weather-sensitive sectors 0.00 0.00 0.20 0.39 0.63 0.87 1.85 2.74 3.65 4.77 4.94 5.11 5.28 5.46 5.64 (US$, millions) Total project economic benefits (US$, millions) 0.00 0.00 0.93 1.85 2.84 3.85 7.85 12.02 15.72 20.03 20.26 20.43 20.57 20.66 20.72 Net project economic benefits (US$, millions) 0.00 –6.07 0.25 0.87 0.68 -3.25 2.77 5.05 12.90 18.04 18.27 18.44 18.57 18.67 18.72 Page 59 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Section 6: Unquantified Project Benefits 23. The project also provides a number of hard-to-quantify benefits, which are excluded from economic analysis calculations. Perhaps the most important project impact is the avoided deaths, injuries, and illness from weather-related disasters. EWSs can prevent loss of human life by giving sufficient lead time for evacuation form hazardous areas and implementation of mitigation actions (for example, sandbagging during floods).68 According to EMDAT data, during 1990–2020, Nepal recorded 5,777 deaths and 2,751 injuries from weather-related disasters, equating to 186 deaths and 89 injuries per year.69 Given the relatively infrequent nature of such disasters (only 15 have occurred since the project commenced in 2013), more time is needed before the impacts of the project on reducing the human toll can be quantified with a degree of certainty; however, the DHM project completion report claims that the mass SMS service for flood alerts is resulting in fewer flood-related casualties.70 24. Improved hydromet and EWS services also contribute to day-to-day safety and general convenience of the public, including recreation, travel and commuting, event management, and preparation for severe weather. A number of studies on the economic value that households attach to public weather forecasts and warnings have been conducted in the past, with a willingness to pay for such services ranging from US$0.09 to US$19 per person per year. Lastly, improved hydromet services can also have a significant environmental impact by improving long-term monitoring of basic environmental indicators and management of local environmental quality.71 Section 7: Key Project Economic Parameters and Sensitivity Analysis 25. The comparison of ex ante and ex post key project economic parameters (in average annual financial terms, with no discounting) is presented in table 4.4. Table 4.5 presents the ex post economic parameters in total economic present value terms, using a 9 percent discount rate. Table 4.4. Ex Ante and Ex Post Economic Estimates - Average Annual Financial Terms, No Discounting Ex Ante Estimates72 Ex Post Estimates Low Estimate High Estimate Low Estimate High Estimate Project costs (US$, millions) 4.90 4.90 2.91 2.91 Project benefits Avoided asset losses (US$, millions) 0.90 17.00 0.72 13.69 Productivity gains in agriculture ((US$, millions) 33.00 7.69 15.39 Productivity gains in other weather-sensitive — 59.50 2.77 27.69 sectors (US$, millions) Total benefits (US$, millions) 33.90 76.50 11.18 56.77 Net benefits (US$, millions) 29.00 71.60 8.28 53.86 Benefit-cost ratio 5.00 14.60 3.85 19.54 68 Hallegatte, S. 2012. A Cost-Effective Solution to Reduce Disaster Losses in Developing Countries - Hydro-Meteorological Services, Early Warning, and Evacuation. 69 https://public.emdat.be/. 70 DHM. 2021. Building Resilience to Climate Hazards Project - Implementation Completion Report. 71 WMO and World Bank. 2015. Valuing Weather and Climate: Economic Assessment of Meteorological and Hydrological Services. 72 PAD Page 60 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Table 4.5. Ex Post Economic Estimates - Total Economic Present Value Terms, 9 Percent Discount Rate Ex Post Estimates Low Estimate High Estimate Project costs (US$, millions) 23.66 23.66 Project benefits Avoided asset losses (US$, millions) 4.26 78.58 Productivity gains in agriculture (US$, millions) 45.55 91.11 Productivity gains in other weather-sensitive sectors (US$, millions) 16.08 160.76 Total benefits (US$, millions) 65.89 330.45 Economic efficiency ENPV at 9% discount (US$, millions) 42.20 306.80 EIRR (%) 40.40 122.20 Benefit-cost ratio 2.80 14.00 26. Overall, even with the most conservative benefits estimates, the project is economically highly viable (ENPV of US$42.2 million with a 9 percent discount rate and EIRR of 40.4 percent). The ex-ante and ex post estimates of benefit-cost ratios are quite similar (5.0–14.6 and 2.8–14.0, respectively). 27. The project is also robust to changes in key parameters even when using the most conservative benefit estimates, as shown in table 4.6. Total project economic costs (expressed in NPV) would have to almost triple for the project to become unviable. Similarly, the productivity gains in agriculture due to the project (conservatively estimated at 0.22 percent) would have to decrease to 0.01 percent, or the discount rate used would have to be increased to 40.4 percent for the ENPV to drop to 0. Even in a situation where no economic benefits are accrued in non-agriculture sectors such as energy and tourism (perhaps the least certain of the sources of quantified economic benefits), the project is highly economically viable (EIRR of 31.35 percent and ENPV of US$26.2 million). Table 4.6. Sensitivity Analysis for the Most Conservative Project Scenario Scenario EIRR (%) ENPV (US$, millions) Base case 40.38 42.24 Economic costs up 20% 34.17 37.51 Economic benefits down 20% 32.83 29.06 No economic benefits in non-agriculture weather-sensitive sectors 31.35 26.16 Sensitivity Indicators and Switching Values Base case Sensitivity Switching Switching Indicator Value Value (% (%) (absolute) change) Total project economic costs (US$, millions) 43.81 0.56 122.05 178.56 Total project economic benefits (US$, millions) 167.74 1.56 60.22 –64.10 Productivity gains in agriculture due to the project (%) 0.22 1.08 0.02 –90.89 Discount rate used (%) 9.00 1.04 40.38 348.67 28. The economic viability of the project is likely to have been underestimated for the following reasons: (a) the current calculations do not take into account the non-quantified benefits described earlier; (b) the EM-DAT losses data for natural disasters in Nepal tend to cover only large events, with Page 61 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) smaller events not recorded;73 consequently, the estimates of avoided losses due to the project are likely to be underestimated; and (c) the current calculations do not take into account the impacts of climate change. By 2050, Nepal is likely to experience increased occurrence of floods, droughts, and other natural disasters due to increases in extreme rainfall (by 35–52 percent), increases in number of consecutive ‘dry’ days (by 3–7 percent), and increased glacial melt.74 Consequently, the avoided losses from natural disasters due to the project are also likely to increase. 29. For future hydromet investments (in Nepal and elsewhere), it is recommended that more resources are committed to recording and quantifying the actual benefits produced. The consequent reduced reliance on benefits estimates from secondary literature will improve the quality of the ex post economic analysis. 73Vladimir Tsirkunov (Lead Specialist, Hydromet Program/GFDRR, World Bank), April 24, 2021 (personal communication). 74USAID (United States Agency for International Development). 2017. Climate Risk Profile - Nepal. https://www.climatelinks.org/sites/default/files/asset/document/2017_USAID%20CCIS_Climate%20Risk%20Profile_Nepal.pdf. Page 62 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Tables from the Economic and Financial Model Table 4.7. Major Financial and Economic Analysis Parameters; Summary Results of EIRR Calculations Economic Pricing Parameter Summary Value Official exchange rate (NPR/US$, December 6, 2012) 85.90 Price basis for economic analysis 2013 constant Project life (PY); and period of operation 15 years: 2013–2027 Price level basis for economic valuation (that is, choice of numeraire) Domestic prices Estimated average domestic taxes and duties on applicable items (%) 13.0 Shadow exchange rate factor (SERF) 1.07 Implied Standard Conversion Factor (SCF) 0.93 Shadow wage rate factor 0.75 Project Viability Analysis Summary Value Project EIRR 40.38 NPV at 9% discount (US$, millions) 42.24 Benefit-cost ratio at 9% discount 2.79 Benefits scenario switch 1 1 = low benefits, 2 = high benefits Source: PAD; https://www.ecovis.com/nepal/tax-guide/;https://www.adb.org/sites/default/files/linked-documents/54107-001-ea.pdf; https://www.adb.org/sites/default/files/linked-documents/54107-001-ea.pdf. Table 4.8. Capital Cost Component Cost at Approval Actual Cost at Project (US$) Closure (US$) Component A: Institutional Strengthening, Capacity Building and Implementation Support of DHM 5,100,000 3,353,437 Component B: Modernization of the Observation Networks and Forecasting 16,450,000 13,985,857 Component C: Enhancement of the Service Delivery System of DHM 3,450,000 1,610,706 Component D: Creation of an Agriculture Management Information System (AMIS) 6,000,000 5,970,000 Counterpart financing 300,000 2,340,000 Total capital cost 31,300,000 27,260,000 Actual cost as share of cost at approval — 87.09% Page 63 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Table 4.9. Shares of Component Expenditure by Cost Category (%) Equipment & Study, Survey Training and Consultant Total Capital Component Civil Works PMU Costs Materials and Design Workshops Costs Costs Component A: Institutional Strengthening, 2 40 10 40 5 3 100 Capacity Building of DHM Component B: Modernization of the 10 75 5 2 5 3 100 Observation Networks and Forecasting Component C: Enhancement of the 10 75 5 2 5 3 100 Service Delivery System of DHM Component D: Creation of an Agriculture 12 35 20 25 5 3 100 Management Information System (AMIS) Counterpart financing 5 10 20 20 5 40 100 Table 4.10. Financial Project Costs Estimates Summary (US$, millions) Equipment & Study, Survey Training and Consultant Total Capital Component Civil Works PMU Costs Materials and Design Workshops Costs Costs Component A: Institutional Strengthening, Capacity Building of 0.07 1.34 0.34 1.34 0.17 0.10 3.35 DHM Component B: Modernization of the 1.40 10.49 0.70 0.28 0.70 0.42 13.99 Observation Networks and Forecasting Component C: Enhancement of the 0.16 1.21 0.08 0.03 0.08 0.05 1.61 Service Delivery System of DHM Component D: Creation of an Agriculture Management Information 0.72 2.09 1.19 1.49 0.30 0.18 5.97 System (AMIS) Counterpart financing 0.12 0.23 0.47 0.47 0.12 0.94 2.34 Total base costs 2.46 15.36 2.78 3.61 1.36 1.68 27.26 Of which Tradeables costs (%) 50 90 70 50 90 10 Domestic resources costs (%) 50 10 30 50 10 90 Page 64 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Table 4.11. Economic Project Costs Estimates Summary (US$, millions)a Component Equipment & Study, Survey Training and Consultant Total Capital Civil Works PMU Costs Materials and Design Workshops Costs Costs Component A: Institutional 0.07 1.41 0.34 1.31 0.18 0.09 3.39 Strengthening, Capacity Building of DHM Component B: Modernization of the 1.37 11.03 0.71 0.27 0.74 0.38 14.49 Observation Networks and Forecasting Component C: Enhancement of the 0.16 1.27 0.08 0.03 0.08 0.04 1.67 Service Delivery System of DHM Component D: Creation of an Agriculture 0.70 2.20 1.21 1.46 0.31 0.16 6.04 Management Information System (AMIS) Counterpart financing 0.11 0.25 0.47 0.46 0.12 0.85 2.26 Total base costs 2.40 16.15 2.82 3.53 1.43 1.52 27.86 Note: a. Adjustment types (financial to economic prices conversion) as follows: Tradeable goods = multiply by SERF (see Table 4.7; SERF = 1.07) 1.07 Domestic goods and services = less taxes and duties of average 13% (that is, multiply by 1/1.13) 0.88 Table 4.12. O&M Costs Component Annual O&M Annual O&M cost, Project Cost, PAD (US$) Completion Report (US$) Component A: Institutional Strengthening, Capacity Building and Implementation Support of DHM 510,000 1,440,000 Component B: Modernization of the Observation Networks and Forecasting 1,645,000 Component C: Enhancement of the Service Delivery System of DHM 345,000 Component D: Creation of an Agriculture Management Information System (AMIS) 600,000 600,000 Total annual O&M cost, financial prices (US$) 3,100,000 2,040,000 Of which Tradeables costs (%) 50 50 Domestic resources costs (%) 50 50 Components A, B, and C 1,407,568 Component D 586,487 Total annual O&M cost, economic prices (US$) 1,994,055 Page 65 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Table 4.13. Project Costs Disbursement Schedule (US$, millions) Project Year PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY8 PY9 PY10 PY11 PY12 PY13 PY14 PY15 Total Calendar Year 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 Project Capital works expenditure schedule (%) 0 22 2 4 8 24 15 21 4 0 0 0% 0 0 0 100 Annual O&M expenditure schedule (%) 0 0 0 0 0 20 40 60 80 100 100 100 100 100 100 – Financial capital cost (US$, millions) 0.00 5.94 0.67 0.96 2.11 6.55 4.18 5.65 1.19 0.00 0.00 0.00 0.00 0.00 0.00 27.26 Financial O&M cost (US$, millions) 0.00 0.00 0.00 0.00 0.00 0.41 0.82 1.22 1.63 2.04 2.04 2.04 2.04 2.04 2.04 16.32 Total financial cost (US$, millions) 0.00 5.94 0.67 0.96 2.11 6.96 5.00 6.88 2.83 2.04 2.04 2.04 2.04 2.04 2.04 43.58 Economic capital cost (US$, millions) 0.00 6.07 0.68 0.98 2.15 6.70 4.28 5.78 1.22 0.00 0.00 0.00 0.00 0.00 0.00 27.86 Economic O&M cost (US$, millions) 0.00 0.00 0.00 0.00 0.00 0.40 0.80 1.20 1.60 1.99 1.99 1.99 1.99 1.99 1.99 15.95 Total economic cost (US$, millions) 0.00 6.07 0.68 0.98 2.15 7.10 5.07 6.97 2.81 1.99 1.99 1.99 1.99 1.99 1.99 43.81 Benefits scenario switch 1 1 = low benefits, 2 = high benefits Table 4.14. Value of Avoided Asset Losses Due to Improved Hydromet Services - ‘Sector-Specific’ Approach Avoided Losses Value Source Average annual losses due to weather-related natural disasters in Nepal, 1990–2017 (US$, millions) 32.5 https://public.emdat.be/ Share of losses that can be avoided via modern EWS (%) 5 PAD report Effectiveness of EWS in Nepal prior to the project, as % of modern EWS 20 World Bank. 2021. The Value of Surface-based Meteorological Observation Data. Annual asset losses avoided due to the project, as a % of potential 80 Annual asset losses avoided due to the project (US$, millions) 1.3 Project Year PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY8 PY9 PY10 PY11 PY12 PY13 PY14 PY15 Calendar Year 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 EWS benefits stream (%) 0 0 5 10 15 20 40 60 80 100 100 100 100 100 100 Asset losses avoided due to the project 0.00 0.00 0.07 0.13 0.20 0.26 0.52 0.78 1.04 1.30 1.30 1.30 1.30 1.30 1.30 (US$, millions) Average annual benefits (US$, millions) 0.72 NPV of benefits (US$, millions, 9% discount) 4.26 Benefits scenario switch 1 1 = low benefits, 2 = high benefits Page 66 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Table 4.15. Value of Avoided Asset Losses From Improved Hydromet Services – Benchmarking Approach Avoided Losses Value Source Annual asset losses that can be avoided via modern EWS, as a % of GDP 0.017 Hallegatte, S. 2012. A Cost-Effective Solution to Effectiveness of EWS in Nepal prior to the project, as % of modern EWS 20 Reduce Disaster Losses in Developing Countries. Annual asset losses avoided due to the project, as a % of potential 80 World Bank 2021. The Value of Surface-based Meteorological Observation Data. Annual asset losses avoided due to the project, as a % of GDP 0.01 Project Year PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY8 PY9 PY10 PY11 PY12 PY13 PY14 PY15 Calendar Year 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 Nepal GDP (US$, millions, 2013 19,271 20,425 21,104 21,228 22,974 24,513 26,227 26,830 27,233 28,858 30,368 31,898 33,500 35,175 36,933 prices)a EWS benefits stream (%) 0 0 5 10 15 20 40 60 80 100 100 100 100 100 100 Asset losses avoided due to the 0.00 0.00 0.14 0.29 0.47 0.67 1.43 2.19 2.96 3.92 4.13 4.34 4.56 4.78 5.02 project (US$, millions) Average annual benefits (US$, 2.33 millions) NPV of benefits (US$, millions, 13.36 9% discount) Source: https://data.worldbank.org/indicator/NY.GDP.MKTP.KD.ZG?locations=NP; https://www.adb.org/countries/nepal/economy; https://www.statista.com/statistics/425729/gross-domestic-product-gdp-growth-rate-in-nepal/. Note: a. 2013–2019 figures were sourced from the World Bank; 2020 and 2021 projections were sourced from the ADB; 2022–2025 projections were sourced from statista.com; and for 2026–2027 projections, real GDP growth was estimated at 5 percent per year. Table 4.16. Productivity Increases from Improved Agromet Services Benefits scenario switch 1 1 = low benefits, 2 = high benefits Productivity Increases Value Source Productivity gains in agriculture from modern agromet services (%) 0.5 PAD report Share of Nepal's agricultural households impacted by the project (%) 44 AMIS Project Completion Report Productivity gains in agriculture due to the project (%) 0.22 Page 67 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Project Year PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY8 PY9 PY10 PY11 PY12 PY13 PY14 PY15 Calendar Year 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 Nepal GDP (US$, 19,271 20,425 21,104 21,228 22,974 24,513 26,227 26,830 27,233 28,858 30,368 31,898 33,500 35,175 36,933 millions, 2013 prices)a Agriculture's share 31 30 29 29 27 25 24 24 23 22 21 20 19 18 17 of GDP (%)b Agromet services 0 0 5 10 15 20 40 60 80 100 100 100 100 100 100 benefits stream (%) Productivity gains 0.00 0.00 0.67 1.34 2.01 2.72 5.48 8.50 11.03 13.96 14.02 14.02 13.98 13.90 13.78 in agriculture due to the project (US$, millions) Average annual 7.69 benefits (US$, millions) NPV of benefits 45.55 (US$, millions, 9% discount) Source: https://data.worldbank.org/indicator/NY.GDP.MKTP.KD.ZG?locations=NP; https://www.adb.org/countries/nepal/economy; https://www.statista.com/statistics/425729/gross-domestic-product-gdp-growth-rate-in-nepal/.Note: a. 2013–2019 figures were sourced from the World Bank; 2020 and 2021 projections were sourced from the ADB; 2022–2025 projections were sourced from statista.com; and for 2026–2027 projections, real GDP growth was estimated at 5 percent per year. b. Figures for 2013–2020 were sourced from https://cbs.gov.np/wp-content/upLoads/2021/01/National-Accounts-of-Nepal-2019_20.pdf; figures for 2021– 2027 were extrapolated (1 percent decline per year). Table 4.17. Productivity Increases from Improved Hydromet Services in Non-Agriculture Weather-Sensitive Sector Benefits scenario switch 1 1 = low benefits, 2 = high benefits Productivity Increases Value Source Productivity gains in weather-sensitive sectors from modern hydromet 0.1 Hallegatte, S. 2012. A Cost-Effective Solution to Reduce Disaster Losses in services (%)a Developing Countries. Effectiveness of hydromet services in Nepal prior to the project, as % of 20 World Bank. 2021. The Value of Surface-based Meteorological modern hydromet services Observation Data. Productivity gains in weather-sensitive sectors due to the project, as a % of 80 potential Page 68 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Productivity Increases Value Source Productivity gains in weather-sensitive sectors due to the project (%) 0.08 Project Year PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY8 PY9 PY10 PY11 PY12 PY13 PY14 PY15 Calendar Year 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 Nepal GDP (US$, millions, 2013 prices)b 19,271 20,425 21,104 21,228 22,974 24,513 26,227 26,830 27,233 28,858 30,368 31,898 33,500 35,175 36,933 Weather-sensitive sectors % of GDPc 23 23 23 23 23 22 22 21 21 21 20 20 20 19 19 Hydromet services benefits stream (%) 0 0 5 10 15 20 40 60 80 100 100 100 100 100 100 Productivity gains in weather-sensitive 0.00 0.00 0.20 0.39 0.63 0.87 1.85 2.74 3.65 4.77 4.94 5.11 5.28 5.46 5.64 sectors due to the project (US$, millions) Average annual benefits (US$, millions) 2.77 NPV of benefits (US$, millions, 9% discount) 16.08 Source: https://data.worldbank.org/indicator/NY.GDP.MKTP.KD.ZG?locations=NP; https://www.adb.org/countries/nepal/economy; https://www.statista.com/statistics/425729/gross-domestic-product-gdp-growth-rate-in-nepal/.Note: a. Weather-sensitive sectors include (a) electricity, gas, and water; (b) construction; (c) transport, storage, and communication; and (d) tourism. b. 2013–2019 figures were sourced from the World Bank; 2020 and 2021 projections were sourced from the ADB; 2022–2025 projections were sourced from statista.com; and for 2026–2027 projections, real GDP growth was estimated at 5 percent per year. c. Figures for 2013-2020 were sourced from https://cbs.gov.np/wp-content/upLoads/2021/01/National-Accounts-of-Nepal-2019_20.pdf; figures for 2021–2027 were extrapolated (1 percent decline per year). Page 69 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Table 4.18. Financial Internal Rate of Return Estimate (sector-specific approach for avoided asset losses) Benefits scenario switch 1 1 = low benefits, 2 = high benefits Project Year PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY8 PY9 PY10 PY11 PY12 PY13 PY14 PY15 Total Annual NPV % of (US$) Average (US$) NPV (US$) Calendar 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 Year Total project 0.00 5.94 0.67 0.96 2.11 6.96 5.00 6.88 2.83 2.04 2.04 2.04 2.04 2.04 2.04 43.58 2.91 23.42 financial costs (US$, millions) Avoided 0.00 0.00 0.07 0.13 0.20 0.26 0.52 0.78 1.04 1.30 1.30 1.30 1.30 1.30 1.30 10.79 0.72 4.26 6.5 asset losses (US$, millions) Productivity 0.00 0.00 0.67 1.34 2.01 2.72 5.48 8.50 11.03 13.96 14.02 14.02 13.98 13.90 13.78 115.41 7.69 45.55 69.1 gains in agriculture (US$, millions) Productivity 0.00 0.00 0.20 0.39 0.63 0.87 1.85 2.74 3.65 4.77 4.94 5.11 5.28 5.46 5.64 41.53 2.77 16.08 24.4 gains in other weather- sensitive sectors (US$, millions) Total project 0.00 0.00 0.93 1.85 2.84 3.85 7.85 12.02 15.72 20.03 20.26 20.43 20.57 20.66 20.72 167.74 11.18 65.89 100.0 financial benefits (US$, millions) Net project 0.00 –5.94 0.26 0.89 0.73 –3.11 2.85 5.15 12.89 17.99 18.22 18.39 18.53 18.62 18.68 124.16 8.28 42.48 financial benefits Page 70 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Project Year PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY8 PY9 PY10 PY11 PY12 PY13 PY14 PY15 Total Annual NPV % of (US$) Average (US$) NPV (US$) (US$, millions) Project Financial Internal Rate of Return (FIRR) 41.02 Net Present Value at 9% discount (US$, millions) 42.48 Benefit-cost ratio at 9% discount 2.81 Table 4.19. Financial Internal Rate of Return Estimate (benchmarking approach for avoided asset losses) Project Year PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY8 PY9 PY10 PY11 PY12 PY13 PY14 PY15 Total Annual NPV % of (US$) Average (US$) NPV (US$) Calendar Year 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 Total project 0.00 5.94 0.67 0.96 2.11 6.96 5.00 6.88 2.83 2.04 2.04 2.04 2.04 2.04 2.04 43.58 2.91 23.42 financial costs (US$, millions) Avoided asset 0.00 0.00 0.14 0.29 0.47 0.67 1.43 2.19 2.96 3.92 4.13 4.34 4.56 4.78 5.02 34.90 2.33 13.36 17.8 losses (US$, millions) Productivity 0.00 0.00 0.67 1.34 2.01 2.72 5.48 8.50 11.03 13.96 14.02 14.02 13.98 13.90 13.78 115.41 7.69 45.55 60.7 gains in agriculture (US$, millions) Productivity 0.00 0.00 0.20 0.39 0.63 0.87 1.85 2.74 3.65 4.77 4.94 5.11 5.28 5.46 5.64 41.53 2.77 16.08 21.4 gains in other weather- sensitive sectors (US$, millions) Total project 0.00 0.00 1.01 2.01 3.11 4.26 8.75 13.43 17.64 22.66 23.09 23.47 23.82 24.14 24.44 191.85 12.79 74.99 100.0 financial benefits (US$, millions) Net project 0.00 –5.94 0.34 1.05 1.00 –2.70 3.75 6.56 14.82 20.62 21.05 21.43 21.78 22.10 22.40 148.27 9.88 51.57 Page 71 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Project Year PY1 PY2 PY3 PY4 PY5 PY6 PY7 PY8 PY9 PY10 PY11 PY12 PY13 PY14 PY15 Total Annual NPV % of (US$) Average (US$) NPV (US$) financial benefits (US$, millions) Project Financial Internal Rate of Return (FIRR) 45.07 Net Present Value at 9% discount (US$, millions) 51.57 Benefit-cost ratio at 9% discount 3.20 Table 4.20. Sensitivity Analysis Using the Most Conservative Estimate of Project Benefits Scenario EIRR (%) ENPV (US$, millions) Base case 40.38 42.24 Economic costs up 20% 34.17 37.51 Economic benefits down 20% 32.83 29.06 No economic benefits in non-agriculture weather-sensitive sectors 31.35 26.16 Sensitivity Indicators and Switching Values Base Case Sensitivity Switching Value Switching Value (% Indicator (%) (absolute) change) Total project economic costs (US$, millions) 43.81 0.56 122.05 178.56 Total project economic benefits (US$, millions) 167.74 1.56 60.22 –64.10 Productivity gains in agriculture due to the project (%) 0.22 1.08 0.02 –90.89 Discount rate used (%) 9.00 1.04 40.38 348.67 Page 72 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS MOALD Comments From: Shib Nandan Pd. Shah Sent: Friday, September 24, 2021 2:19 AM To: Sulochana Nepali Cc: Vladimir Tsirkunov ; Haris Khan ; Hemang Karelia ; Yunziyi Lang ; Rupa Shrestha Subject: Re: FW: Nepal PPCR: BRCHP Implementation Completion letter and ICR Final Draft Post QER Dear Sulochanaji, Thank you for sharing the ICR final draft report PPCR_BRCH project. As per my review, the report is informative and describes and analyzes every aspect of AMIS components. However, during preparation, Ms Lisa and Lang have raised many queries and I have supplied all the clarification. Further I could not find any lacking to improve the report. Now It is OK from AMIS side. With Best Regards Shib DHM Comments From: Jagadish Karmacharya Sent: Monday, October 11, 2021 7:38 AM To: Sulochana Nepali ; Haris Khan ; Yunziyi Lang Cc: sarajubaidya@yahoo.com; Vladimir Tsirkunov ; Hemang Karelia ; Rupa Shrestha ; j_karmacharya@yahoo.com; Shiva Nepal Subject: Re: FW: Nepal PPCR: BRCHP Implementation Completion letter and ICR Final Draft Post QER Dear Sir / Madam, Thank you for sharing the ICR final draft report and sorry for the delay in providing our comments. Please find below DHM’s major comments: a) We argues that the Moderately Unsatisfactory rating of overall outcome should be upgraded to moderately satisfactory for the following reasons: i) Major of the project activities in component A,B &C such as Establishment of Weather Radar, Upper air Radiosonde, Meteorological Network, Lightning detection network, Calibration lab system, re-equipping the hydrological network and hydrological stations, Setting up Forecaster workstations, End to End early warning system, Pilot Air Quality Monitoring system, High performance computing (HPC) system and Upgrading of NWP, Database Management system, Page 73 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Refurbishment/new of DHM Building has all been successfully completed despite the major hurdles during the project implementation period such as Mega Earthquack-2015, Economic blokade-2015, COVID-19 pandemic from early 2020. ii) Moreover, the World Bank also monitored the project performance on a regular basis during the project period, and only 3 out of 14 IP ratings were categorized as moderately unsatisfactory and the rest as moderately satisfactory or higher. So the final overall rating should also reflect this and be marked as moderately satisfactory. iii) If the ranking is based on the Results Framework then this report itself acknowledges that the framework demonstrated several weaknesses as discussed in the M&E section. b) We think that four points should be added in the Key factors that affected implementation and the outcome in Section III B, namely Lack of dedicated team for project implementation, Lack of DHM’s O&M for delivery of project outputs, Limited capacity building activities, Performance of individual consultant. In addition, we have made inline comments in the attached document, Please incorporate those to the extent possible. Best regards, Jagadish Dr Jagadishwor Karmacharya | Deputy Director General | Climate Division | Department of Hydrology and Meteorology, Nepal Page 74 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) ANNEX 6. SUPPORTING DOCUMENTS • Aide Memoire - Building Resilience to Climate Related Hazards - P127508: Feb 2013, May 2013, July 2013, Nov 2014, August 2015, February 2016, April 2017, September 2017, June 2018, and January 2019. • DHM Project Website. http://brch.dhm.gov.np/ • Environmental and Social Management Framework - Building Resilience to Climate Related Hazards Project: (English). Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/en/485841468062634945/Nepal-Building-Resilience-to- Climate-Related-Hazards-Project-environmental-and-social-management-framework. • Implementation Completion and Results Report by DHM on Components A–C (February 2021). • Implementation Completion and Results Report by MOALD on Component D (November 2020). • Implementation Status and Results Report - Building Resilience to Climate Related Hazards - P127508: December 2014, June 2015, December 2015, May 2016, December 2016, June 2017, January 2018, August 2018, March 2019, December 2019, and June 2020. • Project Appraisal Document - Building Resilience to Climate Related Hazards Project - P127508 (English) Washington, D.C.: World Bank Group. (Report No. 73185). http://documents.worldbank.org/curated/en/770011468060865251/Nepal-Building-Resilience-to- Climate-Related-Hazards-Project. • Procurement Post Review Report - Building Resilience to Climate Related Hazards - P127508: Nov 2018, January 2019, April 2019, May 2020, and June 2020. • Procurement Risk Assessment and Performance Report: August 2020. • STEP Procurement Plan by DHM (Feb 2019). http://brch.dhm.gov.np/implementation-plan/ • Procurement Plan: DHM (July 2020), MOALD (Sept 2020). • Restructuring Paper (2018) - Building Resilience to Climate Related Hazards - P127508 (English). Washington, D.C.: World Bank Group. (Report No. RES33528). http://documents.worldbank.org/curated/en/535261534244614255/Disclosable-Restructuring- Paper-Building-Resilience-to-Climate-Related-Hazards-P127508. • Restructuring Paper (2019) - Building Resilience to Climate Related Hazards - P127508 (English). Washington, D.C.: World Bank Group. (Report No.: RES39784). http://documents.worldbank.org/curated/en/715571576039892924/Disclosable-Restructuring- Paper-Building-Resilience-to-Climate-Related-Hazards-P127508. • Strategic Climate Fund Grant Agreement (SCF Grant No. TF013665). • Strategic Climate Fund Loan Agreement (SCF Loan No. TF013557). • Biannual Implementation Progress Reports by the DHM System Integrator: January–July 2019, February–August 2020, and August 2020–April 2021. • Baseline User Satisfaction Survey of DHM Service Delivery System (August 2015). Page 75 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) • Endline User Satisfaction Survey of DHM Service Delivery System (December 2019). • Baseline Survey to Assess Users’ Satisfaction with Delivery of AMIS by MOALD (December 2015). • Endline Survey to Assess Users’ Satisfaction with Delivery of AMIS by MOALD (December 2019). • Final Draft MOALD PPT to PPCR (February 2020). • Final Draft DHM SI Report to PPCR (November 2020). • Draft World Bank-WMO-DHM Technical Assistance Agreement (December 2019). • Weather Forecast Verification Methodology by DHM (February 2021). • DHM O&M Budget Tables: FY19/20, FY20/21. • Draft National Framework for Climate Services in Nepal (November 2020). Page 76 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) ANNEX 7. IILUSTRATIVE PHOTOS Figure 7.1. AWSs at Jomsom Airport and Kankai75 Figure 7.2. AWSs in Daman and Bhairawaha76 Figure 7.3. AWSs for Agromet in Parwanipur, Sarlahi77 75 Implementation Completion and Results Report by DHM on Components A–C (February 2021) 76 Implementation Completion and Results Report by DHM on Components A–C (February 2021). 77 Implementation Completion and Results Report by DHM on Components A–C (February 2021). Page 77 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Figure 7.4. Hydrological Stations78 Figure 7.5. Radar and Radiosonde 79 Figure 7.6. New/Refurbished Building of the DHM Headquarter 80 78 http://brch.dhm.gov.np/gallery/ 79 http://brch.dhm.gov.np/gallery/ 80 http://brch.dhm.gov.np/gallery/ Page 78 of 79 The World Bank Building Resilience to Climate Related Hazards (P127508) Figure 7.7. Media Coverage for Flood Early Warning Figure 7.8. Hydromet Forum 2019, Nepal81 81 http://brch.dhm.gov.np/gallery/ Page 79 of 79