Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) Report Number: ICRR0024105 1. Project Data Project ID Project Name P128940 Pacific Aviation Investment - Tuvalu Country Practice Area(Lead) Tuvalu Transport L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-D1090,IDA-D2410,IDA-D9050,IDA- 31-Dec-2016 22,182,262.68 H7490,IDA-H8960,TF-12703 Bank Approval Date Closing Date (Actual) 13-Dec-2011 15-Jun-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 11,850,000.00 170,000.00 Revised Commitment 24,395,091.24 169,715.40 Actual 22,182,262.68 169,715.40 Prepared by Reviewed by ICR Review Coordinator Group Marc H. Juhel Avjeet Singh Avjeet Singh IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The original project development objective (PDO) was to improve operational safety and oversight of international air transport infrastructure. However, while the project became effective on March 20, 2012, the PDO was revised as early as September 30, 2013, as explained below. Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) The original PDO was modified in the first restructuring of 2013 to reflect investment in road and water infrastructure and changed to “to improve the overall safety and security of air transport and associated infrastructure”. The PDO will be broken down into its two sub-objectives: (i) PDO 1: improve the safety and security of air transport and (ii) PDO 2: improve associated infrastructure. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 30-Sep-2013 c. Will a split evaluation be undertaken? No d. Components Component A: Aviation Infrastructure Improvements (At approval: US$10.23 million including contingencies; At closing: US$20.03 million) This component financed the following:  Rehabilitation of Airport Pavement: Rehabilitation of the Funafuti (FUN) runway, taxiway and apron.  Terminal Building and Control Tower: Construction of a new terminal and control tower.  Air Traffic Control Equipment: Provision of ceilometers, signal lamps, radios, backup generator, along with the necessary communication and surveillance equipment, including ADS-B.  PASNet: Provision of the PASNet secure communications system.  Airfield Maintenance Equipment: Equipment for grass cutting and other basic maintenance.  Automatic Weather Station: Equipment to permit the automatic monitoring and transmitting of weather data.  Obstacle Limitation Survey: WGS84 Obstacle survey for instrument approach;  Security Improvements: Provision of X-Ray equipment and screen facilities for both hold baggage and carry-on baggage.  Laboratory/Testing Equipment: Supply of testing equipment for testing of materials during construction.  Fire Safety: Provision of a fire tender; provision of fire equipment for firemen (jackets, oxygen, etc.).  Consulting Services: The aviation Design and Supervision Consultant. Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) Component B: Aviation Sector Reform and Training (At approval: US$0.62 million including contingencies; At closing: US$0.70 million) This component financed technical assistance to the Ministry of Transport and Communications (MTC), including: (i) support to the MTC to strengthen its capabilities with aviation sector management, policy, safety and security oversight; (ii) a training program, including seconding of staff for industry experience; (iii) a baseline audit of the project airport safety and security and review of progress in the implementation of the ICAO Corrective Action Plan by the Recipient; and, (iv) ongoing safety and security oversight by PASO (financed by the GoA through PRIF). Component C: Strengthening Airport Operations and Management Capacity (At approval: US$0.22 million including contingencies; At closing: US$0 million) This component was to finance the establishment of appropriate operating structures through a study to identify options for creating separate operating company for operating Funafuti airport. However, after considering options it was found that a restructuring of the existing positions and lines of reporting within government would achieve the goal of separating management from regulation of the airport. This component was therefore cancelled during the 2013 Project Restructuring. Component D: Project Support (At approval: US$0.95 million including contingencies: At closing: US$1.87 million) This component financed the support required by various parties involved in the Program:  Incremental Operating Costs: The incremental operating costs incurred by the Government of Tuvalu (GoTv) for the project;  Support to the Technical and Fiduciary Services Unit (TFSU): Grant resources will be used to support the hiring of key technical specialists and consultants in the TFSU to support Tuvalu, including operating costs such as office space and equipment;  Project Support Team (PST): The hiring of staff for the PST;  Project Financial Audit: The cost of financial audits; and,  PASNet Annual Subscription: The cost of annual subscriptions for the operation of the PASNet infrastructure. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. The actual total project cost was US$22,182,263. The project had nine restructurings and four additional financings (AFs) in its 11-year implementation period with a total approved funding for the project at completion of US$22.77 million. The first restructuring in September 2013 changed the PDO and associated indicators to reflect the additional infrastructure activities under the first AF. With it, the first AF (IDA Grant H8960-TV) (AF1) of US$6.06 million scaled up the project to support the resurfacing of access roads to FUN and the building of an 800,000-litre water cistern under the new terminal. The need to resurface the roads was recognized at the time of appraisal; however, funding limitations meant that was not included in the original project and so was addressed in the 2013 AF. Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) The second level 2 restructuring in March 2016 revised the Results Framework and monitoring indicators and changed disbursement estimates and components and cost as well as the implementation schedule, extending the project date by 18 months from December 31, 2016, to June 30, 2018. The associated second AF (IDA Grant D1090-TV) (AF2) of US$2.88 million was to address a funding gap under the project and extend the closing dates for all funding sources (IDA Grants H7490, H8960, and PRIF TF1270). The third level 2 restructuring in October 2017 revised the Results Framework and monitoring indicators to reflect the revised closing dates and include additional indicators related to Gender-Based Violence (GBV) and Violence Against Children (VAC) as well as the runway repairs and resiliency works; extended the closing date of IDA grants (H7490, H8960, and D1090 and PRIF TF1270) to cover the defect liability period for civil works; supported Tuvalu’s admission to ICAO by making ICAO membership fees an eligible expenditure; retroactively made IDA grants H7490, H8960, and D1090 inclusive of taxes to enable pay as you earn (PAYE) and international travel-related taxes to be financed; and changed the disbursement estimates and the components and costs. The objective of the third AF (IDA Grant D2410-TV) (AF3) of US$8.75 million—US$2.90 million equivalent national IDA grant and US$5.85 million equivalent regional IDA grant in October 2017—was to address runway defects and finance (a) associated supervision cost of remediation works; (b) Tuvalu’s participation in the ICAO; (c) activities related to GBV and VAC training, prevention, and support to victims; and (d) OHS and ESMP monitoring tools and trainings for civil works contractors. The fourth level 2 restructuring in May 2018 adapted the Results Framework to allow for comprehensive monitoring of citizen engagement in the project. The fifth level 2 restructuring in April 2020 extended the project closing date from June 30, 2020, to December 15, 2020, and changed the implementation arrangements with only three remaining PAIP projects; the TFSU was closed and the three including Tuvalu adopted new arrangements for the services with transferring all TFSU responsibilities to the TvAIP Project Management Unit (PMU) while engaging individual consultants to provide technical and fiduciary support. The sixth restructuring in September 2020 and seventh restructuring of December 2022 extended the loan closing date. The eighth restructuring changed the Results Frameworks and components and cost. The associated fourth AF (IDA GRANT D905-TV) (AF4) of US$6 million in September 2021 was to cover a cost overrun and to ensure that the project meets its PDO. The final restructuring in April 2023 changed components and cost and reallocated between disbursement categories SDR 8.3 million (US$11.12 million equivalent) of the project financing that was cancelled for rehabilitation of the FUN runway under Component A and recommitted them to a new project, Tuvalu Safe and Resilient Airport Project (TuSRAP), P180674, approved in May 2023. Financing. The World Bank eventually financed the whole project, with IDA credits (US$22,012,548) and Trust Funds grants (US$169,715). The anticipated Borrower contribution (US$4,010,000) did not materialize. Dates. The project was approved on December 13, 2011, and became effective on March 20, 2012. The Mid-Term Review took place on October 16, 2014. The original closing date was December 31, 2016, and the actual closing date was June 15, 2023. Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) The second PDO indicator was changed in the first Implementation Status and Results Report (ISR) in March 2012 from ‘State requirements for safety and security to reach global International Civil Aviation Organization (ICAO) average’ defined in the PAD as institutional oversight functions being performed to ‘Resolution of safety concerns at participating airports reaches global ICAO average’ defined as extent (in percentage) of lack of effective implementation of ICAO Standards and Recommended Practices (SARPs). The second PDO indicator target was then changed in the second restructuring in 2016, from 49% to 40%, arithmetically reducing the target but actually raising the targeted outcome. The first PDO indicator ‘Regulatory certification of safety and security at project airports’ was changed in the first restructuring of 2013 to ‘ICAO certification of safety and security at project airports’ and changed again in the fourth restructuring of 2021 to ‘Certification of safety and security at project airports’ to remove ICAO from the title of the indicator ‘ICAO Audit’ based on lessons learned from other closed projects under the PAIP, which is that the timing of the official ICAO audit would not coincide with the project closing. The following additional indicators were included in the first restructuring in 2013 to cover the IDA Core Indicator requirements: (a) rural roads rehabilitated, (b) non-rural roads rehabilitated, and (c) roads in good and fair condition as a share of total classified roads. Despite the PDO being revised, the project scope was increased and hence, a split evaluation will not be considered. Moreover, during the first one and a half years the project had been effective, before the change in PDO, no considerable civil works were carried out. The US$0.89 million of IDA-H7490 and US$0.07 million of TF-12703 disbursed at the restructuring is about 2 percent of the total approved amount that was used to fund preliminary studies, setting up the project support team (PST), preparing bidding documents, and procuring consultancies. Thus, the Project will be assessed based on the revised PDO to “improve the safety and security of air transport and associated infrastructure”. 3. Relevance of Objectives Rationale Context at appraisal. Tuvalu, a Pacific Island Country (PIC) listed among fragile and conflict-affected situations due to institutional and social fragility, is among the most remote and geographically isolated countries in the world. Similar to other PICs, the country’s development priorities focus on two strategic areas: (a) mitigating economic isolation by encouraging regional and global integration and (b) building resilience against external shocks. Tuvalu is a group of nine coral atolls with a land area of 26 km2 and maximum elevation of 4.5 m. Roughly half the country’s population of 10,000 live on the main atoll, Funafuti. The main island of Fongafale on Funafuti atoll, home to the capital, is at its widest point only 650 m wide, with much of this wider section taken up by the runway for the country’s only airport. Fongafale is home to the country’s hospital, high schools, a branch campus of the University of the South Pacific, radio station, main port, and most of the businesses. The Tuvalu Aviation Investment Project (TvAIP) was the first World Bank-financed project in Tuvalu. The country joined the World Bank Group in 2010 as its smallest member. In early 2011, management identified the regional Pacific Aviation Investment Program (PAIP), which was under preparation with Kiribati and Tonga, as the best opportunity to start operations in Tuvalu. Preparation of the PAIP had started in late 2010 with the approval scheduled for late 2011. The Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) PAIP was a regional series of projects which ultimately supported six PICs and the Pacific Aviation Safety Office (PASO). At project appraisal, Tuvalu’s gross domestic product (GDP) was around US$35 million a year, the smallest of any independent state. About 40.7 percent of the workforce was employed by the Government, retail trade 5.6 percent, and construction 5.5 percent. At appraisal, about 30 Tuvaluans worked in the tourism industry. Tuvalu’s economy was found highly dependent on remittances and considered one of the most economically and environmentally vulnerable countries in the world. Relevance to Government Strategies. At the time of project preparation, TvAIP was aligned with the National Strategy for Sustainable Development Tekakeega II (2005–2015) of the Government of Tuvalu (GovT) with infrastructure development identified as one of the Government’s eight strategic development areas. The GovT’s Tuvalu Infrastructure Strategy and Investment Plan (February 2012) further outlined that infrastructure, including new domestic and international air services, was to be provided where it was economically viable, and current infrastructure should be supported by improved management, operation, and maintenance. At the project’s closing date, the project was still relevant to Tuvalu National Strategy for Sustainable Development 2021–2030 TE KETE, where the National Outcome 18: Improve Shipping, Networking and Harbor Facilities acknowledges the importance of strengthening airports as a priority physical infrastructure required in attaining outcomes under other sectors. Relevance to Bank Strategies. As a new member, Tuvalu did not yet have a Country Assistance Strategy at appraisal. However, the Project was in line with the World Bank Group’s Transport Business Strategy (2008–2012) in addressing critical infrastructure investments to meet international air safety and security requirements and instituting a sustainable financing mechanism for regulatory oversight. At project closing, it was consistent with the World Bank Regional Partnership Framework for Nine Pacific Island Countries (FY2017–FY2023), where Objective 4.2: Increased access to basic services and improved connective infrastructure, specifically highlighted that the World Bank would continue to work to improve operational safety and oversight of international air transport infrastructure. Both PDOs were pitched at an appropriate level of ambition. The first objective, "Improve the safety and security of air transport," is wholly aligned with both the Government’s and the Bank’s strategies, as it caters to a critical development condition for Tuvalu. Whereas most other PICs rely on their airports for bringing tourists to their countries, FUN is Tuvalu‘s strategic lifeline for travel for work, education, health, family connections, and tourism. The inclusion of the second PDO, "Improve associated infrastructure," in a good road network to provide access to the international airport, is an essential element of aviation safety. The ICAO’s SARPs call for roads with the ability to support the heaviest rescue, emergency, and firefighting vehicles to ensure minimum response time in case of aircraft accidents. The construction of a cistern water tank to catch floodwater and provide water to the terminal contributed to the resilience of the new passenger terminal building and enhanced the passenger experience. It also ensured water would be available to the airport fire tender in the event of an emergency. Based on the above rationale, the relevance of the PDOs is rated High. Rating Relevance TBL Rating Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective PDO 1 - Improve the safety and security of air transport Rationale At the time of project appraisal, there was no Theory of Change (ToC). Based on the Project Appraisal Document (PAD) and the Implementation Completion and Results Report (ICR), it is possible to infer that the activities to (i) rehabilitate the runway, taxiway and apron, (ii) build a new terminal and control tower, (iii) install new navigation aids, weather monitoring, safety and security equipment, (iv) conduct baseline safety and security audits of FUN and review the implementation of the Corrective Action Plan (CAP), (v) strengthen the capabilities for aviation sector management, policy, safety and security oversight, (vi) provide training, technical advisory, administrative support to MCT, other line ministries and the Technical and Fiduciary Services Unit (TFSU), and (vii) conduct a study to identify options for the sustainable operation of Funafuti airport, would have the following outputs: (i) rehabilitation of Funafuti pavement infrastructure, (ii) Funafuti terminal and control tower upgrade, (iii) navigation and safety aids fully operational, (iv) FUN airport is certified as per regulatory standards, (v) strengthened capacity of line ministries, regulatory authority and airport management, and (vi) implementation of a regional safety levy for international passengers. In terms of outcome, these activities were to result in (i) resolution of safety concerns at Funafuti airport reaches global ICAO average, (ii) modernization of air traffic management, (iii) certification of safety and security standards at Funafuti airport, and (iv) aviation sector reform. The longer-term outcome would then consist of continued flight services due to improved safety and security of air transport at Funafuti airport. The critical assumptions underpinning this ToC would be: (i) audits are done by an external agency, PASO or ICAO, (ii) adequate regulatory authority capacity to implement CAPs, (iii) global ICAO average is achievable for Tuvalu airport, and (iv) the achieved standards will be satisfactory to the airlines flying into the country. Based on this logical sequence, it is indeed plausible that the effective implementation of outputs will lead to the achievement of the desired outcomes, and potentially of the anticipated longer-term outcome further down the road. Outputs  The runway rehabilitation works at FUN were completed in 2014 but six months after completion in March 2015, isolated sections of the pavement began to blister and had vent cracks, which led to pavement heaving. This failure at the time did not significantly compromise the runway safety as it was in limited areas and was managed through regular maintenance. At the design stage, there were some locations displaying distresses consistent with cavitation under the runway, so before resealing, the contractor injected grout into these areas with varying degrees of success. In March 2017, the University of Auckland was engaged to investigate the pavement issues, funded by quality infrastructure investment through the World Bank. In May 2017, the detailed investigation concluded that there was a correlation between the water table beneath the runway and sub-surface pressure from surface water coupled with sea tidal fluctuations, which was exacerbated by heavy rain and high tides such as those in spring. Further, small voids beneath the pavement became superheated by the Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) sun, resulting in localized pressure ruptures. While the visible cavitations had been addressed by grouting, these micro-cavitations were not visible and could not be grouted. The phenomenon was considered extremely rare and has only been observed and documented in research in a few instances, most notably at Hong Kong SAR, China, airport runway, Albany City Airport in the USA, and the Barrow Island Airport in Australia. Given the nature of these defects and the underlying cause, the remedial works were not covered by the contractor’s defect liability period and outside of the detail designer’s obligations. The updated detailed design and specifications for the remedial works were completed in 2019. The new Tuvalu Safe and Resilient Airport Project (TuSRAP) will support the rehabilitation of the FUN runway.  The apron and taxiway at FUN was successfully upgraded under the project. The existing apron was resurfaced with the application of hot binder to the asphalt and laying of three coat seal of 14/7 mm double coat chip seal treatment and a sand seal locking coat covered by a fuel resistant final coat. The apron was widened with extensions on both sides of the existing pavement with 20/14/7 mm triple coat chip seal.  The supply and installation of the navigation aids package including the non-directional beacon (NDB), automatic weather observation system (AWOS), Precision Approach Path Indicator (PAPI) and the Airfield Lighting and Cabling were completed in May 2017. However, in August 2017, a fire broke out in the container which housed the navigation aids and VSAT equipment, resulting in damage to the equipment. The repurposed shipping container in which the navigation aids were stored was exposed to direct sunlight and may have been inadequately ventilated, leading to overheating of the Uninterruptible Power Supply (UPS) when the VSAT equipment was connected. The project pursued the contractor for replacement, but the contract insurance was no longer applicable.  The construction of the terminal building with climate resilience features, new ATC tower, and completion of the combined flight services center/fire tender shelter was completed. The airport management reported that the new terminal has eased passenger processing while complying with industry regulations to separate departing and arriving passengers. The wide overhanging roof is insulated to minimize solar heat gain and is designed for future installation of a photovoltaic panel array. In addition, rainfall is collected from the whole roof and stored in the new 800,000 L water cistern under the terminal providing a significant new water reserve for the atoll. Although outside of the original project's scope, security screening equipment was procured under the project with assistance from the New Zealand Pacific Security Fund, but was inoperative as at the completion mission in April 2023 owing to difficulties to fit it in the terminal original layout. The terminal will be expanded under the TuSRAP project to accommodate the screening equipment. A new flight services center housing the control tower and fire tender shelter was successfully completed in June 2018. The building not only serves to provide flight operation services but is also a shed for the fire tender truck and the airfield tractor. It also has a storage space for navigation equipment. The building also has an office space and toilet facilities for the fire crew.  The project successfully upgraded FUN fire standards to ICAO category 5; thus, FUN can provide adequate level of aircraft rescue and firefighting for the ATR 72 operations. A category 5 fire tender vehicle was successfully delivered, and the fire crew was equipped with protective safety equipment.  The project facilitated Tuvalu to be the 192nd member of ICAO in October 2017 as a first step in the application to allow ICAO to undertake a USOAP audit. Membership to ICAO will give Tuvalu access to resources and expertise to ensure that its aviation industry adheres to international safety and security standards and harmonizes its regulatory framework with regional and international partners. Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) The project financed the first-year ICAO membership fee in the third AF. Subsequent fees were financed by ICAO itself through its ‘no-country-left-behind’ policy.  Implementation of a regional safety and security levy for departing international passengers was successfully achieved. The PAIP regional safety levy collection from international passengers has successfully been introduced from July 1, 2014, establishing a sustainable financing mechanism for safety and security oversight. Outcomes  Certification of safety and security at FUN airport has not been achieved. PASO, which is a Regional Aviation Safety Oversight Organization (RSOO), recognized by ICAO in 2019, conducted an inspection of the systems at FUN to assess actions needed by the respective authorities in Tuvalu to achieve aerodrome certification. As a result of work carried out during that inspection, a Provisional Part 139 Certificate was issued to FUN (December 4, 2019). During the COVID-19 pandemic border restrictions, PASO conducted an Off-Site Certification Audit (OSCA) of FUN, in August 2021. The audit noted the documentation to meet Civil Aviation Rules Part 139 aerodrome requirements was progressing but not yet sufficient. On February 26, 2023, an on-site audit was carried out, which found that several Part 139 requirements were still not met, and the Part 139 Exposition documents did not fully reflect operations.  The resolution of safety concerns at FUN airport so that it reaches global ICAO average was partially achieved. Safety concerns identified at FUN during appraisal and set as baseline for inspection were largely rectified, but the ICAO Universal Safety Oversight Audit Programme (USOAP) audit was not conducted to determine FUN’s effective implementation score against the global average, due to travel restrictions during the pandemic and later the difficulty to mobilize an ICAO inspection team within the available time window.  Modernization of air traffic management was partially achieved. Air traffic management was modernized by successfully installing Automatic Dependent Surveillance-Broadcast (ADS-B) at FUN. VSAT equipment to connect Tuvalu to the VSAT Aviation Satellite Network for the Pacific (PASNet) was installed but was damaged in a fire incident in the navigation aids container. Consequently, at project closing, the ADS-B was inoperative and awaiting maintenance, and VSAT is to be replaced under the new World Bank-funded project TuSRAP.  Further to the implementation of the regional safety and security levy, funds are being received by the Government, and since July 2014, it has implemented (a) a PAIP Safety and Security Levy of AU$5 on international departing passengers and (b) AU$30 departure tax as part of the ticket price. These are two key actions to help ensure the financial sustainability of safety and security efforts. As of April 2023, the fund had a balance of AU$ 200,000, with an average collection of AU$ 30,000 per year. Rating Modest OBJECTIVE 2 Objective PDO 2 – Improve associated infrastructure Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) Rationale Outputs  Rural roads (8 km) and urban roads (7.4 km), which make up the total classified road network on the main island of Fongafale, totaling 15.4 km road network, which were paved last in 2002 and were reaching the end of their service life, were successfully resurfaced under the project as targeted.  Construction of storage facility under the terminal for water runoff to prevent flooding during heavy rainfall exacerbated by climate change was successfully completed. Outcomes  The newly repaved roads now ensure full access to FUN and key services on a 24-hour basis. Selected roads were constructed with geocell technology and equipped with solar streetlights.  The new water collection and storage facility provides water to service the washrooms in the new terminal, thus improving passenger experience. However, during the last supervision mission it was observed that the storage facility capacity was not sufficient for recent rainfall levels, as it would overflow. In hindsight the storage facility was somewhat under-designed because based on assumptions about the precipitations regime that proved to be too optimistic. A storm water management study is included in the TuSRAP project with the aim to mitigate the risk of flood for both the airport and the neighboring residential areas. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale On PDO 1, one PDO-level indicator was fully achieved (implementation of the new passenger levy), two partially achieved (resolution of safety concerns and modernization of air traffic management), and one not achieved (certification of safety and security standards at FUN airport). On PDO 2, the three PDO-level indicators (rehabilitation of rural and urban access roads, and construction of a water storage facility) were achieved though they were all output oriented. Considering the overarching goal of the project of improving safety and security of air transport, the overall achievements is regarded as modest. Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) Overall Efficacy Rating Primary Reason Modest Low achievement 5. Efficiency Economic Efficiency At appraisal, based on the assumptions that if air traffic services were to be interrupted by 2014 without the project, the loss of half of the travel and tourism GDP and of close to half the annual remittances from Tuvalu’s diaspora would amount to a total annual loss of US$3.7 million until full services are reinstated, the EIRR for the project would be 38.3%, with a NPV of US$4.2 million, assuming a 12 percent discount rate. Total EIRR at appraisal, including the AF and increased scope of works, was estimated as 32.9 percent with an NPV of US$28.3 million, using a discount rate of 12 percent. The economic analysis was based on the project financing aviation infrastructure FUN to ensure ICAO safety and security standards are met, and that aviation services continue without disruption. Without the project, it was expected that the aviation services may not be able to continue, and similar investment will be needed around 2016. At completion, the EIRR is estimated at 14.4% with a NPV of US$1.9 million. This is much lower than anticipated at appraisal. Administrative and Operational Efficiency The project had several aspects in its design and implementation which affected its efficiency. The project experienced several shortfalls within its activities due to complex management structure, procurement delays, unforeseen impact from natural disasters, and the COVID-19 pandemic. The costs for infrastructure investments (Component A) increased due to the runway pavement failure post-rehabilitation. This prolonged the implementation by introducing the uncertainty and increased the scope of investment. Implementation delays, unexpected issues, and increased costs substantially reduced the EIRR to a much lower level than anticipated. The project was extended by nearly 6.5 years and had a total duration of 11.5 years. Therefore, its overall efficiency is rated Moderate. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 100.00 Appraisal  38.30  Not Applicable 100.00 ICR Estimate  14.40  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) 6. Outcome The project’s relevance of objectives is rated High. The project’s efficacy and efficiency are rated Modest. Consequently, the project’s overall outcome is rated Moderately Unsatisfactory. a. Outcome Rating Moderately Unsatisfactory 7. Risk to Development Outcome Continued risk of loss of air connectivity to the island. The most recent NASA Surface Water and Ocean Topography mission collected data on sea levels around the islands of Tuvalu in March 2023 and found that, over the past three decades, the sea level rise for Tuvalu has been substantially higher than the global average. Much of Tuvalu, along with critical infrastructure, will likely be below the average high tide by 2050. Future sea-level rise will worsen the effects of tides, waves, and storms, greatly increasing the frequency and severity of periodic flooding, leaving the runway and other airport pavements vulnerable. The impact of spring tides and rainfall, exacerbated by the effects of climate change, pose a risk to continued integrity of the runway works completed under TvAIP that have ensured continued flight services into the country. Thus the possibility of airport closure and loss of air connectivity to the island remains. Institutional capacity risk. The Directorate of Civil Aviation is currently heavily reliant on a single competent individual who handles all tasks, including regulatory document generation and compliance monitoring. This concentration of responsibilities poses a vulnerability to the regulatory process, as it creates a single point of failure and limits the institutional capacity for effective oversight and enforcement. This can be mitigated through more support from PASO. 8. Assessment of Bank Performance a. Quality-at-Entry The project was strategically relevant to the Government, the Pacific Region, and fully aligned with World Bank guiding documents at the time of preparation. The project was designed as a regional IDA Adaptable Program Loan (APL) aligned with three other projects that were prepared as part of PAIP phase 1 (Kiribati, Tonga, and Samoa). Overall, the background analyses undertaken for the project were adequate. TvAIP was the first ever World Bank project in Tuvalu, and there were only seven months from the identification mission to the Board. The project’s technical, financial, and economic appraisal was thorough, with a similar approach across the PAIP projects. At the time of appraisal, however, the guidelines required that all projects under a regional IDA program have the same set of result indicators; thus, targets were not tailored to the baseline realities of Tuvalu’s circumstances, such as the gap between the intended objective targets and Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) Tuvalu’s aviation sector reality. High-risk elements for achievement of the project’s objectives, primarily related to implementation capacity and sectoral governance constraints, were correctly identified and planned to be mitigated through the establishment of the TFSU to provide procurement, FM, and technical support to the PMU strengthened by international specialists. Procurement-related risks were expected to be mitigated through the coordinated, regional approach to contract packaging and procurement handling arrangements. This approach, however, resulted in a complex management structure that likely made the resolution of the numerous issues that cropped up during implementation more difficult. The procurement risk was also underestimated, with the failure of three successive rounds of bidding for the runway rehabilitation works that failed to yield a bid corresponding to the engineer's cost estimates and acceptance of all requirements. A risk that was not identified during preparation was the climate-induced geotechnical risk, that is, the potential adverse impact of climate change on the geotechnical properties of pavement sub-surface, specifically, in this case, the tidal pressure effect. This was addressed during implementation with the inclusion of an indicator to measure the implementation of repair and resiliency works for FUN runway. Even though the project was strategically relevant, some of the risks were underestimated and there were shortcomings in M&E. Considering the specific challenges of preparing the very first World Bank project in a new member country with distinct vulnerabilities, and as part of a broader regional operation bringing in its own constraints, the quality at entry is rated Moderately Satisfactory. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision At least two formal implementation support missions per year were carried out throughout the project except during the COVID travel ban in 2020. Technical support to respond to the challenging environment of a remote island state was provided through communication with TFSU representatives assigned to TvAIP as well as proactive capacity building and training carried out by the task team through World Bank- led workshops and additional visits from procurement, safeguards, and FM colleagues as well as technical assistance from PASO. Nine restructurings, four AFs, and five project extensions were carried out during implementation to respond to the needs of the project and requests from the Government. As the first World Bank-funded project in Tuvalu, an FCV client country, the project built the client’s project management capacity. The PST benefited from the PAIP ‘PMU in a Box’ which allowed for rapid establishment of the necessary business processes to manage a World Bank-financed project. The PST worked with the PAIP TFSU to manage the project’s implementation progress. Improvements were noted in the performance of PST, especially in their ability to manage unforeseeable events and timelines. The project had seven task team leaders (TTLs) and several changes in key project teams (safeguards and FM) during its 11 years of implementation. There have also been changes within the PMU, coupled with which were periods of adjustment in which the new team leaders were briefed on the project status. While frequent changes of the seven task team leads throughout the project’s 11-year timeline has meant repeated periods of adjustment and familiarization with the project's implementation progress, leading to a temporary disruption in workflow, the TFSU maintained the institutional memory. The PAIP TFSU also provided additional capacity support on project management, procurement, and financial management Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) (FM), including expertise in the areas of planning and budgeting, engineering and equipment specification, contract management, World Bank safeguards policies, M&E, and reporting requirements. However, there is little doubt the numerous staff changes within the Bank team made an already complex situation more difficult to manage. The COVID-19 pandemic significantly affected project implementation. The Government issued a travel ban in February 2020 and all flights to Tuvalu were suspended from March 2020 onwards. This restricted the project team’s ability to travel to Tuvalu, which meant no in-country missions, pre-bid meetings, and site visits could take place nor mobilizing activities for construction. The global impact of the pandemic on supply chains also pushed up prices for materials related to runway rehabilitation works, necessitating additional financings. To sum up, the overall quality of Bank supervision is rated Moderately Satisfactory. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design TvAIP was part of the regional PAIP, leading to the harmonization of the M&E design across the program as mandated by the regional IDA financing requirement at the time of preparation. This necessitated the use of consistent indicators for all participating countries. This is useful for the regional project to achieve comparable measurements across all projects within the program, facilitating the assessment of their impact on poverty alleviation and infrastructure development. However, it did not made it possible to better tailor the project indicators to the specific conditions on the ground in Tuvalu. The PAD did not include an M&E plan with an adequate description of proposed indicators to ensure consistent understanding, and this resulted in ambiguity in the meaning of the second outcome indicator, ‘resolution of safety concerns at participating airports reaches global ICAO average’ and thus unclear meaning of target values. Moreover, indicator 2 and the later change in Indicator 1 (ICAO certification of project airport) relied heavily on an external agency for completion, ICAO in this particular case. Ideally, ICAO effective implementation scores would be updated as countries implement CAPs in response to audit findings and undergo follow-up visits by ICAO. However, the initial step in this process is beyond the project management’s control for ICAO inspectors to be available for the audit, thus delaying implementation of CAPs particularly in low-capacity situations such as that of Tuvalu and for score updates. Furthermore, the audit evaluates various aspects of airport safety and security and adherence to international standards set by the ICAO as per a set checklist, presenting an aggregate that only captures the measurable aspect of the USOAP as per aviation standards but not specifically the results attributable to the project’s work. Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) b. M&E Implementation Tuvalu PST was responsible for TvAIP’s own M&E while embedded in the regional PAIP implementation arrangements. At the time of appraisal, the guidelines required that all projects under a regional IDA program have the same set of result indicators; thus, targets were not tailored specific to baseline realities of Tuvalu’s specific circumstances. The PST worked closely with the DCA and funneled information and data to the TFSU for overall reporting in the quarterly program and project reports which were sufficiently detailed and maintained close monitoring of implementation of activities under the project. The project website tvaip.com maintained current information on the status of procurement activities and included a grievance and complaints logging system to receive grievances from the public. The TFSU worked closely with the PMU to oversee contract management. c. M&E Utilization The project underwent nine restructurings and four additional financings, leading to the revision of the Results Framework to accommodate the changes. Notably, the revised Results Framework incorporated new indicators such as the inclusion of rehabilitation of roads and runway resiliency works to align with the project’s altered objectives. However, a noteworthy observation is that the indicator target of the ICAO airport certification remained unchanged in the outcome indicators despite the project’s progress trajectory until the last two missions while the second indicator, which was dependent on ICAO USOAP audit, remained until project closure. This highlights a discrepancy in adapting the outcome indicators to reflect the revised project goals accurately. Given the shortcomings in M&E design, and failure to adequately address them during implementation, the overall M&E is rated as Modest. M&E Quality Rating Modest 10. Other Issues a. Safeguards The potential environmental impacts identified at appraisal were mostly construction-related, were temporary in nature, and were to be mitigated by implementing sound management of construction activities. In accordance with the World Bank policy for Environmental Assessment (OP/BP 4.01) , the project was assigned a category ‘B’, and an Environmental Management Plan was prepared. At closure, environmental safeguards is rated Satisfactory, with no major issues on construction-related environmental management. Following the demolition of the old terminal building, the Tuvalu National Women’s Council (TNCW), which operated a café, workshop, and retail space (gift store) in the former terminal building, was resettled to an area in the new terminal building. While consultation was undertaken with the TNCW to explain the impacts Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) of the upgrade and to agree on arrangements to relocate its activities to an alternative location, satisfactory resolution has been pending throughout the project duration. A Post-Closure Action Plan (PCAP) has been prepared to ensure measures to address the outstanding resettlement issue will be included in the newly approved TuSRAP. As a result, Social safeguards management at closure was rated Moderately Unsatisfactory. b. Fiduciary Compliance Financial Management was rated Moderately Satisfactory at project closing. Interim financial and audit reports were submitted with some minor delays but were all unqualified. The final interim financial report should cover the four-month grace period and is due to be submitted to the World Bank. The FY2022 audit report was submitted, and the auditors provided an unqualified opinion. The project audit Management Letter did not identify any compliance issues. FM supervision noted that the project had a weak control framework and lacked a robust governance arrangement that undermined risk management, especially on maintaining an adequate timesheet register. The project is still processing final documentation for advance payments made. Procurement: The procurement was rated Moderately Satisfactory at project closing. The project procurement was carried out as per guidelines but faced delays in getting contracts procured, especially with regard to the runway repair design and supervision consultancies. Procurement delays were largely caused by technical issues in relation to Tuvalu's FCV context and the COVID-19 pandemic restraining resource mobility. Tuvalu’s remote location was a major challenge in attracting contractors to carry out works such as the terminal or runway. Three successive rounds of bidding for the runway rehabilitation works failed to yield a bid corresponding to the engineer’s cost estimates and acceptance of all requirements despite inclusion of an AF in the amount of US$6 million, which was approved in September 2021. Further, there were issues with potential contractors’ inability to obtain insurance covers, which may have required using IDA grants to finance ‘self-insurance’ of the project. However, this was not pursued to avoid setting precedent for other infrastructure projects in the Pacific. c. Unintended impacts (Positive or Negative) --- d. Other 11. Ratings Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome Unsatisfactory Unsatisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons The following lessons have been derived and summarized from the ICR. Climate change preliminary assessments. There is a need to adopt an interdisciplinary approach, combining knowledge from hydrology, geotechnical engineering, climate science, and infrastructure management to conduct climate change preliminary assessments into the preparation of all prospective infrastructure projects throughout the Pacific region. This necessity arises from the region's heightened vulnerability to the exacerbating effects of climate change. Aviation sector reform efforts should take a holistic approach. A robust institutional framework, development of both regulatory documentation and human resource capacity building are needed to achieve sustainable, effective implementation of the eight critical elements of a safety oversight system guided by the requirements of compliance with the provisions of ICAO SARPs, procedures, and aviation best practices. This can be achieved in the FCV context by strengthening regional agencies and national regulatory and operational authorities. A proactive maintenance strategy of aviation assets in the Pacific is crucial. Continued monitoring of pavement conditions and assessment of vulnerability to prevailing environmental and climate conditions are required to take timely action and slow degradation to maintain the continuity of flight services into the island states, for which air connectivity is a lifeline. Selection of realistic and measurable indicator targets. Well defined indicators, realistic in the client country context and clearly attributable to the project, will enhance project M&E. Even in the case of regional projects, attention should be paid to the specific conditions of the beneficiary countries, in particular when working in an FCV context. 13. Assessment Recommended? No 14. Comments on Quality of ICR Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Pacific Aviation Investment - Tuvalu (P128940) The ICR is largely in line with the ICR guidelines. It is well-written, candid about the various hurdles the project had to overcome, and is internally consistent. It contains a good description of the project context, of its implementation process, and of its limited achievements or lack thereof. The theory of change is logically reconstituted, since it did not exist at appraisal, and the document is reasonably results-oriented. The ICR adequately reports on the reasons explaining the limited achievement of the targets and provides additional information on project impacts. The ICR provides information on the project’s fiduciary compliance and performance, but it lacks details on the timeliness and qualification of all the Project’s external audits. The lessons are important, based on evidence, and include valuable forward-looking pointers. Overall, the quality of the ICR is rated as Substnatial. a. Quality of ICR Rating Substantial Page 18 of 18