Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) Report Number: ICRR0024159 1. Project Data Project ID Project Name P122007 BO Nat'l Roads & Airport Infrastructure Country Practice Area(Lead) Bolivia Transport L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-49230 31-Dec-2016 80,588,721.14 Bank Approval Date Closing Date (Actual) 05-May-2011 02-Aug-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 109,500,000.00 0.00 Revised Commitment 95,340,585.42 0.00 Actual 80,588,721.14 0.00 Prepared by Reviewed by ICR Review Coordinator Group Ranga Rajan Fernando Manibog Avjeet Singh IEGSD (Unit 4) Krishnamani 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDO) as stated in the Financing Agreement (Schedule 1, page 5) and in the Project Appraisal Document (PAD, page 3) is: " To improve (i) year-round transitability of the San Buenaventura - Ixiamas national road: and (ii) safety, security and operational reliability of the Rurrenabaque Airport." Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 02-Mar-2022 c. Will a split evaluation be undertaken? Yes d. Components There were three components (PAD, page 4). 1. Improving the San Buenaventura - Ixiamas National Road. The estimated cost at appraisal was US$118.20 million. The actual cost was US$67.63 million. The actual cost was lower than the appraisal estimate due to the reduced scope. This component aimed to improve the San Buenaventura - Ixiamas National Road through: (i) an asphalt concrete pavement; (ii) rehabilitating drainage and other road protection measures: (iii) constructing 21 bridges; and (iv) providing support for audits. The scope of this component was reduced through the project restructuring (discussed below). The activity of rehabilitating the San Buenaventura - Ixiamas National Road was dropped and the target for constructing bridges was marginally increased from 21 to 22. 2. Improving the Rurrenabaque Airport. The estimated cost at appraisal was US$6.00 million. The actual cost was US$13.34 million. The actual cost of this component was higher than the appraisal estimate due to the increased project scope. Activities in this component aimed at supporting phase II of the Rurrenabaque airport improvement program through constructing: (i) a new taxiway, apron, control tower, operations building, rescue and firefighting buildings, an access road and a passenger terminal; (ii) installing aviation control rescue and firefighting equipment; and (iii) support for audits. The scope of this component was expanded with the project restructuring to include: (i) paved fire truck access, airport perimeter fencing works, platform construction taxiway and a drainage system; (ii) safety, meteorological and security equipment including medical, firefighting maintenance and security equipment, an ambulance and air navigation system; and (iii) operational support to the Airport and Aviation Services Administration (AASANA). 3. Institutional strengthening of the National Road Agency (ABC), AASANA and other relevant entities. This component was to be financed by the government. The estimated cost at appraisal was US$5.00 million. The actual cost was US$0.69 million. The actual cost of this component was significantly Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) lower than the appraisal estimate as most of these activities were not completed. There were three sub- components: a. strengthening ABC and AASANA's institutional capacity through technical assistance (TA) and training for monitoring and implementing the Good Governance and Anti-Corruption Action Plan. b. strengthening ABC's monitoring capacity through: (i) installing checkpoints to prevent illegal logging; (ii) purchasing equipment for inspections in areas under the Authority on Forest and Land (ABT); (iii) TA for studies aimed at: (1) assessing illegal logging in project areas and recommendations for diminishing it; (2) updating the policy on forest concession and implementation instruments; (iv) acquiring high resolution satellite images for tracing changes in forest cover; and (iv) conducting technical workshops for indigenous population on municipal authorities on sustainable forest management. c. strengthening the National Service for Protected Areas (SERNAP's) monitoring capacity, through: (i) providing personnel, equipment and vehicles for preventing illegal logging and poaching in the eastern border of the Madidi National Park; (ii) constructing a campground facility to serve as headquarters for park rangers; (iii) conducting workshops to strengthen coordination among stakeholders in the National Park: (iv) preparing annual operational plans for the Park and its buffer zone; and (v) TA for designing a financially sustainable mechanism for managing the park and a legal and policy framework for granting ecotourism concessions and promoting tourism. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. The estimated cost at appraisal (inclusive of price and physical contingencies) was US$129.20 million. The actual cost was US$80.58 million. The actual cost was lower than the appraisal estimate due to the significantly reduced scope of component one activities. Project financing. The project was financed by an International Development Association (IDA) credit of US$109.50 million. US$5.49 million of the credit was cancelled (discussed in section 10). The revised estimate of the credit was US$104.00 million. The amount disbursed was US$80.58 million. The ICR (page 40) noted that there were significant exchange rate changes during implementation. Borrower contribution. The borrower contribution was estimated at US$19.70 million. There was no borrower contribution during implementation. Dates. The project was approved on May 5. 2011, became effective 11 months later on April 10, 2012, and was scheduled to close on December 31, 2016. The Mid-Term Review (MTR) was held on January 16, 2017. The project closed six and half years behind schedule on August 2, 2023 for a total implementation period of 12 years. Other changes. There were six Level 2 restructurings. Through the first restructuring on December 28, 2016, the closing date was extended by 19 months from December 31, 2016 to September 30, 2018, due to a failed procurement process for the road sector activity and slow implementation of the Rurrenabaque airport works due to land acquisition issues. Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) Through the second restructuring on July 25, 2018, the closing date was extended by two months from September 30, 2018 to November 30, 2018, to enable the government to close longstanding procurement issues. Through the third project restructuring on October 2, 2018, the closing date was extended by 46 months from October 30, 2018 to August 7, 2022 for completing the airport sector activities. The main change made through the fourth project restructuring on November 12, 2019:  At government request, the Bank reallocated funds from component one to component two activities. While progress in component one activities (road activities) was limited, component two (airport activities) showed progress. The scope of the airport activities was expanded with funds partly reallocated from road sector activities. The following changes were made through the fifth project restructuring on March 2, 2022:  The activity of rehabilitating the San Buenaventura - Ixiamas National Road was dropped and the scope of component one activity was increased to constructing 22 bridges. The PDO - level results indicators on transport costs and its targets were revised to reflect that construction of bridges would still reduce transport costs and improve transitability, although not to the same magnitude as the original scope,  The implementing agency for component two activity was changed from the dissolution of AASANA to a new implementing agency - Bolivian Aviation and Airports (NNBOL). Through the sixth project restructuring on August 2, 2022, the closing date was extended by a year from August 7, 2022 to August 2, 2023 due to the delays in implementation of component one and two activities. Split rating. Although the PDO was unchanged, the scope of component one activity was reduced substantially with the fifth project restructuring. when 64% of the credit was officially disbursed. The ICR uses 50% disbursement for the split rating exercise and does not provide convincing reasons for using 50%. This review is based on a split rating of objectives, when 64% was officially disbursed before restructuring per OPCS guidelines. 3. Relevance of Objectives Rationale Country context. Bolivia is landlocked and geographically divided into three categories: lowlands, valleys and high plateau, with differences in terms of accessibility and traffic volumes. Road sector context. The transport sector, with roads being predominant, played a critical role in integrating diverse areas. The sector faced many challenges resulting from Bolivia's landlocked condition, geography, and vulnerability to climate events. Rain patterns caused rivers to swell, and together with Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) geological conditions, threatened bridges and roads. Other challenges related to weaknesses in the sector management of the national road agency (ABC). Air sector context. Air transport in Bolivia had a significant weight in domestic travel. In 2008, Bolivian airports handled 3.6 million passengers, about 78% of which were domestic. The Rurrenabaque airport played a key role in ensuring connectivity during extreme climatic events and evacuating people during emergencies. The Administration of Airport and Aviation Services (AASANA), responsible for planning and managing Bolivia' airports, was self-financed through airport taxes and the continuous underfunding of this agency adversely affected the airports operation and maintenance. Government strategy. At appraisal, the National Development Plan (PND) approved in 2006, articulated the need for reducing poverty and social exclusion through developing trade and specifically highlighted the need for integrating the national transport system through infrastructure investments. The PDO continues to be relevant to the Government strategy. The PND for 2021-25 articulated the need for promoting national and international integration through roads and identified the Rurrenabaque airport as a crucial airport for Bolivia. World Bank (WB) strategy. The PDO was well-aligned with two pillars of the Interim Strategy Note for 2010 -2011 at appraisal; Pillar I: Productive Development and Support to development of agricultural production along the San Buenaventura- Ixiamas corridor. and Pillar II: Sustainable development by promoting environmentally-friendly tourism. The PDO is well-aligned with the Country Partnership Framework (CPF) for 2023-2026, particularly its high-level objective two of improving income earnings for vulnerable households through improving connectivity. Given that poor road connectivity is a key bottleneck, the CPF recognized the importance of investing in transport infrastructure such as bridges to facilitate access to markets and services. Previous Bank experience. The Bank had previously financed the Road Rehabilitation and Maintenance Project (2002-2011). The scope of this current project was broader than the previously financed project. This project included, in addition to rehabilitating roads and bridges, investments aimed at improving the safety, security, and operational reliability of the Rurrenabaque Airport. While the previous project did not have an institutional component, this project included activities aimed at institutional strengthening of the ABC, AASANA and other related entities such as the Protected Areas National Service (SERNAP) and Forests and Land Authority (ABT). The overall level of ambition of the project was appropriate. The relevance of PDO is rated as High. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) PDO 1. To improve year-round transitability of the San Buenaventura - Ixiamas national road Rationale Theory of change. The causal links between inputs, outputs and outcomes were logical and the intended outcomes were monitorable. The outputs of activities such as improving roads and bridges on the more than 20 river crossings along the road and strengthening the capacity of ABC and other relevant entities, were likely to help in realizing the outcome of improving year-round transitability of the San Buenaventura Ixiamas national road and aid in the long-term development outcome of economic integration. The results framework assumes the following: (i) The completion of works is not disrupted by major natural events: and (ii) The implementing agency is adequately staffed. Outputs.  22 bridges were built (including 21 new bridges and replacing one defective bridge), as compared to the original target of 21.  The project partially improved the road condition through widening, adjusting the alignment area, building culverts, and paving seven kilometers (km) of roads as compared to the target of 114 km.  The project implemented a fire early warning system (EWS) to alert of any impacts that could be caused with the improvement of accessibility brought about by the bridges constructed under the project. Most of the activities aimed at institutional strengthening of the ABC and other transport entities were not implemented. Outcomes. The outputs described above were expected to contribute to: (i) improving the year-round transitability of the road; and (ii) reducing transport cost to road users, through reducing vehicle operating costs and monetary value of travel time.  There were no transit interruptions on the San Buenaventura Ixiamas national road when the project closed even in the rainy season, as compared to the baseline when the road was impassable for 75 days a year.  In the dry season before the project, travel time by cars on the national road was three hours. With project interventions, travel time reduced to two hours. In the rainy season, travel time when the road was virtually impassable before the bridges were constructed, travel time was five hours. With project interventions, travel time in the rainy season reduced to two hours.  Transport costs were reduced for all vehicle types after the project, although short of the original target. Transport cost for cars were reduced by 38% (as compared to the original target of 63%), for pick-ups by 41% (target 68%), for buses by 40% (target 66%), for trailer trucks by 19% (target 51%), for larger trucks by 23% (target 51%), for medium trucks by 23% (target 52%), for large trucks by 36% (target 39%) and for articulated trucks by 37% (target 41%).  The ICR (para 29) noted that bus companies reduced transport fares and increased the frequency of trips on the road. Beneficiaries mentioned that bus fares for a trip from San Buenaventura to Ixiamas were reduced from 50 Bolivian Bolivianos (BOB) before the project to 30 BOB after the project, while bus fares for a trip from La Paz to Ixiamas were reduced from 120 BOB to 90 BOB. Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) While the project did aid in improving the transitability of the road, transport cost reduction was far below the original target. Efficacy of this objective is therefore rated as modest before restructuring. Rating Modest OBJECTIVE 1 REVISION 1 Revised Objective PDO 1. To improve year-round transitability of the San Buenaventura - Ixiamas national road Revised Rationale The objective was not revised. However, the project scope was drastically reduced as the original targets were deemed to be over-ambitious. Outputs. As discussed above. Outcomes.  There were no transit interruptions on the road as compared to baseline when the road was impassable for 75 days a year.  Travel time by cars were reduced from three hours to two hours. In the rainy season, travel time by cars reduced from five hours to two hours.  Transport costs were reduced for all vehicle types. Transport cost for cars reduced by 45% (as compared to the revised target of 48%), for pick-ups by 38% (target 41%), for light buses by 43% (target 46%), for medium buses by 44% (target 47%), for heavy buses by 48% (target 50%), for light trucks by 38% (target 41%), for medium trucks by 35% (target 38%), for large trucks by 36% (target 39%) and for articulated trucks by 37% (target 41%).  As noted above, bus companies reduced transport fares and increased the frequency of trips on the road. The project aided in improving the transitability of the road. Transport costs were reduced for all vehicle types. Although short of the revised target, they were close to the revised targets. Efficacy of this objective is rated as substantial after restructuring. Revised Rating Substantial OBJECTIVE 2 Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) Objective PDO 2. To improve the safety, security and operational reliability of the Rurrenabaque Airport. Rationale Theory of change. The causal links between inputs, outputs and were logical. The intended outcomes were monitorable. The outputs such as a new taxiway, apron, control tower, operational and rescue and firefighting buildings, access road and passenger terminal, equipment for aviation control and rescue and firefighting equipment together with strengthening the institutional capacity of AASANA, were likely to help in improving the safety, security and operational reliability of the Rurrenabaque Airport. The results framework assumes the following: (i) The completion of works is not disrupted by major natural events: and (ii) The implementing agency maintains adequate staff. Outputs. The following activities were completed as targeted.  The taxiway, apron, control tower, operational building, platform, passenger terminals, fire truck access were constructed: A secure perimeter fencing was installed: and equipment for the airport's safe and secure operation were acquired. The activities associated with acquisition of the runway lighting system, equipment for the rescue and firefighting service and a truck to support the monitoring and maintenance of airport infrastructure were eventually dropped. The project design included activities aimed at institutional strengthening of AASANA. Most of these activities, which were to be financed by the government were not completed. Outcomes. The outputs described above were expected to increase the number of inbound and outbound flights from the Rurrenabaque Airport.  The number of inbound and outbound flights from the Rurrenabaque Airport increased from 65% at the baseline to 95% when the project closed, exceeding the target of 90%. The ICR (para 31) noted that in the past, up to 35% of the flights from the airport were cancelled due to the runway surface condition. The ICR notes that after the COVID-19 pandemic, national flights from the airport increased to 295 in 2022 and up to 530 in 2023 and the number of airport passengers increased to 4,912 in 2022 and 10,120 in 2023. The ICR (para 31) acknowledges that the outcomes were not fully attributable to project activities as improvements were made to the airport by AASANA prior to the project. However, it is reasonable to conclude that the project activities significantly contributed to improving the operational reliability and decrease in cancelled flights from the airport due to safety and security reasons. Efficacy of this objective is rated as substantial. Rating Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) Substantial OBJECTIVE 2 REVISION 1 Revised Objective PDO 2. To improve the safety, security and operational reliability of the Rurrenabaque Airport. Revised Rationale The objective and the targets were not revised. Revised Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Efficacy of PDO 1 was modest before restructuring as the transport cost reduction was far below the original targets. Efficacy of PDO 2 is rated as substantial, but marginally so since the ICR (para 31) acknowledges that the outcomes were not fully attributable to project activities as improvements were made to the airport by AASANA prior to the project. However, it is reasonable to conclude that the project activities contributed to improving the operational reliability and decrease in cancelled flights from the airport due to safety and security reasons. The overall efficacy is rated as Modest. Overall Efficacy Rating Primary Reason Modest Low achievement OBJR1_TBL OVERALL EFFICACY REVISION 1 Overall Efficacy Revision 1 Rationale Efficacy of PDO 1 is substantial after restructuring, as the revised targets were realized. Efficacy of PDO 2 is rated as substantial. The overall efficacy is rated as Substantial. Overall Efficacy Revision 1 Rating Substantial 5. Efficiency Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) Economic analysis. A cost-benefit analysis was conducted for component one activities using the Highway Design and Management Model four, version 2 (HDM -4). The activities accounted for 91% of the appraisal estimate and 82% of actual cost. The benefits were assumed to come from improved transitability due to savings in passenger and freight vehicle operating costs (VOCs) and time savings due to travel time reduction for vehicles. The Net Present Value (NPV) at 10% discount rate at closure was US$0.5 million, compared to the NPV of US$25.7 million at appraisal. The ex post Economic Internal Rate of Return (EIRR) was 13%, compared to the ex ante EIRR of 15.5%. The NPV and EIRR at closure were lower than the appraisal estimate due to factors such as increase in construction period, decrease in scope of road network and pavement extension, lower than expected traffic flows and vehicle composition change (with a higher percentage shift from larger vehicles to motorcycles). The ICR (para 37) noted that the Project's impact on Carbon dioxide emissions was minimal. Administrative and operational Issues during implementation. There were several shortcomings in the initial years due to factors such as: (i) an eleven-month delay between approval and effectiveness due to delays in signing the subsidiary agreements with the ABC and AASANA; (ii) a failed procurement process leading to cost escalation; (iii) poor contractor performance, which resulted in the contractor's abandonment of works in April 2017 and subsequent termination of the contract. The abandonment of the original road civil works contract led to sunk costs which eventually led to a significant reduction in project scope; (iv) slow implementation of airport sector activities due to land compensation issues; and (v) cost escalations related to the scaled-up works on bridges, weigh stations and tolling plazas. These factors contributed to cost and time overruns, with the scope of works under component one reduced to the construction of 22 bridges instead of completely improving the San Buenaventura- Ixiamas road and the project closing six and half years behind schedule. The activities pertaining to the institutional strengthening of ABC, AASANA and other transport entities, which were to be financed by the government, were dropped due to lack of government funding. Overall efficiency is rated as Modest mainly due to the low EIRR and NPV at closing and the administrative and operational shortcomings during implementation. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 91.00 Appraisal  15.50  Not Applicable 82.00 ICR Estimate  13.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) 6. Outcome The relevance of the PDO to the Government strategy and Bank strategy is High. Efficiency is Modest due to the operational shortcomings during implementation. Efficacy of the first objective is modest before the restructuring when the scope was significantly reduced was disbursed and substantial after restructuring. Most of the institutional strengthening activities (including the critically important activity focusing on controlling illegal logging) were cancelled. The overall outcome is rated as Moderately Unsatisfactory before restructuring and Moderately Satisfactory after restructuring. Taking the share of disbursements before and after restructuring, overall outcome is rated as Moderately Unsatisfactory. Table: Overall Outcome Rating Rating Dimension Original Scope Revised Scope Relevance of Objective High Overall efficacy Modest Substantial PDO 1 Modest Substantial PDO 2 Substantial Efficiency Modest Outcome rating Moderately Unsatisfactory Moderately Satisfactory Numerical Value of the outcome rating 3 4 Disbursement 64% 36% Weighted Value of the outcome rating 3*0.64=1.92 4*0.36 = 1.44 Final Outcome Rating 3.36 (Moderately Unsatisfactory) a. Outcome Rating Moderately Unsatisfactory 7. Risk to Development Outcome Technical Risk. The technical risk to the sustainability of project investments is Moderate. The ICR (para 82) notes that the National Roads Agency (ABC) had a well-established road conservation and maintenance program through hiring of companies for routine and sometimes periodic maintenance, and microenterprises for cleaning and other minor works, that has proven to work well. According to the clarifications provided by the team, resources for road maintenance come from the following: (i) Vias Bolivia, the national tolling entity, that provides the National Road Agency with 53% of the collected funds for primary road network conservation purposes; (ii) 15% of all hydrocarbon and derived substances direct taxes, including special gasoline and diesel oil taxes); (iii) grants, credits or loans from national government; (iv) grants, credits or loans from international financing organizations; (v) special permits and/or fines resulting from the application of the Law on Dimensions and Load (for heavy traffic); (vi) other complementary resources assigned by the national government for conservation of the primary road network; (vii) extraordinary necessary transfers; and (viii) transfers from the National Account for Road Conservation. In addition, ABC was completing a collection of data regarding the 22 bridges. Likewise for airports, the implementing agency also has a maintenance program. Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) The ICR acknowledges that climate change-induced increased flooding and landslides could threaten the sustainability of investments, particularly roads. Government commitment. There is a risk to the sustainability of investments, given that institutional strengthening activities envisioned under the project (including activities focused on controlling illegal logging) were not completed due to a lack of government funding. 8. Assessment of Bank Performance a. Quality-at-Entry The Bank prepared this project based on the experience from the Bank-financed Road Rehabilitation and Maintenance Project (2002-2011). Lessons incorporated at design included: (i) recognizing that capacity building should not be limited to the implementing agencies, in isolation from the wider context of institutions involved in the sector (discussed in section 3); and (ii) given governance weaknesses, developing a Good Governance and Anti-Corruption Plan (GAC) for helping ABC and AASANA improve governance practices. The implementation arrangements made at the appraisal were appropriate. The ABC, the agency responsible for managing the primary roads, was in charge of implementing the component one activity, and the AASANA was responsible for implementing the component two activity. The Bank team identified several risks at appraisal, including risks with human resources and safeguards and the fiduciary management capacity of the implementing agencies. The mitigation measures included capacity building and support for the executing agencies. Even with these measures, the project risk was rated as High at appraisal (PAD, para 38). The arrangements made at appraisal for safeguards and fiduciary compliance were appropriate (discussed in section 10). There were significant shortcomings at Quality-at-Entry. One, the technical aspects of the project were not ready for effectiveness. given the relatively short preparation time of this project from identification to the Board date. Draft bidding documents of component one and two activities were not updated during preparation, which led to procurement delays and extensions of the closing date, as the designs and bidding documents had to be updated during implementation. Two, there were cost overruns, especially in the road sector activities, and the financing gap led to a reduction in project scope. And three, the design included activities aimed at building and strengthening institutions. These activities were to be wholly financed by the government. The lack of ownership of the agencies led to the non-completion of these activities, and the Bank did not have the leverage to oversee the implementation of these activities. In view of these shortcomings, the Bank's performance at Quality-at-Entry is rated as Moderately Unsatisfactory. Quality-at-Entry Rating Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) Moderately Unsatisfactory b. Quality of supervision The ICR (para 78) noted that supervision missions were carried out at least twice a year and more frequently during critical phases of the project. Virtual follow-up meetings were held bi-weekly during critical phases of the project. Bank supervision team worked closely with the ABC and provided Hands-on Extended Support (HEIS) and this helped in procurement of the 22 bridges after the restructuring of 2020. The Bank also incorporated experts specialized in bridges to the team to counter the risk that the contractors might neglect quality control. This helped in completing the construction of bridges without any quality problems. The Bank also demonstrated flexibility by narrowing the road sector activity to a level that was achievable within a reasonable framework. The support provided by the team aided in safeguards compliance. The continuity of leadership was more or less maintained, with four task team leaders during the actual project implementation period of 12 years. There were moderate shortcomings in supervision. Towards the end of the project, the Bank's focus was on completing component one and two activities, while paying less attention to component three (institutional strengthening of ABC, AASANA and other entities: Moreover, better prioritization of acquisitions could have helped in enabling the implementing agency to acquire illumination for the airport's night flights. The Bank was also not able to receive timely updated values for the indicators in the Results Framework. Overall Bank performance is rated as moderately unsatisfactory, due mainly to the shortcomings at Quality- at-Entry. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Unsatisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The key outcome indicator linked to the road sector interventions measured transitability of the road through savings in vehicle operating cost and reduced travel time. This indicator did not however directly address the outcome of whether there was year-round transitability during the rainy season. Regarding the airport sector, the indicator measured the number of inbound and outbound flights operated from the Rurrenabaque Airport, as per the published schedule. While this indicator was appropriate, it was not fully attributable to the project activities, as the number of inbound and outbound flights from the airport would also depend on the interventions that had previously taken place on the airport. There were no appropriate indicators for the envisioned institutional strengthening activities. ABC and AASANA were to be responsible Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) for data collection and monitoring the results indicators from project inception until project closing, with support from a consultant financed under the project. b. M&E Implementation An appropriate indicator - that tracked year-round transitability on the road as compared to in the baseline when the road was impassable for 75 days a year during the rainy season - was introduced with the project restructuring. The ICR (para 64) noted that M&E implementation faced significant challenges, due to the rotation of both the technical and managerial staff in the implementing agencies. As there was no dedicated M&E specialist in the teams working on the project, reporting responsibilities were with Project supervision consultant. Although the consultant submitted progress reports, these reports only provided information on physical and financial progress (and not on the PDO level results indicators). c. M&E Utilization Throughout the Project lifetime, indicators in the Results Framework were not routinely updated and reported in detail. Hence, the M&E data did not become a valuable tool for monitoring. Given the shortcomings in design and implementation, there was little scope for the M&E system to be utilized for course corrections during implementation. In sum, M&E is rated as Modest, mainly due to the shortcomings in M&E design and implementation. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as a Category B (partial assessment) project under the WB safeguard policies. Six safeguard policies were triggered at appraisal: Environmental Assessment (OP/BP 4.01); Natural Habitats (OP/BP 4.04); Forests (OP/BP 4.36); Physical Cultural Resources (OP?BP 4.11); Indigenous Peoples (OP/BP 4.10); and Involuntary Resettlement (OP/BP 4.12). Environmental Assessment, Natural Habitats, Forests and Physical Cultural Resources. The potential environmental impacts of project activities were expected to be site-specific, reversible and mitigable. These impacts included; (i) deforestation and forest degradation; (ii) biodiversity loss; (iii) illegal logging; (iv) degradation of natural habitats; (iv) damage/destruction of physical cultural resources. An Environment Impact Assessment (EIA) report and an Environment Mitigation Plan (EMP) were prepared and publicly- disclosed at appraisal to address these safeguards (PAD, para 64). Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) The ICR (para 69) observed that compliance with environmental safeguards was generally satisfactory, although there were some problems during implementation. Special environmental safeguards initiatives were implemented such as socio-environmental trainings and a flora and fauna rescue program. The liabilities in the environmental liabilities restoration program were resolved by closure and there were no issues with the safeguards on Natural Habitats, Forests and Physical Cultural Resources. Indigenous Peoples (IP). ABC conducted a Social Assessment (SA) and prepared and publicly-disclosed an Indigenous Peoples Plan (IPP) at appraisal to address issues pertaining to IP (PAD, para 54). The ICR (para 67) noted that there were some delays in the implementation of the IPP, as a result of which the execution of the IPPs was extended until the end of the project. The ICR noted that the issues with indigenous peoples were resolved to date. Involuntary Resettlement. ABC prepared and publicly-disclosed a Resettlement and Compensation Plan (PRIPA) to address adverse impacts relating to land acquisition and involuntary resettlement (PAD, para 57). The ICR (para 67) notes that the compliance with social safeguards was satisfactory, although there were shortcomings during implementation. A post-closing action plan was developed to enable the ABC to complete outstanding Project social safeguards commitments. Regarding the Grievance Redress Mechanism (GRM), although there were some pending cases at project closure, they have been resolved to date. b. Fiduciary Compliance Financial Assessment (FA). The Bank conduced a FA of ABC and AASANA at appraisal. The PAD (para 50) noted that as the project design was straightforward with clearly defined activities, the project was to rely on the existing financial arrangements at ABC and AASANA. While ABC had implemented externally- financed projects, AASANA had limited experience with such projects. The overall FM risk was rated as substantial at appraisal (PAD, para 51). The ICR (para 74) noted that FM of the Project encountered numerous challenges due to the constrained capabilities of the implementing entities. According to the information provided by the team, the unqualified audits were submitted, although with delays. Procurement Assessment. The Bank conducted a procurement assessment of ABC and AASANA at appraisal. The AASANA had no experience of implementing externally-financed projects. Several mitigation measures were incorporated at design such as developing a procurement plan. The ICR (para 72) noted that the Project faced procurement problems in several instances of delays due to a combination of factors including, a failed bidding process for civil works, limited capacity of procurement staff in AASANA and frequent staff turnover. There was a case of mis procurement for the supervision contract for road sector activity. Both the original contractor's and supervision firm's contracts were investigated by the World Bank's Integrity Vice-Presidency. The outcome of the investigation was released on September 12, 2019, resulting in a 28 month disbarment of the supervision firm. The Bank undertook remedial measures, including partial cancellation of funds in the amount of US$5.49 million, partial refund and reprimand. Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) c. Unintended impacts (Positive or Negative) There were no unintended impacts. d. Other Not applicable. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment The weighted final outcome rating after the project Moderately Moderately Outcome restructuring when 64% of the Satisfactory Unsatisfactory credit was disbursed, is moderately unsatisfactory. Bank performance is rated as Moderately Moderately moderately unsatisfactory mainly Bank Performance Satisfactory Unsatisfactory due to the shortcomings at Quality-at-Entry. Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons The ICR draws the following main lessons from the experience of implementing this project, with some adaptation of language. 1. Preparation of necessary technical studies for major transport infrastructure investments at design would help in minimizing implementation delays. In this project, the road designs were not prepared during preparation. This contributed to delays following effectiveness, as final engineering designs had to be prepared. 2. Allocation of Bank financing could help in ensuring that capacity building activities are carried out in countries with implementation capacity constraints. The design of this project had several activities aimed at building institutional capacities of the implementing agencies and other relevant entities in the country context. However, Bank financing was not allocated for these activities and hence did not have leverage to ensure that these activities are implemented. Most of these activities were not completed due to lack of government funding. Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BO Nat'l Roads & Airport Infrastructure (P122007) 3. Adopting timely actions for addressing the situation can help in countries with capacity constraints. In this project, at Government request, the Bank provided Hands-on Implementation Support (HEIP) to help the government in implementing construction of bridges. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is well-written and clear. The theory of change articulated in the text clearly shows the clear causal links between the project activities (inputs), outputs and the intended outcomes. The analysis provided in the text is adequate for assessing project performance. The ICR draws reasonably good lessons from the experience of implementing this project. The photographs provided in Annex Seven helps the reader to visualize the changes made after the project in the project-intervened areas. There were shortcomings. The ICR acknowledges that 64% of the Bank financing was officially disbursed before the project restructuring when the project scope was significantly decreased. The ICR, however, uses a 50% disbursement figure before restructuring. The ICR states that this is in part due to the delay in decision- making by the GOB regarding the restructuring (para 40), which is not a convincing reason for using a 50% figure for the split rating exercise. Another minor shortcoming is the length of the ICR. The main body of the text at 26 pages is significantly more than the recommended length of 15 pages. a. Quality of ICR Rating Substantial Page 17 of 17