Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00006700 IMPLEMENTATION COMPLETION AND RESULTS REPORT IBRD LOAN 8562-PE ON A LOAN/CREDIT/GRANT IN THE AMOUNT OF US$50 MILLION TO THE REPUBLIC OF PERU FOR THE SUPPORT TO THE SUBNATIONAL TRANSPORT PROGRAM PROJECT JULY 31, 2024 Transport Global Practice Latin America and Caribbean Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective May 30, 2024) Currency Unit = Peruvian Nuevo Sol (PEN) PEN 3.76 = US$1 US$1 = PEN 0.266 FISCAL YEAR July 1 - June 30 Regional Vice President: Carlos Felipe Jaramillo Country Director: Issam A. Abousleiman Regional Director: Maria Marcela Silva Practice Manager: Bianca Bianchi Alves Task Team Leader(s): Eric R. Lancelot ICR Main Contributor: Cecilia Nallely Escalante Hernandez ABBREVIATIONS AND ACRONYMS CBA Cost-Benefit Analysis CPF Country Partnership Framework CPS Country Partnership Strategy DGAAM Dirección de Asuntos Ambientales (Directorate for Environmental Affairs) E&S Environmental and Social EIA Environmental Impact Assessment FONIE Fondo para la Inclusión Económica en Zonas Rurales (Fund for Economic Inclusion in Rural Areas) GDP Gross Domestic Product GoP Government of Peru GPS Global Positioning System ICR Implementation Completion and Results Report IDB Inter-American Development Bank IE Impact Evaluation IPP Indigenous Peoples Plan IRR Internal Rate of Return ISR Implementation Status and Results Report LA Loan Agreement LG Local Government M&E Monitoring and Evaluation MOP Operations Manual (Manual de Operaciones del Proyecto) MTC Ministry of Transport and Communications NPV Net Present Value PAD Project Appraisal Document PATS Programa de Apoyo al Transporte Subnacional (Subnational Transport Support Program) PCAP Post-Closure Action Plan PDEL Plan de Desarrollo Económico Local (Local Economic Development Plan) PDO Project Development Objective PIU Project Implementing Unit PVD Provías Descentralizado (Rural Roads Agency) PVPP Participatory Provincial Road Plan RAP Resettlement Action Plan RED Roads Economic Decision Model RF Results Framework RP Restructuring Paper RUC Road User Cost SSP Strategic Sector Plan VOC Vehicle Operating Cost TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ................................................................9 II. OUTCOME .................................................................................................................... 13 A. RELEVANCE OF PDOs ............................................................................................................ 13 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 14 C. EFFICIENCY ........................................................................................................................... 20 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 21 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 22 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 23 A. KEY FACTORS DURING PREPARATION ................................................................................... 23 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 23 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 25 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 25 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 27 C. BANK PERFORMANCE ........................................................................................................... 28 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 29 V. LESSONS AND RECOMMENDATIONS ............................................................................. 30 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 32 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 40 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 43 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 44 ANNEX 5. EFFICACY DETAILS ................................................................................................ 47 ANNEX 6. COMMENTS FROM PVD ....................................................................................... 52 ANNEX 7. ROADS FINANCED AND MAPS .............................................................................. 54 The World Bank Support to the Subnational Transport Program Project (P132515) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P132515 Support to the Subnational Transport Program Project Country Financing Instrument Peru Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Republic of Peru Provias Decentralizado Project Development Objective (PDO) Original PDO The Project development objectives are to (i) facilitate sustainable road access of Peru's rural populations to services, (ii) reduce transport costs on rural roads linked to priority logistics corridors; and (iii) strengthen decentralized road management. Page 1 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 50,000,000 34,273,313 34,273,313 IBRD-85620 Total 50,000,000 34,273,313 34,273,313 Non-World Bank Financing 0 0 0 Borrower/Recipient 4,790,000 0 0 Total 4,790,000 0 0 Total Project Cost 54,790,000 34,273,313 34,273,313 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 11-Dec-2015 08-Jul-2016 10-Apr-2019 31-Mar-2021 31-Dec-2023 Page 2 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 10-Mar-2021 19.48 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Safeguard Policies Triggered Change in Legal Covenants Change in Institutional Arrangements Change in Implementation Schedule 30-Jun-2021 23.85 Change in Results Framework Change in Components and Cost Reallocation between Disbursement Categories Change in Institutional Arrangements Change in Implementation Schedule 04-Apr-2022 25.25 Change in Components and Cost Change in Loan Closing Date(s) Change in Procurement Change in Implementation Schedule 24-Feb-2023 31.47 Change in Loan Closing Date(s) 24-May-2023 31.47 Change in Results Framework Change in Components and Cost Cancellation of Financing Reallocation between Disbursement Categories Change in Safeguard Policies Triggered Change in Implementation Schedule KEY RATINGS Outcome Bank Performance M&E Quality Moderately Unsatisfactory Moderately Unsatisfactory Modest RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 24-Mar-2016 Satisfactory Satisfactory 0 02 08-Oct-2016 Satisfactory Satisfactory 0 Page 3 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) 03 26-Apr-2017 Satisfactory Moderately Satisfactory 0 04 30-Oct-2017 Satisfactory Moderately Satisfactory .74 05 13-Apr-2018 Moderately Satisfactory Moderately Satisfactory .74 06 15-Oct-2018 Moderately Satisfactory Moderately Unsatisfactory 3.74 07 12-Apr-2019 Moderately Satisfactory Moderately Unsatisfactory 3.74 08 21-Oct-2019 Moderately Satisfactory Moderately Unsatisfactory 11.59 09 13-Apr-2020 Moderately Satisfactory Moderately Unsatisfactory 11.59 10 18-Nov-2020 Moderately Satisfactory Moderately Satisfactory 15.89 11 02-Jun-2021 Moderately Satisfactory Moderately Satisfactory 19.48 12 07-Dec-2021 Moderately Satisfactory Moderately Satisfactory 23.85 13 09-Aug-2022 Moderately Satisfactory Moderately Unsatisfactory 28.45 Moderately 14 01-May-2023 Moderately Unsatisfactory 31.47 Unsatisfactory 15 26-Jun-2023 Moderately Satisfactory Moderately Satisfactory 31.47 SECTORS AND THEMES Sectors Major Sector/Sector (%) Transportation 100 Public Administration - Transportation 10 Rural and Inter-Urban Roads 90 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Economic Policy 14 Trade 14 Trade Facilitation 14 Private Sector Development 100 Jobs 100 Page 4 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Public Sector Management 20 Public Administration 20 Municipal Institution Building 20 Social Development and Protection 27 Social Inclusion 27 Indigenous People and Ethnic Minorities 9 Other Excluded Groups 9 Participation and Civic Engagement 9 Urban and Rural Development 38 Rural Development 38 Rural Infrastructure and service delivery 38 ADM STAFF Role At Approval At ICR Regional Vice President: Jorge Familiar Calderon Carlos Felipe Jaramillo Country Director: Alberto Rodriguez Issam A. Abousleiman Director: Pierre A. Guislain Maria Marcela Silva Practice Manager: Aurelio Menendez Bianca Bianchi Alves Task Team Leader(s): Anca Cristina Dumitrescu Eric R. Lancelot Cecilia Nallely Escalante ICR Contributing Author: Hernandez Page 5 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. At appraisal, Peru had experienced a decade of strong economic performance—but with persistent disparities between rural and urban areas. Between 2005 and 2014, the poverty headcount fell from 55.6 to 22.7 percent, but the rural poverty rate in 2014 was more than 30 percentage points higher than in urban areas, standing at 46 percent. 2. In 2011, the Government of Peru (GoP) launched an ambitious plan to overcome social inequalities. Its main objectives were to improve access to services, reduce extreme poverty, improve environmental monitoring, and enhance the connectivity of rural areas. This agenda was supported by the Strategic Sector Plan 2012–2016 (SSP), launched by the Ministry of Transport and Communications (MTC) with the aim of establishing logistic platforms and corridors to facilitate trade and promote social inclusion. The SSP identified a US$14 billion gap in transport infrastructure, almost half of which (US$6.6 billion) was in rural road networks. 3. The Subnational Transport Support Program (Programa de Apoyo al Transporte Subnacional, PATS) was designed to support the Government’s trade and social inclusion agenda. The purpose of the PATS was to invest in rural roads that link the poorest and most remote districts in the country, and support physical, institutional, and regulatory investments to improve transport on selected roads feeding into logistics corridors. The PATS built on previous programs and introduced a stronger focus on road safety, socio-environmental compliance, and gender. Beyond the provision of reliable infrastructure, PATS was also expected to benefit the population by strengthening the technical capabilities of local governments (LGs) and road infrastructure institutions, including investment planning, while supporting the decentralization agenda. 4. The PATS, like previous rural road programs over the past 20 years, benefited from World Bank and the Inter-American Development Bank (IDB) support in its design and financing.1 This long-term collaboration allowed for the development of innovative pilots that were progressively expanded over time, such as pairing of road improvement with rural development activities, the promotion of decentralization, more in-depth citizen engagement, community-based rural road maintenance, and the advance of the micro-enterprise model, with a strong focus on gender. 5. The Project was aligned with the Country Partnership Strategy (CPS) for Peru (FY12–FY16) (Report No: 66187-PE), in particular Results Area 2.2: Better transport and infrastructure to reduce inequality, under Strategic Objective 2: Connecting the poor to services and markets. The Project was also fully aligned with the Government’s strategy to (a) extend the rural roads network, (b) expand and modernize road corridors, (c) consolidate and modernize sector institutions, and (d) provide services and promote innovation and standardization of processes through enhanced capacity building within the sector. 1 The projects supported the following: PE-Rural Roads Rehabilitation and Maintenance Project - P037047 (1995–2001); PE-Second Rural Roads Project - P044601 (2001–2006); PE-Peru Decentralized Rural Transport Project - P095570 (2007–2013); and PE-Regional Transport Infrastructure Decentralization - P078813 (2006–2014). Page 6 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Theory of Change (Results Chain) 6. The PATS, implemented by the Rural Roads Agency (Provías Descentralizado, PVD), was designed as a US$600 million investment supported by US$500 million from the GoP and two loans of US$50 million each, one from the World Bank and the other from the IDB (Program financing is detailed in Table 2). As per the Project Appraisal Document (PAD), the Results Framework (RF) originally encompassed the whole PATS Program scope, relative to the US$600 million investment, despite the World Bank contributing to only 8.3 percent of the total investment. The PATS aimed to rehabilitate/improve 2,200 km of rural roads (vías vecinales); maintain, either through periodic or routine maintenance, 5,000 km of roads; and deepen the decentralization agenda. However, by 2021 it became clear that each financier was implementing its project independently, and the World Bank carried out a restructuring in 2021 that split the World Bank investments (US$50 m) from the overall PATS. As a result, while the original Project Development Objective (PDO) was maintained, the 2021 restructuring reduced the Project scope to what the World Bank was effectively financing. Details on the restructuring are found in Section B. Figure 1: Theory of Change - PATS Note: E&S = Environmental and social; LLDW = Local and Logistics Development Window; M&E = Monitoring and evaluation. Project Development Objectives (PDOs) 7. PDO 1, facilitate sustainable road access of the Borrower’s rural populations to services; PDO 2, reduce transport costs on rural roads linked to priority logistics corridors; and PDO 3, strengthen decentralized road management. Key Expected Outcomes and Outcome Indicators 8. The Project had four indicators to measure the achievement of the PDOs: • PDO 1: (a) Share of rural population with access to an all-season road Page 7 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) • PDO 2: (a) Reduced truck transit time from production to distribution centers on feeder roads integrated to logistics corridors; and (b) Reduced vehicle operating cost for trucks • PDO 3: (a) Share of roads in good and fair condition as a share of total classified roads (core). 2 Components 9. The components described in the following paragraphs are as per the original PAD, which considered the investments of US$600 million for the PATS, also noting the World Bank contribution under each component. The final amounts presented correspond to those executed after assigning only the costs of the US$50 million Project. Details on the changes in components’ costs throughout the restructurings are presented in Table 3. 10. Component 1: Infrastructure for Integration and Social Inclusion (estimated original cost: US$405.8 million3, of which World Bank financing: US$35 million; actual cost: US$18.37 million). Road financing under this component included a strong focus on road safety, social and environmental compliance, and climate change adaptation. For both subcomponents, the loan financed technical design and detailed engineering studies and well as execution of civil works. • Subcomponent 1.1: Infrastructure for Social Inclusion (estimated original cost: US$169.7 million, of which World Bank financing: US$27.7 million; actual cost: US$18.13 million). This subcomponent aimed at improving 1,100 km of rural roads that promote accessibility and social inclusion of the poorest areas of the country. • Subcomponent 1.2: Integration of the Feeder Network into Selected Logistic Corridors (estimated original cost: US$236.1 million, of which World Bank financing: US$7.3 million; actual cost: US$0.24 million). The second subcomponent aimed at improving 1,100 km of rural roads linked to five selected logistic corridors: Chiclayo-Moyobamba-Tarapoto- Yurimaguas-Iquitos (Corridor 1); Matarani-Arequipa-Juliaca-Puno-Puente Inambari (Corridor 5); Cusco Puerto Maldonado-Iñapari (Corridor 8); Cusco-Juliaca-Puno- Desaguadero (Corridor 11); and Tarapoto-Aucayacu-Tocache-Tingo Maria (Corridor 12). 11. Component 2: Rural Road Infrastructure Maintenance (estimated original cost: US$151.9 million, of which World Bank financing: US$10 million; actual cost: US$10.12 million). This component aimed to provide support for the efficient and sustainable decentralized maintenance of the rural road network, through (a) technical design for periodic and routine maintenance contracts, (b) centralized and decentralized execution of civil works for periodic and routine maintenance of about 5,000 km of rural roads,4 and (c) supervision of related civil works. 12. Component 3: Decentralized Road Management (estimated original cost: US$26.7 million, of which World Bank financing: US$4.65 million; actual cost: US$5.78 million). The component activities included institutional strengthening of subnational level in road transport management and PVD; capacity building program for microenterprises to carry out routine maintenance; the design and implementation 2 The total original financing of the Project was US$600 million, of which the World Bank was contributing US$50 million, the IDB US$50 million, and the GoP US$500 million. The PAD, in the datasheet, shows the financing breakdown, and the components’ description also refers to the whole Program. Costs of the Project are presented in Table 3. 3 The amounts correspond to those presented in the PAD, which were rounded up. 4 According to the first Operations Manual (MOP) (2016), the Project planned to carry out routine maintenance of 2,200 km of roads rehabilitated under Component 1, 2,350 km of periodic and routine maintenance, and an additional 450 km through a pilot scheme for level of service under Component 2. Page 8 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) of a Local and Logistics Development Window; and program M&E, including implementation of an information and reporting system. 13. Component 4: Project Management (estimated original cost: US$15.7 million, of which World Bank financing: US$0.35 million; actual cost: US$0 million). B. SIGNIFICANT CHANGES DURING IMPLEMENTATION 14. The Project went through five restructurings. The first one (Restructuring Paper [RP]1), in March 2021, extended the closing date from March 2021 to April 2022. The second (RP2), in June 2021, reduced the scope to what the World Bank was effectively financing in the Project. The third (RP3), in April 2022, extended the closing date to February 2023. The fourth (RP4), in February 2023, corresponded to the last extension to the closing date to December 2023. The fifth (RP5), in May 2023, canceled part of the loan. The assessment of the Project mainly considers changes introduced under RP2 and RP5. Revised PDOs and Outcome Targets 15. The PDOs were not revised, but the outcome indicators and targets were changed, as shown in Error! Reference source not found.. Descriptions of indicators are available in Annex 5. Table 1. Changes in PDO Indicators and Outcome Targets Revised Revised Original Revised PDO Indicators Baseline Target Target Target Baseline (RP2) (RP5) PDO 1. Original: share of rural population with access to an all- 43 51.5 Replaced (see below) season road (%) PDO 1. Revised: people with access to an all-season road — — 0 30.225 24.73 financed by the project (#, thousand) PDO 2. Original: Reduced truck transit time from production to distribution centers on feeder roads integrated to logistics 0 25 Replaced (see below) corridors (%) Revised to measure PDO 16: change in the vehicle travel time — — 0 50 — on rural roads financed by the project (%, reduced) PDO 2. Original: Vehicle operating cost for trucks (US$) 0.79 0.42 Replaced (see below) PDO 2. Revised: Vehicle operating cost for trucks on rural roads — — 0.59 0.48 — with access to logistics corridors financed by the project (US$) PDO 3. Original: Roads in good and fair condition as a share of 55 60 Replaced (see below) total classified roads (%) PDO 3. Revised: percentage of local governments with a fair or — — 0 30 — good institutional capacity in road management (%) 5 RP2 presents a target of 25.3, but RP5 notes that the agreed methodology was not followed and introduces 30.22 as “original target”. 6 RP2 linked this indicator to both PDO 1 and PDO 2, but the Implementation Completion and Results Report (ICR) considers the revised indicator relevant for measuring PDO 1. Page 9 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Revised Components 16. The component descriptions were revised in RP2, as follows (changes to the Loan Agreement [LA] in bold): Component 1: Infrastructure for Integration and Social Inclusion 17. Subcomponent 1.1 (ii): centralized execution of civil works for rehabilitation and improvement selected works for this subcomponent, including road safety materials and related road users training.7 18. Subcomponent 1.2: Improving the transitability of rural roads linked to and/or feeding into logistic corridors as described in the MOP, through (i) centralized technical design and detailed engineering studies for roads rehabilitation and improvement; (ii) implementation of safeguards mitigation measures as needed for the civil works in this subcomponent.8 Component 2: Rural Road Infrastructure Maintenance 19. Provision of support for the efficient and sustainable maintenance of the rural road network, through: (i) technical design for periodic and routine maintenance contracts; (ii) centralized execution of civil works for periodic maintenance selected works9; and (iii) supervision of related civil works.10 Component 3: Decentralized Road Management 20. Provision of support for the efficient and sustainable decentralized transport management agenda, through: (i) the strengthening of the capacity at the subnational (Regions and Municipalities) in road transport management; (ii) the institutional strengthening of PVD; (iii) the development and implementation of a capacity building program for routine maintenance; (iv) the design and implementation of a Local Development Window; and (v) Program monitoring and evaluation.11 Component 4: Project Management 21. Component 4 was dropped. World Bank resources under this component were initially envisioned to cover only the financial audit12. Other Changes 22. Project scope (roads). While this was initially envisioned as a coordinated project, during implementation, the World Bank, IDB, and the GoP had different supervision strategies for civil works and therefore the World Bank had little influence over the works carried out as part of the entire program. Hence, through RP2, the scope of Project was adjusted to include only the interventions to be financed with World Bank resources. Accordingly, the target for kilometers of roads under Component 1 was adjusted from 2,200 km in the original Project to 172 km in RP2. That Component 1 target was reduced to 7 The reference to 1,100 km was dropped from the component description. The original design of the Program envisioned that only the GoP resources would be transferred to LGs, so the RP2 refocused the Project scope to only what the World Bank was financing (with the exclusion of the GoP resources). 8 The names of the corridors prioritized under Subcomponent 1.2 were dropped, the execution of designs and technical studies was to be centralized, and the Project no longer financed civil works and their supervision under this subcomponent. 9 RP2 dopped the reference to routine maintenance. The original design envisioned this activity as financed with GoP resources, given that the execution requires transferring funds to LGs. 10 The references to 5,000 km and routine maintenance activities were dropped. 11 In other words, RP2 dropped reference to microenterprises, the word ‘Logistics’ from the Local and Logistics Development Window concept, and the implementation of an information and reporting system. 12 The Mid-Term Review mentions that, because of discrepancies between procurement policies of the World Bank and the IDB, the original plan of financing 50/50 the external audits, was dropped. The MTR identified $279,851 of unassigned resources, as only the final audit was to be fully financed with WB resources. RP2 dropped this component and reallocated the resources to C1. Page 10 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) 102.57 km under RP5. The target for kilometers under maintenance (Component 2) was adjusted from 5,000 km in the original Project to 190 km in RP2. Annex 7 shows a list of roads that changed through the restructurings. 23. Methodology for prioritizing road interventions under Component 1 and Component 2. The methodologies for prioritizing interventions under Components 1 and 2 were briefly described in the PAD and detailed in the original MOP (2016). For Subcomponent 1.1, the original methodology for prioritization and selection of roads ranked them based on a composite indicator from a database of eligible roads.13 The methodology in the revised MOP (November 2018) dropped the composite indicator and used a simpler weighted criteria of complementarity, integration, and accessibility. For Subcomponent 1.2, the methodology expanded the universe of eligible roads reflected in the MOP and then confirmed under RP2 (see paragraphs 10 and 18). 24. Institutional arrangements. RP2 dropped the assignation of resources and the corresponding accountability14 to local and regional governments. As World Bank resources were not envisioned to be transferred to the LG, a transfer mechanism was not needed anymore. Also, it included the Special Interventions Directorate (Gerencia de Intervenciones Especiales, GIE) as an implementing actor, and eliminated the Steering Working Group to reduce bureaucracy.15 25. Changes in the closing date. The closing date was extended three times: from March 31, 2021 in the original Project to April 4, 2022, in RP1; then to February 28, 2023, in RP3; and finally to December 31, 2023, in RP4. 26. Project financing and component costs (reallocations and cancellation).16 The Project financing and component costs (and allocations by Disbursement Category) were adjusted as shown in Table 3. Further, the RP5 cancelled US$12.8 million of the World Bank loan (25.6 percent), and US$2.9 million was reimbursed after the closing date. See Table 2 and Table 3. 27. Changes in the RF. Changes to the RF included new indicators, redefinition and dropping of indicators, and reducing of targets in line with the Project restructurings. Details are included in Annex 5. 13 These eligibility criteria included roads that connected to the most basic services, connected population centers, provided access to markets, were 10–40 km in length, had targeted improvements with insignificant E&S impacts, had pre-investments studies, had technical specifications for cheaper pavement, and cost PEN 420,461 or less per kilometer. 14 The LA mentioned that PVD, before the provision of support to any participating province under the Project, shall enter into a financing agreement (convenios de financiamiento), setting forth the financing arrangements for road rehabilitation, improvement, periodic and routine maintenance works on a yearly basis. This was aimed as a means to formalize the decentralization process, which involved transfers of funds from PVD to LGs. World Bank and IDB funds were only to be executed at the central level. 15 The establishment of a Steering Working Group was a legal covenant. The group’s goal was to provide ‘general oversight and guidance on the strategic aspects of the Project’; however, project implementation noted there were other mechanisms in place that play that role. 16 The LA recognized each component as a different disbursement category. Page 11 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Table 2. Original Budget Approved for PATS (US$) Source: Adapted from original MOP. Table 3. Changes in Component Costs/Allocations by Disbursement Category in the Loan Original (US$, Revised Revised Revised millions) RP2 RP3 RP5 Component Actual World (June 30, (April 4, (May 24, PATS Bank 2021) 2022) 2023) Component 1: Infrastructure for Integration and 405.73 35.00 36.28 35.46 21.20 18.37 Social Inclusion Component 2: Rural Road Infrastructure 151.88 10.00 10.00 10.08 10.50 10.12 Maintenance Component 3: Decentralized Road Management 26.69 4.65 3.72 4.46 5.50 5.78 Component 4: Project Management 15.70 0.35 0.00 0.00 0.00 0.00 Total 600.00 50.00 50.00 50.00 37.20 34.27 Note: From RP2 on, the table shows only World Bank financing (the GoP financed the taxes). Final amounts are based on the Informe de Cierre (final report) provided by the PATS Project Implementing Unit (PIU) at PVD. Rationale for Changes and Their Implication on the Original Theory of Change 28. Changes in higher management at PVD caused a slowdown in road selection. In May 2017, PVD’s new management brought a new vision of the PATS and requested a revision of the road selection methodology previously agreed upon. It took until November 2018 to agree on a new methodology, which caused a two-year delay in implementation. 29. By the time of the first restructuring (2021), PVD had presented an accelerated implementation plan (2020–2022) and demonstrated a dramatic turnaround in the quality of management. The initial delays in implementation were exacerbated by COVID-19 and the rainy season; however, at the time of the first extension, the World Bank and most of the National Program interventions were already identified. An extension of the closing date was then necessary to realize the expected investments. RP1 Page 12 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) established a set of milestones17 to be achieved to evaluate further extensions and extended the closing date by 12 months. 30. The Project scope was reduced to World Bank-financed activities under RP2 because each stakeholder was moving forward rather independently.18 In addition, by 2021, it was evident that the targets for the overall program were not achievable before the closing date. Accordingly, the changes introduced by RP2 affected component descriptions and costs (reduced), disbursement estimates (reduced), results targets (reduced) and indicators (replaced), implementation arrangements, economic analysis, and safeguards arrangements. 31. A new extension was necessary to fully catch up on delays related to the implementation of resettlement plan and slow procurement processes. While PVD did not fully comply with the milestones set at RP1,19 by the time of RP3, progress was considered sufficient to grant a new extension. Two roads under Component 1 had already been completed, two were under implementation, and four roads were to start procurement in April 2022. Five more roads were selected for partial initial financing based on the assumption that there would be unused funds from roads not completed by the closing date. RP3 extended the closing date by 11 months (until February 2023) and envisioned an additional conditional 10 months extension until December 2023 provided that further progress in implementation was achieved. With the inclusion of the new roads, the E&S risk was upgraded to Substantial. 32. The GoP requested a partial cancellation of loan funds in the amount of US$12.8 million because procurement for some roads (which would be partially financed) had not yet started 10 months before the closing date. The cancellation reduced the scope of the project and affected the outcome targets and intermediate indicators (see Table 1 and annex 5), costs and allocation of funds, disbursement estimates, safeguards, and economic analysis, under RP5. 33. All these changes directly affected the achievement of PDO 2 and indirectly the sustainability of works carried out through Component 1 and Component 2, as routine maintenance was no longer part of the Project scope after RP2. Specific changes in outputs/activities are marked in the Theory of Change (figure 1). II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 34. The PDOs remain relevant to the World Bank’s Country Partnership Framework (CPF, FY23– FY27)20 as well as the Government’s priorities. The Project spanned three CPS/CPF cycles. Relevance to CPS/FY12–FY16 is addressed in paragraph 5. The PATS remained relevant to the CPF FY17–FY21 (Report No. 112299-PE) launched in 2017, under its Pillar 1, Productivity for Growth, and its Objective 1: Improve 17 The milestones were (a) six works under Component 1 under procurement or implementation, (b) 13 technical studies under Component 1 completed, (c) seven periodic maintenance works completed under Component 2, (d) 24 Local Economic Development Plans (PDELs) completed under Component 3, and (d) an impact evaluation (IE) baseline completed under Component 3. 18 The first version of the MOP established that each bank (World Bank and IDB) would finance a different portfolio of interventions, either alone or in combination with counterpart financing. The IDB loan closed in August 2023, executing US$47.6 million. 19 The RP mentioned faulty procurement processes, and PVD staff mentioned the lack of compliance by contractors, which resulted in several rounds of bidding and some subsequent defaults on contracts. 20 The CPF FY23–FY27, discussed by the Board of Executive Directors on January 31, 2023 (Report No. 179046-PE). Page 13 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) connectivity at critical corridors of the territory. For the current CPF (FY23–FY27), the PDOs contribute to CPF Objective 3: Enhance the delivery of public services” under the High-Level Outcome 2 of Improved access to quality public services. In addition, PDO 3, Strengthen decentralized road management, contributes to the objectives set under Objective 4: Strengthen institutional effectiveness at national and subnational levels. The Project was designed with the objective of facilitating sustainable road access to services through the improvement of rural road infrastructure, either to rehabilitation or maintenance. 35. The PDOs remain relevant to the Government’s priorities.21 The GoP, through PVD, continues the implementation of the PATS with its own resources. As of January 2024, PVD had completed 237 km under the road improvement/rehabilitation component and carried out periodic maintenance of 2,247 km. The Government continues to emphasize the need to reduce inequalities and territorial disparities, as established in its Strategic Plan for National Development 2050. Also, Peru’s National Infrastructure Plan for Competitiveness and Sustainability (2022–2025) emphasizes the need to enhance the logistics sector through investments in the secondary road network (departmental). The PATS continues to have a strong focus on social inclusion; however, the design of the Project also brought the concept of strengthening the logistic corridors taking into consideration the role that the tertiary network (‘vecinal’) plays in the development of value chains. In addition, discussions on a new rural roads’ operation have shown the interest in supporting the logistics corridors agenda, and lessons are drawn from the PATS. 36. With the launch of ProRegion22 (202123), PATS remained the only national program targeting the tertiary network. While the ProRegion programs (phases 1 and 2) target the secondary network, last- mile investments are still needed to ensure adequate transitability. 37. Based on the above, relevance of the PDOs is rated as High. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 38. This ICR assesses the achievement of three separate development objectives. PDO 1 is evaluated against the original and RP2 and RP5 targets. PDO 2 and PDO 3 are evaluated against the original and RP2 targets. RP2 reduced the scope of the original Project to interventions financed with World Bank resources. RP5 further reduced the scope of the Project by cancelling funds under Component 1 and reducing the PDO 1 outcome targets accordingly. 39. The split evaluation carried out in this ICR implies that: (a) Original targets are compared against the achievement of the overall PATS Program by December 2023;24 this includes accounting for achievements with resources from the GoP, 21 The GoP continues implementing the PATS through PVD. The final draft of the closing report from PVD shows a committed total amount of US$646 million, with the GoP’s contribution amounting to US$559.3 million. The PATS-PIU presented the following as accomplishments by January 2024: 237 km of rehabilitation/improvements out of 2,200 km expected and 2,247 km of periodic maintenance of the 2,800 km expected. PVD has also informed the World Bank that the technical studies developed under the PATS, some with World Bank financing, will be and are being used to continue implementation. 22 ProRegion is part of the Subnational Logistic Development Plan, which promotes regional integration and competitiveness through the intervention of the roads of the subnational road network that belong to the feeder network of each selected logistic corridor. https://www.gob.pe/36800-programa-proregion. 23 Year in which a financing agreement between the GoP, IDB, and CAF (Banco de Desarrollo de América Latina) was signed. A second phase was signed in 2023. 24 Data from December 2023 are used as it is the date for data submitted through the final progress report presented by PVD to the World Bank. Page 14 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) IDB, and World Bank. In terms of kilometers, this is 233 km rehabilitated/maintained25 and 2,111 km that received periodic maintenance; (b) Targets under the second restructuring are compared against achievements of only World Bank investments at closing date; (c) A third split is carried out for PDO 1, which changed targets for one PDO indicator under RP5. RP2 and RP5 targets, in terms of kilometers, are compared to the completed works of 57.96 km roads rehabilitated and 190 km in periodic maintenance. PDO 1: Facilitate sustainable road access of the Borrower’s rural populations to services Original PDO 1 with original targets: Modest Revised PDO 1 with RP2 targets: Modest Revised PDO 1 with RP5 targets: Substantial Table 4. Key Outcome Indicators Actuals at Target (achievement) Closure Unit Baseline Original RP2 RP5 PDO 1: Facilitate sustainable road access of the Borrower’s rural populations to services Share of rural population with access Percentage 43 51.5 (47%) — — — — 47 to an all-season road People with access to an all-season road Number, thousands 0 — — 30.22 (66%) 24.73 (81%) 20.07 financed by the project Change in the vehicle travel time on rural Percentage 0 — — 50 (99%) No change 49 roads financed by the project 40. The Project facilitated sustainable access of the rural population to services from a Modest to a Substantial extent, assessed against the original, RP2, and RP5 targets, as shown in Table 4 and discussed below. The Project improved access either through civil works for improvement/rehabilitation or through periodic maintenance. The improvements in access were also evident from the increase in the average safe speed for vehicles. 41. The original Program was expected to increase the share of population with access to an all- season road by 8.5 percentage points by intervening 5,000 km of roads.26 Rough estimations, based on the Program’s progress by the end of 2023, show that with the 2,344 km of interventions, approximately half of the original target was achieved (47 percent).27 The achievement in this case is rated Modest. 42. The Project estimates that more than 20,070 people now have access to an improved road through civil works and more through periodic maintenance. RP2 introduced an outcome indicator to 25 The contribution is 111.13 km from IDB, 63.93 km from the GoP, and 57.96 km from the World Bank. 26 The 5,000 km to be completed counted the rehabilitation and further maintenance of 2,200 km of roads and periodic maintenance of 2,800 km of roads. It is assumed that the 8.5 percentage point increase was expected to be achieved through 5,000 km of roads. 27 The rationale for this calculation: the PATS had targets of 2,200 km improved/rehabilitated and 2,800 km of periodic maintenance. Carrying out these works (5,000 km) was increasing the target by 8.5 percentage points, from 43.0 to 51.5 percent. The proportional achievement based on the completed 2,344 km by December 2023 would correspond to an increase of 3.98 percentage points (3.98 / 8.5 = 0.47), therefore an achievement of 47 percent toward the target; details are in annex 5. Page 15 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) measure the number of people with access to an all-season road. Compared with the RP2 target (30.2228), the achievement was Modest (66 percent), and against the RP5 target (24.73), the achievement was Substantial (81 percent). It is expected that a larger number of people will benefit from the Project in the coming months, achieving the target, as works on at least one additional road were advanced at closing. In a short survey29 on the three completed roads, 65 percent of those interviewed felt the Project’s investments met their needs. Also, the implementing agency reports that approximately 50,086 people are beneficiaries of roads that underwent periodic maintenance. 43. The Project reduced the travel time on improved roads, facilitating access for the population in the catchment area. This indicator was Substantially achieved, based on an increase in speed from 15 km/hour (speed estimated for rural roads at baseline) to an average speed of 29.5 km/hour,30 almost fully achieving the target of safely reducing travel time by 50 percent. Intermediate Indicators and Outputs 44. On the physical component, by the end of 2023, the Government Program achieved approximately 11 percent of the original target in terms of kilometers of rural roads, for social inclusion or connecting with logistic corridors, improved/rehabilitated, and 42 percent of the target of km of roads under the maintenance component. By December 2023, PVD had improved/rehabilitated 233 km of roads and performed periodic maintenance on 2,111 km. The Program also has approximately 1,495 km of roads with technical studies, ready to be implemented for road rehabilitation. 45. The Project rehabilitated 57.96 km of roads (34 percent of the RP2 rural road improvement targets and 57 percent of the RP5 targets).31 Of the two roads still under construction, one road is in an advanced stage and could be completed soon.32 The Project completed 100 percent of the periodic maintenance estimated under the RP2 target (190 km; same target in RP5) and reached 75 percent of the intended direct and indirect beneficiaries. Under the RP5 targets, the Project reached 96 percent of the direct/indirect beneficiaries, of which about 50 percent were women. 46. Besides works, the Project also financed technical studies which comply with both the technical (for example, road safety) and E&S requirements of the World Bank.33 A total of 47 technical studies were financed through the World Bank loan, with 15 being for road rehabilitation and 32 for periodic maintenance; some of those are being used to continue advancing the Government program. 28 See footnote 5. 29 As part of the Project, a representative survey should have been carried out; however, of the expected over 900 surveys, only 263 were applied, rendering the results to be used only as anecdotal. 30 Speed was measured at the end of the rainy season, which affected the results in at least one road. Also, PVD did not measure the improvements on speed from roads under Component 2. 31 The target set at RP5 accounted finishing four roads and 70 percent completion of one road (Puente Checca). Following the same logic, Puente Chico is at 74.9 percent progress and could be included in the actual to increase it to 76.5 km and the achievement toward the progress to 74.6 percent. 32 There is no clear indication about when Puente Checca will be completed, as PVD initiated a process to terminate the works contract due to poor performance of the contractor. On Puente Chico, a preliminary date in July 2024 was discussed, but the contract has not restarted after the rainy season. 33 Lack of compliance of the technical studies with E&S standards of the World Bank was a source of many delays. Page 16 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Table 5. Intermediate Outcome Indicators Original Target RP2 Revised RP5 Revised Original Measure Unit Baseline Target Target Actual (achievement) (achievement) (achievement) Direct Project Number, x 0 1,100 — Replaced (see below) No data Beneficiaries thousands Direct and indirect project beneficiaries Number, 0 — — 157.52 (75%) 128.78 (96%) 124.32 of roads financed by thousands the Project Of which females x (Direct Project Percentage 0 51 — Replaced (see below) No data Beneficiaries) Percentage of direct and indirect project beneficiaries of roads Percentage 0 — — 51 (98%) 49.97 (99%) 49.89 financed by the project that are female. x Roads rehabilitated. Kilometers 0 2,200 (11%) 172 (34%) 102.57 (57%) 57.96 Length of roads under x routine and periodic Kilometers 0 5,000 (42%) Replaced (see below) 2,111.66 maintenance Number of km under periodic maintenance Kilometers 0 — — 190 (100%) — — 190 financed by the project Percentage of beneficiaries who feel that project Percentage 0 — — 70 (93%) — — 65 investments reflect their needs PDO 2: Reduce transport costs on rural roads linked to priority logistics corridors Original PDO 2 with original targets: Negligible Revised PDO 2 with RP2 targets: Negligible Table 6. Key Outcome Indicators Target (achievement) Original Actuals at Unit Baseline RP2 RP5 Project Closure PDO 2: Reduce transport costs on rural roads linked to priority logistic corridors Reduced truck transit time from production to distribution centers on feeder roads integrated Percentage 0 25.0 — — — — — No data to logistics corridors Vehicle operating cost for trucks US$ 0.79 0.42 — — — — — No data Vehicle operating cost for trucks on rural roads with access to logistics corridors financed by the US$ 0.59 — — 0.48 (-65%) No change 0.84 project Page 17 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) 47. By the Project’s closing date, no roads had been financed under Subcomponent 1.2. Therefore, in a strict sense, the second part of the PDO, to reduce transport costs on rural roads linked to priority logistics corridors, was not achieved. However, two of the three roads rehabilitated under Subcomponent 1.1 are connected to logistic corridors: Talavera is connected to logistics corridor 10 and Vinchos with logistics corridor 9. 48. The Project did not reduce transport costs, when measured using vehicle operating cost (VOC), on rural roads linked to priority logistics corridors. Under RP2, PDO 2 was linked to only one indicator, vehicle operating costs for trucks on rural roads with access to logistics corridors. Information provided by PVD shows that the target of reducing VOC for trucks in Talavera and Vinchos was not achieved, with VOC ranging between US$0.84 and US$1.05 per km. Complementing with information from the short survey, most people in Vinchos34 agreed that the cost of traveling was reduced, with people mentioning cheaper fares and less fuel needed for transportation. 49. Roads rehabilitated under the Project with World Bank financing were selected under the criteria of social inclusion, and therefore, it is not evident if the roads used for this assessment supported specific value chains or had a role in the economic development around logistics corridors. Both roads had low traffic numbers (Talavera with a median daily traffic of 96 vehicles and Vinchos with 38), and trucks are not the most representative mode of transport. PVD also noted that the target VOC was too low —not adequate for the type of roads in remote rural areas that experience frequent heavy rains. PDO 3: Strengthen decentralized road management Original PDO 3 with original targets: Modest Revised PDO 3 with RP2 targets: Substantial Table 7. Key Outcome Indicators Target (achievement) Actuals at Original Closure Unit Baseline RP2 Project PDO 3: Strengthen decentralized road management Roads in good and fair condition as a share of Percentage 55 60 (46%) — — 57.3 total classified roads LG with a fair or good institutional capacity in Percentage 0 — — 30 (197%) 59 road management 50. The Project strengthened decentralized road management from a Modest to a Substantial extent. The original indicator intended to measure the share of the network in good and fair condition, based on the assumption that, as maintenance is mostly under the mandate of LGs, a stronger capacity of local actors would translate to better conservation of the network. Rough estimates, using the Government Program’s achievement by December 2023, could put this indicator at 57.3 percent.35 Most of the interventions in periodic maintenance, funded with PVD resources, were executed at the LG level, which provides evidence of strengthen capacity and decentralization of resources and roadworks. 34 The survey interviewed 69 people in the surrounding areas of Vinchos. 35 Calculations assume that the improving/maintenance of 5,000 km of roads would produce an increase in the share of roads in good/fair condition of 5 percentage points (from 55 to 60 percent). As 2,344 km of roads were improved/maintained by December 2023, the proportional increase is 2.3 percentage points, putting this indicator in 57.3 percent and achieving 46 percent of the target. Page 18 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) 51. Between 2018 and2019, the World Bank collaborated with PVD in conducting more than 80 workshops on the methodology for preparing Participatory Provincial Road Plans (PVPPs), which play a key role in the decentralization agenda. By the closing date, the World Bank had financed the development of 15 PVPP.36 The World Bank Project also supported 24 PDELs, which are being implemented with technical assistance from PVD. The Project also financed equipment37 to facilitate management of the road network in all 194 LGs.38 Table 8. Intermediate Outcome Indicators RP2 Revised RP5 Revised Original Target Original Measure Unit Baseline Target Target Actual (achievement) (achievement) (achievement) Number of Projects approved x Number 0 24 — Replaced (see below) No data under LLDWs Local Economic Development Plans (PDEL) delivered to Number 0 — — 24 (100%) — — 24 Local Governments financed by the project Percentage of PGs included in the program that have x Percentage 0 80 — Deleted No data executed 80% of the multiyear road program Number of microenterprises x certified to perform routine Number 0 200 — Deleted No data maintenance services Number of Participatory Provincial Road Plans (PVPPs) implemented under the Number 0 — — 16 (93.7%) — — 15 program and financed by the project Gender 52. The Project made progress toward mainstreaming gender in road management at subnational level, particularly by ensuring that women were involved in the development of the PVPPs. Before the second restructuring, the PATS produced39 a Road Management Gender Action Plan (GAP) for LGs with the objective of strengthening their institutional capacity to promote women’s participation throughout the life cycle of road management.40 53. RP2 dropped activities referring to microenterprises, which were the entry point for the gender agenda in previous projects, mostly because activities implemented by microenterprises were executed by LG with transfers of resources. Road Safety 36 The Government program, in total, financed 166 PVPPs—155 finalized by December 2023. 37 194 LGs were equipped with total stations, global positioning system (GPS), laptops, and motorbikes. 38 During the second semester of 2023, the PATS also conducted four large workshops that helped LGs to develop guidelines, incentives, and M&E mechanisms. PVD conducted 23 workshops on building a road condition inventory. 39 This activity was finalized before the restructuring of the Project, the World Bank reviewed it and gave its no objection to the Plan. 40 There were three regional workshops in the north (Trujillo), center (Huancayo), and south (Cusco). Page 19 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) 54. The Project supported road safety improvements in road design. The technical design of the three completed roads included informative and speed signaling, guardrails, and other measures. The World Bank closely supervised road safety improvements during missions. In addition, the technical designs of roads under construction include road safety aspects. Justification of Overall Efficacy Rating 55. The overall original efficacy is Modest, the overall efficacy based on the RP2 is Modest, and the overall efficacy based on the RP5 is Substantial. C. EFFICIENCY 56. An ex post economic analysis of the Project replicated the analysis done during preparation. The Project used cost-benefit analysis (CBA) for the economic analysis of the original Program activities focused on Components 1 and 2, which totaled 95 percent of overall investment, and 90 percent of the World Bank loan. At completion, an ex-post analysis was conducted for the roads completed, using the Roads Economic Decision Model Version 4.00 (RED), using similar assumptions to the ones at RP2 except for km completed. Data (roadworks cost and maintenance cost, traffic volume) were provided by the implementing agency reports based on the most updated information available at the time of completion. 57. The Project’s cost efficiency is considered Substantial based on the internal rate of return (IRR) of 14.4 percent and the net present value41 (NPV) (see Table 9). The changes in NPV and IRR values at completion stage could be attributed to changes in traffic volume and assumptions on vehicle operation costs. Table 9. Comparison of Appraisal/Restructuring and Completion Stage Economic Analysis IRR (%) Economic Est./Actual NPV at 9% Analysis Length Est./Actual Cost Appraisal Scope Cost per Km Traffic (AADT) (US$, Stage (km) (US$, millions) vs. (US$) in 2024 millions) completion Component Restructuring 102.57 15.9 155,200 <50 10.8 13.4 1 Completion 57.96 11.5 162,955 54 5.0 13.4 Component Restructuring 190.7 10.5 55,060 <50 1.1 17.8 2 Completion 190.7 9.0 47,356 29 2.7 17.2 Restructuring 293.27 — — — 11.9 16.1 Total Completion 248.66 — — — 7.7 14.4 (Scenario: 172.8 km completed - RP2) NPV at 9% IRR (%) Analysis Length Est. Cost (US$, Economic cost per km Scope (US$, Stage (km) millions) (US$) millions) Component 1 172.8 km 172.8 34.35 162,955 20.1 16.2 41 The World Bank invested lessthan US$6.5 million in works for the completed roads (~65 percent of the works’ cost). The total financed by the World Bank, adding technical studies and supervision, is around US$7.3 million. Page 20 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) 58. The Project disbursed 92 percent of the loan, and the implementing agency had to reimburse the World Bank US$2.93 million of unused funds.42 59. The project experienced delays since Board approval,43 the initial one referring to the lack of a portfolio of interventions. The project experienced almost a two-year delay between the development and approval of the methodology for road selection and the completion of the portfolio of interventions. The methodology, in particular, was affected by the changes in management within PVD. 60. Efficiency was negatively affected by the lack of clarity on the responsibilities of co-financiers and on the processes of approval, mainly regarding safeguard instruments. One of the key results of RP2 was that the processes for approval and no-objection were clarified, as, for example, issues with safeguards, as well as topics on procurement and financial management, were dealt with the corresponding co-financier. However, RP2 only occurred in 2021, after the Project had been going on for almost five years. The three project extensions to the closing date allowed a larger share of disbursement, but by RP2, 70 percent of the total amount had already been disbursed. Assessment of Efficiency and Rating 61. Based on the above, the efficiency is rated Modest. D. JUSTIFICATION OF OVERALL OUTCOME RATING Table 10. Original Project RP2 RP5 Relevance of PDO High Efficacy Modest Modest Substantial PDO 1 Modest Modest Substantial PDO 2 Negligible44  Negligible → PDO 3 Modest  Substantial → Efficiency Modest Moderately Moderately Moderately Satisfactory Outcome Ratings Unsatisfactory Unsatisfactory 3 3 4 Row Totals Disbursement 23.85 7.62 2.8 34.27 Share of disbursement 0.70 0.22 0.08 1.00 Weighted value of outcome 2.1 0.7 0.3 rating Final Outcome Rating 3.08 Moderately Unsatisfactory 42 As of December 2023, PVD informed that the overall national program had disbursed approximately 38 percent of the total budgeted amount but implemented only 11 percent of the rehabilitation/improvement works. However, the slow pace of disbursement and implementation was related to Component 1, as Component 2 was able to implement 80 percent of the targets estimated. 43 The project was declared effective seven months after Board approval. 44 The PATS program, by December 2023, completed some roads linked to logistics corridors. The World Bank will contribute to the achievement of the outcomes through the implementation of the technical studies that were financed through the World Bank loan. Moreover, some roads within World Bank financing are connected to logistics corridors. Page 21 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) 62. The Project objectives continue to be highly relevant to the World Bank Group’s strategies and Peru’s development plans and strategic vision. The Project achieved its objectives from a Modest to Substantial extent; however, World Bank Project activities continue to be implemented six months after the closing date, and the GoP’s program continues its implementation. The efficacy evaluation revealed that governance and the accessibility objectives were Modestly to Substantially achieved, but the objective supporting the logistics corridor agenda were negligibly achieved. The efficiency rating is also Modest. Based on the information presented above, the overall outcome rating is set as Moderately Unsatisfactory. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 63. As mainstreaming gender was part of the focus of the Project, the discussions and achievements have been included in the efficacy section. Institutional Strengthening 64. This topic has been included in the efficacy section. Mobilizing Private Sector Financing 65. The project did not mobilize private sector financing. Poverty Reduction and Shared Prosperity 66. The roadworks completed under Components 1 and 2 were in Ayacucho (3 roads), Apurimac (5 roads), and Junin (2 roads). At Project approval, Ayacucho and Apurimac had poverty levels above 37 percent.45 The Government Program intended to perform an IE to measure the impact of road rehabilitation and maintenance on poverty measures; however, baseline data collection was finalized only in 2022. Other Unintended Outcomes and Impacts 67. Data-driven solutions. The Project revealed the need for a more data-driven/objective approach to selecting interventions and a multisector approach. The World Bank prepared (2023) an infrastructure sector assessment (infraSAP) focused on regional development and logistic corridors. The findings are relevant for the development of future projects in rural areas, including new engagements in rural road projects. Also, the infraSAP tested innovative approaches for data collection (for example, use of big data for road condition assessment) and produced a wealth of data. 68. Climate resilient infrastructure. The difficulties experienced under the Project (and the Government Program) due to frequent landslides, heavy storms, floods, and flashfloods increased the interest in incorporating robust information for decision-making in the design of rural roads. The Project financed a study ‘Building resilience to climate change-induced events in the rural road network of Peru’46 and a workshop to seven different government organizations that work in Peru on disaster risk 45 Ministerio de Desarrollo e Inclusión Social. https://sdv.midis.gob.pe/RedInforma/Reporte/Reporte?id=17. 46 ‘Construyendo resiliencia a eventos asociados al cambio climático en la red de caminos vecinales del Perú’ (2019). Page 22 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) management with the objective to build an index to assess the exposure of rural roads to different climate-induced hazards characteristic of Peru.47 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 69. The Project definition (scope and outcome targets) was not well aligned with what the World Bank could effectively achieve. The Project entailed a more substantial financial contribution from the Government over a shorter period than in previous projects. The risk was recognized in the PAD, but mitigation measures were limited to the co-financier’s joint accountability for the achievements of the full set of objectives. In addition, there were foundational elements for implementation, such as the methodologies for road selection and specific road sections subject to improvement/maintenance, not yet established at preparation, that delayed the Project’s readiness and slowed its overall execution by about two years. 70. The Project was a continuation of the engagement of the World Bank in the rural roads sector and was appropriate to the context and the Government’s agenda focused on logistic corridors development. However, evidence suggests that there was low capacity to implement the logistics corridors approach in the tertiary network. The design of the Project was similar to past World Bank roads projects in the country, but the PATS introduced a focus on logistics, going beyond supporting roads with a social inclusion profile and incorporating a dimension of competitiveness by enhancing the road links between the rural production areas and logistic corridors. This innovation was made explicit by the introduction of the PDO. Nevertheless, the lack of achievement of objectives supporting the logistics agenda (PDO 2) and PVD’s lack of experience in collecting information relevant to measure the impact on logistics corridors demonstrates that stronger capacity building would have been important.48 71. Adequacy of risk and mitigation measures. The PAD identified risks under the categories of implementation and sustainability, political and governance, and technical design categories, and some of these risks materialized. The PAD identified lack of coordination among entities as a substantial risk. This risk materialized as coordination among entities responsible for safeguards turned out to be critical. The PAD failed to develop sufficient strategies to mitigate issues of staff turnover, deficiencies on contract management and supervision of scattered works in remote areas, and discrepancies in the implementation of the World Bank safeguards versus the local law. B. KEY FACTORS DURING IMPLEMENTATION Factors Subject to the Control of the Government and the Implementing Agency 72. Frequent changes in leadership and priorities had a negative impact on the implementation schedule. PVD had more than 10 Executive Directors over the course of 2021–22, and the PIU Coordinator was sometimes changed along with the Executive Director. This caused significant issues regarding coordination and management of key Project activities, such as the lack of agreement on technical 47 Centro Nacional de Estimación, Prevención y Reducción del Riesgo de Desastre (CENEPRED), Instituto Geológico, Minero y Metalúrgico (INGEMMET), Servicio Nacional de Meteorología e Hidrología (SENAMHI), the Fund for Economic Inclusion in Rural Areas (Fondo para la Inclusión Económica en Zonas Rurales, FONIE), Instituto Nacional de Defensa Civil (INDECI), PVD, and the Ministerio del Ambiente (MINAM). 48 By the end of 2023, only 50 km of roads connected to logistic corridors were completed under the program. Page 23 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) approaches or the continuity of decisions made regarding the application of safeguards. Moreover, the key government agencies also experienced high staff turnover. 73. The process to approve E&S management documents was complex and caused delays. The works-related documents were subject to several reviews from different agencies or ministries, on top of reviews by the World Bank and the MTC Directorate for Environmental Affairs (Dirección de Asuntos Ambientales, DGAAM).49 The Project acknowledge the DGAAM’s limited capacity to quickly react to the increased workload expected from the PATS. Therefore, the PATS supported the DGAAM by hiring E&S specialists; however, these staff were soon devoting time to activities outside the PATS. The allocation of human resources was therefore ineffective and led to a strain on the DGAAM’s ability to effectively handle its Project-related responsibilities. 74. Discrepancies between the PIU and the World Bank safeguards teams delayed implementation even of subprojects that were considered ready. The first works to be financed by the Project were inherited from FONIE; however, the technical documents did not comply with the World Bank safeguards. As a result, the environmental assessment had to be revised and updated, which delayed the start of implementation by about six months. Also, studies developed under the Project proved to be complex, and submissions were initially of poor quality. Because of the review process and responsibilities of each co-financier, there was a constant duplication of efforts by the World Bank team and PVD’s E&S staff. In addition, as noted above, the absence of a clear strategy to solve conflicts between the implementation of the safeguards (World Bank versus national law) was a key factor in the delays. 75. There were frequent issues with procurement, and contract management proved to be complicated. The PAD recognized the risk of advertising a large number of contracts (for both studies and works) and proposed having packages of contracts to make contract management more convenient and attractive to qualified contractors. However, interviews with stakeholders revealed the lack of a robust roster of consultants/firms able to comply with the terms of reference, causing the same firms to bid on several contracts and then defaulting or doing unsatisfactory work. Another issue was the lack of consensus on proactively resolving contractual matters, which also caused delays. In addition, contract management and record keeping were inadequately executed at PVD and the local offices in the regions. 76. The supervision of individual road interventions turned out to be complex, and contract defaulting was common. It was a challenge to supervise awarded contracts and coordination between the zone offices50 and headquarters. Some works contracts were terminated due to noncompliance of the contractors, and some left debts with local workers and suppliers, with contractors often citing cash-flow issues as the reason. For works supervision, the World Bank and PVD tested a pilot to supervise works remotely using an imagery tool in Puente Checca51 and provided two trainings on the use of this tool. However, PVD did not utilize it further. Factors Subject to World Bank Control 77. While the World Bank supported the implementing agency with continuous training, staff changes during implementation slowed the Project’s momentum and complicated the implementation process. The PATS had a total of five task team leaders, with three transitions between preparation and 49 Depending on the nature of the interventions, different agencies such as the National Forestry and Wildlife Service (SERFOR) or the Ministry of Production (PRODUCE) had to provide ‘binding opinions’, which delayed the approval of the studies for road improvements/rehabilitations. 50 PVD has in its structure ‘Oficinas Zonales’ (zone or area offices) that are closer to the area where projects are located. 51 Puente Checca is short for ‘Mejoramiento Del Camino Vecinal Puente Checca - Puente Asuncion - Chitibamba - Emp. Cu 126 (Comunidad Huinchiri - Quehue)’ road rehabilitation work. Page 24 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) the midterm review (MTR). After the MTR, the Project gained traction because of its natural maturity (for example, completion of technical studies) and consistent leadership, which led to an increased rate of disbursement and faster implementation. The safeguards team also underwent multiple changes, with the MTR identifying six different social specialists since the Project’s inception. The World Bank Regional Safeguards Adviser (RSA) was involved in the Project, addressing the discrepancies between the national law and World Bank safeguards, which required significant time. Factors outside the Control of the Government, the Implementing Agency, and the World Bank 78. Political demonstrations and instability between 2016–2020 and 2022–2023. The PATS implementation period was characterized by a climate of political instability, with widespread public demonstrations and corruption scandals at different Government levels. The first period of political instability was exacerbated by the COVID-19 pandemic. The second period created uncertainty in leadership at the level of ministries and government agencies, including PVD. Both periods had a negative impact on the implementation schedule. 79. Outbreak of COVID-19. On March 15, 2020, the President of Peru announced a mandatory countrywide lockdown due to COVID-19. The lockdown forbade nonessential business, and the Project’s civil works were stopped. Project activities resumed slowly, but implementation delays were significant and cumulative. 80. Natural-induced hazards in the project areas. Some of the Project areas, including those where World Bank interventions were financed, suffered from regular landslides and heavy rains that prevented works from advancing or being completed. The frequent naturally induced impacts also led to the need to amend the works contracts, to allow the contractors to deal with the aftermath. In most cases, civil works contractors had to stop operations for approximately four months a year due to the rainy season. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 81. The M&E and RF for the Project seemed simple at the time of approval and directly related to the intended outcomes. The PAD stated the intention to track the results of the overall Program, which is reflected in the original RF. In the original Project, the indicators included in the PAD were to be enriched with other indicators collected as part of the Program’s M&E, including indicators to be collected through the IE. The original MOP described the strategy to monitor the indicators, assigned the responsibility of monitoring to PVD’s M&E office, provided a template intended to capture all the indicators to be followed as part of the Program (that is, from each co-financier), and established that other indicators were going to be captured in PVD’s systems.52 This approach was deemed advantageous as it meant streamlining the follow-up and tracking process into PVD and maintaining efficiency in PVD’s operational framework. 82. The original PDO indicators followed relevant best practices at the time to measure the outcomes of the Project. Measuring the Rural Access Index and the VOC was the standard in rural roads projects. Today, it is understood that these indicators must be better customized to be practical. One 52 PVD would use the ‘Sistema de Planificación, Monitoreo y Evaluación del PATS’ available through SIGAT, which is PVD’s suitable and efficient financial information system that works in parallel with the SIAF (Integrated System of Financial Management). Page 25 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) outcome-related indicator lacked specific information on the methodology for data collection and the evaluation of the indicator against the baseline. For the indicator on VOC, PVD lacked the tools to measure input data (that is, international roughness index), which added a layer of difficulty when evaluating the outcomes at closing. M&E Implementation 83. As the Project unfolded, the RF was modified to a more operational vision of the Project, which was adequate to the context of Project implementation. Through the second restructuring, all four PDO- level indicators were revised to some extent. It is important to note that before the MTR (2019), the implementing agency had not reported on indicator progress. This lack of reporting was likely a result of the limited progress up to that point. However, this absence of data also posed a challenge for enhancing the indicators during RP2, as the task team lacked concrete evidence to assess the effectiveness of the indicators. 84. As of June 2024, the implementing agency submitted a draft completion report. PVD started the planning for collecting data on the final outcomes at least six months before the closing date of the Project. However, issues with hiring of consultants were exacerbated by the need to review the methodologies to collect data and measure the indicators. Also, the indicator on decentralization lagged by at least one year, and therefore the latest information is from 2022.53 Additionally, the indicator on citizen engagement is vague and subject to a variety of interpretations. The data collection of PDO indicators also revealed that PVD lacked the equipment and resources to collect the information more regularly. 85. PVD reported on the progress of the intermediate indicators through the progress reports submitted every six months and, more regularly, on the follow-up virtual meetings. Still, it is now known that there was a need to have complementary intermediate indicators to better identify issues in achieving the PDOs. For example, an indicator that signals sustainability of interventions, such as the condition of the roads completed or the incorporation of these roads into a routine maintenance scheme, would have been beneficial, as visual evidence and interviews have confirmed that some roads are already deteriorating. 86. The IE was restricted to collecting baseline data. The Project envisioned an IE that measured the impacts of the overall Program on poverty outcomes. As early as 2016, the World Bank started developing a methodology to conduct a robust IE, informed by the lessons learned from experience. Factors affecting the Project also took a toll on the IE (for example, composition of treatment and control groups, hiring a firm versus the National Institute of Informatics and Statistics). Baseline data were processed and finalized in June 2022. M&E Utilization 87. The RF’s intermediate outcome indicators were used by both the World Bank and the implementing agency to assess the progress of the Project. The World Bank closely supervised progress on the indicators and, despite some difficulties related to data sources, considered them sufficient to evaluate the expected outcomes. The PDO indicators, however, could have been better monitored, and agreements on the methodology for data collection and evaluation against the baseline could have been done earlier. Despite this delay, the indicators accurately represented the reality on the ground. 53 PVD’s draft closing report mentions that this indicator is no longer measured using the methodology described in RP2. Page 26 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) 88. The intermediate indicators were straightforward and most were easy to measure. PVD and the World Bank were able to follow the progress of the indicators established by RP2 and RP5, most of which focused on specific outputs that were easily measurable. However, the indicator on citizen engagement was insufficiently utilized, as it was not a regular practice of PVD to collect this type of information. This indicator, if regularly collected for all types of interventions (road improvements and maintenance), could have provided better guidance on the impacts of the works on the lives of the citizens. Justification of Overall Rating of Quality of M&E 89. The overall rating of quality of M&E is Modest. There were significant shortcomings in the M&E systems design and implementation. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 90. The Project was classified as category B, as it was not expected to have significant or irreversible impacts or risks. The Project triggered the World Bank's safeguard policies, specifically those pertaining to Environmental Assessment (OP 4.01), Natural Habitats (OP 4.04), Forests (OP 4.36), Physical and Cultural Resources (OP 4.11), Indigenous Peoples (OP 4.10), and Involuntary Resettlement (OP 4.12). In compliance with OP 4.01, Environmental Impact Assessments (EIAs) were conducted for each subproject. An Indigenous Peoples Planning Framework (IPPF) was established, leading to the development of an Indigenous Peoples Plan (IPP) for subprojects involving indigenous communities. Additionally, a Resettlement Policy Framework (RPF) was developed for the entire Project, to cover OP 4.12, and was followed by the preparation of several Resettlement Action Plans (RAPs) for each subproject. 91. Compliance with safeguards policies was hindered by the client’s high staff turnover, the low quality of reports produced by consultants, and the limited understanding of the World Bank safeguards. The quality of the studies required for compliance with OP 4.10 and OP 4.12 was subpar, and the need for frequent revisions caused substantial delays. Additionally, there was initial resistance from PVD to comply with the World Bank safeguards, due to the gaps with the national regulation,54 for example, on resettlement compensation,55 which contributed to further complications and postponements in Project execution. Moreover, some of the concepts in the approved terms of reference for the safeguard documents were sometimes interpreted in different manners, by the PIU or the consultants, which contributed to constant revisions and delays. 92. Contractors lacked expertise in implementing the World Bank safeguards, and with a combination of poor performance and climatic conditions, the Project closed with a Post-Closure Action Plan (PCAP). During the execution of the works, contractors encountered challenges in fulfilling their commitments to the works and safeguards, mostly related to the implementation of the RAP and IPP. In addition, works expanding beyond the project closing date, due to either poor contractor performance or climatic conditions, led to the current requirement for a PCAP for three subprojects. Pacobamba and Puente Checca have unresolved issues related to OPs 4.10 and 4.12, while Puente Chico has pending matters concerning OP 4.12. 54 For example, for the IPP, the process of consultation with indigenous communities in Peru is captured in law Nº 29785. The ‘consulta previa’ is a dialogue between the State and the indigenous peoples, the purpose of which is to reach agreements on administrative or legislative measures that may affect the collective rights of the peoples. The agreements reached in the process are mandatory for both parties. This process can be carried out only by the Government, and it is advised and overseen by the Ministry of Culture. 55 In the national law, resettlement is paid at ‘market value’, while the World Bank’s OP 4.12 establishes a ‘total replacement cost’. Under the World Bank policy, properties are paid at a higher value, which poses a conflict to the implementing agency that feared retaliation from Peru’s Comptroller. Page 27 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Procurement 93. Procurement was rated Moderately Satisfactory for most of the Project’s life. Procurement processes financed with World Bank funding started only two years after the signing of the LA. The PIU recorded the information on Systematic Tracking of Exchanges in Procurement (STEP) throughout the Project; however, it needed constant reminder from the World Bank to update the system. 94. High staff turnover called for constant training and made it difficult to accumulate knowledge and experience. Procurement processes suffered from constant changes in the staff forming the evaluation committees and several bidding processes were not successful. The implementing agency also noted that, it was difficult to identify contractors/consultants that comply with the requirements established, for example, the use of the same specialists for several bids, leading to low-quality final products and/or the same contractor winning several works and not being able to fulfill all the responsibilities. Financial Management 95. Financial management was mostly rated Satisfactory. At approval, financial management risk was considered Moderate, based on the assessment regarding the solidness of the institution, which had qualified and experienced staff in working with World Bank-funded projects, and on the fact that the World Bank financing was only to be used for activities to be executed at a centralized level. The rating was temporarily downgraded to Moderately Satisfactory due to an issue with budget allocation from the Ministry of Economic and Finance (MEF), which caused a five-month delay. That prevented the initiation of any procurement process, including the hiring of consultants to strengthen the PIU. Overall, expenses were consistently documented, and the PIU supplied information that enabled effective monitoring of the Project’s progress. Interim financial reports (IFRs) were mostly submitted on time, and audits, which predominantly resulted in unmodified (clean) opinions, were also conducted and delivered to the World Bank in a timely manner. C. BANK PERFORMANCE Quality at Entry 96. The design of the PATS drew on lessons learned and insights gained from decades of efforts in the rural road sector, partnership with the IDB, and support to the GoP. The Project was fully aligned with the priorities of the World Bank and the GoP in terms of investments in infrastructure in the rural roads sector. However, an important challenge is that the RF envisioned a Program while the Project interventions were restricted to specific works. Further, there were no measures nor instruments to incentivize actions beyond the World Bank financing to achieve the PDO although previous projects had similar approaches, with adequate results. 97. The Project did not present a clearly defined set of roads to be intervened at entry, and while the main elements of the methodology had been agreed, changes in government and lack of coordination between the relevant institutions generated delays in the selection of roads. 98. The lengthy process to develop and approve technical road documents (for example, expedientes técnicos and EIA) was a cause of delay. The PAD specified that the World Bank and IDB would supervise the overall Program’s safeguards, to ensure good practices, but it did not clearly define the level of responsibility for each safeguards’ team regarding supervision and approval of documents. This ambiguity led to an excessive workload and, consequently, delayed response times from the World Bank, which hindered the Project’s progress. Page 28 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) 99. The World Bank did recognize the frequent climate-induced hazards in the country, which is a critical factor affecting the rural road network. However, the Project did not include disaster risk management mechanisms56 or specific measures to enhance the infrastructure’s resilience in its design. This omission represents a significant shortcoming, as the Project missed the opportunity to ensure the network’s long-term sustainability and put in place a strategy to make rural road infrastructure more resilient to climate and natural induced hazards. Issues of climate-induced hazards were addressed through a thorough technical assistance during implementation, however, recommendations were not implemented because it meant an increase in the cost of the roads. Quality of Supervision 100. The World Bank provided continuous and adequate support and training to the client throughout the implementation. The World Bank ensured a minimum of two supervision missions annually. When the need was identified, the World Bank involved high-level specialists (engineers; procurement, environmental, and social aspects; and so on). As noted, the World Bank team also underwent several changes throughout the life of the Project, which affected the implementation and supervision of the activities. Issues related to World Bank safeguards were a recurrent source of delay and misunderstandings, but the PIU acknowledged, during interviews, that the World Bank’s safeguard requirements, while stringent, were accompanied by support and training. 101. The Project could have changed the PDO, based on the information that no works were carried out under Subcomponent 1.2. PDO 2 was not achieved and based on the information on the actual nature of roads,57 it was foreseeable. Other changes during restructurings, such as extending the closing date and revising the indicators, were relevant and allowed the Project to achieve modest results. 102. At closing, during data collection to measure the PDO indicators, it was evident that roads completed under the Project had not received adequate maintenance, and their condition was suboptimal. The data collected, including imagery, showed at least one road already in deteriorated conditions. The PAD envisioned that roads financed through the PATS should have received routine maintenance58, and the World Bank requested PVD, during missions, to develop a strategy for maintenance. Remote supervision tools (for example, Geospatial Environmental Monitoring System [GEMS]) could have played a critical role in this aspect, and while a tool for remote monitoring was piloted, it was not further implemented. Justification of Overall Rating of Bank Performance 103. The overall rating of the Bank performance is Moderately Unsatisfactory. D. RISK TO DEVELOPMENT OUTCOME 104. Providing sustainable and reliable rural road infrastructure to alleviate poverty, reduce regional disparities, and increase the competitiveness of the country is still a priority in the agenda of the GoP. However, the default of adequate rural road maintenance is a risk to the development outcome: at the time of this ICR, one of the three roads rehabilitated under the restructured Project was deteriorating, and some others were already in unacceptable conditions. The World Bank continues its engagement with 56 Contingent Emergency Response Component (CERC) was not an option/instrument available for Project design at the time of approval of the Project. 57 The MTR and RP2 mention that some of the financed roads (under social inclusion) connect to logistic corridors. 58 According to the MOP, some roads financed through previous rural roads projects could also received routine maintenance. Page 29 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) the implementing agency, and it is working on the concept of a new rural roads operation with a strong focus on sustainability through maintenance schemes. V. LESSONS AND RECOMMENDATIONS 105. When defining World Bank-financed operation, it is essential to align the Project objective and PDO indicators to what the Project can effectively leverage. In the present case, the Project was an Investment Project Financing (IPF) of limited amount aimed at supporting a much wider Program, but the World Bank had limited leverage on Program activities that were not directly connected to what it was financing. This resulted in a restructuring that substantially revised the M&E framework. Achieving results at Program level would have probably required a result-based financing approach and/or the use of an instrument like a Program-for-Results. Also, the M&E system, including the RF indicators should follow SMART59 principles, so that it can be utilized effectively throughout Project implementation. Revising the RF indicators, if necessary, will help evaluate the Project appropriately at completion. 106. Readiness for implementation for infrastructure projects should include a robust analysis that considers the implementing agency capacity and the World Bank technical and cross-support teams to ground expectations. The lack of an established methodology and identified interventions were the main source of implementation delays of the project. This made the Project vulnerable to political influence and hindered its readiness for implementation. To improve this, it is crucial to establish, before approval, a transparent and participatory process for road selection, with clearly defined criteria based on objective attributes. This will help ensure that interventions are identified and earmarked in a timely manner, reducing the risk of delays. 107. It is crucial to prioritize maintenance with a focus on climate change resilience in all rural road projects. This is particularly important in countries like Peru that are highly vulnerable to frequent and intense climate-induced hazards and have difficult landscape. Future project designs should focus on promoting and innovating in rural road maintenance schemes, enhancing the resilience of rural road infrastructure, and incorporate mechanisms for effective disaster risk management to ensure long-term sustainability of the network. Routine maintenance planning and funds need to be included in the project design to ensure sustainability of the investments. 108. Support mechanisms for effective contract management, information and record keeping and ensure there is a clear plan of action to solve issues during contract execution. Supporting an information system/contract management platform that is able to collect, integrate, and keep records related to contract management is critical to ensure information is readily available to stakeholders to make effective decisions. In addition, there is a need to identify, in advance, issues that delay contract execution and develop a plan of action to proactively solve these situations. 109. In-person site visits, paired with remote supervision tools, are critical to rapidly respond to project implementation developments and ensure compliance with infrastructure requirements and the World Bank’s safeguards policies. Remoteness of the types of interventions such as those financed through the PATS made frequent site visits difficult and expensive. However, it is the isolation and remoteness that makes these investments more critical. Therefore, World Bank projects should ensure low-cost remote supervision tools are included in project design, along with regular training. In addition, early review of designs is important to ensure these comply with World Bank standards in terms of road 59 Specific, measurable, achievable, relevant, and time-bound. Page 30 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) safety and protection of vulnerable road users and considering climate-induced hazards. It would be helpful to increase field visits during study preparation and the execution of works and include interviews with the contractor and the supervision firms for each subproject. 110. The gaps between the World Bank safeguards and national law that caused confusion and delayed implementation could be addressed by developing, at preparation, road-specific safeguard documents for the first round of investments, documenting the process of approval of each document and the path to obtain no-objection, and developing a strategy to solve the controversies. The Project suffered several delays due to the implementation of the safeguards and the discrepancy between World Bank and national laws. It is necessary that the PIU and the technical teams are regularly trained and provided with tools that help them navigate these gaps. Additionally, having a clear and documented process for approving subproject-required safeguard documents and terms of reference can help streamline the review and approval. 111. Strength the institutional capacity and a common understanding and applicability of E&S safeguards and develop a strategy to solve controversies. The Project suffered several delays during the review of safeguard documents due to a poor understanding of safeguards, low-quality performance of the consultancy services hired by the PIU, and their own quality control before sending documents to the World Bank. It is recommended to offer permanent trainings not only on safeguards but also on methodological and formal aspects of the preparation of technical studies or management documents. Additionally, have a clear process to solve controversies, ensuring that the PIU agrees that is the World Bank who interprets its own policies and requirements. 112. Promote, through the project, the professionalization of the workforce and provide conditions of contract stability to incentivize capable staff to stay throughout the project life. High staff turnover led to poor coordination, management issues, and delays in decision-making, ultimately affecting the implementation and outcomes of the Project. Investing in the development and retention of skilled staff is crucial; develop clear job descriptions related to project activities, and the project should promote not only the professionalization of the PIU but provide conditions of contract stability. This can help ensure continuity and consistency in project implementation. . Page 31 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Facilitate sustainable road access of Peru’s rural populations to services Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion People with access to an all- Number 0.00 30.22 24.73 20.07 season road financed by the (Thousand) project. 30-Jun-2021 30-Jun-2021 29-Dec-2023 Comments (achievements against targets): - This indicator was introduced through the second restructuring. The target presented in RP2 was calculated using the wrong methodology (25) and is therefore discarded. Through RP, the target is updated to 24.73k people and it mentions the adequate target based on the scope of RP2. - The project provided access to rural roads to 20,070 people. Achievement towards the target is 81 percent. - The methodology uses the information from the technical studies to assess the number of beneficiaries. Objective/Outcome: Reduce transport costs on rural roads linked to priority logistic corridors Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 32 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Change in the vehicle travel Percentage 0.00 50.00 49.90 time on rural roads financed by the project. 21-Jun-2021 30-Jun-2021 29-Dec-2023 Comments (achievements against targets): - This indicator was introduced through the second restructuring. The original indicator measured travel time for trucks, this indicator expands to measure any type of vehicle and do not restrict the trips on roads connected to logistic corridors, but traveling to any road. - Through RP2, the target is set at a reduction of 50 percent of travel time, taking as baseline 15 km/hr. The target represented the expectation to increase speeds to 30 km/hr . - Calculation is based on information provided by PVD on the current speed on the three completed roads. Speed in Talavera is 30.9 km/h (14.28km in 27.74 min), Speed in Chuchin is 30.7 km/hr (20.98km in 40.98 min),Speed in Vinchos is 27.56 km/hr (22.73km in 49.47 min),* Average speed is 29.72km/hr, - The methodology did not specify how to aggregate the values, therefore a simple average of speed was taken and used against the baseline of 15km/hr. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Vehicle operating cost for Amount(USD) 0.59 0.48 0.85 trucks on rural roads with access to logistics corridors 30-Jun-2021 30-Jun-2021 29-Dec-2023 financed by the project Comments (achievements against targets): - This indicator was modified through the second restructuring. The wording makes the direct link to roads that are connected to logistic corridors prioritized. Page 33 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) - Through RP2, the target is updated to 0.48 (USD) and it was maintained. - PVD calculated the VOC for trucks using two methods, one rends a result of US$ 0.84 and the second method US$ 1.055. This means that the target was not achieved, as VOC was increased based on both models. - The methodology for calculation establishes that it comes from HDM-4, however, PVD notes that the baseline comes from quantitative and qualitative interviews with transport operations in logistic corridors, which will not correspond to the situation of the roads assessed. Moreover, PVD mentions that trucks are not the most representative vehicle on the roads assessed. Objective/Outcome: Facilitate sustainable road access of Peru’s rural populations to services Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of local Percentage 0.00 30.00 59.00 governments (LG) with a fair or good institutional capacity 30-Jun-2021 30-Jun-2021 29-Dec-2023 in road management. Comments (achievements against targets): - This indicator was introduced through the second restructuring. The original indicator measured the share of the rural road network in good and fair condition. - Through RP2, the target is introduced at 30 percent and it was maintained. - PVD reported at the end of 2021 (valid for 2022) that 59 percent of the LG had good/fair capacity as measured by. - This indicator was surpassed. Page 34 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) A.2 Intermediate Results Indicators Component: Infrastructure for Integration and Social Inclusion Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct and indirect project Number 0.00 152.52 128.78 124.34 beneficiaries of roads (Thousand) financed by the project 30-Jun-2021 30-Jun-2021 30-May-2023 30-Dec-2023 Percentage of direct and Percentage 0.00 51.00 49.97 49.89 indirect project beneficiaries of roads financed by the project that are female Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Roads rehablitated Kilometers 0.00 172.00 102.57 57.96 30-Jun-2021 30-Jun-2021 30-May-2023 29-Dec-2023 Comments (achievements against targets): Page 35 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Description of changes can be found in Annex 5. - The target was reduced through RP2 and RP5. - The 102.57 considered 4 roads to be completed and accounted 70 percent of one road (Puente Checca) to be completed after the closing date. Component: Rural Road Infrastructure Maintenance Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of km under Kilometers 0.00 190.00 190.00 periodic maintenance financed by the project. 30-Jun-2021 30-Jun-2021 29-Dec-2023 Comments (achievements against targets): Component: Decentralized Road Management Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Local Economic Development Number 0.00 24.00 24.00 Plans (PDEL) delivered to Local Governments financed 30-Jun-2021 30-Jun-2021 23-Jun-2023 by the project. Page 36 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of Participatory Number 0.00 16.00 15.00 Provincial Road Plans (PVPP) implemented under the 30-Jun-2021 30-Jun-2021 29-Dec-2023 program and financed by the project. Comments (achievements against targets): This indicator was previously marked as achieved, but information from the client's closing report only shows that 15 PVPP were effectively financed with WB resources. Nevertheless, there were over 100 PVPPs financed in the Program's framework. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of beneficiaries Percentage 0.00 70.00 65.00 who feel that project investments reflect their 30-Jun-2021 30-Jun-2021 01-Apr-2024 needs. Comments (achievements against targets): Page 37 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) B. KEY OUTPUTS BY COMPONENT PDO 1: Facilitate sustainable road access of the Borrower’s rural populations to services 1. Share of rural population with access to an all-season road - dropped Outcome Indicators 2. People with access to an all-season road financed by the project - 20,070 3. Change in the vehicle travel time on rural roads financed by the project - 49.9% 1. Direct Project Beneficiaries - dropped 2. Direct and indirect project beneficiaries of roads financed by the Project - 124.340 3. Of which female - 49.89% Intermediate Results Indicators 4. Roads rehabilitated - 57.96 km 5. Length of roads under routine and periodic maintenance - dropped 6. Number of km under periodic maintenance financed by the project - 190 km 7. Percentage of beneficiaries who feel that project investments reflect their needs - 65% 1. Roads rehabilitated Key Outputs by Component 2. Roads under periodic maintenance (Linked to the achievement of the 3. Technical studies developed for rehabilitation and periodic maintenance - 47 Objective/Outcome 1) 4. Road safety measures installed along the roads 5. Compliance with safeguards (implementation of indigenous plans, resettlement plans, and so on) PDO 2: Reduce transport costs on rural roads linked to priority logistic corridors 1. Reduced truck transit time from production to distribution centers on feeder roads integrated to logistics corridors - dropped Outcome Indicators 2. Vehicle operating cost for trucks - dropped 3. Vehicle operating cost for trucks on rural roads with access to logistics corridors financed by the project - 0.85 Intermediate Results Indicators 1. Roads rehabilitated - (repeats from PDO 1) Key Outputs by Component 1. Technical studies developed for rehabilitation of roads linked to Logistic Corridors (Linked to the achievement of the Page 38 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Objective/Outcome 2) PDO 3: Strengthen decentralized road management 1. Roads in good and fair condition as a share of total classified roads - dropped Outcome Indicators 2. Percentage of local governments (LG) with a fair or good institutional capacity in road management - 59% 1. Number of Projects approved under LLDWs - dropped 2. Local Economic Development Plans (PDEL) delivered to Local Governments financed by the project - 24 3. Percentage of PGs included in the program that have executed 80% of the multiyear road program - Intermediate Results Indicators dropped 4. Number of microenterprises certified to perform routine maintenance services - dropped 5. Number of Participatory Provincial Road Plans (PVPPs) implemented under the program and financed by the project - 15 Key Outputs by Component 1. Equipment transferred to local governments (GPS, total stations, motorcycles, laptops) - 194 of each (Linked to the achievement of the 2. Training in climate resilient roads (4 sessions) Objective/Outcome 3) 3. Training in remote monitoring (Videologger) 2 trainings - 4 sessions Page 39 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Anca Cristina Dumitrescu Task Team Leader(s) Selene del Rocio La Vera Procurement Specialist(s) Nelly Ikeda Financial Management Specialist Sergio Garcia Monroy Team Member Jorge Minaya Osorio Team Member Steven Farji Weiss Team Member Satoshi Ogita Team Member Jose Simon Rezk Team Member Ana Lucia Jimenez Nieto Team Member Maria Virginia Hormazabal Team Member Mariana Margarita Montiel Counsel Alonso Zarzar Casis Social Specialist Raul Tolmos Social Specialist Aracelly G. Woodall Team Member Supervision/ICR Eric R. Lancelot Task Team Leader(s) Juan Carlos Martell Rivera, Leonel Jose Estrada Procurement Specialist(s) Martinez, Selene del Rocio La Vera Enrique Antonio Roman Financial Management Specialist Cristina Manget Sanchez Sacristan Team Member Luis Francisco Malaspina Quevedo Social Specialist Page 40 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Damian Pedro Saint Martin Zimmerspitz Team Member Andrea Victoria Calderon-Urquizo Carbonel Environmental Specialist Maria Jose Carreras Gamarra Environmental Specialist Piero Antonio Galluccio Chiri Social Specialist Claudia Lopez Rodriguez Team Member Carla Albertina Jerez Abascal Procurement Team Nancy Itami Okumura Warth Procurement Team Li Qu Team Member Sonia Rousseau-Lopez Team Member Javier Morales Sarriera Team Member Licette M. Moncayo Team Member Ana Silvia Aguilera Team Member Marisol Gisel Noriega Ramos Team Member Marco Antonio Zambrano Chavez Environmental Specialist Catarina Isabel Portelo Counsel Maria Virginia Hormazabal Team Member Paula Genis Team Member B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY13 1.585 30,423.02 FY14 9.063 89,694.99 FY15 35.325 233,403.79 FY16 19.680 113,337.23 FY17 0 4,113.34 Page 41 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) FY18 0 2,592.23 Total 65.65 473,564.60 Supervision/ICR FY14 0 2,086.00 FY15 0 446.60 FY16 19.127 112,559.72 FY17 41.278 226,775.66 FY18 29.507 208,151.82 FY19 30.491 223,396.54 FY20 26.071 153,843.56 FY21 25.249 140,790.60 FY22 39.362 196,432.94 FY23 17.132 150,084.12 FY24 25.078 173,061.69 Total 253.30 1,587,629.25 Page 42 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Closing Percentage of Components (US$, millions) (US$, millions) Approval Infrastructure for Integration and 35.00 18.37 52.00 Social Inclusion Rural Road Infrastructure 10.00 10.12 100.00 Maintenance Decentralized Road Management 4.65 5.78 124.00 Project Management 0.35 0.00 0.00 Total 50.00 34.27 68.54 Page 43 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) ANNEX 4. EFFICIENCY ANALYSIS The Scope of Road Economic Analysis 1. A CBA ensures the project roads generate sufficient economic benefits that warrant the investments. At appraisal and restructuring,60 the economic analysis of PATS, conducted by PVD, assessed the economic viability of the two major physical components, namely Component 1 and Component 2. The economic analysis of those components was conducted based on a conventional consumer surplus approach, which measures benefits accruing principally to existing road users in terms of reduction of VOCs and time cost for passengers and freight. The analysis also considered changes in costs of road maintenance and rehabilitation works due to different types of road surfaces. Road Economic Analysis at Project Restructuring 2. In undertaking economic evaluation at appraisal/Project restructuring, the following assumptions were applied. The basic characteristics of the project roads are summarized in Table 4.1. . • The base alternative of the evaluation is ‘Do nothing’: the case of not implementing the project but carrying out only basic maintenance. For Component 1, the project alternative considers road rehabilitations. For Component 2, the project alternative considers periodic maintenance to keep good road conditions. • The discount rate was 9 percent, and the evaluation period was 20 years. • All the project roads initially had older pavements. All the project roads were two way with one lane in each direction in rural areas. Table 4.1. Basic Characteristics of Project Roads at Restructuring (before project) Est. Traffic (AADT) at No. Scope Length (km) Width (m) Surface Roughness (IRI)a project start year 1 Component 1 102.57 7.0 Paved 16 <50 2 Component 2 190.70 7.0 Paved 16 <50 Note: a. Weighted average of field survey results in 2024 3. The estimated cost for the improvement is summarized in Table 4.3. (compared with completion). The economic feasibility results such as NPV, expected internal rate of return (EIRR), and sensitivity analysis checks are provided in Table 4.3. (compared with completion). The IRR of the project roads with base scenario was 16.1 percent and the positive NPV (US$11.9 million), which shows financial feasibility. Road Economic Analysis at Project Completion 4. The completion stage economic analysis was conducted for the same roads for which the economic analysis was conducted at appraisal/restructuring. The Road User Cost Model Version 5.01 (RUC) (as part of RED) was used to compute annual road agency and users’ costs for each project alternative over the evaluation period. In the models, the quantities of resources consumed and vehicle speeds are calculated first and then multiplied by unit costs to obtain total VOCs, travel time costs, and CO2 emissions. The resources consumed and vehicle speeds are related to traffic volume and composition, road surface type, geometric characteristics, and roughness. 60 The restructuring presented economic analyses calculated on the principle of proportional magnitude. Page 44 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) 5. In undertaking economic evaluation at completion, the following assumptions were changed from appraisal/restructuring and applied in the RUC model as part of the RED calculation. Unless specified, the same assumption was used at appraisal and completion. • The project roadworks commenced in 2018 and the start year varies by section. All sections were completed in 2023. The duration of each section is one year. The economic evaluation period is 20 years. • The length of roadworks under Component 1 was reduced to 57.96 km. • A conversion factor of 0.82 was applied to convert financial costs into economic costs to remove taxes from financial costs. • The actual costs (roadworks cost and maintenance cost), the traffic volume, and the average traffic composition are taken from PVD’s traffic survey report in 2024, based on the most updated information available at the time of evaluation. • Given a lack of detailed economic data, the default vehicle fleet economic unit costs and characteristics in RUC and RED are used. • The average daily traffic annual increase rate was 3.2 percent over the evaluation period, based on the average gross domestic product (GDP) growth rate in Peru (the depression between 2020 and 2021 was excluded) and an elasticity of traffic growth to GDP growth of 1.2. Table 4.2. Vehicle Fleet Economic Unit Costs and Characteristics Parameters Car Medium Delivery Vehicle Truck Light Truck Medium New vehicle cost (US$) 17,950 26,100 103,000 126,300 New tyre cost (US$) 49 83 149 369 Fuel cost (US$/liter) 1.162 1.162 1.162 1.162 Lubricant cost (US$/liter) 6.965 6.965 6.965 6.965 Maintenance labor cost (US$/hour) 2.89 2.89 2.89 2.89 Crew cost (US$/hour) 0 1.34 2.58 3.41 Overheads (US$/year) 0 0 0 0 Interest rate (%) 14 14 14 14 Value of passenger working time (US$/hour) 1.6 1.6 0.96 0.96 Value of passenger non-working time (US$/hour) 0 0 0 0 Cargo time (US$/hour) 0 0.12 0.09 0.09 Annual utilization (km) 25,000 40,000 60,000 90,000 Annual utilization (hours) 480 960 1,440 2,400 Service life (years) 10 8 8 10 Percent of time for private use (%) 100 0 0 0 Number of passengers (#) 3 3 1 1 Work-related passenger trips (%) 75 0 0 0 Gross vehicle weight (tons) 1.37 2.18 6.86 15.40 Source: Parámetros Requeridos y Opcionales para el uso del HDM - Oficina de Inversiones - OGPP-MTC (conducted by PIU in 2024). 6. Based on the information outlined in the tables and assumptions made, the economic analysis was performed using RED for all the project roads at the completion stage. It was assumed that during the construction, road user benefits (VOC and time) produced. The obtained economic feasibility results such as NPV and IRR are listed in Table 4.3. , which provides a comparison of economic analysis results of Page 45 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) the appraisal stage and the completion stage. The project roads are economically justified at completion stage. Table 4.3. Comparison of Appraisal and Completion Stage Economic Analysis Est./Actual Economic Est./Actual NPV at 9% Length IRR Scope Analysis Stage Cost (US$, Cost per Km Traffic (AADT) (US$, (km) (%) millions) (US$) in 2024 millions) Restructuring 102.57 15.9 155,200 <50 10.8 13.4 Component 1 57.96 11.5 125,949 Completion 54 5.0 13.4 Restructuring 190.70 10.5 55,060 <50 1.1 17.8 Component 2 Completion 190.70 9.0 47,356 29 2.7 12.5 Restructuring 293.27 — — — 11.9 16.1 Total Completion 248.66 — — — 7.7 14.4 7. The comparison clearly shows that the NPV and IRR values are smaller at completion stage compared with the appraisal stage but still economically efficient (IRR ≥ 9 percent). 8. The changes in NPV and IRR values at completion stage could be attributed to the increased traffic volume and assumptions on VOCs and changes in traffic volume on the project roads at completion stage. 9. The comparison indicates that IRR at project completion is at a similar level as at appraisal. Given the roadworks changes, this analysis suggests that, based only on the IRR, the economic efficiency of the project was Substantial. Page 46 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) ANNEX 5. EFFICACY DETAILS 1. The PDO indicators were modified through the second restructuring. The table below shows the original indicator and linked PDO and provides the definition of each of the indicators. Table 5.1. Original and Revised Outcome Indicators Revised Revised Original Revised PDO Indicators Baseline Target Target Target Baseline (RP2) (RP5) PDO 1. Original: share of rural population with access to an all- season road (%) 43 51.5 Replaced (see below) PDO 1. Revised: people with access to an all-season road — — 0 30.22 24.73 financed by the project (#, thousand) PDO 2. Original: Reduced truck transit time from production to distribution centers on feeder roads integrated to logistics 0 25 Replaced (see below) corridors (%) Revised to measure PDO 1: change in the vehicle travel time on — — 0 50 — rural roads financed by the project (%) PDO 2. Original: Vehicle operating cost for trucks (US$) 0.79 0.42 Revised (see below) PDO 2. Revised: Vehicle operating cost for trucks on rural roads — — with access to logistics corridors financed by the project (US$) 0.59 0.48 — PDO 3. Original: Roads in good and fair condition as a share of 55 60 Replaced (see below) total classified roads (%) PDO 3. Revised: percentage of local governments with a fair or — — 0 30 — good institutional capacity in road management (%) Table 5.2. PDO Indicator Details PDO 1. Original: share of rural population with access to an all-season road (%) [dropped] Share of rural population with access to an all-season is the proportion of rural people in the program area who live within 2 km of an all-season road (Rural Access Index). Baseline 43 percent. Source: Assumed PVD with World Bank estimations. Target 51.5 percent. Source: From the MTR, it is considered that 5,000 km would regain transitability as part of the Program, split as 2,200 km on rehabilitation/improvement and 2,800 on periodic maintenance. The total of 5,000 are assumed to move the share of rural population 8.5 percentage points. Achievement As of December 2023, the PATS carried out works on approximately 2,344 km of roads (2,111 km periodic maintenance, 233 km improvement/rehabilitation). Accounting for the expectation of 5,000km, the achievement as of Dec/2023 will proportionally suppose a 3.98 percentage points progress towards the target. Calculations: 5,000 km → 8.5 pp, then 2,344 → 3.98 pp, therefore, baseline (43%) + progress (3.98pp) = 46.98%. The progress of 3.98 pp of the expected 8.5 pp puts the achievement at 46.8%. PDO 1. Revised: people with access to an all-season road financed by the project (#, thousand) Number of people in the program area who live in a rural setting within 2 km of an all-season road that is financed by the restructured project (RP2) Baseline 0 people. Source: not applicable. Target RP2 30,220 people.61 Source: n/a. This target is presented in RP5, mentioning that a previous calculation was not following the agreed methodology. 61 The target is based on the eight roads to be financed with World Bank resources. This reduced the ambition of the project outcome. RP2 presents a target of 25.3 but RP5 notes that the agreed methodology was not followed; therefore, it is assumed that 30.22 used as ‘original target’ in RP5 is the correct/comparable target. Page 47 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Target RP5 24,730 people. Source: From the expedientes técnicos – PVD reporting. This calculation excludes the beneficiaries of one road that was known would not be completed by the closing date. Only accounts for people benefiting from roads under Component 1. Achievement The achievement, as reported by PVD, is 20,070 people, based on the information from the expedientes técnicos (technical studies). The calculation is mentioned in World Bank documents to have been calculated by PVD and WB teams. PDO 2. Original: Reduced truck transit time from production to distribution centers on feeder roads integrated to logistics corridors (%) [dropped] This indicator used the information from the ‘Study Analysis of Logistics in Peru 2015’ which established that it took “2 hrs to drive an average distance of 25 km or about 13 km/hr. The improved road condition will reduce the time by a minimum of 30 minutes, that is, improving the distance coverage to about 17 km/hr.” Baseline 0%. Source: not applicable. Target 25%. Source: Assumed estimation PVD-World Bank. Achievement This indicator was supposed to be measured in the 1,100 km of roads under Subcomponent 1.2; as no roads were built under that subcomponent with World Bank funding, the achievement was not calculated. Revised to measure PDO 1: change in the vehicle travel time on rural roads financed by the project (%) This indicator compares the average speed before and after the interventions on rural roads with access to the logistic corridors of the program. According to PVD, the average speed before the intervention is 15 km/hour. The goal for the average speed at the end of the intervention is 30 km/hour. This objective corresponds to a reduction in travel time of 50%. * RP2 puts this indicator as linked to PDO 1 and PDO 2; however, as there were no roads built under subcomponent 1.2, the ICR only assesses this indicator for PDO 1 and uses all three roads completed, regardless of those being linked to logistics corridors or not. Baseline 0%. Source: not applicable. Target 50%. Source: Estimation PVD-World Bank. Achievement This indicator was almost fully achieved when compared to the baseline estimated speed of 15 km/hour. The indicator had a flaw as it did not mention how the aggregate was to be calculated nor the method of measuring the speed. The baseline was calculated as part of the restructuring, but it seems it included information from the logistics study, that used different methods and a larger sample of roads to measure speed. PVD used different methods to collect information in the three completed roads. Based on the average speed from 2 methods (floating vehicle and ‘plate method’), it was calculated that the average speed is of 29.7 km/hour, almost fully achieving the target of reducing travel time by 50 percent. The average of the different methods is used, as the ‘floating vehicle’ which would be the best method, was measured only using one type of vehicle. The average of methods is: speed in Talavera is 30.9 km/hour (14.28 km in 27.74 min), speed in Chuchin is 30.7 km/hour (20.98 km in 40.98 min), speed in Vinchos is 27.56 km/hour (22.73 km in 49.47 min). The simple average speed is 29.72 km/hour. PDO 2. Original: Vehicle operating cost for trucks (US$) [dropped] The baseline and target vehicle operating costs were based on the information provided by the MTC in the PATS’ economic analysis. It was calculated as the weighted average for two-axle trucks moving on the rural roads linked to logistics corridors (Subcomponent 1.2). Baseline US$0.79. Source: MTC Target US$0.42. Source: Estimation PVD-World Bank/MTC. Achievement This indicator was dropped and there is no information from trucks moving on rural roads linked to logistic corridors improved through the program. PDO 2. Revised: Vehicle operating cost for trucks on rural roads with access to logistics corridors financed by the project (US$) This measures the VOC for trucks in interventions financed with the IBRD loan. Baseline US$0.59. Source: MTC Target US$0.48. Source: Estimation PVD-World Bank/MTC. Achievement The revised indicator measures substantively the same result. The revised indicator was made more consistent with the component name. PVD calculated the VOC for trucks using two methods, one rends a result of US$0.84 and the second method US$1.055. This means that the target was not achieved, as VOC was increased based on both models. The methodology for calculation establishes that it comes from HDM-4, however, PVD notes that the baseline comes from quantitative and qualitative interviews with transport operations in logistic corridors, which will not correspond to the situation of the roads Page 48 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) assessed. Moreover, PVD mentions that trucks are not the most representative vehicle on the roads assessed. PDO 3. Original: Roads in good and fair condition as a share of total classified roads (%) [dropped] The percentage of the total classified rural road network in the program area that is in good and fair condition depending on the road surface and the level of roughness. Baseline 55%. Source: n/a Target 60%. Source: Estimation PVD-World Bank Achievement The MTR proposed a methodology to calculate this indicator, and proposed a new target, which was much lower, using a conservative assumption that 50 percent of the interventions of the program would change from poor to fair/good condition, and that all other roads stay in fair condition throughout the duration of the program. This was never used, and the indicator was dropped. The calculation presented in this ICR comes from an estimation, based on the assumption that the roads completed until December 2023, of 2,111 km of periodic maintenance and 233 km of rehabilitation and maintenance, would count toward the indicator, and uses the expected 5,000 km to be intervened through the PATS for the 5-pp increase. The calculation then assumes that all things equal, the interventions in 2,344 km would render a progress of 2.3 pp toward the target, putting the achievement in 46 percent (2.3/5 = 0.46). PDO 3. Revised: percentage of local governments with a fair or good institutional capacity in road management (%) It will be calculated by using the results of the categorization reports for the 194 LGs, through the Road Management Tracking and Monitoring System (Sistema de Seguimiento y Monitoreo de la Gestión Vial [PVD-GFGVD]). Training is prioritized to LGs with low institutional capacity, and then the categorization reports are applied to monitor their progress (bad, fair, or good). It is expected that at least 30% of LGs end up with fair or good institutional capacity in road management by project closing. Baseline 0%. Source: not applicable Target 30%. Source: Estimation PVD-World Bank Achievement This indicator was last reported by PVD in 2022. The methodology to calculate this indicator within the agency changed, and now it follows the management of financial resources at LG level. The indicator was achieved, and, as the activities and consultancies have continued, is assumed to be maintained. Page 49 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Table 5.3. Changes in Intermediate Indicators Revised Target Revised Target Original Measure Unit Baseline Original Target Actual Changes (RP2) (RP5) Direct Project Number, Intermediate Indicator (IR) x 0 1,100 No datab Replaced (see below) No dataa Beneficiaries thousands 1, ‘Direct project Direct and indirect beneficiaries’ was revised project beneficiaries of Number, to ‘Direct and Indirect 0 157.52 75% 128.78 96% 124.34 roads financed by the thousands project beneficiaries of Project roads financed by the Of which females project’ in RP2, and the x (Direct Project Percentage 0 51 — Replaced (see below) No data target was recalculated Beneficiaries) and reduced from 1.1 million to 157,520 and Percentage of direct further down to 128,780 in and indirect project RP5. The target for the beneficiaries of roads Percentage 0 51 98% 49.97 99.8% 49.89 supplementary indicator, financed by the project ‘women beneficiaries’, was that are female aligned with it. IRI2, ‘Roads Rehabilitated’ was reduced in RP2 from x Roads rehabilitated Kilometers 0 2,200 11%c 172 34% 102.57 57% 57.96 2,200 km to 172 km, and through RP5 to 102.57 km. Length of roads under IR4, ‘Number of km under x routine and periodic Kilometers 0 5,000 42% Replaced (see below) 2,111.66 routine and periodic maintenance maintenance’ was revised in RP2 to refer only to Number of km under ‘periodic’ maintenance, periodic maintenance Kilometers 0 190 100% — — 190 and the target was financed by the project reduced from 5,000 km to 190 km Percentage of beneficiaries who feel that project Percentage 0 70 93% — — 65 Added RP2 investments reflect their needs Page 50 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Revised Target Revised Target Original Measure Unit Baseline Original Target Actual Changes (RP2) (RP5) IR5, ‘Number of projects approved under Local and Logistics Development Number of Projects Windows’ was replaced by x Number 0 24 Replaced (see below) approved under LLDWs ‘Local Economic Development Plans (PDEL) delivered to LGs financed by the project’ in RP2 Local Economic Development Plans (PDEL) delivered to Number 0 24 100% — — 24 Added RP2 Local Governments financed by the project Percentage of PGs included in the program x that have executed 80% Percentage 0 80 — Deleted No data Dropped of the multiyear road program Number of microenterprises x certified to perform Number 0 200 — Deleted No data Dropped routine maintenance services Number of Participatory Provincial Road Plans (PVPPs) Number 0 16 94% — — 15 Added RP2 implemented under the program and financed by the project a,b: The PAD mentions this will be calculated as part of the impact evaluation. It is unclear which roads are to be accounted. c: based on the achievement of 233 km of the PATS Program. Page 51 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) ANNEX 6. COMMENTS FROM PVD 1. The following comments were received from PVD. A draft English version was shared along with a Spanish version as a courtesy. Page 52 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Page 53 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) ANNEX 7. ROADS FINANCED AND MAPS 1. Table 7.1 describes the roads financed or expected to be financed at the time of each restructuring. Roads are marked as completed (finalized before the closing date), works in progress are those that started civil works, studies in progress are those roads that had already consultants assigned working on their technical documents, with the assumption that the civil works would be financed through the project. Procurement delayed refers to roads that did not start procurement of civil works as expected for various reasons. Dropped roads were those excluded from the project through the restructurings. 2. RP3 mentions that as P. Checca, P. Angasmayo, Allaspina, and Huambo works were not going to be completed by the closing date, the available funds could be allocated to finance initial works on other roads. Then, it was agreed that five more roads could be partially financed (initial works) as detailed in the table. In RP5, all roads that had not started procurement or did not have ongoing works were dropped. Table 7.1. Planned Rehabilitation of Roads under Subcomponent 1.1 Works under Component 1 Status at the time of restructurings Roads RP2 RP3 RP5 As of April-24 Talavera (14.28 km) Completed Completed Completed Completed Chuchin (20.98 km) Works in progress Completed Completed Completed Vinchos (22.7 km) Works in progress Works in progress Completed Completed P. Chico (24.85 km) Works in progress Works in progress Works in progress 74.9% P. Checca (28.24 km) Studies in progress Procurement delayeda Works in progress† 10.5% P. Angasmayo (24.37 km) Studies in progress Procurement delayeda Dropped — Inkacancha (10.52 km) Studies in progress — — — Allaspina (26.7 km) Studies in progress Procurement delayeda Dropped — Huambo (22.6 km) — Procurement delayeda Dropped — Pacobamba (28.83 km) Suspended/Dropped Studied for Re-bidding Dropped — Dv. F. Puerto Viejo (5.2 km) — Studies in progressb Dropped — Nva. Esperanza (24.96 km) — Studies in progressb Dropped — Nvo. San Juan (11.8 km) — Studies in progressb Dropped — Chapina (22.4 km) — Studies in progressb Dropped — Targets at RPs 172 km 172 km62 102.57 km 57.96 km (actual) Note: a. RP3 mentions that these four roads would not be completed by the proposed closing date (February 2023) but by December 2023. b. These new projects were expected to be partially financed with World Bank resources but not expected to be completed by the closing date. † RP5 mentions that P. Checca was not expected to be completed by the closing date (December 2023), but the target for the indicator on km rehabilitated accounted for the expected 70 percent progress on roadworks. 62 The RP3did not introduce changes in the RF, however, it does present a change in the economic analysis using a total of 139 km of roads. RP3 considers 4 completed roads and 9 unfinished roads by the closing date. Page 54 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Figure 7.1. Talavera - Road Completed Page 55 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Figure 7.2. Vinchos - Road Completed Figure 7.3. Chuchin - Road Completed Page 56 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Figure 7.4. Puente Checca - Road in Progress Figure 7.5. Puente Chico - Road in Progress Page 57 of 58 The World Bank Support to the Subnational Transport Program Project (P132515) Figure 7.6. Talavera and Vinchos - Roads Completed onnected to Logistic Corridors Source: Pictures provided by PVD as part of the data collection. Page 58 of 58