Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) Report Number: ICRR0024274 1. Project Data Project ID Project Name P132515 PE Support to the Subnational Transport Country Practice Area(Lead) Peru Transport L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-85620 31-Mar-2021 34,273,312.66 Bank Approval Date Closing Date (Actual) 11-Dec-2015 31-Dec-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 50,000,000.00 0.00 Revised Commitment 34,273,312.66 0.00 Actual 34,273,312.66 0.00 Prepared by Reviewed by ICR Review Coordinator Group Vibecke Dixon Fernando Manibog Avjeet Singh IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Financing Agreement (Schedule 1, page 6), the Project Development Objective (PDO) was “to (i) facilitate sustainable road access of the Borrower’s rural populations to services; (ii) reduce transport costs on rural roads linked to priority logistics corridors; and (iii) strengthen decentralized road management”. The PDO in the PAD (para 18, page 6) was nearly identical, only changing “the Borrower” for “Peru”; “The Project development objectives are to (a) facilitate sustainable road access of Peru’s rural populations to services, (b) reduce transport costs on rural roads linked to priority logistics corridors, and (c) strengthen Page 1 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) decentralized road management”. For the purpose of the analysis in this ICRR, the PDO will be parsed as follows: 1. To facilitate sustainable access of Peru’s rural populations to services 2. To reduce transport costs on rural roads linked to priority logistics corridors 3. To strengthen decentralized road management b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? Yes d. Components The Subnational Transport Support Program (Programa de Apoyo al Transporte Subnacional, PATS) was designed to support the Peruvian Government’s trade and social inclusion agenda. The purpose of the PATS was to invest in rural roads that link the poorest and most remote districts in the country, and support physical, institutional, and regulatory investments to improve transport on selected roads feeding into logistics corridors. The PATS built on previous programs and introduced a stronger focus on road safety, socio-environmental compliance, and gender. Beyond the provision of reliable infrastructure, PATS was also expected to benefit the population by strengthening the technical capabilities of local governments (LGs) and road infrastructure institutions, including investment planning, while supporting the decentralization agenda. The PATS was designed as a US$600 million investment supported by US$500 million from the GoP and two loans of US$50 million each, one from the World Bank and the other from the IDB. The project had four components. The description of the components in the PAD encompassed the full investment of US$600 million, while noting the World Bank contribution under each of them, as follows: Component 1: Infrastructure for Integration and Social Inclusion (estimated total cost at appraisal: US$405.8 million, of which World Bank financing; US$35 million; actual cost US$18.37 million). This component included a focus on road safety, social and environmental compliance and climate change adaptation. For both subcomponents, the loan financed technical design and detailed engineering studies as well as execution of civil works. There were two subcomponents: i. Subcomponent 1.1: Infrastructure for Social Inclusion (estimated total cost at appraisal: US$169.7 million, of with World Bank financing was US$27.7 million, actual cost US$18.3 million). This subcomponent was to improve 1,100 km of rural roads to promote accessibility and social inclusion of the poorest areas of the country. ii. Subcomponent 1.2: Integration of the Feeder Network into Selected Logistic Corridors (estimated total cost at appraisal: US$236.1 million, of which World Bank financing: US$7.3 million, actual cost Page 2 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) US$0.24 million). This subcomponent was to improve 1,100 km of rural roads linked to five selected logistic corridors. Component 2: Rural Road Infrastructure Maintenance (estimated total cost at appraisal: US$151.9 million, of which World Bank financing US$10 million, actual cost: US$10.12 million). This component was to provide support for the efficient and sustainable decentralized maintenance of the rural road network, through a) technical design for periodic and routine maintenance contracts; b) centralized and decentralized execution of civil works for periodic and routine maintenance of about 5,000 km of rural roads; and c) supervision of related civil works. Component 3: Decentralized Road Management (estimated total cost at appraisal: US$26.7 million, of which World Bank financing US$4.65 million, actual cost US$5.78 million). Component activities included institutional strengthening of the Rural Roads Agency Provías Decentralizado (PVD) and subnational level institutions in road transport management, capacity building program for microenterprises to carry out routine maintenance; the design and implementation of a Local and Logistics Development Window; and program M&E including implementation of an information and reporting system. Component 4: Project Management (estimated total cost at appraisal; US$15.7 million, of which World Bank financing US$0.35 million, actual cost US$0 million). This component was to cover the project’s incremental administrative costs, including staff, audits, information dissemination, program activities related to training, and operational support through consulting services. The only activity to be financed by the World Bank under this component was an external audit and the cost was to be shared with IDB. Restructurings The project underwent 5 restructurings (the ICR does not specify whether these were level 1 or level 2 restructurings), with the following key revisions: R1 March 10th, 2021; Change in Results Framework, in Components and Cost, in Loan Closing Date, in Safeguard Policies Triggered, in Legal Covenants, in Institutional Arrangements and in Implementations Schedule. R2 June 30th, 2021; Change in Results Framework, in Components and Cost, in Institutional Arrangements, in Implementations Schedule and Reallocation between Disbursement Categories. This restructuring reduced the measured and reported scope to what the World Bank was effectively financing in the Project. R3 April 4th, 2022; Change in Components and Cost, in Loan Closing Date, in Procurement and in Implementation Schedule. R4 February 24th, 2023; Change in Loan Closing Date. R5 May 24th, 2023; Change in Results Framework, in Components and Cost, in Safeguard Policies Triggered, in Implementation Schedule, Cancellation of Financing and Reallocation between Disbursements Categories. This restructuring reduced some of the indicator targets further. The Results Framework (RF) in the Project Appraisal Document (PAD) originally encompassed the whole PATS Program scope, relative to the US$600 million investment, despite the World Bank contributing to Page 3 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) only 8.3 percent of the total investment. The PATS aimed to rehabilitate/improve 2,200 km of rural roads; maintain, either through periodic or routine maintenance, 5,000 km of roads; and deepen the decentralization agenda. It became clear in 2021, however, that each financier was implementing its project independently, and the World Bank carried out a restructuring in 2021 that split the World Bank investments (US$50 m) from the overall PATS. As a result, while the original Project Development Objective (PDO) was maintained, the 2021 restructuring reduced the Project scope to what the World Bank was effectively financing. Revised components The component descriptions were revised in the second restructuring, as follows: Component 1: Infrastructure for Integration and Social Inclusion (R2 revised cost: US$36.28 million, R3 revised cost US$35.46 million, R5 revised cost US$ 21.20 million, actual cost US$18.37 million) i. Subcomponent 1.1: centralized execution of civil works for rehabilitation and improvement selected works for this subcomponent, including road safety materials and related road users training. ii. Subcomponent 1.2: Improving the transitability of rural roads linked to and/or feeding into logistic corridors as described in the MOP, through (i) centralized technical design and detailed engineering studies for roads rehabilitation and improvement; (ii) implementation of safeguards mitigation measures as needed for the civil works in this subcomponent. Component 2: Rural Road Infrastructure Maintenance (R2 revised cost US$10 million, R3 revised cost US$10.08 million, R5 revised cost US$10.5 million, actual cost US$10.12 million) Provision of support for the efficient and sustainable maintenance of the rural road network, through: i. technical design for periodic and routine maintenance contracts; ii. centralized execution of civil works for periodic maintenance selected works; and iii. supervision of related civil works. Component 3: Decentralized Road Management (R2 revised cost US$3.72 million, R3 revised cost US$4.46 million, R5 revised cost US$5.5 million, actual cost US$5.78 million) Provision of support for the efficient and sustainable decentralized transport management agenda, through: i. the strengthening of the capacity at the subnational (Regions and Municipalities) in road transport management; ii. the institutional strengthening of PVD; iii. the development and implementation of a capacity building program for routine maintenance; iv. the design and implementation of a Local Development Window; and v. Program monitoring and evaluation. Component 4: This component for Project Management was dropped. World Bank resources under this component were initially envisioned to cover only the financial audit. The indicators and indicator targets were also changed and reduced;  The original PDO indicator “Share of rural population with access to an all-season road” (target 51.5%) was replaced by “People with access to an all-season road financed by the project” (target 30,220. This target was further reduced in the 5th restructuring (R5) to 24,730 people). Page 4 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515)  The original PDO indicator “Reduced truck transit time from production to distribution on feeder roads integrated to logistics corridors” (target 25%) was replaced by “Change in the vehicle travel time on rural roads financed by the project (target 50%).  The original PDO indicator “Vehicle operating cost for trucks” (target US$0.42 from a baseline of US$0.79) was replaced with “Vehicle operating cost for trucks on rural roads with access to logistics corridors financed by the project (target US$0.48, revised baseline US$0.59). This revised indicator selectively measures successful instances of project-financed logistics corridors and ignores cases where access to project-financed logistics corridors failed or was negligible.  The Original PDO indicator “Roads in good and fair condition as a share of total classified roads” (target 60%, from a baseline of 55%) was replaced by “Percentage of local governments with a fair or good institutional capacity in road management (target: 30%). This revised indicator is vague and replaced an original indicator that was specific and measurable. A detailed overview of corresponding changes in component costs is provided in table 3 on page 12 of the ICR. Split evaluation A double split evaluation will be done based on the changes introduced in the 2nd and 5th restructurings, as only these two restructurings entailed significant changes to the Results Framework and reductions in targets. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Cost The planned total project cost was US$600 million, of which the IBRD contribution was planned to be US$50 million. Actual IBRD cost was US$34.72 million. The final total cost of the project is not reported, as the ICR only reports on the WB share of the project. The Project and component financing shares were adjusted as shown above in the description of the revised components. Further, the R5 cancelled US$12.8 million of the World Bank loan (25.6 percent), and US$2.9 million was reimbursed after the closing date. Financing The project was cofinanced with the Government of Peru (GoP) and the Inter-American Development Bank (IBD), where the GoP’s share was US$500 million, and the IDB and WB would contribute with US$50 million each. Borrower Contribution Borrower contribution at appraisal was US$500 million. Dates The project was approved on December 11th, 2015 and effective on July 8th, 2016. The Mid-Term-Review date was April 8th, 2019. The original closing date was March 31st, 2021 and the actual closing date was December 31st, 2023, i.e., 33 months’ extension (2 years and 9 months). The project was restructured 5 times, on the following dates: March 10th, 2021, June 30th, 2021, April 4th, 2022, February 24th, 2023 and May 24th, 2023. Page 5 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) 3. Relevance of Objectives Rationale Country Context At appraisal in 2015, despite a decade of strong economic performance, there were still persistent disparities between rural and urban areas in Peru. Between 2005 and 2014, the poverty rate was reduced from 55.6 to 22.7 percent, but the rural poverty rate in 2014 was more than 30 percentage points higher than in urban areas, at 46 percent. In 2011, the Government of Peru (GoP) launched an ambitious plan to overcome social inequalities. Its main objectives were to improve access to services, reduce extreme poverty, improve environmental monitoring, and enhance the connectivity of rural areas. Alignment with Government priorities The PDO was well aligned with the GoP’s Strategic Sector Plan 2012-2016 (SSP), which was launched by the Ministry of Transport and communications (MTC) with the aim to establish logistic platforms and corridors to facilitate trade and to promote social inclusion. The PDO was fully aligned with the Government’s strategy to (a) extend the rural roads network, (b) expand and modernize road corridors, (c) consolidate and modernize sector institutions, and (d) provide services and promote innovation and standardization of processes through enhanced capacity building within the sector. The PDO was also well aligned with Government priorities at closure. Peru’s Strategic Plan for National Development 2050 emphasizes the need to reduce inequalities and territorial disparities, and Peru’s National Infrastructure Plan for Competitiveness and Sustainability (2022–2025) focuses on the need to enhance the logistics sector through investments in the secondary road network. The implementation of PATS still continues and still has a strong focus on social inclusion. The design of the project also brought the concept of strengthening the logistic corridors taking into consideration the role that the tertiary network plays in the development of value chains. Alignment with World Bank strategies The project spanned three World Bank Country Portfolio Strategy (CPS)/Country Portfolio Framework (CPF) cycles, and remained relevant to the World Bank’s CPS/CPF strategies throughout implementation including at closure. At appraisal, the PDO was well aligned to the World Bank’s Country Partnership Strategy (CPS) for Peru FY12-16, in particular results area 2.2 concerning better transport and infrastructure to reduce inequality, under Strategic Objective 2: “Connecting the poor to services and markets”. The PDO remained relevant to the CPF FY17–FY21 under its Pillar 1:” Productivity for Growth”, and its Objective 1: “Improve connectivity at critical corridors of the territory”. Page 6 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) At closure, the PDO was well aligned to the CPF (FY23–FY27) Objective 3: “Enhance the delivery of public services” under the High-Level Outcome 2 of “Improved access to quality public services”. In addition, PDO 3, “To strengthen decentralized road management” remained relevant to the objective set under Objective 4: “Strengthen institutional effectiveness at national and subnational levels”. Previous World Bank Experience The PATS, like previous rural road programs in Peru over the past 20 years, benefited from World Bank and the Inter-American Development Bank (IDB) support in its design and financing. The World Bank had a long-term collaboration with the Government of Peru in the transport sector, which allowed for the development of innovative pilots that were progressively expanded over time, such as pairing of road improvement with rural development activities, the promotion of decentralization, more in-depth citizen engagement, community-based rural road maintenance, and the advance of the micro-enterprise model, with a strong focus on gender. Level of PDO The three PDOs were pitched at a intermediate outcome level (facilitate access to services and reduce transport costs, strengthen decentralized road management). The Theory of Change (ToC) and the narrative in the PAD highlighted that the expected higher level objectives would include improved agricultural livelihoods, increased agricultural income, increased competitiveness and improved economy. Summary The PDOs were well aligned with both national and World Bank strategies and priorities throughout the project period, including at closure. The Relevance is therefore rated High. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To facilitate sustainable access of Peru’s rural populations to services Rationale Theory of Change The PAD did not include an explicit Theory of Change (ToC), as that was not a requirement at the time. The ICR provides a reconstructed ToC. The project’s inputs such as technical design and detailed engineering studies for roads rehabilitation; rehabilitation and improvement of rural road for social inclusion; rehabilitation Page 7 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) and improvement of rural road connected to logistics corridors; technical design for periodic and routine maintenance contracts; works for routine maintenance; and civil works for periodic maintenance were expected to lead to outputs such as improved rural roads for social inclusion in the poorest rural areas, and rural roads under routine and periodic maintenance, both with road safety, gender and E&S considerations. These outputs were in turn expected to lead to the outcome level result of Peru’s rural population’s sustainable road access to services. The critical assumptions included a) all parties’ financing would be readily available; b) the implementing agency’s capacity and personnel would be maintained throughout the project; c) political stability would be maintained; and that d) no major climate-induced disaster would occur in the project areas. The ToC is clear and convincing and does not contain any logical gaps. Outputs The following reflect the original output indicators with the original target values. Since these were replaced at the second restructuring, there is no data on achievements against these targets. Output level targets under this objective were thus not achieved.  Direct project beneficiaries, target 1.1 million o Of which females, target 51%  Rural roads rehabilitated, target 2,200 km (The cumulative length of all roads reopened to motorized traffic, rehabilitated, or upgraded under the project).  Length of roads under routine and periodic maintenance, target 5,000 km Outcomes The following reflect the original outcome indicators with the original target values. Since these were replaced at the second restructuring, there is limited data on achievements against these targets. Outcome level targets under this objective were thus not achieved.  Share of rural population with access to an all-season road, target 51.5% (Proportion of rural people in the program area who live within 2 km (20-minute walk) of an all-season road)  Reduced truck transit time from production to distribution centers on feeder roads integrated to logistics corridors, target 25%. (At appraisal it took 2 hours to drive an average distance of 25 km or about 13 km/h (2015). Improved road condition was expected to reduce time by a minimum of 30 minutes, that is, improving the distance coverage to about 17 km/h).  Roads in good and fair condition as a share of total classified roads, target 60%. (The percentage of the total classified rural road network in the program area that is in good and fair condition depending on the road surface and the level of roughness). Page 8 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) There is insufficient information to report on the original targets, i.e., achievements against these targets are not documented and the rating of this objective is therefore Negligible. Rating Negligible OBJECTIVE 1 REVISION 1 Revised Objective To facilitate sustainable access of Peru’s rural populations to services Revised Rationale Theory of Change The revisions to the indicators and targets did not affect the ToC. Outputs  57.96 km roads rehabilitated financed by the project, not achieving the R2 target of 172 km. The target was significantly reduced from the original 2,200 km. The original target values reflected the targets for the whole program, while the reduced targets reflected only the WB-financed part of the program.  190 km under periodic maintenance financed by the project, achieving the R2 target of 190 km. The target was significantly reduced from the original 5,000 km. The original target values reflected the targets for the whole program, while the reduced targets reflected only the WB-financed part of the program. Outcomes The ICR reported the following achievements at outcome level:  20,070 people with access to an all-season road financed by the project, almost achieving the R2 target of 20,220 people.  Vehicle travel time was reduced with 49.9%, achieving the R2 target of 50%.  There were 124,340 direct and indirect project beneficiaries of roads financed by the project, not achieving the R2 target of 152,520 beneficiaries. o 49.89% of which were female, almost achieving the R2 target of 51%  65% of beneficiaries felt that the project investments reflected their needs, almost achieving the R2 target of 70% beneficiary satisfaction. While the project did achieve the R2 target for roads under periodic maintenance and managed to reduce travel time by almost half (nearly achieving the R2 target), it was far from achieving R2 targets for kilometers Page 9 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) of roads rehabilitated, and did not fully reach the R2 targeted beneficiary numbers. Achievement of Objective 1 Revision 1, to facilitate access of Peru’s rural population to services is therefore rated Modest. Revised Rating Modest OBJECTIVE 1 REVISION 2 Revised Objective To facilitate sustainable access of Peru’s rural populations to services Revised Rationale Theory of Change The revisions to the indicators and targets did not affect the ToC. The results achieved under Objective 1 Revision 1 are also valid under this revision. The achievements compared to indicators with reduced targets (R5) were as follows: Outputs  57.96 km roads rehabilitated financed by the project, not achieving the R5 target of 102.57 km. Outcomes  There were 20,070 people with access to an all-season road financed by the project, not achieving the R5 target of 24,730 people.  There were 124,340 direct and indirect project beneficiaries of roads financed by the project, almost achieving the R5 target of 128,780 beneficiaries. o 49.89% of which were female, almost achieving the R5 target of 49.97% Taking into consideration the results achieved under R2 which are also valid for R5, the project did achieve the target for roads under periodic maintenance and managed to reduce travel time by almost half (nearly achieving the target), it was far from achieving R5 targets for kilometers of roads rehabilitated, and did not fully reach the R5 targeted beneficiary numbers. Achievement of Objective 1 Revision 2, to facilitate access of Peru’s rural population to services is therefore rated Modest. Page 10 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) Revised Rating Modest OBJECTIVE 2 Objective To reduce transport costs on rural roads linked to priority logistics corridors Rationale Theory of Change The project’s inputs such as technical design and detailed engineering studies for roads rehabilitation; rehabilitation and improvement of rural road for social inclusion; rehabilitation and improvement of rural road connected to logistics corridors; technical design for periodic and routine maintenance contracts; works for routine maintenance; and civil works for periodic maintenance were expected to lead to outputs such as improved rural roads for social inclusion in the poorest rural areas, improved feeder rural roads connecting with logistic corridors, rural roads under routine and periodic maintenance, all three with improved road safety, gender and E&S considerations; technical studies for rural road rehabilitation, and improved level of service maintenance contracts. These outputs were in turn expected to lead to the outcome of reduced transport costs on rural roads linked to priority logistics corridors. The critical assumptions are the same as those listed under the original objective 1. The ToC is clear and convincing and contains no logical gaps. Outputs No output indicator was identified under this objective. Outcomes The following reflects the original outcome indicator with the original target value. Since this was replaced at the second restructuring, there is no data on achievement against this target. The outcome level target under this objective was thus not achieved.  Vehicle operating costs for trucks, target 0.42 US$ (baseline 0.79 US$) (The baseline and target vehicle operating costs were based on information provided in the PATS economic analysis. It is calculated as the weighted average for two-axle trucks moving on the rural roads linked to logistics corridors.) There is insufficient information to report on the original targets, i.e., achievements against these targets are not documented and the rating of this objective is therefore Negligible. Page 11 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) Rating Negligible OBJECTIVE 2 REVISION 1 Revised Objective To reduce transport costs on rural roads linked to priority logistics corridors Revised Rationale Theory of Change The revisions to the indicators and targets did not affect the ToC. Outputs Vehicle operating cost for trucks on rural roads with access to logistics corridors financed by the project increased from the baseline of US$0.59 to US$0.85, not achieving the R2 target of US$0.48. Outcomes No outcome indicator was identified under this revised objective. The target was not achieved, as vehicle cost increased instead of decreased under the project. Hence the achievement of the objective is negligible. Revised Rating Negligible OBJECTIVE 2 REVISION 2 Revised Objective To reduce transport costs on rural roads linked to priority logistics corridors Revised Rationale The indicator related to this objective was not revised in R5, achievement is therefore the same as under the 2nd restructuring, i.e., negligible. Revised Rating Page 12 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) Negligible OBJECTIVE 3 Objective To strengthen decentralized road management Rationale Theory of Change Inputs such as institutional strengthening of the Rural Roads Agency PVD, capacity building for regional and provincial governments, design and implementation of Local and Logistics Development Windows (LLDW); and M&E and implementation of information and reporting system were expected to lead to outputs such as projects developed and approved under LLDWs; stronger capacity of provincial governments to execute multiyear road programs; and microenterprises certified to perform routine maintenance services. These outputs were in turn expected to lead to the outcome of strengthened decentralized road management. The critical assumptions are the same as those listed under the original objective 1. The ToC is clear and convincing and contains no logical gaps. Outputs The following reflect the original output indicators with the original target values. Since these were replaced at the second restructuring, there is limited data on achievements against these targets. Outcome level targets under this objective were thus not achieved.  Number of projects approved under LLDWs, target 24 (Component 3) (24 provinces with their local economic development plan elaborated and approved, i.e., 12 Local Development Windows and 12 Logistic Development Windows)  Percentage of Province Governments included in the program that have executed 80% of the multiyear road program, target 80% (From the 37 provinces included in the PATS, Component 3).  Number of microenterprises certified to perform maintenance services, target 200 (Number of microenterprises trained and certified for maintenance services in a program designed with universities). Outcomes No outcome indicator was identified for this objective. There is insufficient information to report on achievements against the original targets, i.e., achievements against these targets are not documented and the rating of this objective is therefore Negligible. Rating Page 13 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) Negligible OBJECTIVE 3 REVISION 1 Revised Objective To strengthen decentralized road management Revised Rationale Theory of Change The revisions to the indicators and targets did not affect the ToC. Outputs  24 Local Economic Development Plans (PDEL) were delivered to Local Governments financed by the project, achieving the R2 target of 24.  15 Participatory Provincial Road Plans (PVPP) implemented under the program and financed by the project, almost achieving the R2 target of 16. Outcomes 59% of local governments had fair or good institutional capacity in road management, exceeding the R2 target of 30%. The project achieved and almost achieved the R2 output targets under this objective, and exceeded the R2 outcome target of percentage of local governments with fair or good institutional capacity in road management, thus substantially achieving the objective of strengthened decentralized road management Revised Rating Substantial OBJECTIVE 3 REVISION 2 Revised Objective To strengthen decentralized road management Revised Rationale None of the indicators related to this objective was revised in R5. Achievement is therefore the same as under the 2nd restructuring. Achievement of this objective is thus Substantial. Revised Rating Page 14 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale All three original objectives were negligibly achieved, as the indicators were replaced in the second restructuring and thus no relevant data was available at closure to compare achievements to original indicator targets. Original overall efficacy is thus Negligible. Some of the project difficulties were due to delays in budget allocation from the Ministry of Economic and Finance (MEF) which caused implementation delays. Furthermore, procurement processes started late, only two years into implementation. There was a need for constant training of staff due to very high turnover which also made it difficult to accumulate knowledge and experience within the PIU. This also led to several unsuccessful bidding processes. It furthermore turned out to be difficult to identify contractors/consultants who would comply with the requirements established. Overall Efficacy Rating Primary Reason Negligible Insufficient evidence OBJR1_TBL OVERALL EFFICACY REVISION 1 Overall Efficacy Revision 1 Rationale Under Revision 1 (i.e., R2), the first objective to facilitate access of Peru’s rural population to services was rated Modest due to low achievement. The second objective to reduce transport costs was rated negligible as vehicle costs increased under the project, and the third objective to strengthen decentralized road management was rated Substantial as the project almost achieved, achieved and exceeded the R2 targets under this objective. With Modest, Negligible and Substantial ratings of the three objectives, the Overall Efficacy rating for Revision 1 is rated Modest. Overall Efficacy Revision 1 Rating Primary Reason Modest Low achievement OBJR2_TBL OVERALL EFFICACY REVISION 2 Overall Efficacy Revision 2 Rationale Under Revision 2 (i.e., R5), the first objective to facilitate access of Peru’s rural population to services was rated Modest because while the project did achieve the target for roads under periodic maintenance and managed to reduce travel time by almost half (nearly achieving the target), it was far from achieving R5 targets for kilometers of roads rehabilitated, and did not fully reach the R5 targeted beneficiary numbers. The second objective to reduce transport costs was rated negligible as vehicle costs increased under the project, and the third objective to strengthen decentralized road management was rated Substantial as there were no Page 15 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) further revisions to the related indicator targets, so the achievements and rating remain the same as for Revision 1, i.e., Substantial. With Modest, Negligible and Substantial ratings of the three objectives, the Overall Efficacy of Revision 2 is Modest. Overall Efficacy Revision 2 Rating Primary Reason Modest Low achievement 5. Efficiency Economic Efficiency The ex post economic analysis of the Project done for the ICR replicated the analysis done during preparation. The economic analysis at appraisal encompassed the whole project and the results showed that over 10 years, the net present value (NPV) at a 9 percent discount rate was estimated to be US$271.4 million and the related internal rate of return (IRR) was estimated to be 16.1 percent (PAD p 14 and PAD Annex 6). This was estimated for the whole project, and not only for the WB financed part of the project. Hence, a new economic analysis was done at the second restructuring with estimations covering only the WB financed part of the program. Pre-and post- comparison of the figures will thus take the R2 values as pre-project values. The Project used cost-benefit analysis (CBA) for the economic analysis of the original Program activities focused on Components 1 and 2, which covered 95 percent of the overall investment, and 90 percent of the World Bank loan. The economic analysis done at the second restructuring estimated an internal rate of return (IRR) at 16.1 percent and the net present value (NPV) at a 9 percent discount rate, to be US$11.9 million. At completion, an ex-post analysis was conducted for the roads completed, using the Roads Economic Decision Model Version 4.00 (RED). Data (roadworks cost and maintenance cost, traffic volume) were provided by the implementing agency reports based on the most updated information available at the time of completion. The Project’s discounted net worth at completion had an estimated internal rate of return (IRR) of 14.4 percent and the net present value (NPV) at a 9 percent discount rate of US$7.7 million, as outlined in Table 9 on page of the ICR. The ICR states (p 20) that the changes in NPV and IRR values at completion stage could be attributed to changes in traffic volume and assumptions on vehicle operation costs. This shows a modest economic efficiency. Administrative efficiency The project experienced an almost two-year delay between the development and approval of the methodology for road selection and the completion of the portfolio of interventions. The methodology, in particular, was affected by the changes in management within the Rural Roads Agency (PVD). The project’s efficiency was negatively affected by the lack of clarity on the responsibilities of co-financiers and on the processes of approval, mainly regarding the safeguard instruments. One of the key results of R2 was that the processes for approval and no-objection were clarified, as, for example, issues with safeguards, as well as aspects of procurement and financial management, were dealt with the corresponding co-financier. However, R2 only occurred in 2021, after Page 16 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) the Project had been going on for almost five years. The three project extensions to the closing date allowed a larger share of disbursement, but by R2, 70 percent of the total amount had already been disbursed. Efficiency is rated Modest due to the project’s marginal economic efficiency and the delays and substantial shortcomings in administrative efficiency Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance is rated High, the Original Efficacy is Negligible, while the R2 and R5 Efficacy both are rated Modest, and Efficiency is rated Modest. This gives an overall Outcome rating of Unsatisfactory. Original Objective 2nd Restructuring (R2) 5th Restructuring (R5) Relevance High Objective 1 Negligible Modest Modest Objective 2 Negligible Negligible Negligible Objective 3 Negligible Substantial Substantial Efficacy Negligible Modest Modest Efficiency Modest Moderately Moderately Outcome rating Unsatisfactory Unsatisfactory Unsatisfactory Rating value 2 3 3 Disbursements US$ 23.85 7.62 2.8 Share of disbursement 0.70 0.22 0.08 Weighted value of 1.40 0.66 0.24 outcome rating Overall outcome rating 1.40+0.66+0.24=2.3, rounded down to 2 = Unsatisfactory Page 17 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) The overall outcome rating is thus Unsatisfactory. a. Outcome Rating Unsatisfactory 7. Risk to Development Outcome Political Risk: The project’s implementation period was characterized by a climate of political instability in Peru, with widespread public demonstrations and corruption scandals revealed at different Government levels. There is a risk that the political instability may continue and thus potentially affect the government’s commitment to ensuring sustained development outcomes of this project. Environmental Risk: Some of the Project areas, including those where World Bank interventions were financed, suffered regular landslides and heavy rains that prevented works from advancing or being completed. The frequent naturally induced impacts also led to the need to amend the works contracts, to allow the contractors to deal with the aftermath. In most cases, civil works contractors had to stop operations for approximately four months a year due to the rainy season. There is thus a significant environmental risk to the sustainability of the project outcomes. Sustainability Risk: During the collection of data for the ICR, it became apparent that some of the roads were already deteriorating and that no plan for maintenance of the roads had been put in place. There is thus a significant risk to the sustainability of the project outcomes. 8. Assessment of Bank Performance a. Quality-at-Entry While the design of the project drew on lessons learned and insights gained from decades of efforts in the rural road sector, partnership with the IDB, and support to the GoP and the Project was fully aligned with the priorities of the World Bank and the GoP in terms of investments in infrastructure in the rural roads sector, a significant shortcoming was the misalignment between the Results Framework covering the entire program and the project interventions which were restricted to specific works. The design lacked the means to incentivize actions beyond the World Bank financing to achieve the PDO. A clearly defined set of roads to include into the project was not set at entry. Changes in government and lack of coordination between the relevant institutions generated delays in the selection of roads. The PAD specified that the World Bank and IDB would supervise the overall Program’s safeguards but the level of responsibility for each safeguards’ team regarding supervision and approval of documents was not defined. This ambiguity led to an excessive workload and delayed response time from the World Bank, which affected project progress. Page 18 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) Despite the frequent climate-induced hazards affecting the rural road network in Peru, the project did not include disaster risk management mechanisms or specific measures to enhance the infrastructure’s resilience in its design. This omission is another a significant shortcoming, as the project missed the opportunity to ensure the long-term sustainability of the road network and to facilitate a strategy to make rural road infrastructure more resilient to climate and natural induced hazards. Issues of climate-induced hazards were addressed through thorough technical assistance during implementation. The related recommendations were not implemented, however, because it encompassed an increase in the cost of the roads. Due to these significant shortcomings in design, quality at entry is rated Moderately Unsatisfactory. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision The Task Team conducted 13 Supervision Missions during the 7 years of the project’s life, i.e., two supervision missions annually except from in 2022, when only one mission was conducted. The World Bank provided continuous and adequate support and training to the client throughout the implementation. When the need was identified, the World Bank involved high-level specialists like engineers and procurement-, environmental-, and social specialists. There was significant turnover of staff which affected the implementation and supervision of the activities. Issues related to World Bank safeguards were a recurrent source of delay and misunderstandings despite significant support and training. The Task Team missed the opportunity to revise the PDO based on the information that no works were carried out under Subcomponent 1.2. and to more accurately reflect the Development Objectives of the World-Bank financed project. PDO 2 was thus not achieved and based on the available information on the actual nature of the roads, this non-achievement was foreseeable. Other changes during restructurings, such as extending the closing date, revising the indicators and reducing the target values, were relevant. During the data collection to measure the PDO indicators at closing, it became apparent that roads completed under the project had not received adequate maintenance, and that their condition was suboptimal. The PAD envisioned that roads financed through the project should have received routine maintenance, and the World Bank requested PVD, during missions, to develop a strategy for maintenance. This was not developed during the course of the project. The lengthy process to develop and approve technical road documents was also a cause of delay. Due to these significant shortcomings, the Quality of Supervision is rated Moderately Unsatisfactory Page 19 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Moderately Unsatisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design While the M&E design at appraisal was relatively simple, it was designed to track the results of the overall program and did not focus on the WB financed part of the operation. This was also reflected in the Results Framework (RF). The indicators included in the PAD were expected to be enriched with additional indicators collected as part of the Program’s M&E, including indicators to be collected through a planned Impact Evaluation (IE). The M&E office of the Rural Roads Agency (PVD) was to be responsible for the monitoring and to capture all the indicators for each co-financier, and to establish additional indicators during implementation. The original PDO indicators followed relevant best practices at the time to measure the outcomes of the Project. At the time of appraisal, measuring the Rural Access Index and the Vehicle Operating Cost was the standard in rural roads projects. However, it is now understood that such indicators must be better customized to be practical. For the indicator on VOC, PVD lacked the tools to measure input data and this added a layer of difficulty when evaluating project outcomes at closing. b. M&E Implementation The original M&E design was not able to track the outputs and outcomes related to the World Bank part of the operation as all the original indicators and targets in the Results Framework took the whole Project into account. The indicators were revised and targets reduced in the Second Restructuring (R2) to more accurately reflect the expected outcomes from the World Bank financed part of the operation. The targets were further reduced in the Fifth Restructuring (R5). The implementing agency did not report on indicator progress prior to the MTR in 2019. This absence of data posed a challenge for enhancing the indicators during R2, as the task team lacked concrete evidence to assess the effectiveness of the indicators and to set realistic targets. The implementing agency, PVD, started the planning for collecting data on the final outcomes at least six months before the closing date of the Project. There were, however, some issues with the hiring of consultants which were exacerbated by unclarities regarding the methodologies to collect data and measure the indicators. The indicator on decentralization lagged by at least one year and the latest information available was from 2022, and the indicator on citizen engagement was vague and subject to a variety of interpretations. The data collection of PDO indicators revealed that PVD lacked the equipment and resources to collect the information more regularly. PVD reported on the progress of the intermediate indicators through the progress reports submitted every six months and in virtual meetings. The RF lacked relevant indicators to appropriately measure the expected outcomes of the project such as for example indicators that measure sustainability, i.e., the condition of the roads completed or the incorporation of these roads into a routine maintenance scheme. Having such indicators would have Page 20 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) been beneficial, as visual evidence and interviews confirmed, according to the ICR (page 26) that some roads were already deteriorating at the time of the ICR. The planned Impact Evaluation did not materialize beyond a baseline study which was only finalized in June 2022. c. M&E Utilization The RF’s intermediate outcome indicators were used by both the World Bank and the implementing agency to assess the project progress, especially after the revisions to the RF in R2 and R5, where the revised indicators mostly focused on specific outputs that were easily measurable. However, the indicator on citizen engagement was insufficiently utilized, as it was not a regular practice of PVD to collect this type of information. M&E Quality is rated Modest as there were significant shortcomings to the M&E design and implementation M&E Quality Rating Modest 10. Other Issues a. Safeguards The Project was classified as category B, as it was not expected to have significant or irreversible impacts or risks, and it triggered the following safeguard policies; Environmental Assessment (OP 4.01), Natural Habitats (OP 4.04), Forests (OP 4.36), Physical and Cultural Resources (OP 4.11), Indigenous Peoples (OP 4.10), and Involuntary Resettlement (OP 4.12). In compliance with OP 4.01, Environmental Impact Assessments (EIAs) were conducted for each subproject. An Indigenous Peoples Planning Framework (IPPF) was established, leading to the development of an Indigenous Peoples Plan (IPP) for subprojects involving indigenous communities. Additionally, a Resettlement Policy Framework (RPF) was developed for the entire Project, to cover OP 4.12, and was followed by the preparation of several Resettlement Action Plans (RAPs) for each subproject. Compliance with safeguards policies was hindered by the client’s high staff turnover (PVD had more than 10 Executive Directors over the course of 2021–22, and the PIU Coordinator was sometimes changed along with the Executive Director), the low quality of reports produced by consultants, and the limited understanding of the World Bank safeguards. The quality of the studies required for compliance with OP 4.10 and OP 4.12 was subpar, and the need for frequent revisions caused substantial delays. Additionally, there was initial resistance from PVD to comply with the World Bank safeguards, due to the gaps with the national regulation, for example, on resettlement compensation, which contributed to further complications Page 21 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) and postponements in Project execution. Moreover, some of the concepts in the approved terms of reference for the safeguard documents were sometimes interpreted in different manners, by the PIU or the consultants, which contributed to constant revisions and delays. The project closed with a Post-Closure Action Plan (PCAP). Works expanding beyond the project closing date, due to either poor contractor performance or climatic conditions, led to the current requirement for a PCAP for three subprojects. Two sites have unresolved issues related to OPs 4.10 and 4.12, and one site has pending matters concerning OP 4.12. The project implementation was thus not in compliance with the WB safeguard policies. b. Fiduciary Compliance Financial Management The financial management risk was considered to be Moderate at approval, as the implementing agency had qualified and experienced staff in working with World Bank-funded projects, and as the World Bank financing was only to be used for activities at a centralized level. During implementation, there was an issue with budget allocation from the Ministry of Economic and Finance (MEF). This caused a five-month delay which prevented the initiation of any procurement process, including the hiring of consultants to strengthen the PIU. Overall, expenses were consistently documented, and interim financial reports (IFRs) were mostly submitted on time. The audits predominantly had clean opinions and were also submitted to the World Bank in a timely manner. Procurement Procurement processes financed with World Bank funding started only two years after the signing of the LA. The Implementing Agency recorded the information on Systematic Tracking of Exchanges in Procurement (STEP) throughout the Project; however, it needed constant reminder from the World Bank to update the system. There for a need for constant training of staff due to very high turnover which also made it difficult to accumulate knowledge and experience within the PIU. This also led to several unsuccessful bidding processes. It furthermore turned out to be difficult to identify contractors/consultants who would comply with the requirements established. c. Unintended impacts (Positive or Negative) None Page 22 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) d. Other None 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment With 77% of the funding disbursed prior to restructuring Moderately when development outcome was Outcome Unsatisfactory Unsatisfactory negligible, the overall outcome rating is Unsatisfactory. (See split rating table in Section 6). Moderately Moderately Bank Performance Unsatisfactory Unsatisfactory Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons The following lessons are based on the ones presented in the ICR, with some adjustment to language: If the project’s objective and PDO indicators are not well aligned to the activities financed by the World Bank and to what the Project can effectively leverage, there is a risk that the project will a) not be able to achieve the set targets (as they are irrelevant to the project) and b) will not be able to measure the outcomes, as was the case here. In the present case, the project was an Investment Project Financing (IPF) of limited amount aimed at supporting a much wider program, but the World Bank had limited leverage on program activities that were not directly connected to what it was financing. This resulted in a restructuring that substantially revised the M&E framework. A robust analysis that considers the implementing agency capacity and the contributions of World Bank technical and cross-support teams could ground expectations better and may contribute to higher readiness for implementation for infrastructure projects. The lack of an established methodology and identified interventions were the main source of implementation delays of this project. It made the Project vulnerable to political influence and hindered its readiness for implementation. To improve this, a transparent and participatory process for road selection, with clearly defined criteria based on objective attributes can be established at appraisal. This could ensure that interventions are identified and earmarked in a timely manner, reducing the risk of delays. Page 23 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) If a focus on climate change resilience is not ensured in the project design and implementation, there is a high probability that project roads may not be maintained and may deteriorate quickly. This is particularly important in countries like Peru that are highly vulnerable to frequent and intense climate-induced hazards and have difficult landscapes. Future project designs may focus on promoting and innovating in rural road maintenance schemes, enhancing the resilience of rural road infrastructure, and incorporating mechanisms for effective disaster risk management to ensure long-term sustainability of the network. Routine maintenance planning and funds may be included in the project design to ensure sustainability of the investments. Developing road-specific safeguard documents for the first round of investments at preparation may contribute to smoother safeguards processes in the cases where there are gaps between the World Bank safeguards and national law. Such discrepancies may cause confusion and delay implementation if a strategy to solve the controversies is not developed. This Project suffered several delays due to the implementation of the safeguards and the discrepancy between World Bank and national laws. Additionally, having a clear and documented process for approving subproject-required safeguard documents and terms of reference can help streamline the review and approval. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is well written and presents a good analysis including assessing the quality of evidence. Due to the weak M&E system though, there is lacking evidence to demonstrate significant outcomes from the project. The ICR is clear and candid with regards to the project’s challenges and shortcomings and provides a comprehensive narrative when elaborating on what did and did not work well. The report is internally consistent and generally follows the OPCS guidelines. One shortcoming is the excessive length of the report with its 31 pages, which is nearly double the recommended length stated in the guidelines. The quality of the ICR is thus rated Substantial. a. Quality of ICR Rating Substantial Page 24 of 25 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PE Support to the Subnational Transport (P132515) Page 25 of 25