Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005385 IMPLEMENTATION COMPLETION AND RESULTS REPORT Loan No. 8255-YF ON A LOAN IN THE AMOUNT OF EUR73.8 MILLION (US$ 100 MILLION EQUIVALENT) TO THE Republic of Serbia FOR THE SERBIA ROAD REHABILITATION AND SAFETY PROJECT May 27, 2021 Transport Global Practice Europe And Central Asia Region Official Use CURRENCY EQUIVALENTS (Exchange Rate Effective {Apr 05, 2021}) Currency Unit = Euro Serbian Dinar 1 = Euro 0.0085 US$ 1 = Euro 1.18 FISCAL YEAR January 1 - December 31 Regional Vice President: Anna M. Bjerde Country Director: Linda Van Gelder Regional Director: Lucio Monari Practice Manager: Karla Gonzalez Carvajal Task Team Leader(s): Murad Gurmeric, Svetlana Vukanovic ICR Main Contributor: Aliya Karakulova Official Use ABBREVIATIONS AND ACRONYMS ARAP Abbreviated Resettlement Action Plan COS Corridors of Serbia CPS Country Partnership Strategy CXHP Corridor X Highway Project DLI Disbursement-Linked Indicator EBRD European Bank for Reconstruction and Development EC European Commission ECA Europe and Central Asia EHS Environmental Health and Safety Guidelines EIB European Investment Bank EIRR Economic Internal Rate of Return EMP Environmental Management Plan ESIA Environmental and Social Impact Assessment EU European Union FM Financial Management GoS Government of Serbia HDM-4 Highway Development and Management Tool ICR Implementation Completion and Results Report IFR Interim Financial Report IRI International Roughness Index M&E Monitoring and Evaluation MoT Ministry of Transport NPV Net Present Value NRNRP National Road Network Rehabilitation Program PAC Project Audit Consultant PAD Project Appraisal Document PDO Project Development Objective PBMC Performance-based Maintenance Contract PERS Public Enterprise Roads of Serbia PIT Project Implementation Team RNC Road Network Condition RoW Right of Way RPF Resettlement Policy Framework RRSP Road Rehabilitation and Safety Project TEN-T Trans European Transport Network TRP Transport Rehabilitation Project VOC Vehicle Operating Cost WBG World Bank Group Official Use TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 10 II. OUTCOME .................................................................................................................... 13 A. RELEVANCE OF PDOs ............................................................................................................ 13 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 13 C. EFFICIENCY ........................................................................................................................... 18 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 20 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 20 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 22 A. KEY FACTORS DURING PREPARATION ................................................................................... 22 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 22 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 24 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 24 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 25 C. BANK PERFORMANCE ........................................................................................................... 29 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 30 V. LESSONS AND RECOMMENDATIONS ............................................................................. 30 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 33 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 40 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 42 ANNEX 4. EFFICIENCY ANALYSIS/EX-POST ECONOMIC ANALYSIS .......................................... 43 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 47 ANNEX 6. SUPPORTING DOCUMENTS (PHOTOS AND REPORTS)............................................ 48 ANNEX 7. RESULTS OF BENEFICIARY SURVEYS ...................................................................... 53 ANNEX 8. JOB CREATION UNDER THE PROJECT .................................................................... 58 ANNEX 9. DLI TARGETS (ORIGINAL, REVISED, AND ACHIEVED) .............................................. 60 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P127876 SERBIA ROAD REHABILITATION AND SAFETY PROJECT Country Financing Instrument Serbia Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Republic of Serbia Public Enterprise Roads of Serbia Project Development Objective (PDO) Original PDO The proposed PDO is to improve the condition and safety of the National Road Network for road users by supporting the Republic of Serbia in the implementation of the first phase of its National Road Network Rehabilitation Program. Page 1 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 100,000,000 76,428,862 63,201,425 IBRD-82550 Total 100,000,000 76,428,862 63,201,425 Non-World Bank Financing 0 0 0 Borrower/Recipient 157,480,051 149,699,233 26,664,558 EC: European Investment 135,525,000 112,050,324 62,748,069 Bank Total 293,005,051 261,749,557 89,412,627 Total Project Cost 393,005,051 338,178,419 152,614,052 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 26-Apr-2013 24-Oct-2014 13-Nov-2017 31-Oct-2018 31-Dec-2020 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 13-Jan-2015 0 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Disbursements Arrangements Change in Financial Management Change in Procurement Change in Implementation Schedule 26-Jun-2018 27.24 Change in Results Framework Change in Loan Closing Date(s) Cancellation of Financing Reallocation between Disbursement Categories Change in Disbursements Arrangements Change in Implementation Schedule 24-Apr-2020 58.67 Change in Loan Closing Date(s) Page 2 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Moderately Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 12-Oct-2013 Satisfactory Moderately Satisfactory 0 02 04-May-2014 Satisfactory Moderately Satisfactory 0 03 30-Nov-2014 Moderately Satisfactory Moderately Satisfactory 0 04 24-Jun-2015 Moderately Satisfactory Moderately Satisfactory 19.91 05 28-Dec-2015 Moderately Satisfactory Moderately Satisfactory 19.91 06 07-Jul-2016 Moderately Satisfactory Moderately Unsatisfactory 27.24 07 27-Feb-2017 Moderately Satisfactory Moderately Satisfactory 27.24 Moderately 08 11-Nov-2017 Moderately Unsatisfactory 27.24 Unsatisfactory Moderately 09 29-May-2018 Moderately Unsatisfactory 27.24 Unsatisfactory Moderately 10 05-Dec-2018 Moderately Satisfactory 27.24 Unsatisfactory 11 31-May-2019 Moderately Satisfactory Moderately Satisfactory 58.67 12 17-Dec-2019 Moderately Satisfactory Moderately Satisfactory 58.67 13 24-Jun-2020 Moderately Satisfactory Moderately Satisfactory 58.67 Page 3 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) SECTORS AND THEMES Sectors Major Sector/Sector (%) Transportation 100 Public Administration - Transportation 7 Rural and Inter-Urban Roads 47 Other Transportation 46 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Economic Policy 95 Trade 95 Trade Facilitation 95 Private Sector Development 1 Jobs 1 Job Creation 1 Public Sector Management 2 Public Administration 2 Administrative and Civil Service Reform 2 Urban and Rural Development 2 Urban Development 1 Urban Infrastructure and Service Delivery 1 Rural Development 1 Rural Infrastructure and service delivery 1 ADM STAFF Role At Approval At ICR Regional Vice President: Philippe H. Le Houerou Anna M. Bjerde Country Director: Ellen A. Goldstein Linda Van Gelder Page 4 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Director: Laszlo Lovei Lucio Monari Practice Manager: Juan Gaviria Karla Gonzalez Carvajal Murad Gurmeric, Svetlana Task Team Leader(s): Moustafa Baher El-Hefnawy Vukanovic ICR Contributing Author: Aliya Karakulova Page 5 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. At Project appraisal, Serbia, while undergoing a remarkable period of political change, was facing a double- dip economic recession. Following the global economic crisis and low GDP growth in 2010 and 2011, unemployment continued to be high reaching a peak level of 25.5 percent in April 2012. The main policy challenges were balancing growth-enhancing policies with fiscal consolidation. In March 2012 Serbia gained EU "candidate" status for membership, which reflected the significant progress Serbia had made in its reforms. 2. Serbia is at the crossroads of South-Eastern Europe; having an efficient transport network is essential for the country's economic growth. The international roads and railways crossing Serbia represent the shortest link between Western and South-Eastern Europe and Greece, Turkey, and further east. Corridor X of the Trans-European Network (TEN- T) connects Austria/Hungary, Slovenia/Croatia, Serbia and Bulgaria/North Macedonia/Greece. 3. At appraisal, the road network was a major asset for Serbia but connectivity and road conditions needed improvement to support economic growth. In 2012 the road network estimated to have had an asset value of US$13 billion, extending over 40,800 km, including about 9,500 km of national roads. Road transport was the dominant mode in Serbia, with a share of 80 percent of total cargo transport and about 74 percent of total passenger transport. The 2012-2013 Global Competitiveness Report ranked Serbia 122nd out of 144 countries on road quality. The poor quality of roads manifested itself in high vehicle operating costs and inadequate road safety, reducing Serbia's overall trade competitiveness. 4. At appraisal, the Ministry of Transport (MoT) was responsible for policy, while the Pubic Enterprise for State Road Management (Public Enterprise "Roads of Serbia" or PERS) was responsible for construction, maintenance, operation and management of national roads. In 2013 MoT initiated a National Road Network Rehabilitation Program (NRNRP) to improve the quality and safety on priority national roads thus improving the connectivity of the entire road network. With the support of the EU, MoT had developed a Transport Strategy and Master Plan that envisaged a €5 billion investment in maintenance and rehabilitation up to 2027. Over 50 percent of the national road network was identified as a high priority; and financial support from several IFIs was sought to implement the rehabilitation works. 5. The Serbia Road Rehabilitation and Safety Project (RRSP) was a part of a larger IFI coordinated effort to support Serbia in the implementation of the first phase of NRNRP. The first phase of NRNRP was co-financed by the Government of Serbia (GoS) and three International Financial Institutions (IFIs), namely the WB, European Investment Bank (EIB), and the European Bank for Reconstruction and Development (EBRD), as summarized in Table 1. The first phase of NRNRP refers to two projects: (i) the project co-financed by GoS, EIB, and WB (the Project of this ICR or RRSP) with two closing dates – one for WB (December 31, 2020) and the other one for EIB (June 2022), and (ii) the project financed in parallel solely by EBRD (the EBRD Project). The Republic of Serbia signed the financial agreements with all three IFIs on November 27, 2013, and on June 19, 2014 all three agreements were ratified. Page 6 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Table 1. The First Phase of NRNRP’s Initial Sources of Financing and Implementation Structure Loan amount, EUR, Source of Financing % of Total Cost M The Republic of Serbia 116.20 29.80 The European Investment 100.00 25.64 Bank The World Bank 73.80 18.92 The European Bank for Reconstruction and 100.00 25.64 Development TOTAL 390.00 100.00 6. RRSP built on the achievements and the strong engagement of the IFIs in the sector over the past decades. Nearly half of EIB's total lending to Serbia of €3.1 billion between 2001 and 2010 period has been to upgrade the road network. EBRD has been the country's second-largest investor since 2001 committing almost €2.5 billion to the country with transport infrastructure being the main focus area. The World Bank Group provided about USD479 million to the roads sector of Serbia through the Transport Rehabilitation Project (TRP/P075207) and Corridor X Highway Project (CXHP/P108005). TRP (approved in May 2004 and closed in Sept 2012) assisted the GoS in reducing cost and improving safety for road users through (a) improving the condition of selected road sections of the main and regional road network; (b) improving the effectiveness of road maintenance through the introduction of performance-based contracting in two pilot regions; and (c) introducing safe road design principles, road safety audits and targeted improvement measures. CXHP (approved in July 2009 and closed in Sept 2019) increased transport efficiency and improved traffic safety on the three project sections of Corridor X and improved road management and road safety in Serbia. 7. The RRSP was consistent with, and part of, the Country Partnership Strategy (CPS) for Serbia for FY12-15, which was closely aligned with the European Commission's (EC) Europe 2020 Growth Strategy. As outlined in the CPS, the Bank's interventions in Serbia were designed to advance the EU agenda of smart and inclusive growth. To continue to improve the quality of people's lives, Serbia aimed at restoring strong, sustainable, and job-creating growth through improved competitiveness. The CPS found the quality of Serbia's transport infrastructure to be inadequate as a result of underinvestment in maintenance. RRSP aimed to contribute to promoting competitiveness through improved and safe road connectivity, thus encouraging economic development in regions not directly linked to the main transport corridors. Theory of Change (Results Chain) 8. Figure 1 presents the project’s results chain based on the project description, project development objectives, results framework, and higher-level objectives to which the project was expected to contribute, as presented in the Project Appraisal Document (PAD) and developed for the purposes of this ICR. Page 7 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Figure 1. Results Chain Project Development Objectives (PDOs) 9. The PDO, as stated in the Loan Agreement of the project, was to improve the condition and safety of the National Road Network for road users by supporting the Republic of Serbia in the implementation of the first phase of its National Road Network Rehabilitation Program. There were no changes to the PDOs during project implementation. Key Expected Outcomes and Outcome Indicators 10. The Project’s key outcome indicators were aligned with the PDO. There were two key indicators, each for the improved road condition and improved road safety (Table 2). Page 8 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Table 2.PDO Indicators and Targets PDOs Outcome Indicators (i) To improve the condition of the 1. Improvement in road conditions measured by a reduction in the national road network average International Roughness Index (IRI) for the roads in the first phase of NRNRP (number) (ii) To improve the safety of the national 2. Reduction in road traffic accidents with fatalities on the improved roads road network in the first phase of NRNRP (percentage) Components 11. Per the description of the Project from the Loan Agreement, the Project includes the following components and associated costs, which are summarized in Table 3. Component 1: Road Rehabilitation and Safety Investments (Total cost €263.35 million, including IBRD financing of €67.97) Provision of goods, works, consultants’ services, and non-consulting services for carrying out rehabilitation and safety works, including: (i) Periodic maintenance and rehabilitation works; (ii) Partial pavement widening; (iii) Traffic signalization improvement; (iv) Structural renewal; (v) Setting up ancillary road connections and drainage systems pursuant to criteria set forth in the Project Operational Manual (POM); and (vi) Incorporation of road safety measures as recommended by road safety audits, pursuant to criteria set forth in the POM. Component 2: Institutional Strengthening (Total cost €3.45 million, including IBRD financing of €0.81) Provision of goods, consultants’ services, non-consulting services for: A. Strengthening safety of roads within the National Road Network Program, through: (i) Carrying out road safety inspections; (ii) Implementation of low-cost safety measures, including traffic calming measures, and road furniture within the national road network; (iii) Carrying out road safety awareness campaigns; (iv) Supporting the development and implementation of strengthened enforcement of road safety regulations; and (v) Carrying out training and accreditation of personnel on safety audits and inspections. B. Strengthening road rehabilitation and planning processes through: (i) Carrying out a road conditions survey of the national road network; (ii) Conducting an update of the national roads database; (iii) Establishment of a practice of developing multi-year national roads maintenance plans that prioritize Page 9 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) activities based on economic, social, and regional criteria set forth in the POM; and (iv) Supporting the implementation of updated design standards for national roads maintenance and rehabilitation. C. Strengthening national roads maintenance management through: (i) Supporting the development of a strategic plan for the adoption of a tendering process of performance-based maintenance contracts (PBMC); (ii) Supporting the preparation of model bidding documents for PBMC; (iii) Provision of training to Project Implementing staff and prospective contractors on PBMC; and (iv) Provision of implementation support of PBMC. Component 3: Project Detailed Design, Project Supervision, Management, Monitoring and Audits (Total cost €18.4 million, including IBRD financing of €4.83 million). A. Support the design and Project supervision activities for rehabilitation and safety works to be carried out on the national road network. B. Carrying out activities for Project management support and capacity building of the Project Implementing Entity in areas such as procurement, financial management, environmental and social safeguards, and annual program planning. C. Carrying out monitoring activities, and Project Audits, including (i) the Independent Performance Audit; and (ii) the Project financial audit. Table 3.Project Cost, Euro millions Project Components Project Cost IBRD Percentage of Actually Percentage (EIB/IBRD/RoS) financing IBRD Financing disbursed of IBRD At appraisal Financing At Closing 1. Road rehabilitation and safety 229 59.1 39.3 investments 2. Institutional Strengthening 3 0.7 1.2 3. Detailed design, supervision, 16 4.2 12.7(1) management and monitoring 25 42 (2) Total Baseline Costs 248 64 53.2 Physical contingencies 25 6.4 3.02 Price Contingencies 12.6 3.2 Total Project Costs 285.6 73.6 56.2 Interest during construction (IBRD) 4.25 Front-End-Fees (IBRD) 0.18 0.18 0.18 Total Financing Required 290 73.8 56.4 (1) Includes Euro 2.5 million for implementing the recommendations of road sector reform developed under the Corridor X Highway Project. (2) Although EIB and GoS portion of the Project is ongoing through June 2022, EIB’s disbursed amount as of April 2021 stands at Euro 23.8 million or 17% of total project cost; and GoS’s disbursed amount stands at Euro 55.9 million or 41% B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Page 10 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Revised PDOs and Outcome Targets 12. While PDO remained the same throughout the project lifetime, the intermediate results indicators and corresponding outcome targets were revised during project restructurings in 2015, 2018, and 2020. Revised PDO Indicators 13. As part of restructurings, the target values for the intermediate outcome indicators have been revised to reflect the revised implementation schedule and closing dates. Target dates for the PDO indicators and intermediate indicators, dependent on the completion of the works, have been revised to reflect the new priority delivery schedule. In addition, as part of the first restructuring, the baseline value and target value for the indicator: Roads in good and fair condition as a share of total National Roads were added; the baseline and target values for the core indicator: Size of the total classified network were revised to reflect new network definitions; and target values for the Roads rehabilitated have been significantly reduced. Revised Components 14. The names of the components remained relevant throughout the life of the project. Three Level 2 restructurings in 2015, 2018, and 2020 included several changes to the activities covered by the project, which are summarized below. 15. Project’s first restructuring of 2015 responded to the multiple delays in Project effectiveness associated with government changes and parliamentary elections; and addressed the needs resulting from the massive floods of May 2014, which called for a revision of priorities for road network rehabilitation. The restructuring involved the following: (i) change to Component 1: the total length of road design works has been reduced from 800 – 810km to 795 km. The total length of sections of 795 km co-financed by the World Bank and EIB included about 125 km of emergency rehabilitation and maintenance works. (ii) changes to the annual financing percentages for particular road segments, to reflect the reduced availability of financing from Public Enterprise Roads of Serbia (PERS) during Year 1 of implementation due to the floods; (iii) additional procurement methods to allow for the use of Design and Build for the Year 1 program, which covers priority works contracts under Component 1. (iv) change in the project implementation schedule and associated update of the Results Framework and Disbursement Linked Indicators (DLIs). (v) extension of the Loan Closing Date by 14 months to December 31, 2019. 16. Project’s second restructuring of 2018 included a) further reduction in the road sections that are co-financed by the World Bank from 795 km to 248.6 km; b) extension of the Loan Closing Date by 6 months to June 30, 2020 to leave enough time for verification of the DLIs regarding the road sections to be completed until December 31, 2019 by the Project Audit Consultant (PAC), and c) modification of DLIs/DLI disbursement caps and certain PDO indicators to match the revised implementation program to rescope the project in line with what could be realistically accomplished by the project closing date, and d) partial cancellation of €17.4 million out of the total loan amount of € 73.8 million. The following changes were made to Components and Costs: (i) The cost of Component 1 was reduced by €17 million and the works programs for years 3 and 4 were deleted from the revised Project Implementation Program, along with certain road sections from the programs of Years 1 and 2, which were experiencing delays concerning the design completion. With the restructuring, 5 projects from Year 1 (total length of 95 km) and 4 projects from Year 2 (total length 28 km), thus 9 projects with a total length of 123 km continued to be financed under the loan. Page 11 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) (ii) The cost for Component 2 was increased by € 0.5 million to reflect the actual costs. (iii) The cost for Component 3 was increased by €8.5 million, in part due to the addition of a €2.5 million activity to support the implementation of the recommendations of the technical assistance on the road sector reform being financed under the Corridor X Highway Project; and in part due to changes in the scope of road design services. (iv) Contingencies were decreased by €6.6 million to include only the remaining works as overall procurement of the project has advanced with realistic cost figures obtained. 17. Project’s last restructuring of 2020 included the extension of the Loan Closing Date by 6 months to December 31, 2020 due to Covid-19 to allow for completion of a) ongoing construction works; b) remaining design works; and c) Road Weather Information and Weigh-In-Motion Systems, the two important road-reform-related work items added during the 2018 project restructuring and assure full attainment of the PDO before project closing. Other Changes 18. There were no other changes. Rationale for Changes and Their Implication on the Original Theory of Change 19. The 18 months of effectiveness delay due to the Government changes and parliamentary elections and devastating floods in May 2014 resulted in the first project restructuring in 2015 to allow for using loan proceeds to rehabilitate flood-damaged national roads (the Emergency Works Program) at the request of the GoS. The Project’s second restructuring of 2018 came as a result of the findings of the Mid-Term Review Mission in 2017 and included revision of the project scope and partial cancellation of loan proceeds. Despite the length of the rehabilitated roads was reduced significantly from the original target, the Project’s key contribution was on the roads sector reform: the introduction of the Road safety audits and the PBMC on 3,000 km of the road network - was a transformational change for the Serbian road sector, shifting focus from investments in new infrastructure to maintenance – all that made achievement of the PDOs possible. The main contributors to the need for Project’s second Restructuring were: • A number of reforms (which the RRSP was promoting) necessitated a more thorough due diligence that increased the timeline for completion of design and construction works. E.g.(a) A new Law of January 2017 necessitated the development of a planning instrument/master plan as a basis for initiating land acquisition even for road rehabilitation projects. Additional few months of project life were taken to develop the instrument. (b) The adoption of the new law on the road safety audits for the first time in Serbia took place after the design works under the Project were almost completed; it took additional 4 months to get formal approval from the road safety auditors for completion of the designs. • Delays in procurement, which resulted from the need to secure two no objections (one from EIB and one from the Bank) due to the delay in agreement on the Principles of Collaboration between the two institutions and that would result in EIB delegating procurement decisions to the WB. • During project preparation, the Bank team and PERS believed that the project would be able to tap into the large market for consultants and contractors in the Balkans, which would enable PERS to increase the number of km that it typically rehabilitates per year. However, this did not materialize. Instead the limited pool of local companies was overloaded especially with several large national road projects under implementation. Page 12 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) • Poor performance by several designers and contractors resulting in significant delays, which led to the termination of contract. • While the consultants supporting the Project Implementation Team (PIT) have played a positive role in reducing some of these delays, PIT was short of procurement and implementation staff. 20. The Project’s last restructuring in 2020 addressed the need to extend the Loan Closing Date by 6 months to allow for completion of project activities impacted by Covid-19. There were no major implications on the original theory of change, except for the change of the Results Framework and Disbursement Linked Indicators (DLIs) to reflect the revised Project implementation schedule. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 21. The Country Partnership Framework for Serbia for 2016-2020 identifies infrastructure as one of the top priority areas. The Project is aligned with CPF objectives 1d - More efficient public transport companies and 2d -Enhanced infrastructure networks. The RRSP PDO supports the CPF’s aim of helping Serbia take advantage of the country’s geographic opportunities, transform its economy and improve the livelihoods of its citizens through improved infrastructure networks and more efficient PERS. 22. To this date the RRSP’s PDO remains relevant with the National Road Network Rehabilitation Program, as the overall project (WB/EIB/GoS) implementation, which started in 2014 and apart from the World Bank financing, which ended on December 31, 2020, the Project implementation lasts through June 2022, which is the EIB and GoS completion date for the Project. EBRD’s parallel financing project is also ongoing with the closing date of December 31, 2021. 23. The 2020 Country Economic Memorandum “Serbia’s New Growth Agenda” emphasizes the role of the infrastructure as enabler of the private sector growth: “The focus of government capital expenditures should be on infrastructure that helps the private sector to grow and connect to markets, including in education, transport, energy, water, and environment.” 24. Considering the above, the relevance of the PDO is rated High. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 25. The efficacy of the PDO has been assessed for each of the two objectives: to improve the (i) condition and (ii) safety of the National Road Network for road users by supporting the Republic of Serbia in the implementation of the first phase of its National Road Network Rehabilitation Program. PDO (i): to improve the condition of the National Road Network for road users by supporting the Republic of Serbia in the implementation of the first phase of its NRNRP Page 13 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) 26. The Project’s objective of improving the condition of the National Road Network for road users by supporting the Republic of Serbia in the implementation of the first phase of its NRNRP was achieved though through the revised targets, as summarized in Table 4. Table 4. Targets and Achievements for PDO (i) PDO Indicators Baseline End Target Achieved 2014 2020 2020 Improvement in road conditions measured by a reduction in the average IRI for the roads in the first 5.34 2.5 1.9 phase of NRNRP (IRI number) Intermediate Results Indicators Baseline End Target Achieved 2014 2020 2018-2020 Roads in good and fair condition as a share of total 58.79 61.30 63.7 classified roads (%) Roads rehabilitated, Non-rural (km) 0 389.80 (1)(2) 416.5(3) Road Condition Survey Completed on National Roads 0 9,000 15,076 (km) PBMC Contracts in Implementation (km) 600 3,000 3,000 Implementation of a circular for the preparation of rehabilitation program based on the socioeconomic & No Yes Yes regional considerations (1) A revised target from original target of 800km under RRSP (1,125 km if EBRD’s sections are added) (2) Of which 248.6 km proposed for the WB-financing (3) Of which 254.5 km actually rehabilitated from the WB-financed portion, per the last verification report of March 2021. 27. While there was a significant reduction in the number of kilometers rehabilitated under the Project, from original target of 800km (1,125 km if EBRD’s sections are added) to 416.5 km, simple percentage of roads rehabilitated is not the only parameter in assessing the success of the RSSP, as the project was transformative in implementing reforms that are normally difficult to achieve in a traditional roads project. The project objective was met by shifting the focus from building new roads to a more efficient planning, maintenance and operation of the national road network, improving the capacity of relevant agencies and institutions, introduction and mainstreaming of good practices and supporting institutional changes. Project’s results-based elements (DLIs) were well suited for this type of institutional reforms. Original, revised, and achieved targets for all DLIs are presented in Annex 9. 28. The project’s important contribution was in bringing prominence to road assets operation and maintenance through introduction of Performance-Based Maintenance Contracts (PBMC) and improving capacity for planning and managing maintenance works. Six PBMCs were signed and implemented covering 3,000 km of Category I and II state roads, which was a requirement for the Project DLI. This is indeed an achievement, that is rarely made possible through a traditional road project where focus is on the number of km of new roads built. The procurement of the PBM works was carried under the national Public Procurement Law. The type of PBMC introduced was “hybrid”, which combined features of both method- and performance-based contracts. Some services were paid on a unit rate basis, while others were linked to meeting performance indicators. About 27% of the specified maintenance activities were paid on a lump-sum basis while the rest of activities (73%) were paid based on measured quantities and unit rates. The model of PMB contract related mainly to the routine (summer and winter) maintenance activities for the duration of three years. Page 14 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) 29. Road Maintenance Circular (RMC) has been developed as a part of RRSP to collect and process the information on necessary road maintenance works submitted by the PERS’s Engineers, Supervising Engineers, and Road Maintenance Companies in charge of everyday maintenance activities on the Category I and II state roads. RMC, being another requirement for the Project DLI, was adopted by the PERS on January 19, 2021 and was prepared per the latest version of the Reference system of the Serbian state road network approved on March 16, 2020. The RMC Consultancy started on April 1, 2020 and completed by July 31, 2020, with the following main tasks accomplished: 1. Preparation of methodology for prioritization of the maintenance (rehabilitation) works; 2. Preparation of Road Maintenance Circular (RMC); 3. Preparation of Guide for Road Maintenance Circular (RMC); 4. Preparation of Road Maintenance Circular (RMC) Final Report. 30. Based on the 2020 road network condition survey of national roads, the average International Roughness Index (IRI) weighted by section length over the Project implementation period has improved from IRIavg=5.34 m/km to IRIavg=1.90 m/km, which exceeds the original target of IRIavg=2.5 m/km. Measurements of IRI have been performed on all 16 road sections in a total length of 254.5 km financed by the World Bank. It includes six RRSP Emergency Work Contracts (127.2 km) that have been completed in 2016 -2018 and 10 RRSP Regular Work Contracts (127,3 km), which have been completed in 2020. 31. As per the 2020 Annual Performance Audit Report, a total of 416.5 km of roads were rehabilitated and Certificates of Completion issued under the RRSP (WB, EIB, and GoS financing). This is significantly lower than the original target of about 800km (or 1,125 km if EBRD financing is added), but exceeds the revised target of 389.8km by 6%. Section III (Key Factors that Affected Implementation and Outcome) explains the reasons for the reduction in the scope of works related to the delays in the design works, procurement and mobilization of contractors, safeguards compliance, and Covid- 19. Out of total of 416.5 km roads rehabilitated, the contribution from the World Bank extended to the road sections with a total length of 254.5 km (which exceeds the revised DLI of 248.6 km), including 127.3 km on national highways (Regular Works) in Vojvodina region, West and East Serbia. 32. Roads in good and fair condition as a share of total classified roads has increased from 58.79% of 9,000km to 63.7% of 15,076km, or from 5,220km to 9,603km (increase by 4,383km). A share of total classified roads in PAD refers to 9,000km, supported by the component on Road Network Condition Survey (RNCS) on national roads of 9,000 km (another intermediate results indicator). Later, the Restructuring papers updated the number to 16,900 km, which is Serbia's total national road network. In 2018 some 11,456 km of National Roads were surveyed. Some 64.7% of the surveyed state road network was in good and fair condition. In 2020 some 15,076 km of national road network was completed with 63.7% of the surveyed state road network being in good and fair condition (IRI ≤4.0 m/km). The RRSP was initially targeting to rehabilitate about 800 km, but rehabilitation (due to a number of reasons referred to in Section III) has only been possible on a section of 416,5 km (with WB’s direct contribution of 254.5 km). This represents about 10% of the total network improvements of 4,383km, however, it is directly related to the original project design where WB’s contribution was a small percentage of the total network of national roads. 33. Beneficiary Feedback Survey results for Emergency and Regular Works Sections show that the majority of interviewed beneficiaries see benefits of rehabilitation works. Since different Questionnaires on the degree of satisfaction of users were applied on Emergency and Regular Works due to differences in approaches by Supervision Consultants, the average score for the entire Project could not be generated. However, survey results for Emergency Works show that an Page 15 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) average of 93% of users found their expectations to be met or surpassed. Results of surveys conducted for Regular Works show that 60% of users to be very satisfied or satisfied with overall project implementation and results. More details are provided in Annex 7. 34. Overall, WB’s contribution translated in the support to the GoS in setting up the solid foundation for the institutional reform to improve the condition of the road network through introduction of more efficient road maintenance approaches. It should be noted that the EIB and GoS portion of the RRSP is still ongoing and aims to complete more rehabilitation works until the closing date of June 2022. EBRD’s parallel financing project is also ongoing with the closing date of December 31, 2021. PDO (i): to improve the safety of the National Road Network for road users by supporting the Republic of Serbia in the implementation of the first phase of its NRNRP 35. The Project’s objective of improving the safety of the National Road Network for road users by supporting the Republic of Serbia in the implementation of the first phase of its NRNRP was fully achieved, as summarized in Table 5. Table 5. Targets and Achievements for PDO (ii) PDO Indicators Baseline End Target Achieved 2014 2020 2020 Reduction in road traffic accidents with fatalities on 100 90 78 the improved roads in the first phase of NRNRP (%) Intermediate Results Indicators Baseline End Target Achieved 2014 2020 2018-2020 Safety Inspection of National Roads beyond the first phase of NRNRP (km) 0 1,000 1,016 36. Based on the annual Traffic Police reports for 2015-2020, road accidents on the RRSP sections had a decreasing trend throughout the project implementation period. Data for 2020 was only available for the period 01-Jan-2020 to 30- Sep-2020, with 24 road accidents with fatalities recorded on the RRSP sections. If the progress of the first nine months continued till the end of the year, the projected value of the road crashes with the fatal outcome on the RRSP road section could be up to 32, which is 78% compared to the base year, exceeding the original target, as seen from Table 6. Due to the lockdown and reduced number of traveling due to the Covid-19, there were even better results in road safety, and it is expected that the number of fatal accidents would not likely reach the number 38, which was the PDO indicator target. Page 16 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Table 6. Types of Road Accidents, RRSP, 2015-2020 Table 7. The Decreasing Trend for Project years 1-6 Year All Type Of Traffic Accidents 50 Accident with heavy minor material s 45 fatalities injures injures damage 40 2015 979 41 142 346 450 35 2016 1055 35 109 360 551 2017 1131 45 105 392 589 30 25 y = -1.4857x + 41.867 2018 1209 29 144 409 627 R² = 0.2229 2019 1132 38 161 392 541 20 2020(1) 1029 32 97 393 507 0 2 4 6 8 Total 6535 220 758 2292 3265 with fatalities Linear (with fatalities) (1) Note: Data for 2020 is prepared based on the available data for the first 9 months. Data has been projected to cover the whole year. Originally received data for the first 9 months are: All crashes 772; crashes with fatalities 24; crashes with heavy injury 73; crashes with minor injury 295 and crashes with material damage 380. 37. As shown in Table 7, comparing with 2015 as a base year, only accidents with material damage have increased, while all other types of accidents have decreased during 2016-2020. In 2015 traffic police started a new method for the collection of road crash data, and in the first years, the crashes including material damage were not presented correctly. They were underreported during the years. This however does not relate to the accidents with injury and fatalities, since reporting on these types of accidents has always been a priority and obligatory exercise, hence, the data on such crashes has been reported accurately. 38. Overall, road safety was incorporated in all phases of the Project implementation. At the road design stage, the designers were demanded to develop road designs with safety features, civil works were implemented observing road safety parameters and finally a road safety audit was introduced as an independent assessment of road safety interventions. All roads for which designs were developed under the RRSP were based on safe road principles of designing and were audited by the independent Road Safety Auditors. The measures that were incorporated related not only to motorized traffic participants, but also to vulnerable categories of road users. Subsequently, all roads that have been rehabilitated as part of the Emergency and Regular Works program included road safety features, thus contributing to the safer national road network. A Road Safety Inspection was done on a total of 1,016 km of the road network, as presented in the Final Report of January 26, 2018. As part of the Project, the certification of road safety auditors was done for the first time in the Republic of Serbia that complemented the holistic approach to road safety management. All project DLIs related to the road safety have been completed, including implementation of the low-cost safety measures on 622 km of rehabilitated road sections, which exceeded the targeted number of 500 km. Annex 6 includes photos of the condition and safety of the roads before and after the Project. Page 17 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) 39. The Project has also implemented a series of workshops at schools to raise road safety awareness among the vulnerable traffic participants and ensure sustainability via training of teachers and transfer of knowledge to future generations of students. Road Safety Awareness Raising Campaign in schools in the vicinity of Project roads was implemented in 20 elementary schools within 4 regional centers (Belgrade, Novi Sad, Nis, and Uzice). Throughout the Campaign (i) 148 teachers completed the training; (ii) 281 parents were trained; (ii) 441 students were trained by teachers in three months, including 217 second-grade students who participated as pedestrians and 224 fourth-grade students who participated as cyclists. As a result of the campaign, more positive behavior of children pedestrians in traffic was achieved by 48.1%, and more positive behavior of children cyclists as participants in traffic was achieved by 36.1 percent. Figure 2. RRSP Road Safety Awareness Campaign Justification of Overall Efficacy Rating 40. As described above, all revised targets and DLIs have been achieved, the road condition and safety of the National Road Network for road users has been improved as part of the implementation of the first phase of Serbia’s NRNRP. Although delays in implementation justified MU rating for the project’s progress towards achievement of PDO during Nov 2017-Dec 2018, the project has reached its objectives after the acceleration in 2019. Accordingly, the overall efficacy of the project is rated as Substantial based on the above discussions on the achievement of PDOs 1 and 2. C. EFFICIENCY 41. The efficiency of the project has been assessed along two main dimensions: (i) the design and implementation efficiency; and (ii) economic efficiency Assessment of Efficiency and Rating Aspects of design and implementation that influenced efficiency 42. At appraisal, only two risks were rated as substantial, namely project Design and Delivery Monitoring and Sustainability. During implementation, however, a third risk - Program and Donors - turned out to be a substantial constraining factor. 43. Design of the implementation arrangements. According to the PIT, at the appraisal stage, it was obvious that the RRSP would be one of the most complex projects implemented in the road sector in Serbia. The complexity of the project was not caused by the nature of the Works (as those might be considered simple since they covered road rehabilitation within the road reserve area without significant or durable impact on the environment of the site), but it stemmed from the complex implementation arrangements between the co-financiers under the Project, each coming with its requirements and policies, as well as from applying the DLI approach, which was new for both the PIT and Bank teams. 44. Non-harmonized procedures of the co-financiers. Lack of agreement among the IFIs significantly affected the Project implementation and caused delays that could not have been compensated in the later stage of the Project. At preparation, a joint agreement between the World Bank and the EIB stipulated that the RRSP (sub-project co-financed by Page 18 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) WB, EIB, and GoS) would be implemented based on the uniform principles and that the policies that would govern the project, would be those of the World Bank, except in cases where the EIB has stricter requirements, where such EIB requirements would apply. This type of intended approach was also specified in the PAD and POM. It was understood by all parties that this type of approach was not unique and has been already applied on various joint projects of the World Bank and EIB, but unfortunately, it took years of RRSP implementation for the IFIs to reach the agreement on the manner, in which the Project would be implemented on behalf of the Banks. 45. DLI-based approach to financing. At the time of the Project preparation, the idea of the introduction of the DLIs to trigger the conversion of the advances into disbursement and enable withdrawal of the new advances was new for the WB-funded projects implemented by PERS. Although it was expected that the timely achievement of DLIs related to the institutional strengthening such as road safety certification, road safety campaigns in schools, and introduction of PBMC would be difficult, the project implementation showed that cooperation between the institutions functioned well and that other stakeholders provided their support and enabled that those DLIs were rather easy to meet. At the same time, the fulfillment of the targeted number of kilometers that should have been easily reached, due to the accumulated delays in implementation of the design works was behind the schedule in all years of the project implementation. Economic Efficiency 46. An ex-post analysis was done using the Highway Development and Management Model (HDM-4) to evaluate the economic internal rates of return (EIRR) of the activities completed before project closing. It follows the same methodology used at the appraisal stage and focuses on the economic appraisal of Component 1, for which investments amounted to 92 percent of project costs at appraisal. The overall economic benefits of the project derive from: (a) savings in travel time; and (b) savings in vehicle operating cost (VOC), which are both due to the improved quality of the roads. In addition, a separate ex-post economic analysis including road safety benefits was done. Details of the economic analysis are presented in Annex 4. 47. At appraisal, the first 18-month (2013-2015) program of the first phase of NRNRP covered 22 homogeneous road sections grouped under 16 prospective contracts with a total length of 308 km amounting to Euro 103.3 million of investment value. Of these 22 road sections, some were excluded from the project, some were financed by EBRD, and for others the contracts are ongoing, thus, the ex-post economic analysis excluded those road sections. The remaining nine road sections, financed by WB/EIB/PERS, were evaluated for the ICR. 48. The total ex-post actual NPV is satisfactory at Euro 30.6 million, which can be compared to the total NPV of Euro 61.7 million estimated at appraisal, representing a 50 percent decrease on the project net benefits compared to the appraisal estimates. The total ex-post EIRR satisfactory at 17.6 percent, for the road sections evaluated for the ICR, which can be compared with the total EIRR estimated at appraisal that is 24.4 percent. The lower total NPV and EIRR found at the ICR stage are due mainly to lower than expected annual traffic growth on the road sections (lockdowns due to Covid- 19 might be one of the reasons). The ex-post EIRR (17.6 percent) is higher than the adopted discount rate of 12 percent and the ex-post NPV is positive, therefore, the ex-post economic analysis indicates that while there was a reduction in the project NPV and EIRR compared to the appraisal estimates, the ex-post actual NPV and EIRR of the project are still satisfactory Indicating that the project had a good economic justification. 49. The efficiency of the project is rated Substantial, taking into account the design and implementation aspects and results of the cost-benefit analysis. Page 19 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) D. JUSTIFICATION OF OVERALL OUTCOME RATING 50. The overall project outcome is rated as Satisfactory, based on the ratings of Relevance of the PDO, Efficacy, and Efficiency as High, Substantial, and Substantial, respectively. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender and Poverty Reduction and Shared Prosperity 51. Except for collecting data related to the number of female and male workers, respondents in the surveys and PAPs, there was no specific analysis on gender related issues under the Project. 52. Creation of Jobs. Apart from creating better connectivity thus the possibility for greater economic development, shorter travel time and more comfortable ride for drivers, one of the direct outcome of the project was creation of jobs. Job opportunities were created on construction sites, but also in the areas related to the workers’ accommodation, nutrition, transport, etc. The RRSP created new employment under the first- and second-year Regular Work contracts in the amount of 929 person-months (28 female and 901 male employees). The total number of person- months employment for the respective work contracts is 9,431 person-months, out of which 480 female and 8,951 male employees. Detailed data is provided in Annex 8. Furthermore, all Emergency Works Contracts Contractors have engaged and trained local manpower with focus and prioritization on the younger population. The share of the local population in the total workforce of all Emergency Works Contracts was 10-20 percent. 53. Road Safety. Total 220 traffic fatalities occurred during the Project life, i.e. 2015-2020. Female death on the RRSP represents 25 percent, while the majority of victims were men with 75 percent. Likewise, among the drivers, the female fatalities were significantly rarer compared to the male fatalities. The road user group 65+ had the highest fatality rate with the share of 23 percent. The 65+ road user group had the highest share as the pedestrians in road crashes. Also endangered were users in the group 25-34, which could be explained by higher participation in traffic and lack of experience. Having in mind that only first 6 months were taken from the 2020, distributions through years does not shows abnormal deviation. 54. The needs of the local community were assessed at the early stages of road design development. This approach, together with the diagnostic survey of the section done by the PERS in advance of design works enabled that particular context of the road section is taken into consideration and that proposed improvements also embrace the requests of the local community. Public consultations and established grievance mechanisms, cooperation with local authorities, and local municipalities resulted in technical Figure 3.Road Accidents by Age groups solutions developed by the designers, which included improvements requested by the local community (construction of bus stops, sidewalks, accesses, etc.). Total 73 public presentations and consultations with stakeholders (36 EMP presentations, 33 presentations of Preliminary Design Solution, and 1 Page 20 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) presentation each for ARAP, RPF, EMFD, and RRSP) were conducted under the RRSP. These presentations were attended by a total of 882 people. 55. Examples of good practice were demonstrated under the RRSP. This included a donation of milled asphalt from the MoT to the Municipalities for the maintenance of local roads, donation of removed/cut trees that were in the belt of the projected works and which were the property of the MoT, to schools as firewood, and provision of labor and machinery by the Contractor for replacement of old utilities in the possession of the Municipalities, since local utility companies were often incapable of executing works themselves. Institutional Strengthening 56. RRSP has strengthened the capacity of the PERS, as a road managing agency, in the planning, design, construction, operations and maintenance of road infrastructure, specifically: - More efficient planning and use of funds through the introduction of a multi-year PBMC model, based on clearly defined economic, social and regional criteria and adoption of the updated design standards for the maintenance and rehabilitation, and development of road maintenance circular; - Improved road rehabilitation and planning processes by conducting a road condition survey, and as a result an updated data on road condition of the national road network and establishment of the updated State Road Data Base; - Provision of goods to enable better road management (road weather information system and weight-in motion system); - Improved road safety management capacity through road safety inspections and the implementation of low-cost measures for the enforcement of safety. - 57. The capacity of the local consultants and contractors in the Balkans has been improved significantly on the aspects of road safety. All the designs produced by the consultants were based on safe road principles of designing and underwent scrutiny from multiple agencies and audited by the independent Road Safety Auditors. 58. The Project also brought a more responsible attitude towards the safeguards and due care that Project activities are implemented in an environmental and social friendly manner (through the preparation of EMPs, training of contractors and supervision staff, continuous monitoring and reporting). Through RRSP, the Contractors learned how to properly plan and implement social safeguards, and the local population advanced their knowledge on their role in decision-making on aspects related to their living and working environment. Mobilizing Private Sector Financing N/A Other Unintended Outcomes and Impacts N/A Page 21 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 59. Key factors during preparation that affected implementation and outcome included (i) financing a tranche of a Project; (ii) the use of one set of guidelines for the Project by multiple IFIs; (iii) DLI approach; and (iv) road construction market in the Balkans. 60. (i) Unlike traditional transport projects where the WB directly finances specific road sections and activities in a program, in RRSP the Bank-financed a certain percentage of all the activities under the Project. So for example, for the rehabilitation of the 35-40 road sections (16 contracts) that were to be financed under Component 1, the Bank financed about 20 percent of each section. Since EIB, the WB and the RoS pooled their financing, the Bank did not monitor every single contract but performed annual project performance reviews covering at least 20 percent of the number of contracts. 61. (ii) At preparation, one set of guidelines was to be used for the implementation of the Project and apply to EIB, RoS and WB financing. A Resettlement Policy Framework (RPF) and an Environmental Management Framework (EMF) acceptable to RoS, EIB and WB, and consistent with the Bank's guidelines, have been prepared. EIB was also to follow the procurement guidelines of the WB. EBRD was to follow the same set of environmental and social guidelines in the implementation of its EBRD Project that is being financed in parallel. 62. (iii) RSSP was the first ever IPF in Serbia and ECA region to test the DLI approach with both the WB and the Client teams lacking any experience in the results-based projects. Disbursement for Components 1 and 2 was linked to agreed disbursement-linked indicators (DLIs). Except for a few binary choice DLIs (i.e. DLIs linked to whether a certain measure was undertaken or not), all other DLIs were scalable to provide flexibility in disbursement to match implementation progress whether faster or slower than agreed annual targets. Rolling advance loan proceeds were to be utilized to facilitate the use of the results-based approach given that the Borrower's budgetary process and limited resources did not allow for pre-financing by RoS and later disbursement by EIB and the WB. For the WB, loan advances were to be converted into disbursements when expenditures reported were reviewed and accepted as eligible, when the independent audit consultant report has reviewed and verified that DLI targets have been partially or fully met or exceeded and upon the Bank's review. Contract reviews and DLI verification were to be carried out by a Project Audit Consultant (PAC). 63. (iv) During project preparation, the Bank and the Client teams believed that the project would be able to tap into the large market for consultants (designers, road condition survey, PAC) and contractors in the Balkans, which would enable PERS to increase the number of km that it typically rehabilitates per year. However, this did not materialize. Instead, the limited pool of local companies was overloaded especially with several large national road projects under implementation. B. KEY FACTORS DURING IMPLEMENTATION 64. Key factors during implementation that resulted in inefficiencies and delayed progress included (i) GoS’s slow internal decision-making process (effectiveness took 18 months) that delayed the timely start of project activities and natural disaster (floods) that prompted changes to the project design; (ii) lack of harmonized procedures of co-financing Page 22 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) IFIs; (iii) low quality of initial outputs prepared by design consultants that significantly delayed the start of the civil works; (iv) challenges of DLI approach and finding the Project Audit Consultant of required expertise to verify the fulfillment of the DLIs; and (v) impact of Covid-19 on the timely completion of works and low-cost road safety measures. Earlier sections of this ICR (Change, Efficiency) had touched upon many factors, hence to avoid repetition, this section explains (iii) and provides more details on (ii), (iv), and (v). 65. (ii) Lack of agreement between the co-financiers on procurement procedures delayed Project results for almost one year: (a) Total 74 contracts were awarded as part of the overall NRRM Program that included the RRSP (WB/EIB/GoS) and EBRD Projects, as shown in Figure 4. The actual completion of procurement took significantly longer periods than required by the EIB for certain phases (e.g. 37 days for submission of EoI for Consultant’s selection Vs 14 days deadlines according to the Guidelines). Furthermore, the necessity to obtain two no-objections (one from World Bank and one from EIB) further extended the time for procurement, given that 63 out of 74 contracts were awarded using WB/EIB co-financing. The additional delay was caused by the fact that even contracts that under the WB thresholds were identified as “post review” contracts had to be “prior reviewed and approved” by the EIB. (b) The additional challenge was the exclusion of the EIB financing for the services procured through the CQS method of the WB. The EIB did not recognize this type of selection within its procurement procedures. Figure 4. Total Contracts Awarded The problem arose at the mid-selection of Consultants for the 1st year designs, which per the WB thresholds could be applied for the selection (less than USD 300,000 were to be provided through CQS). Resolution of the problem lasted several months and in the end resulted in a transfer of the financing of the Services to WB/RoS funds. This also affected the financial management of the Project as it triggered the reallocation of loan funds and changes to the project financing plans. As a result of this lack of agreement among IFIs, the award of the design contracts for 158 km of road rehabilitation works for the 1st year of regular works of the project was delayed for almost a year, and not only for the RRSP (WB/EIB/RoS subproject), but also for the EBRD parallel financing Project as Works under the EBRD project were to be based on the designs developed under the design services financed by the RSSP (World Bank/EIB/RoS share). 66. (iii) The quality and delays of outputs by the design consultants have significantly postponed the start of the rehabilitation works. Accumulated delays in design preparation led to further delays in the overall project progress and in some cases resulted in the exclusion of road sections from the Project scope due to the impossibility to finish or even start the Works within the project life. Introduction of financial penalties for poor quality and unjustified delays could have remedied the situation. However, apart from the standard of performance, there are not many contractual clauses that deal with the Consultant’s quality of execution of the Services. The only available means for the Client are either to suspend the payments under the contract (which was not a highly efficient measure, since the design contracts were lump-sum and payments were to be made when certain output is achieved) or to terminate the contracts. There was little maneuver for the Project but to having the designs even with several months of delays compared to the option of having another selection with almost the same market players (which after several months would potentially lead to the very same situation). 67. (iv) DLI Approach. Another difficulty that could have not been foreseen at the stage of the project design was a failure in the provision of the Project Audit Consultancy (PAC) Services to verify the fulfillment of the DLIs. The specific nature of the Services and complexity of the assignment, together with the required expertise of the key staff was a real challenge. The initial procurement resulted in only one Proposal being submitted and evaluation showed that the Technical Page 23 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Proposal did not pass the minimum technical score. The rebidding was done, which doubled the procurement time envisioned, and resulted in services that started with a significant delay compared to the initial project schedule. This in turn led to a delayed first disbursement record under the Project. 68. (v) As the rapid onset of Covid-19 pandemic and the cessation of most economic activities made it impossible to reach the June 30, 2020 target anymore, the loan closing date was extended to December 31, 2020. Even with the extension granted, it was a challenging period for the remaining construction works on eight road sections, as the first couple of months of the pandemic were characterized by widespread lockdowns and strong limitations on all workspace activities. Nevertheless, work continued in all sections with approximately 50 percent of the labor force. The WB Team collaborated early on with PERS/PIT regarding the safety measures to be taken in the construction sites as well as reporting of the Covid-19 cases, with the first Covid-19 case reported in the IB15 Vrbas-Srbobran section on April 9, 2020. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 69. Due to the Project’s complexity stemming from the multiple co-financing arrangements and DLI approach, it was challenging to identify monitoring indicators. Some of the indicators in the results framework (RF) were revised several times to reflect properly the difference between what constituted National Road Network Rehabilitation (NRNR) Program scope (RoS, EIB, WB, and EBRD financing) and what could be attributed to the RRSP’s scope (RoS, EIB, and WB financing). Some of the examples include (i) the initial 1,125 km of rehabilitated roads included in the RF referred to the overall NRNR program and not RRSP (only 800km of 1,125 km should have been attributed to RRSP originally); (ii) baseline value for Improvement in road conditions measured by a reduction in the average IRI has changed from 5.42 in PAD to 5.56 in 2015 Restructuring paper and 4.94 in 2018 Restructuring paper to 5.34 in the ISR in order to identify relevance only to the WB- financed sections; (iii) the total classified road network was initially indicated as 9,000km in PAD and later changed to 16,900km in the restructuring paper; (iv) baseline value for PBMC’s indicator has changed from 600km to 0km; (v) baseline and target values for the share of the roads in good and far condition were missing in PAD and were updated in the restructuring papers. M&E Implementation 70. Progress towards the achievement of the targets was monitored through regular project progress reports by PIT and WB’s ISRs. Despite the challenges of overseeing in effect multiple sub-projects financed by three IFIs unified under the umbrella of the NRNR Program; overall, PIT managed to collect and analyze M&E data in a methodologically sound manner. This justified a consistent Moderately Satisfactory and Satisfactory rating for the project’s M&E throughout the project life. M&E Utilization Page 24 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) 71. The M&E indicators were used by both PIT and the WB teams during supervision activities to gauge and assess performance and to prepare response measures to ensure the project was on track towards achieving its development objectives. PITs M&E reports were used for this ICR. Justification of Overall Rating of Quality of M&E 72. The overall quality of M&E is rated Substantial. The indicators were adequate to track the achievement of the PDO, and regular monitoring by the Client was conducted. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental safeguards compliance 73. The Project was classified as Environmental Category “B” and triggered the OP 4.01 (Environmental Assessment). The World Bank Environmental Health and Safety (EHS) Guidelines also applied to the Project. As an environmental Category B project, the anticipated negative impacts were of small to medium scale, easily mitigable, site- specific, and predominantly confined to the construction phase. The primary safeguards instruments were the Environmental and Social Management Framework (ESMF) and site-specific Environmental and Social Management Plans (ESMP). ESMF was disclosed, both nationally and by the World Bank in January 2013. ESMPs for sections Uzice-Pozega- Kratovska Stena, Arandjelovac-Krcevac and Zabalj intersection-Zrenjanin were disclosed locally in November and December 2012 and in the Bank’s Info Shop in January 2013.The ESMPs stipulated all civil works contracts to include requirements for implementation of the specific measures as per ESMPs provisions and good construction practices. 74. The ratings for environmental safeguard compliance during the project implementation varied from MU to S, and the project was rated satisfactory for the last two ISRs, beginning with December 17, 2019. In the first two years of project implementation, there were few major non-compliances with site-specific ESMPs – related to traffic safety, cultural heritage protection, waste management, and health and safety of workers. These were dealt with satisfactorily upon the Bank raising these issues. In the last two years, there were no major non-compliances in ESMPs’ implementation, the quality of supervision was greatly improved in the second part of the project implementation, reaching desired level and quality in 2019. There were also issues related to inadequate quality for several section-specific ESMPs at the initial two years of the project implementation, resulting in several versions of the same document being prepared and reviewed by the Bank. In the last two years, the remedial measures were undertaken by the Client and contractors in question – and the quality of prepared ESMPs improved gradually, reaching the desired quality, and the overall performance and compliance improved. The capacity of PIT related to the environmental safeguards improved throughout the project, both at technical level and in its staffing. The PIT was operating fully satisfactory during the last 18 months of the project implementation. Social safeguards compliance 75. The project is compliant with OP 4.12 requirements, and its social risk management performance has been rated satisfactory due to: 1) PERS’s adequate management of minor social impacts related to expropriation and clearance of public land in line with the project-specific Resettlement Policy Framework (RPF) and Abbreviated Resettlement Action Plan (ARAP); 2) proactive measures taken to avoid economic displacement and minimize impacts on private assets (located on both private and public land); and 3) appropriate use of willing-buyer/willing-seller method to meet minor land Page 25 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) acquisition requirements outside the Right of Way (RoW). Below is a summary of the three social risk management methods applied during the project. 76. Social screening and management of land-related impacts have been done according to OP 4.12 requirements. At preparation, the project triggered OP 4.12 due to the expectation that minor land acquisition would be needed during adjustment of road alignments to accommodate road safety and other enhancement measures, such as the incorporation of sidewalks, bus stops, and drainage. Prior to appraisal, in 2012, PERS prepared an RPF consistent with the World Bank's OP 4.12 and the co-financing IFIs policies on Involuntary Resettlement. All road sections subject to implementation and/or design under RRSP (38 in total) have been screened for social impacts in line with the screening guidelines provided in the RPF. Results of the social screening have identified only one Third Year Program section (IB 21 Ivanjica-Sjenica) with anticipated minor expropriation (38.41 m2 for expropriation and 274.21 m2 for relocation of fences) and need for clearance of public land within the RoW. The Bank financed the design preparation for IB 21 Ivanjica-Sjenica and the client prepared an Abbreviated Resettlement Action Plan (ARAP) to record assets, discussions with affected persons on mitigation measures and compensation eligibility as per OP 4.12. The Bank-approved ARAP, disclosed on November 17, 2020, will be implemented after the project closing date and before the rehabilitation of the section, as the RRSP financed only the Project Design for the respective section. 77. Measures were taken to avoid economic displacement and reduce impacts on private assets. Design change requests from local communities, roadside businesses and property owners have been accommodated to the extent that the requests aligned with the project’s road safety objectives. On some sections, private assets built within public RoW (fences/auxiliary structure) have been relocated on private property at contractors’ expense in line with guidelines provided in the RPF. On IB 23 Pakovraće (Markovica) - Užice Figure 5. Road safety and Roadside Businesses before and after the section, roadside businesses complained about a loss of direct IB23 road rehabilitation access from main road IB 23 as a result of rehabilitation activities, which included the introduction of curbs. On 18 February, 2020 PERS submitted a report to the World Bank, which documented that while some commercial facilities would lose the convenience of direct access from IB 23, access was ensured for all businesses in the final design through (i) maintenance of all existing access roads; (ii) incorporation of designated access roads and public parking lots for businesses, and (iii) in some cases, lowering of curbs to accommodate continued access directly from the IB 23 road. The report showed that the final road rehabilitation design was based on a thorough road safety analysis taking into consideration the safety of all traffic participants while facilitating continued operation of formal and informal roadside businesses. Figure 5 shows the road safety situation on IB 23 with respect to roadside businesses before and after the road rehabilitation. 78. Willing-Buyer/Willing-Seller land transactions have been appropriately used and documented. During the implementation of two Emergency Works, minor land acquisition needs (four parcels/0.04 hectares) for the construction of bus stops and a pedestrian walkways have been conducted through willing-buyer/willing-seller transactions. In such cases, the owners could reject the negotiated offers without the risk of being expropriated if negotiations failed and the project would have chosen another location and therefore the WB OP 4.12 did not apply. Direct purchase agreements have been diligently documented and owners compensated prior to land take. On First- and Second Year Program sections, no Willing-Buyer/Willing-Seller transactions were recorded. Page 26 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) 79. Retroactive expropriation was needed to update the ownership records for the footprint of existing roads rehabilitated under RRSP. Where necessary documentation was missing, PERS retroactively updated the ownership records for existing roads built 40 years ago, so that the road RoW could be legally established. This ensured that PERS could obtain usage permits from the Ministry of Construction, Transport and Infrastructure (former Ministry of Transport) for rehabilitated roads once these have been completed. This “retroactive expropriation” did not affect the land usage or ownership by the local population, and did not negatively affect their livelihoods in any way. This affected 4 out 6 Emergency Works sub-projects and on 7 out of 9 First -Program sections, and has resulted in the expropriation of 35ha 24a 9m2 of land in private property and payment of 803,769.00 USD in compensation. An additional 83ha 58a 98m2 of state-owned land was expropriated for the same purpose. Since no expenses arouse when transferring property between state institutions, the expropriation of state-owned land didn’t imply the payment of compensation. No disputes over the ownership of these lands have been recorded during the retroactive expropriation process. 80. Local communities were engaged in the design process and subproject-level Grievance Redress Mechanisms (GRMs) have been strengthened over time. All preliminary designs were subject to public consultations and wherever possible designs have been changed to respond to the needs of local communities. There were functioning and accessible GRMs at the construction sites and at the supervision consultants’ offices where affected communities could submit construction- Not in line with traffic Solved positively related grievances/requests. During the duration of the project, a Out of the Project domain total of 316 requests and complaints were recorded and resolved, as 9% Unfounded shown in Figure 6 - 237 positively (75%) and 79 negatively (25%). Unfounded 8% From 79 complaints that were negatively resolved, 26 (8%) were Out of the Project negatively resolved because they were unfounded, 29 (9%) were domain negatively resolved because they were beyond the project domain, Not in line with Solved traffic safety and 24 (8%) were negatively resolved because they were not in line positively 75% with traffic safety. Management of grievances improved over time with acknowledgments being sent to claimants upon complaint Figure 6. Grievances under RRSP receipt. Financial management 81. Financial Management (FM) performance has been assessed as satisfactory throughout the project implementation period. FM function has been adequately staffed with experienced and qualified finance professionals who are permanent employees of the PERS. The latest Bank’s FM supervision was conducted in November 2020, and no major issues were identified. Appropriate internal controls and procedures were instituted and applied in practice. Disbursement and flow of funds were smooth and undisrupted. 82. The external audit has been conducted by Deloitte, a private audit firm acceptable to the Bank. The financial reports audited are consolidated for all sources of funds (WB, Government, EIB, EBRD). The audit report for the year ended December 31, 2019 was received in May, 2020. The auditor issued the unmodified (clean) audit opinion. The previous years’ audit opinions were also unmodified. The Management Letter raised a few observations, and they were addressed promptly. 83. The semi-annual interim unaudited financial statements (IFRs) consolidated for all sources of funding (WB, Government, EIB, EBRD) have been submitted to the Bank within due dates (45 days after the end of the reporting Page 27 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) semester) and assessed as reliable throughout. The latest IFR received relates to the first semester of 2020, and it was assessed as reliable as well. Procurement 84. Procurement processes were conducted in compliance with the procedures of the Procurement and Consultant Guidelines with some deviations along the way due to the huge volume of contracts being procured and implemented, as shown in Figure 7. The Project Implementation Team (PIT) within the Public Enterprise “Roads of Serbia” conducted procurement function and project management role through its Project Coordinator, Assistant Executive Director, Financial Specialist and Procurement Specialist. Figure 8 shows the number of stated and awarded contracts per year during the NRNR Program life. TOTAL CONTRACTS 95 – EUR 391.55 mil. AWARDED CONTRACTS ON-GOING PROCUREMENTS PLANNED PROCUREMENTS 9 – EUR 53.91 mil. 74 – EUR 235.92 mil. 12 – EUR 101.72 mil. WB/EIB EIB EBRD EIB EBRD EIB 63 - EUR 2 – EUR 9 – EUR 7 – EUR 2 – EUR 12 – EUR 131.86 mil. 26.94 mil. 77.12 mil. 32.16 mil. 21.75 mil. 101.72 mil. Figure 7. Procurement Arrangements 85. Procurement plans were updated regularly and used as a monitoring tool. Two post reviews conducted by the Bank (May 2017 and December 2019) during the life of the project confirmed overall satisfactory procurement performance with some deviations. Recommendations in 22 the Bank’s post review reports to improve 25 procurement and to build capacity were 20 15 15 15 13 13 implemented accordingly. It should be 15 12 11 10 9 9 mentioned that there were three (3) 10 8 8 6 6 6 turnovers of Bank procurement specialist 4 5 under the project in addition to the turnovers 0 of Task Team Leaders. Although these 2014 2015 2016 2017 2018 2019 2020 2021 2022 turnovers happened for a variety of valid reasons, they disrupted project Started procurements Contracts awarded implementation to a certain degree. Figure 8. Number of Stated and Awarded contracts per year during the Project life 86. Procurement risks were well managed through the implementation of mitigation measures such as (a) regular procurement sessions and workshops, capacity building activities for the PIT staff; (b) procurement processing and monitoring of progress through STEP; and (c) close supervision by the Bank team. The project, like all other projects, was affected by the COVID-19 pandemic. In this regard, the Bank team applied some flexibilities on procurement: for example, (a) public bid opening was allowed using Zoom with appropriate measures in place to protect the integrity of the process, including a video recording of the whole process which is shared with all participants along with the Minutes of Bid Opening; and (b) contracts are extended to give time for suppliers/contractors/consultants to complete implementation of the relevant contracts which were affected by the lockdown. Despite the disruptions caused by then ongoing pandemic all Page 28 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) planned procurement have been fully completed by the project closing. Overall, the performance of PIT was satisfactory. PIT was able to manage all procurement despite the challenges faced. C. BANK PERFORMANCE Quality at Entry 87. The WB team guided the preparation of the project, keeping in mind the broader development aspects of the overall National Road Network Rehabilitation Program and Serbia’s needs for connectivity development. DLI approach was assessed to be the most well suited for bringing in the needed reforms in the roads sector. RSSP was the first ever IPF in Serbia and ECA region to test the DLI approach. In the hindsight it is obvious that a few critical aspects (like difficulties in hiring Project Audit Consultant) were not foreseen simply because both the WB team and the Client have never implemented results-based projects before. 88. The team rightly identified key challenges for smooth implementation: project design and delivery monitoring and sustainability, however, as discussed in the ICR’s earlier sections, Program and Donors risk has been underestimated at the time of preparation. This risk however arose very prominently on procurement-related aspects, when the lack of agreement among IFIs came as a critical barrier hindering the project advancement. 89. While the WB team carried out a good appraisal of the economic analysis and implementation capacity related to the environmental, social, and fiduciary aspects, a more thorough analysis of local and regional market for bidders of design and rehabilitation works was lacking, which translated into the challenges of insufficient competition, poor quality of works, and major delays. As discussed in the earlier section, the WB team ensured that the WB’s fiduciary and safeguards policies were complied with and appropriate risk management measures were incorporated in the project design. 90. Due to the Project’s complexity stemming from the multiple co-financing arrangements and DLI approach, it was challenging to identify monitoring indicators attributable to each IFI. Some of the examples include (i) the initial 1,125 km of rehabilitated roads included in the RF referred to the overall NRNR program and not RRSP (only 800km of 1,125 km should have been attributed to RRSP originally); (ii) baseline value for Improvement in road conditions measured by a reduction in the average IRI has changed from 5.42 in PAD to 5.56 in 2015 Restructuring paper and 4.94 in 2018 Restructuring paper to 5.34 in the ISR in order to identify relevance only to the WB- financed sections; (iii) the total classified road network was initially indicated as 9,000km in PAD and later changed to 16,900km in the restructuring paper, as a result of change in road classification, which was introduced after the WB Loan was signed; (iv) baseline value for PBMC’s indicator has changed from 600km to 0km; (v) baseline and target values for the share of the roads in good and far condition were missing in PAD and were updated in the restructuring papers. Quality of Supervision 91. The WB team proactively identified and resolved threats to the achievement of relevant development outcomes and remained responsive to changes during implementation. Overall, the task team undertook 14 implementation missions with on average two full-fledged supervision missions conducted per year. To assist the PIT in addressing challenges during implementation, missions undertook site visits and discussions with contractors and consultants. Each mission carried out a systematic review of the status of all aspects of the project. Page 29 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) 92. The WB team took the appropriate decision to restructure the project, when it was needed, and ensure the appropriate use of resources through category reallocations and cancellation of funds. The restructurings enabled the project to respond to Covid-19 and achieve its objectives. The team also supported the Borrower in ensuring the sustainability of project-created assets through the continued support from EIB and EBRD parallel financed Projects. 93. The efforts of the Bank team in coordinating activities with IFIs to overcome the challenges of double no- objections was not very efficient during the first years of project implementation, which created significant delays in project advancement. Justification of Overall Rating of Bank Performance 94. For the reasons outlined in the previous paragraphs, the overall rating for WB performance is Moderately Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 95. At project completion, the risk to development outcome is considered low, particularly in view of the continuation of the NRNRP, despite the completion of the WB financing, through the EIB/RoS co-financing project and EBRD parallel financed Project, for which the closing dates are in 2022. V. LESSONS AND RECOMMENDATIONS 96. RRSP provides valuable experience, and lessons learned from the challenges of this complex project will certainly increase the capacity of the PERS to deal with similar projects and would contribute to an easier implementation of similar future projects. The following lessons and recommendations are offered to the Teams at the Bank and beyond: 97. The project provides an example of successful introduction of Performance-Based Maintenance contracts, an objective that is normally quite challenging to be achieved during a Project life . The lesson learned from the RRSP is to start modest with a hybrid approach and move as the capacity locally builds gradually. The procurement of the PBM works was carried under the national Public Procurement Law, and each of six contracts was singed for 500 km road section with the duration of three years. The hybrid type of PBMC combined features of both method- and performance-based contracts. Some services were paid on a unit rate basis, while others were linked to meeting performance indicators. About 27 percent of the specified maintenance activities were paid on a lump-sum basis while the rest of activities (73 percent) were paid based on measured quantities and unit rates. 98. Procurement in a Project co-financed by multiple IFIs. Although IFIs co-financing in a single project leverages additional financial resources for bigger impact, the Project demonstrated several challenges involved with such an arrangement. While at the time of project preparation, one set of guidelines, including on procurement aspects, was agreed to be used for the implementation of the Project and apply to all co-financing IFIs (EIB and WB), in reality, this has not been implemented. The project experienced significant delays in procurement due to the need to secure two no-objections (one from EIB and one for the WB) and different approaches to the “contract prior- and post- reviews”. A lesson learned is for Teams at the preparation stage to sign a carefully elaborated agreement on the principles of collaboration between the institutions, and ideally delegating procurement decisions to one of the institutions (e.g. WB). Page 30 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) 99. Financing a portion of the Project. In a situation when WB finances only a certain percentage of all the activities under the project (unlike traditional transport projects where the Bank directly finances specific road sections and activities in a project), the Teams should be aware that the ability to attribute project’s success or failure to the WB financing becomes a challenge. In addition, at project level monitoring every single activity/contract might not be feasible. RRSP was managed by a team of high-quality experts in PIT that made this arrangement work, however, if the capacity of the implementing agency is low, the Teams should not use such arrangement and instead go for a stand-alone project. 100. DLI Approach. When deciding on whether to use the DLI approach, the Teams should take into account many factors, such as experience of the teams (both at the WB and Client) and the availability of Project Audit Consultancy (PAC) or Independent Verification Agent (IVA) Services in the market. The specific nature of the Services and complexity of the assignment, together with the required expertise of the key staff could be a real challenge. Under RRSP the initial procurement resulted in only one Proposal being submitted for PAC services and evaluation showed that the Technical Proposal did not pass the minimum technical score. The rebidding for PAC services was done, which doubled the procurement time envisioned and resulted in services that started with a significant delay compared to the initial project schedule. This in turn led to a delayed first disbursement record under the Project. 101. Road Design Consultancy Services. Whenever possible the Teams should seek for the road design documents to be ready before the Project Effectiveness or a more realistic timeframe for the development of designs should be envisaged within the Project life. The quality and delays of outputs by the design consultants have significantly delayed the start of the rehabilitation works under the RRSP. Accumulated delays in design preparation in some cases resulted in the exclusion of road sections from the Project scope due to the impossibility to finish or even start the Works within the project life. Some of the lessons learned include: • Preparation of design with road safety features procedures took much more time than originally estimated. • Input data for designing was often unreliable, especially on the cadastral of underground utilities in settlements. That caused a need to do the details of design solutions on the spot, per the conditions in the field. Moreover, the excavation of trial pits for determining the location and the position of utilities required special expertise and led to the slowdown of the work progress and variation orders. • Although the PERS has taken all measures at the stage of the design preparation (the survey of the interested parties, the presentation of the preliminary design to the interested parties, the presentation of the design) and made public calls through the media, some Project affected people expressed their interest or concerns only after the machines have arrived at the construction site. • Market capability, capacity, and competition should be taken into serious consideration. Local designers and contractors were more interested in familiar locally financed projects rather than IFIs’s financed projects due to IFIs’s unfamiliar requirements and procedures, which often were regarded as cumbersome. • The capacity of local designers and contractors became inadequate in doing multiple contracts simultaneously. 102. Contract Management. The introduction of financial penalties for Consultant’s poor quality and unjustified delays could have remedied the situation. However, apart from the standard of performance, there are not many contractual clauses that deal with the Consultant’s quality of execution of the Services. The only available means for the Client are either to suspend the payments under the contract (which was not a highly efficient measure, since the design contracts were lump-sum and payments were to be made when certain output is achieved) or to terminate the contracts. Page 31 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) RRSP proceeded with having the design even with several months of delays compared to the option of having another selection with almost the same market players (which after several months would potentially lead to the very same situation). In addition, to overcome the designs delay issue, for some of the Works under the EIB financing, and after the termination of the design contract, the agreement was reached to proceed with the Design and Build model. 103. Coordination between the contractor and design consultant under the Design and Build contract. At the time of implementation of the emergency works component, coordination between the contractor and designer was a serious issue. Designs were delayed, and the issue was even more complicated by the fact that almost the same 2-3 designer companies were engaged by the contractors under the design-build contract. It is obvious that although most of the contractors do have relevant experience on international projects and international models of contracts, it is still desirable to conduct additional trainings and workshops to raise the capacity within the contractor companies to comply with international standards and requirements. 104. In the face of more frequent and intense climatic events occurring in the region and worldwide, it is a must to plan, design, construct and effectively operate and maintain climate resilient infrastructure in a cross - sector integrated/coordinated manner to minimize vulnerability of the physical assets to natural hazards. In May 2014 a state of emergency has been declared in the whole country of Serbia after the heaviest rain and worst floods since records began some 120 years ago. Overflowing rivers have burst into towns and villages, cutting off whole communities, while landslides have buried houses. The cross - sectoral coordinated work by MoT, Ministry of Emergency and other bodies was key in responding to the events, and extended to the integrated planning and preparedness too. Lessons learned from RRSP were incorporated in future roads projects by Serbian MoT. Given significant weather- and disaster-related risks, it is advisable to include the Contingency Emergency Response Component (CERC) in transport projects to allow for immediate response in emergencies. 105. Project Implementation during Covid-19. One of the important lessons under the RRSP was the readiness to embrace changes, no matter how difficult they are, and avoid project paralysis. Covid-19 and limitations introduced by the declaration of the state of an emergency certainly negatively affected the Project activities. Import of goods and conduct of fieldwork, travels of foreign staff were disabled or implemented with major difficulties. All these required adoption and implementation of measures that would mitigate the effects of those difficulties. One of the examples of agility was public opening of bids for the supply of goods in international bidding procedures conducted online through live streaming on Zoom platform. . Page 32 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: To improve the condition and safety of the Serbian National Road Network for road users Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Reduction in road traffic Percentage 100.00 90.00 78.00 accidents with fatalities on the improved roads in the 30-Apr-2014 31-Oct-2018 30-Dec-2020 first phase of NRNRP Comments (achievements against targets): Indicator met, and exceeded the target. Based on the traffic police report for the first 9 months of 2020 (data for the last quarter is not ready) there were 24 fatal road crashes on the improved roads of the RRSP road network, which represents a 22% reduction compared to the baseline indicator. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improvement in road Number 5.34 2.50 1.90 Page 33 of !Syntax Error, ! Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) conditions measured by a 30-Oct-2014 31-Oct-2018 30-Dec-2020 reduction in the average IRI for the roads in the first phase of NRNRP Comments (achievements against targets): The International Roughness Index (IRI) has improved from 5.34 to 1.90, which exceeded the original target of 2.5 by 24%. Note: a higher IRI value indicates a rougher road surface. A.2 Intermediate Results Indicators Component: Road Rehabilitation and Safety Investments (264 million Euros) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Roads in good and fair Percentage 58.79 63.50 61.30 63.70 condition as a share of total classified roads 30-Oct-2014 31-Dec-2019 31-Dec-2020 31-Dec-2020 Size of the total classified Kilometers 9000.00 9000.00 16,900.00 15,076.00 network Comments (achievements against targets): Roads in good and fair condition as a share of total classified roads has increased from 58.79% of 9,000km to 63.7% of 15,076km, or from 5,220km to 9,603km (increase by 4,383km). A share of total classified roads in PAD refers to 9,000km, supported by the component on Road Network Condition Survey (RNCS) on national roads of 9,000 km (another intermediate results indicator). Later, the Restructuring papers updated the number to 16,900 km, which is Page 34 of !Syntax Error, ! Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Serbia's total national road network. In 2018, 11,456 km of National Roads were surveyed. Some 64.7% of the surveyed state road network was in good and fair condition. In 2020 some 15,076 km of the national road network was completed with 63.7% of the surveyed state road network being in good and fair condition (IRI ≤4.0 m/km). Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Roads rehabilitated, Non- Kilometers 0.00 1125.00 389.80 416.50 rural 15-Dec-2014 30-Oct-2018 31-Dec-2020 30-Dec-2020 Comments (achievements against targets): As per the 2020 Annual Performance Audit Report, a total of 416.5 km of roads were rehabilitated and Certificates of Completion issued. This is a significant reduction from the 800km target set up in the PAD for RRSP and the 1,125km target if EBRD's sections are added, but this exceeds the revised target of 389.8km by 6%. Component: Institutional Strengthening (3.10 million Euros) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Road Condition Survey Kilometers 0.00 9000.00 15,076.00 Page 35 of !Syntax Error, ! Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Completed on National 30-Oct-2014 31-Oct-2018 31-Dec-2020 Roads Comments (achievements against targets): 11,456 km of Road Network Condition Survey on National Roads was completed by PBMC Road Network Survey in 2018. 15,076 km of Road Network Condition Survey on National Roads was completed in 2020, which is 167.5% higher than the targeted 9,000 km. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Safety Inspection of National Kilometers 0.00 1000.00 1,016.00 Roads beyond the first phase of NRNRP 30-Oct-2014 31-Oct-2018 15-Jun-2020 Comments (achievements against targets): Within the contract for Road Safety Inspection, the Consultant performed an inspection of 1,016 km as presented in the Final Report from January 26, 2018. Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at Page 36 of !Syntax Error, ! Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Target Completion PBMC Contracts in Kilometers 0.00 3000.00 3,000.00 Implementation 30-Oct-2014 31-Oct-2018 15-Jun-2020 Comments (achievements against targets): Six (6) PBM Contracts in total length of 3,000 km were signed and implemented, which is equal to the targeted 3,000 km. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Implementation of a circular Text None, except for on IFI Implementation of a Road Maintenance for the preparation of funded projects RMC Circular (RMC) is rehabilitation program based prepared and ready socioeconomic & regional for use. considerations 30-Oct-2014 31-Oct-2018 15-Jun-2020 Comments (achievements against targets): Road Maintenance Circular (RMC) is prepared and ready for use. The PERS’s PIT prepared a draft of the General Manager’s Decision for implementation of a Road Maintenance Circular and the Decision is waiting for checking by PERS’s Legal Department and the official signature of the General Manager of PERS. Page 37 of !Syntax Error, ! Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 To improve the condition of the Serbian National Road Network for road users 1. Improvement in road conditions measured by a reduction in the Outcome Indicators average IRI for the roads in the first phase of NRNRP. 1. Roads in good and fair condition as a share of total classified roads (%) 2. Road Condition Survey Completed on National Roads (km) Intermediate Results Indicators 3. Roads rehabilitated, Non-rural (km) 4. PBMC Contracts in Implementation (km) 5. Implementation of a circular for the preparation of rehabilitation program based socioeconomic & regional considerations. 1. The IRI has improved from 5.34 to 1.90, which exceeded the original target of 2.5 by 24%. Note: a higher IRI value indicates a rougher road surface. 2. Roads in good and fair condition as a share of total classified roads has increased from 58.79% to 63.7%. 3. Road Maintenance Circular (RMC) is prepared and ready for use. The PERS’s PIT prepared a draft of the General Manager’s Decision for Key Outputs by Component implementation of a RMC. (linked to the achievement of the Objective/Outcome 1) 4. Six PBM Contracts in total length of approx. 3,000 km were signed and implemented. 5. As per the Annual Performance Audit Report, a total of 416.5 km of roads were rehabilitated and Certificates of Completion issued. This exceeds the revised target of 389.8km by 6%. 6. 15,076 km of Road Network Condition Survey on National Roads is completed by RNCS Consultant in 2020, which is 167.5% of the targeted 9,000 km. 63.7% of the surveyed state road network was Page 38 of !Syntax Error, ! Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) in good and fair condition (IRI ≤4.0 m/km) as part of the total surveyed national road network. Objective/Outcome 2 To improve the safety of the Serbian National Road Network for road users 1. Reduction in road traffic accidents with fatalities on the improved Outcome Indicators roads in the first phase of NRNRP. 1. Safety Inspection of National Roads beyond the first phase of Intermediate Results Indicators NRNRP (km) 1. Based on the traffic police report for the first 9 months of 2020, there were 24 fatal road crashes on the improved roads of the RRSP Key Outputs by Component road network, which represents a 22% reduction compared to the (linked to the achievement of the Objective/Outcome 2) baseline indicator. 2. 1,016 km of roads have been inspected by the road safety expert, as presented in the Final Report from January 26, 2018. Page 39 of !Syntax Error, ! Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Moustafa Baher El-Hefnawy Task Team Leader(s) Kashmira Hoshang Daruwalla Procurement Specialist(s) Aleksandar Crnomarkovic Financial Management Specialist I. U. B. Reddy Social Specialist Nikola Ille Environmental Specialist Sandra Kdolsky Social Specialist Supervision/ICR Murad Gurmeric, Svetlana Vukanovic Task Team Leader(s) Benedicta T. Oliveros Procurement Specialist(s) Aleksandar Crnomarkovic Financial Management Specialist Graciela A. Tejeda Team Member Nikola Ille Environmental Specialist Desanka Stanic Procurement Team Andoria Indah Purwaningtyas Procurement Team Ifeta Smajic Social Specialist B. STAFF TIME AND COST Page 40 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 9.703 54,927.83 FY13 47.301 244,160.79 FY14 .125 277.00 Total 57.13 299,365.62 Supervision/ICR FY13 1.655 6,938.40 FY14 17.747 92,512.72 FY15 28.020 130,402.41 FY16 19.905 86,166.71 FY17 33.161 129,207.86 FY18 31.398 113,809.11 FY19 29.836 121,382.01 FY20 22.832 88,423.64 Total 184.55 768,842.86 Page 41 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) ANNEX 3. PROJECT COST BY COMPONENT Amount at Approval Actual at Project Components Percentage of Approval (Euro,M) Closing (Euro,M) Road Rehabilitation and Safety Investments (264 59.1 39.3 66 million Euros) Institutional Strengthening 0.7 1.20 171 (3.10 million Euros) Project Detailed Design, Supervision, Management 4.2 12.7 302 and Monitoring (18.4 million Euros) Front-End-Fee 0.18 0.18 100 Physical Contingencies 6.4 3.02 31 Price Contingencies 3.2 Total 73.80 56.4 76 Page 42 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) ANNEX 4. EFFICIENCY ANALYSIS/Ex-Post Economic Analysis The ex-ante economic analysis focused on the road rehabilitation under Component 1 of the Project. The economic analysis was undertaken for the entire length of the first eighteen months of road works financed by the World Bank, EIB and EBRD. The economic analysis was done using the Highway Development and Management Model (HDM-4) using the conventional approach of comparing the estimated road user and agency benefits and costs in the "do-something" scenario, when the road is rehabilitated, against the "do-minimum" scenario, representing the continuation of the current maintenance regime. The main project benefits are savings in vehicle operating costs and passenger time costs. The first 18-month (2013-2015) program of first phase of NRNRP covered 22 homogeneous road sections grouped under 15 prospective contracts with a total length of 308 km amounting to Euro 103.3 million of investment value. Of these 22 road sections, some were excluded from the project, some were financed by EBRD, and for others the contracts are ongoing, thus, the ex-post economic analysis excluded those road section. The remaining nine road sections, financed by WB/EIB/PERS, were evaluated for the ICR. The table below shows the status of the first 18-month program at the ICR Stage. Table 1: First Year Program Roads Evaluated at Appraisal Road Section Code Road Section Description Financing Status at ICR Stage M-4S (0218-0219) Arandjelovac 3 - Krcevac WB/EIB/PERS Included on ICR M-21V (0441-0443) Iriski Venac-Ruma 2 (Putinci) / Irig 2 (Vrdnik)-Ruma 2 (Putinci) WB/EIB/PERS Included on ICR M-21V (0444-0450) Ruma 2 (Putinci)-Sabac 6 / Ruma 2 (Putinci)-Ruma 1 (autoput) WB/EIB/PERS Included on ICR M-5S (0252) Uzice 3 - Pozega WB/EIB/PERS Included on ICR M-5S (0253) Pozega - Kratovska Stena WB/EIB/PERS Included on ICR M-9S (0365) Vlasotince - Svodje WB/EIB/PERS Included on ICR M-23S (0619-0621) Gornja Trnava (ili Topola 2) - Cerovac WB/EIB/PERS Excluded from Project M-23S (0622) Cerovac - Kragujevac 5 WB/EIB/PERS Excluded from Project M-4S (0212-0215) Lazarevac 4 - Arandjelovac 1 WB/EIB/PERS Included on ICR M-21V (0439) Paragovo - Iriski Venac WB/EIB/PERS Excluded from Project M-21V (0440) Iriski Venac - Paragovo WB/EIB/PERS Excluded from Project M-21.1S (0481) Pozega - Arilje WB/EIB/PERS Contract is on-going M-21.1S (0482-0483) Arilje - Ivanjica WB/EIB/PERS Contract is on-going M-8S (0342-0343) Stavalj - Susica WB/EIB/PERS Excluded from Project M-4S (0188-0191) Zavlaka 2 (Likodra) - Valjevska Kamenica WB/EIB/PERS Included on ICR M-25S (0702-0703) Kladovo - Brza Palanka WB/EIB/PERS Included on ICR M-7V (0312) Zabalj 1 - Zrenjanin 1 (obilaznica) EBRD Not Included on ICR M-1V (0003) Adasevci - Kuzmin 1 (autoput) EBRD Not Included on ICR M-1V (0004) Kuzmin 1 (autoput) - Adasevci EBRD Not Included on ICR M-21S (0471) Bela Zemlja - Knezevici EBRD Not Included on ICR M-21S (0470) Uzice 3 - Bela Zemlja EBRD Not Included on ICR M-25S (0719-0722) Vratarnica - Knjazevac 2 EBRD Not Included on ICR The ex-post economic analysis of the project for the ICR was done also with the HDM-4 model adopting the same key parameters defined at appraisal, such as the discount rate, evaluation period, vehicle fleet characteristics and maintenance costs. The ex-post economic analysis was done considering the actual length of the road works, the actual investment costs and the actual traffic observed on the project road Page 43 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) sections from 2014 to 2019. The table below presents the nine road sections evaluated for the ICR, the length of the road sections estimated at appraisal and the actual length, and the ratio between the actual and appraisal lengths. The total length of the nine road sections evaluated for the ICR was estimated at appraisal to be 131.2 km, while the actual total length is 142.0 km, representing an 8 percent increase in the total road sections length. Table 2: Roads Evaluated at ICR and their Length (km) Appraisal Actual Actual / Road Section Code Road Section Description Length (km) Length (km) PAD M-4S (0218-0219) Arandjelovac 3 - Krcevac 8.7 8.7 1.00 M-21V (0441-0443) Iriski Venac-Ruma 2 (Putinci) / Irig 2 (Vrdnik)-Ruma 2 (Putinci) 9.3 9.2 0.98 M-21V (0444-0450) Ruma 2 (Putinci)-Sabac 6 / Ruma 2 (Putinci)-Ruma 1 (autoput) 6.4 6.4 1.00 M-5S (0252) Uzice 3 - Pozega 23.4 27.7 1.19 M-5S (0253) Pozega - Kratovska Stena 7.9 9.4 1.19 M-9S (0365) Vlasotince - Svodje 12.7 12.5 0.99 M-4S (0212-0215) Lazarevac 4 - Arandjelovac 1 13.7 17.8 1.30 M-4S (0188-0191) Zavlaka 2 (Likodra) - Valjevska Kamenica 24.8 27.0 1.09 M-25S (0702-0703) Kladovo - Brza Palanka 24.3 23.3 0.96 Total 131.2 142.0 1.08 The table below presents the road works costs estimated at appraisal, the contract costs and the actual costs including cost increases. In total, the actual costs are 5 percent lower than the appraisal cost estimates. The total actual costs are 0.4 percent higher than the total contract costs. Table 3: Roads Evaluated at ICR Costs (M Euros) Appraisal Contract Actual Contract / Actual / Road Section Code Cost (M Euros) Cost (M Euros) Cost (M Euros) Appraisal Appraisal M-4S (0218-0219) 3.4 3.5 3.2 1.05 0.95 M-21V (0441-0443) 3.3 2.8 3.0 0.85 0.92 M-21V (0444-0450) 2.3 2.0 2.1 0.86 0.93 M-5S (0252) 6.1 8.1 8.4 1.33 1.37 M-5S (0253) 3.6 2.7 2.8 0.75 0.78 M-9S (0365) 7.2 6.2 6.2 0.86 0.86 M-4S (0212-0215) 10.5 6.2 6.2 0.59 0.59 M-4S (0188-0191) 13.0 8.7 7.8 0.67 0.60 M-25S (0702-0703) 9.0 14.9 15.5 1.65 1.72 Total 58.4 55.1 55.3 0.94 0.95 The table below presents the road works costs per km estimated at appraisal, the contract costs per km and the actual costs per km including cost increases. On average, the actual costs per km are 88 percent of the appraisal costs per km. However, there is a high variation between the project road sections comparing the actual and appraisal costs per km, with some road sections having actual costs per km at around half the appraisal estimates and other road sections having actual costs per km around double the appraisal estimates. Page 44 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Table 4: Roads Evaluated at ICR Costs per Km (M Euros/km) Appraisal Contract Actual Actual / Road Section Code Cost (M Euros/km) Cost (M Euros/km) Cost (M Euros/km) Appraisal M-4S (0218-0219) 0.387 0.404 0.367 0.95 M-21V (0441-0443) 0.352 0.305 0.329 0.93 M-21V (0444-0450) 0.353 0.305 0.329 0.93 M-5S (0252) 0.261 0.292 0.302 1.16 M-5S (0253) 0.458 0.292 0.302 0.66 M-9S (0365) 0.570 0.499 0.499 0.87 M-4S (0212-0215) 0.767 0.347 0.348 0.45 M-4S (0188-0191) 0.523 0.321 0.290 0.55 M-25S (0702-0703) 0.372 0.640 0.667 1.79 Average 0.445 0.388 0.390 0.88 The table below presents the Average Annual Daily Traffic (AADT), in vehicles per day, estimated at appraisal in 2010 and 2019 and the actual traffic in 2014 and 2019. In 2019, the average actual traffic on the road sections is 69 percent of the average traffic estimated at appraisal. At appraisal, the traffic annual growth from 2010 to 2019 was estimated to be on averaged around 3.5 percent per year, with the actual traffic annual growth from 2014 to 2019 was 2.5 percent per year. Thus, there were less actual project benefits associated to the reduction of road user costs on the road sections as compared with the appraisal estimates. Table 5: Roads Evaluated at ICR Traffic (AADT, vehicles per day) Appraisal 2010 Appraisal 2019 Actual 2014 Actual 2029 2019 Actual / Road Section Code Traffic (AADT) Traffic (AADT) Traffic (AADT) Traffic (AADT) Appraisal M-4S (0218-0219) 4,100 5,805 2,251 2,863 0.49 M-21V (0441-0443) 9,677 13,701 8,361 10,303 0.75 M-21V (0444-0450) 9,831 13,917 8,361 10,303 0.74 M-5S (0252) 10,371 14,684 9,516 10,972 0.75 M-5S (0253) 8,450 11,963 9,516 10,972 0.92 M-9S (0365) 3,593 5,088 2,697 1,055 0.21 M-4S (0212-0215) 3,082 4,327 2,644 3,052 0.71 M-4S (0188-0191) 2,318 3,254 1,301 1,699 0.52 M-25S (0702-0703) 1,914 2,663 1,784 1,877 0.70 Average 5,258 7,427 4,534 5,130 0.69 The table below compares the Economic Internal Rate of Return (EIRR) and Net Present Value (NPV), at 12 percent discount rate, estimated at appraisal and the actual EIRR and NPV values based on the actual length, costs and traffic data on the road sections. The total ex-post actual NPV is satisfactory at Euro 30.6 million, which can be compared to the total NPV of Euro 61.7 million estimated at appraisal, representing a 50 percent decrease on the project net benefits compared to the appraisal estimates. The total ex-post EIRR satisfactory at 17.6 percent, for the road sections evaluated for the ICR, which can be compared with the total EIRR estimated at appraisal that is 24.4 percent. The lower total NPV and EIRR found at the ICR stage is due mainly to lower than expected annual traffic growth on the road sections. The ex-post EIRR (17.6 percent) is higher than the adopted discount rate of 12 percent and the ex-post NPV is positive, Page 45 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) therefore, the ex-post economic analysis indicates that while there was a reduction on the project NPV and EIRR compared to the appraisal estimates, the ex-post actual NPV and EIRR of the project are still satisfactory Indicating that the project had a good economic justification. Table 6: ICR Ex-Post Economic Appraisal Summary Appraisal Appraisal Actual Actual EIRR NPV EIRR NPV NPV Actual / Road Section Code (%) (Euro million) (%) (Euro million) Appraisal M-4S (0218-0219) 27.2% 5.6 16.8% 1.3 0.23 M-21V (0441-0443) 37.3% 10.1 32.0% 7.0 0.69 M-21V (0444-0450) 36.6% 6.3 30.1% 4.1 0.65 M-5S (0252) 33.5% 14.7 23.9% 10.2 0.69 M-5S (0253) 31.0% 2.9 37.6% 3.9 1.35 M-9S (0365) 20.0% 5.2 13.5% 0.7 0.13 M-4S (0212-0215) 16.2% 3.4 17.6% 3.6 1.07 M-4S (0188-0191) 18.8% 7.1 17.3% 3.3 0.46 M-25S (0702-0703) 21.3% 6.4 8.0% -3.4 -0.53 Total 24.4% 61.7 17.6% 30.6 0.50 The ex-post economic analysis for the ICR was done without including road safety benefits due to the reduction in road fatalities with the project to be able to compare the results with the economic analysis done at appraisal that did not include road safety benefits. In addition, a second ex-post economic analysis including road safety benefits was done considering: (i) a 22 percent reduction on road fatalities per year observed on the project roads after the road improvements; (ii) there were around 17 fatalities per year on the roads subject to the economic analysis before the project; and (iii) an economic cost per fatality of Euro 0.43 million per fatality. Including road safety benefits the overall ex-post EIRR is 21.0% and the NPV, at 12 percent, is Euro 42.21 million. Page 46 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Thank you for sharing the ICR with us. From our view, the Report is well prepared and properly reflects the circumstances related to implementation of the RRSP. Few minor comments from our side are the following: - Page 2 of the ICR: Information on Project financing is not consistent in terms of USD/EUR currencies. The Project currency is EUR and relevant shares of EIB and EBRD financing are 100 million EUR each. For easier understanding of the comment, we are attaching a table showing the Project financing per Banks’ involvement1; - Page 13, Para 22: Closing Date for the EBRD sub-project is December 31, 2021, although there might be an extension (the request still has not been initiated by the RoS); - On page 15, Para 31, there is a reported error in the document, please check - Page 16, Para 36 – EIB Project Closing Date is June 2022 - Page 26, Para 76 the correct figures are 38.41 m2 for expropriation and 274.21 m2 for relocation of fences. Kindly note that relocation of fences still has not taken place as Works on the section Ivanjica – Sjenica have not been contracted yet. - Pg 26, Para 77 the Figure 5 seems to be wrong, instead of the presented ones we believe that the following two should be placed as evidences of roadside business before and after rehabilitation on IB23 Before After - Page 29, Para 90 – just to clarify that change in number of km of national road network was a result of change in road classification which was introduced after the WB Loan was signed; 1The ICR’s pages 2 and 10 have been updated as per the Borrower’s latest data on financing/disbursed amounts from the GoS and EIB sources. Page 47 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) ANNEX 6. SUPPORTING DOCUMENTS (PHOTOS AND REPORTS) A. Visual Representation of the RRSP Activities: Before and After Figure 4 Improvement of road condition and safety on section IB29 Prijepolje-Sjenica: Before and After Page 48 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Page 49 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Figure 5. Improvement of road condition and safety on section IB39 Vlasotince-Svođe from km 58+876 to km 46+362, L=12.514 km Page 50 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Figure 6 IB 29, Section: Prijepolje - Sjenica 3 (Medare), from km 39+100 to km 52+900, L=13.800 km Page 51 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) B. Documents used for the purposes of this ICR Government of Serbia, MoT/PERS/PIT Implementation Completion Results Report, January, 2021 Project Audit Consultant’s Final Report, March 2021 Project Website https://www.putevi-srbije.rs/index.php/en/road-rehabilitation-and-safety- project World Bank, Country Economic Memorandum Report Serbia’s New Growth Agenda, 2020 World Bank, Project Appraisal Document Report No: 72287-YF, 2013 World Bank, Restructuring Paper Report No: RES31560, 2018 World Bank, Restructuring Paper Report No: RES41232, 2020 World Bank, Implementation Supervision and Results Reports World Bank, Aide Memoires Page 52 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) ANNEX 7. RESULTS OF BENEFICIARY SURVEYS Polls on degree of road users’ satisfaction2 Survey findings for road sections EWC 01 and EWC 06 No. of Positive Negative Section: Question regarding: respondents: response response Support to the project 20 Traffic disruption 20 EWC 01: IB 27 Information on works 20 Arandjelovac 4 Pollution 20 20 (Orasac) - Krcevac, Noise 19 1 length: 8.713 km Comfort of road use 20 Traffic safety 20 Desirability of the project 20 Support to the project 20 Traffic disruption 20 Information on works 20 Pollution 20 20 Noise 18 2 Comfort of road use 20 EWC 06: IB 28 Mali Traffic safety 20 Zvornik - Gracanica, Desirability of the project 20 length: 23.620 km + Support to the project 15 Additional 1.8 km Traffic disruption 14 1 Information on works 15 Pollution 13 2 15 Noise 13 2 Comfort of road use 15 Traffic safety 14 1 Desirability of the project 15 431 (98%) 9 (2%) 55 440 answers positive negative respondents respons response 2For all EWC sections, polls on degree of road users’ satisfaction have been carried out by the Contractors as a part of As-Built Designs. Since it was not defined which questionnaire should be used, there were two different questionaries on EWCs (the Contractors on EWCs 01 and 06 used one type of the questionnaire and the Contractors on EWCs 02-05 used different one). Page 53 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Summary for EWC 01 and EWC 06 Positive response Negative response Support to the project 55 (100%) 0 (0%) Traffic disruption 54 (98%) 1 (2%) Information on works 55 (100%) 0 (0%) Pollution 53 (96%) 2 (4%) Noise 50 (91%) 5 (9%) Comfort of road use 55 (100%) 0 (0%) Traffic safety 54 (98%) 1 (2%) Desirability of the project 55 (100%) 0 (0%) Summary for EWC 01 and EWC 06 431 (98%) 9 (2%) Survey findings for road sections EWC 02, EWC 03, EWC 04 and EWC 05: No. of Surpassed Fulfilled Partially Unfulfilled Section: Question regarding: Inadequate respondents: expectations expectations fulfilled expectations Quality of works 3 5 2 Road improvement 4 6 Cooperation with local EWC 02: IB 27 4 6 population Lazarevac 4 - Arandjelovac 10 Safety and health of 2 7 1 1, length: local residents 17.607 km Neatness and 3 5 2 cleanliness Environmental 3 6 1 protection Quality of works 2 6 2 Road improvement 2 8 Cooperation with local EWC 03: IB 27 4 5 1 population Zavlaka 2 - Pricevici, 10 Safety and health of 2 6 2 length: 27.018 local residents km Neatness and 2 5 3 cleanliness Environmental 2 6 2 protection EWC 04: IB 27 Quality of works 1 6 3 Krst - Zavlaka 10 Road improvement 6 4 2, length: Cooperation with local 5 5 Page 54 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) 20.279 km population Safety and health of 2 8 local residents Neatness and 2 8 cleanliness Environmental 4 6 protection EWC 05: IB 26 Quality of works 6 13 5 Banja Road improvement 9 12 3 Koviljaca - Cooperation with local Mali Zvornik, 7 12 5 population length: 14.150 Safety and health of km and IB 28 24 4 17 3 local residents section: Mali Neatness and Zvornik - 4 17 3 cleanliness Ljubovija 1, length: 13.526 Environmental 3 19 2 km protection 54 40 324 answers 86 (27%) 198 (61%) 0 (0%) 0 (0%) respondents (12%) Surpassed Fulfilled Partially Unfulfilled Summary of EWCs 02-05 Inadequate expectations expectations fulfilled expectations Quality of works 12 (22%) 30 (56%) 12 (22%) Road improvement 21 (39%) 30 (56%) 3 (5%) Cooperation with the local population 20 (37%) 28 (52%) 6 (11%) Safety and health of local residents 10 (19%) 38 (70%) 6 (11%) Neatness and cleanliness 11 (20%) 35 (65%) 8 (15%) Environmental protection 12 (22%) 37 (69%) 5 (9%) Summary of EWCs 02-05 86 (27%) 198 (61%) 40 (12%) 0 (0%) 0 (0%) Survey findings for Regular works road sections IB23PU, IB39VS, IB21IR, IB29PS, IB15VS, IB10PK, IB27MS, IB39LB, and IB23/28KBZU: Page 55 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Final Total number of participants who answered conclusion of Total the applied Contract ID NO number of No “Neither Questionnaire Road Section participants “Very high” “High” high nor “Low” “Very low” on degree of (%) (%) (%) low” (%) (%) satisfaction of (%) users 1 RRSP/RRW- Successfully IB23PU/2018-09 completion of Road Section 19 2 6 4 3 4 the works to IB23PU Pakovrace (100%) (11%) (31%) (22%) (15%) (21%) moderate (Markovica) - Uzice satisfaction of from km 10+485 to the users. km 47+586 2 RRSP/RRW- Successfully IB39VS/2017-08 completion of Road Section 9 2 3 2 1 1 the works to IB39VS Vlasotince - (100%) (22%) (33%) (23%) (11%) (11%) moderate Svodje from km satisfaction of 58+876 to km the users. 46+362 3 RRSP/RRW- Successfully IB21IR/2017-09 completion of Road Section 34 12 11 4 2 1 the works to IB21IR Irig – Ruma (100%) (35%) (32%) (12%) (6%) (3%) satisfaction of from km 26+680 to the users. km 47+586 4 RRSP/RRW- IB29PS/2018-02 Successfully Road Section completion of 17 11 1 2 1 2 IB29PS Prijepolje – the works to (100%) (64%) (6%) (12%) (6%) (12%) Sjenica 3 (Medare) satisfaction of from km 39+100 to the users. km 52+900 5 RRSP/RRW- IB2328KBZU/2018- Successfully 03 completion of Road Section 20 0 4 6 3 7 the works to IB23/28KBZU (100%) (0%) (20%) (30%) (15%) (35%) moderate Knezevici – Bela satisfaction of Zemlja – Uzice the users. from km 145+140 to km 160+374 Page 56 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) 6 RRSP/RRW- Successfully IB15VS/2018-06 22 5 7 7 1 1 completion of Road Section IB15 (100%) (22%) (32%) (32%) (6%) (6%) the works to Vrbas (Zmajevo) - 11 legal 1 2 7 1 0 moderate Srbobran (Feketic) entities (13%) (18%) (63%) (6%) (0%) satisfaction of from km 78+298 to 33 in total 6 (18%) 9 (27%) 14 (42%) 2 (6%) 1 (3%) the users. km 87+790 7 RRSP/RRW- IB10PK/2018-04 Road Section IB10 Border APV (Pancevo) – Successfully Pancevo (Kovin) completion of and Pancevo 5 6 6 0 1 the works to (Kovin) – Border 18 (100%) (28%) (33%) (33%) (0%) (6%) moderate APV (Pancevo) satisfaction of from km the users. 10+185,00 to km 13+047,80, (LHS) and from km 10+190,32 to km 13+040,97 (RHS) 8 RRSP/RRW- IB39L/2018-05 Road Section Successfully IB39LB Leskovac completion of South (Connection 34 20 9 1 1 1 the works to with A1) – (100%) (58%) (27%) (3%) (3%) (3%) high Leskovac satisfaction of (Bratimilovce) the users. from km 68+241 to km 74+740 9 RRSP/RRW- IB27MS/2018-07 Road Section Successfully IB27MS completion of Intersection 19 5 9 2 1 2 the works to Markovac - (100%) (27%) (51%) (11%) (4%) (7%) high Svilajnac from km satisfaction of 189+753 to km the users. 195+961, L=6,208km The percentage of RRSP WB FINAL 97% is CUMULATIVE RESULTS 203 63 58 41 14 20 resulted by FROM THE BENEFICIARY (100%) (31%) (29%) (20%) (7%) (10%) not marked FEEDBACK SURVEYS options of the Questionnaire Page 57 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) ANNEX 8. JOB CREATION UNDER THE PROJECT Cumulative number of new created person-months of employment for first and second year Regular Work contracts: No of person-months No of person-months of new employment of new employment Cumulative person-months of created last month Road created from Start new employment from Start Start No. (from December 1st Section Date till November Date till December 31st 2020 Date: 2020 till December 30th 2020 31st 2020) Men Female Men Female Male Female Total Septembe 1 IB21IR 75 1 - - 75 1 76 r 11th 2018 August 2 IB39VS 55 1 - - 55 1 56 31st 2018 January 3 IB23/28KBZU 10 6 - - 10 6 16 04th 2019 January 4 IB29PS 43 - - - 43 - 43 04th 2019 June 07th 5 IB23PU 38 20 - - 38 20 58 2019 May 22nd 6 IB15VS - - - - - - - 2019 June 06th 7 IB10PK 680 - - - 680 - 680 2019 June 19th 8 IB39LB - - - - - - - 2019 July 24th 9 IB27MS - - - - - - - 2019 Total number of new created 901 28 - - 901 28 929 employment (person-months) Page 58 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) The Labour influx for first and second year Regular Works Contracts: Total number of Total number of person-months person-months Total cumulative number of employment (from employment (from person-months employment Road Start Date till December 1st 2020 (from Start Date till December Start No. Section November 30th till December 31st 31st 2020) Date: 2020) 2020) Men Female Men Female Male Female Total Septemb 1 IB21IR 1053 99 - - 1053 99 1152 er 11th 2018 August 2 IB39VS 2020 47 - - 2020 47 2067 31st 2018 IB23/28KBZ January 3 1603 132 70 6 1673 138 1811 U 04th 2019 January 4 IB29PS 637 40 - - 637 40 677 04th 2019 June 07th 5 IB23PU 766 38 - - 766 38 804 2019 May 22nd 6 IB15VS 267 23 - - 267 23 290 2019 June 06th 7 IB10PK 836 31 - - 836 31 867 2019 June 19th 8 IB39LB 1016 34 68 2 1084 36 1120 2019 July 24th 9 IB27MS 615 28 - - 615 28 643 2019 Total number of employment 8813 472 138 8 8951 480 94313 (person-months) 3 Total number of employment (person-months) includes the number of new created employment as per previous table Page 59 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) ANNEX 9. DLI TARGETS (ORIGINAL, REVISED, AND ACHIEVED) ORIGINAL DLI TARGETS Total Disbursement as 12/2018 Disbursement 12/2015 12/2016 12/2017 a percentage of extension Amount (million total loan Euros) 1. DLI 1: Road DLI 1.1 DLI 1.2 DLI 1.3 DLI 1.4 Rehabilitation & Safety Km of road rehabilitated and safety measures 125 203 576 795 incorporated under the Project (cumulative) Disbursement amount due 13.87 11.38 7.34 1.78 34.38 46.58% (million Euros) / percept 2. DLI 2: Road safety actions DLI 2A: Road safety DLI 2A.1 DLI 2A.2 DLI 2A.3 DLI 2A.4 inspections contract signing: Road Safety 750 km inspected & 1,000 km inspected 700 km safety and km of roads covered by inspection Contract 250 km safety & 500 km safety measures road safety inspection and Signed measures measures implemented low cost road safety implemented implemented measures implemented (cumulative) DLI 2B: Capacity building DLI 2B.1 DLI 2B.2 and awareness campaigns 20 persons certified awareness to do road safety campaigns in 20 audit/inspection schools in vicinity of project roads Page 60 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) Disbursement amount due 6.93 5.69 3.67 0.89 17.19 23.29% (million Euros) / percent 3. DLI 3: Maintenance DLI 3.1Adoption of Management reforms revised maintenance & rehabilitation design standards; DLI 3.2 Preparation DLI 3.4 DLI 3.6 of strategic plan for PBMC for an PBMC for an the introduction of additional 1,000 km additional 1,0500 PBMC; competitively km competitively tendered; tendered; DLI 3.3 DLI 3.5 DLI 3.7 PBMC for 1,000 km PBMC for 1,000 km PBMC for an competitively in implementation additional 1,000 km tendered in implementation Disbursement amount due 6.93 5.69 3.67 0.89 17.19 23.29% (million Euros) / percent TOTAL Disbursement 27.734 22.762 14.685 3.569 68.75 93.16% amount Page 61 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) REVISED DLI TARGETS Total Disbursement Disbursement as a 12/2015 12/2016 12/2017 12/2018 12/2019 Amount (mln percentage of Euros) total loan 1. DLI 1: Road Rehabilitation & DLI 1.1 DLI 1.2 DLI 1.3 DLI 1.4 Safety Km of road rehabilitated and safety measures 50.25 127.02 157.02 248.60 incorporated under the Project (cumulative) Disbursement amount due (million Euros) / 8.00 1.60 3.10 8.60 21.30 37.9% percept 2. DLI 2: Road safety actions DLI 2A: Road safety DLI 2A.1 DLI 2A.2 DLI 2A.3 DLI 2A.4 inspections contract Road Safety 750 km 1,000 km 500 km safety signing: and km of inspection inspected inspected & 100 measures roads covered by road Contract km safety implemented safety inspection and Signed measures low cost road safety implemented measures implemented (cumulative) Page 62 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) DLI 2B: Capacity DLI 2B.1 DLI 2B.2 building and 20 persons awareness awareness campaigns certified to do campaigns in 20 road safety schools in audit/inspection vicinity of project roads Disb. amount due (million Euros) / 4.00 0.80 1.55 4.30 10.65 18.9% percent 3. DLI 3: Maintenance DLI 3.1Adoption Management reforms of revised maintenance & rehabilitation design standards; DLI 3.2 DLI 3.3 & DLI 3.4 Preparation of & DLI 3.5 strategic plan PBMC for 3,000 for the km competitively introduction of tendered; PBMC; DLI 3.6 DLI 3.7 PBMC for 1,500 PBMC for an km in add.1,500 km in implementation implementation Disb. amount due (million Euros) / 4.00 0.80 1.55 4.30 10.65 18.9% percent TOTAL Disbursement 16.00 3.20 6.20 17.20 42.60 75.8% amount Page 63 of 64 Official Use The World Bank SERBIA ROAD REHABILITATION AND SAFETY PROJECT (P127876) ACHIEVED DLI TARGETS (FROM THE PAC REPORT DATED MARCH, 2021) Page 64 of 64 Official Use