Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00005553 IMPLEMENTATION COMPLETION AND RESULTS REPORT IBRD-78890; IBRD-86020; IDA-47320 ON A CREDIT IN THE AMOUNT OF SDR46.2 MILLION (US$70.0 MILLION EQUIVALENT) AND A LOAN IN THE AMOUNT OF US$171.6 MILLION AND A LOAN IN THE AMOUNT OF US$140.0 MILLION TO THE REPUBLIC OF AZERBAIJAN FOR THE THIRD HIGHWAY PROJECT December 10, 2021 Transport Global Practice Europe And Central Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective {December 01, 2021}) Currency Unit = Azerbaijani Manat (AZN) AZN Manat 1.70 = US$ 1 US$ 1.399500 = SDR 1 FISCAL YEAR July 1 - June 30 Regional Vice President: Anna M. Bjerde Country Director: Sebastian-A Molineus Regional Director: Charles Joseph Cormier Practice Manager: Karla Gonzalez Carvajal Task Team Leader(s): Nijat Valiyev ICR Main Contributor: Manzoor Ur Rehman, Muhammad Zulfiqar Ahmed ABBREVIATIONS AND ACRONYMS ADB Asian Development Bank M3 Highway No. 3. Alat-Astara-state border with the Islamic Republic of AHP3 Third Highway Project (P118023) Iran ARS AzerRoadService OJSC, former name M4 Highway No. 4. Baku-Shamakhi- of the road administration. Yevlakh AZN Azerbaijan Manat MED Ministry of Economic Development (now Ministry of Economy) Bank The World Bank MMU Motorway Management Unit of the COVID-19 Coronavirus Disease road administration. CPF Country Partnership Framework MoF Ministry of Finance CPS Country Partnership Strategy MoT Ministry of Transport (now the Ministry of Digital Development and EA Environmental Assessment Transport) EIA Environmental Impact Assessment NPV Net Present Value EIRR Economic Internal Rate of Return PAD Project Appraisal Document EMP Environmental Management Plan PDO Project Development Objective GBV Gender-Based Violence PIU Project Implementation Unit GDP Gross Domestic Product Project Third Highway Project (P118023) Govern- Government of the Republic of RAP Resettlement Action Plans ment Azerbaijan RMU Regional Maintenance Units of the HDM-4 Highway Development and road administration. Management (software tool) RPF Resettlement Policy Framework IBRD International Bank for Reconstruction and Development SAAAR The State Agency of Azerbaijan ICR Implementation Completion and Automobile Roads, current name of Results Report the road administration. IDA International Development SDR Special Drawing Rights Association SLA Service Level Agreement IFI International Financial Institution TRACECA Transport Corridor Europe-Caucasus- IRI International Roughness Index Asia (an Intergovernmental Commission) ISR Implementation and Status Results VAT Value Added Tax Report M Roads Magistral roads. Roads forming the VOC Vehicle Operating Costs primary road network in Azerbaijan. TABLE OF CONTENTS DATA SHEET ............................................................................................................................ I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 1 A. CONTEXT AT APPRAISAL .........................................................................................................1 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) .......................................5 II. OUTCOME ...................................................................................................................... 8 A. RELEVANCE OF PDOs ..............................................................................................................8 B. ACHIEVEMENT OF PDOs (EFFICACY) ........................................................................................9 C. EFFICIENCY ........................................................................................................................... 14 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 17 E. OTHER OUTCOMES AND IMPACTS ......................................................................................... 17 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 21 A. KEY FACTORS DURING PREPARATION ................................................................................... 21 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 21 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 23 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 23 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 25 C. BANK PERFORMANCE ........................................................................................................... 26 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 28 V. LESSONS AND RECOMMENDATIONS ............................................................................. 28 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 30 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 38 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 40 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 41 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 46 ANNEX 6. SUPPORTING DOCUMENTS .................................................................................. 50 The World Bank Third Highway Project (P118023) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P118023 Third Highway Project Country Financing Instrument Azerbaijan Investment Project Financing Original EA Category Revised EA Category Full Assessment (A) Full Assessment (A) Organizations Borrower Implementing Agency Ministry of Finance State Roads Agency Project Development Objective (PDO) Original PDO The Project Development Objective is "to contribute to a more efficient and safer Baku-Shamakhi road and higher quality roadservices as part of the upgrading to motorway standards, and to improve the management of the nascent motorway network." Revised PDO “To contribute to a more efficient and safer Baku-Shamakhi and Yenikend-Shorsulu roads and higher quality road services as part ofthe general network upgrading to motorway standard, and to improve the management of the nascent motorway network.” PDO as stated in the legal agreement To contribute to more efficient and safer Baku-Shamakhi and Yenikend-Shorsulu roads and higher quality road services as part of the general network upgrading to motorway standard, and to improve the management of the nascent motorway network. The World Bank Third Highway Project (P118023) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 171,600,000 171,600,000 171,600,000 IBRD-78890 70,000,000 69,818,498 64,685,309 IDA-47230 140,000,000 75,000,000 75,000,000 IBRD-86020 Total 381,600,000 316,418,498 311,285,309 Non-World Bank Financing 0 0 0 Borrower/Recipient 114,550,000 161,843,088 153,702,412 Total 114,550,000 161,843,088 153,702,412 Total Project Cost 496,150,000 478,261,586 464,987,721 KEY DATES FIN_TABLE_DAT Approval Effectiveness MTR Review Original Closing Actual Closing 25-May-2010 08-Nov-2010 06-Nov-2013 31-Mar-2015 31-Mar-2021 The World Bank Third Highway Project (P118023) RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 04-Dec-2013 26.33 Change in Project Development Objectives Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Financing Plan Change in Implementation Schedule 28-Mar-2016 131.82 Additional Financing Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Implementation Schedule 31-Dec-2019 306.34 Change in Loan Closing Date(s) 05-Jun-2020 309.34 Change in Components and Cost Change in Loan Closing Date(s) Cancellation of Financing Reallocation between Disbursement Categories Change in Implementation Schedule KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 23-Sep-2010 Satisfactory Satisfactory 0 02 31-Jan-2011 Satisfactory Satisfactory 0 03 30-May-2011 Satisfactory Satisfactory 0 04 31-Dec-2011 Satisfactory Moderately Satisfactory 3.02 05 17-Jun-2012 Moderately Satisfactory Moderately Unsatisfactory 5.19 Moderately 06 26-Dec-2012 Unsatisfactory 8.03 Unsatisfactory Moderately 07 26-Jun-2013 Unsatisfactory 10.38 Unsatisfactory The World Bank Third Highway Project (P118023) 08 26-Sep-2013 Moderately Satisfactory Moderately Satisfactory 26.14 09 28-Apr-2014 Moderately Satisfactory Moderately Satisfactory 33.39 10 26-Nov-2014 Moderately Satisfactory Moderately Unsatisfactory 66.72 11 11-Jun-2015 Moderately Satisfactory Moderately Satisfactory 97.62 12 18-Dec-2015 Moderately Satisfactory Moderately Satisfactory 117.78 13 10-Feb-2016 Moderately Satisfactory Moderately Satisfactory 131.82 14 10-Aug-2016 Moderately Satisfactory Moderately Satisfactory 153.06 15 08-Feb-2017 Moderately Satisfactory Moderately Satisfactory 173.45 16 11-Aug-2017 Satisfactory Satisfactory 196.40 17 15-Feb-2018 Satisfactory Satisfactory 231.96 18 22-Aug-2018 Satisfactory Moderately Satisfactory 249.98 19 19-Feb-2019 Satisfactory Satisfactory 273.84 20 31-Jul-2019 Satisfactory Satisfactory 287.29 21 26-Dec-2019 Moderately Satisfactory Moderately Satisfactory 303.14 22 25-Jun-2020 Moderately Satisfactory Moderately Satisfactory 309.34 23 27-Dec-2020 Satisfactory Moderately Satisfactory 310.51 SECTORS AND THEMES Sectors Major Sector/Sector (%) Transportation 100 Public Administration - Transportation 2 Rural and Inter-Urban Roads 95 Other Transportation 3 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Private Sector Development 32 Jobs 32 Job Creation 32 The World Bank Third Highway Project (P118023) Public Sector Management 2 Public Administration 2 Administrative and Civil Service Reform 2 Social Development and Protection 1 Social Inclusion 1 Other Excluded Groups 1 Fragility, Conflict and Violence 0 Forced Displacement 0 Urban and Rural Development 64 Urban Development 32 Urban Infrastructure and Service Delivery 32 Rural Development 32 Rural Infrastructure and service delivery 32 Environment and Natural Resource Management 1 Environmental policies and institutions 1 ADM STAFF Role At Approval At ICR Regional Vice President: Philippe H. Le Houerou Anna M. Bjerde Country Director: Asad Alam Sebastian-A Molineus Director: Peter D. Thomson Charles Joseph Cormier Practice Manager: Henry G. R. Kerali Karla Gonzalez Carvajal Task Team Leader(s): Jacques Bure, Elizabeth C. Wang Nijat Valiyev ICR Contributing Author: Muhammad Zulfiqar Ahmed The World Bank Third Highway Project (P118023) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. The Third Highway Project was appraised in 2010. The Republic of Azerbaijan’s oil-based economy had been booming with an annual average GDP growth rate of 21.6 percent over the preceding five years but had slowed to 9.3 percent by 2009 in the global economic crisis . Half of the 9.1 million population lived in urban areas, predominantly in or around the capital Baku. In 2010, per capita income 1 was US$5,370, with just over nine percent of the population falling below national poverty lines. 2. The Government’s ambitious development agenda aimed to develop natural resources and use hydrocarbon wealth to combat poverty and transform into a diversified, sustainable market economy. It recognized that growing the non-oil sector was critical for improving the living standards of the low-income population and achieving the objectives of the 2008 State Program on Poverty Reduction and Economic Development2. Diversification of the economy was also at the core of the then-current Country Partnership Strategy3, aligning well with the pillar “supporting sustainable and balanced growth of the non-oil economy”. 3. Infrastructure investment was essential to implement Azerbaijan’s development agenda, particularly in the transportation sector which contributed 6.6 percent of GDP in 2009 . The sector was dominated by road transport carrying 60 percent of freight and almost all passengers, so an efficient road network was critical to regional development. In addition, the convenient location of Azerbaijan on the crossroad of major international traffic arteries, the Silk Road and North-South corridor, highlighted the strategic importance of the transportation sector and potential for becoming a prosperous transit hub. 4. The Government’s priority was to reconstruct the primary road network4 and upgrade much of it to international motorway standards, including sections of the M4 Baku-Shamakhi-Yevlakh road. The M4 connects the Mountainous Shirvan Economic Region to the capital and forms part of the strategic East-West corridor. It was expected that improving the M4 would enhance regional development in an area with potential for tourism and light industrial production. The Government and Bank consequently agreed to include upgrading a 100 km section of the M4, from Baku to Shamakhi, as the major component of the Third Highway Project. This action would increase the road’s capacity by widening it to a four-lane dual carriageway and improving its suitability for commercial traffic. 5. The Third Highway Project began the next phase of the Bank support to the road sector of Azerbaijan5 and was expected to provide a platform for the Bank and Government to further improve the road network and road sector institutions, incentives, and policies and realize the institutional changes agreed during the earlier stages. The rationale for the Bank’s engagement with the project also included an expectation to advance 1 World Bank Data 2010 Atlas method. 2 State Program on Poverty Reduction and Sustainable Development Republic of Azerbaijan, Approved by Decree #3043 of the President of the Republic of Azerbaijan, dated September 15, 2008. 3 CPS FY07-FY10. 4 Classified as magistral (M) roads in Azerbaijan. The term is used interchangeably with “highway” and “motorway”, although not all of Azerbaijan’s Magistral roads are high speed dual-carriageways. 5 The first phase began with the Highway Project launched in 2001. This Project provided US$40 million equivalent to reconstruct a section of M2, to provide technical assistance for rural road maintenance and begin road sector modernization. Phase two was substantive and broad totaling US$675 million to improve over 450 km of major roads through the Second Highway Project. Page 1 of 58 The World Bank Third Highway Project (P118023) policy dialogue to address issues specific to the developing motorway network and strengthening road sector institutions. At Entry, the Bank already had long running involvement and experience in the transport sector of Azerbaijan and was well placed to support the Government through the next stage in the road sector development. The Bank also aimed at building on its substantial experience in advancing the road network maintenance reforms in the region and beyond. 6. Financing of the Third Highway Project provided the opportunity to improve the sustainability of the road sector through institutional strengthening. Despite the ongoing multibillion-dollar capital investment, the management and maintenance of the road network required improving if assets were to be sustainable. At the time of appraisal, road maintenance was fractured with over sixty local or regional maintenance units responsible for all roads, from gravel local roads to newly constructed highways. These units were poorly equipped and, being arranged in local administrative areas, could not apply a systemic approach to maintaining arterial highways. Significant institutional development activities were included in the project to address this issue, initially exploring different options for managing and maintaining higher value road assets. The project would then support the piloting of a preferred option on sections of the existing motorways. 7. The project was designed when Azerbaijan had a poor road safety record, recording the highest number of road fatalities per 10,000 vehicles amongst Europe and South Caucasus countries6. The number of people hurt or killed in traffic accidents was expected to rise as car ownership and traffic increased. The Third Highway Project, through the design of the motorway guidelines and the road safety measures implemented between Baku and Shamakhi, was expected to help mitigate traffic accidents on motorways due to traffic growth. Theory of Change (Results Chain) 8. The Third Highway Project aimed to improve the capacity, safety, and quality of the Baku-Shamakhi road and strengthen institutional capacity to manage the primary road network. At appraisal, the results chain derived from the 2010 Project Appraisal Document (PAD) linked the project’s actions to the desired outcomes, as shown graphically on the next page. 9. Critical assumptions relevant to the results chain were identified during project preparation and included in the PAD. (i) Transport links to the economically important Mountainous Shirvan and Sheki-Zagatala regions required improvement due to their high agricultural and tourism growth potential. (ii) The continued commitment from the Government to motorway reform, including improved maintenance. (iii) Technical design challenges could be economically solved, largely within the existing road corridor, particularly in the mountainous sections. (iv) The growing network of the upgraded motorways required a dedicated mechanism for their enhanced management and maintenance. (v) The constrained implementation capacity of the road administration and MoT would be strengthened to manage the growing project portfolio. (vi) The timely acquisition of land following the new Land Expropriation Law and Bank requirements and based on the experience of the previous operations. 6 Project Appraisal Document (PAD) April 2010, et al. Page 2 of 58 The World Bank Third Highway Project (P118023) Results Chain Activity Output Outcome Long Term Outcomes Reduced road user costs along Baku-Shamakhi highway. Upgrade a 100 km section of the M4 road Fewer accidents along 100 km of modern dual- between Baku and Baku-Shamakhi highway. carriageway highway. Shamakhi from two lanes to four lanes. A higher quality primary Sustainable and road network. Policy options on the balanced growth of the management and non-oil economy financing for motorways. through. A mechanism for Enhance regional Support the project motorway management Motorway Guidelines for development in the management capacity of and financing. motorway construction, Shamakhi area. the road agency. operations, and maintenance. Improved serviceability Institutional and sustainability of the A pilot motorway development of the ARS road network. in motorway operation The study, design and management unit and maintenance. pilot-implementation of responsible for a sustainable operations and management and maintenance units. financing scheme for The establishment of a motorway operations dedicated motorway and maintenance. An operations and network management maintenance pilot on and operation sections of the existing mechanism, separate motorways. from lower level road categories. Project Development Objectives (PDOs) 10. The PDO statement, as provided in both the loan and the financing agreement, was “to contribute to a more efficient and safer Baku-Shamakhi road and higher quality road services as part of the general network upgrading to motorway standard, and to improve the management of the nascent motorway network”. Key Expected Outcomes and Outcome Indicators 11. The compound PDO statement has been unpacked into four outcomes . (i) a more efficient Baku-Shamakhi road. (ii) a safer Baku-Shamakhi road. (iii) higher quality road services as part of the general network upgrading to motorway standard. Page 3 of 58 The World Bank Third Highway Project (P118023) (iv) improved management of the nascent motorway network. 12. At Entry, the four outcomes specified in the PDO statement were captured by three PDO Indicators. TABLE 1. Outcomes and PDO Indicators Baseline End Target Outcome PDO Indicator Name (2010) (2014) Efficiency: Reduction in road user costs (i) a more efficient Baku-Shamakhi road. 100% 80% from Baku to Shamakhi. Safety: Reduction of Personal Injury per (ii) a safer Baku-Shamakhi road. 100-million vehicle-km along the Baku- 100% 70% Shamakhi road. (iii) higher quality road services as part of Percentage of M Roads in Good and Fair the general network upgrading to Condition as a share of total M Roads of the 55% 70% motorway standard. network. (iv) improved management of the nascent Outcome partially recorded by PDO Indicator (iii) above. motorway network. 13. The fourth outcome, “improved management of the nascent motorway network”, did not have a unique PDO indicator and only tangentially contributed to the indicator used for the third PDO. Activities associated with this outcome were largely preliminary (piloting motorway maintenance and a pilot motorway management unit) or advisory, such as presenting financing and maintenance recommendations to stakeholders. Consequently, meaningful outcome indicators would have been challenging to identify at Entry and dependent on the then developing Government policy toward maintenance reform. As Government plans firmed up during implementation, the decision was made to move from pilot schemes to actual full-fledged implementation of agreed reforms. The PDO was modified accordingly (see the following chapter on Significant Changes During Implementation). Notwithstanding the above, progress towards the fourth outcome was well monitored by four Intermediate Outcome Indicators in the original Results Framework. Components 14. The Third Highway Project comprised three components. Component 1 focused on dualling 100 km of the M4 Baku-Shamakhi road, while Component 2 targeted institutional strengthening. A third component supported project management. TABLE 2. Component Costs US$ million excluding VAT Cost at Cost7 Actual Cost8 Project Cost by Component Entry incl. AF incl. AF Component 1: (at Entry) Upgrading a 100 km section of existing two lane M4 296.00 --- --- Baku-Shamakhi road into a four-lane motorway. Component 1 (Revised 2016): Upgrading a 94 km section of M4 Baku- Shamakhi and 48 km section of M3 Yenikend-Shorsulu roads into a four-lane --- 419.40 374.43 motorway. Component 2: Institutional development (management and financing 5.00 25.00 15.60 Azerbaijan’s motorways; institutional development of the ARS). 7 “Cost at Entry” shown for Component 1 after restructuring and additional financing in 2016. Data from Minutes of Loan Negotiation. 8 Total Actual Cost from Audited Financial Statement to April 30, 2021. The Financial Statement does not disaggregate VAT from the Government contribution, so the actual cost shown above is calculated assuming VAT was applied to all activities at 18 percent. Page 4 of 58 The World Bank Third Highway Project (P118023) Component 3: Project Management including technical assistance and 1.00 4.00 3.27 financing additional costs for project implementation. Contingency --- 28.70 --- Total For Components 302.00 477.10 393.30 Front-end Fee 0.43 0.78 0.78 Total Cost 302.43 477.88 394.08 15. At Entry, the percentage of expenditures financed by the Bank was 80 percent net of VAT. The percentage of Bank financing varied during implementation, and at close, the two IBRD loans and the IDA credit had contributed 79 percent of the actual project cost, net of VAT (at 18 percent). The project was initially partly financed by an IDA credit of SDR46.20 million, which at the time of project preparation was the equivalent of US$70.00 million. However, exchange rate fluctuations, particularly in 2014 and 2015, resulted in the equivalent dollar amount being reduced to US$64.69 million. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 16. The Third Highway Project evolved during implementation, undergoing restructuring in 2013 and 2016 when additional financing was introduced. More minor changes, not affecting the scope of the project, were made in 2019 and 2020 to extend the closing date, cancel a portion of the financing, and support the request from the government for the preparation of the new Bank financed road sector operation. Project Change Timeline NOVEMBER 2010 PROJECT IMPLEMENTATION MARCH 2021 10 11 12 13 14 15 16 17 18 19 20 21 22 2013 2016 2019 2020 Restructuring Additional Closing Date Amendment Financing Revised 17. Restructuring in 2013 was undertaken as part of a holistic reassessment and technical due diligence of the World Bank financed road sector portfolio in Azerbaijan, which identified opportunities for more efficient use of available resources and necessary modifications to the scope of both the Second 9 and Third Highway Projects. The changes agreed by the Bank and Government opened prospects for satisfactory implementation of both projects, greater exchange of learning between projects, improved disbursements, achievement of Project Development Objectives, and the timely close-out of the Second Highway Project. It was also an opportunity to increase the ambition of the Third Highway Project. 18. The construction of 48 km of new motorway on the M3 corridor from Yenikend to Shorsulu was introduced to the project. This activity had previously been part of the parallel Second Highway Project, but the civil works contracts had been terminated in mid-2013 with less than a third of the work completed due to the 9 P094488 Page 5 of 58 The World Bank Third Highway Project (P118023) contractor’s non-performance. The section was a high priority for the Government, but insufficient time remained to retender and complete the work under the Second Highway Project, which was due to end in mid- 2015. The restructuring also reduced the length of M4 motorway to be dualled by about 25 km10 due to complex geotechnical issues that could not be resolved within the project's timescale. At the request of the Government, the savings from this were reallocated to the M3 Yenikend to Shorsulu section. 19. The 2013 restructuring enabled institutional development to be progressed and build on the Ministry of Transport and road administration’s decision to follow recommendations on modernization of motorway management and maintenance practices. These recommendations, resulting from technical assistance under the project, included establishing a new motorway management unit and making preparations to manage motorway operation and maintenance using service level agreements with maintenance units. 20. The co-financing share of the Government was increased from 20 percent to 25 percent (excluding VAT) to ensure sufficient funding to implement the new project activities. The Bank financing stayed at the original US$241.6 million but now represented 75 percent of the project costs. The restructuring also extended the project closing date from March 31, 2015 to September 30, 2016 to allow time to complete the additional project activities. 21. Additional Financing in 2016 supported the scaling up and restructuring of the Third Highway Project by financing the dualling of the remaining sections of the Baku to Shamakhi road to motorway standard and expanding the key institutional activity on modernization of motorway management and maintenance system. The additional resources enabled the two remaining single carriageway sections of the M4 road between Baku and Shamakhi to be included for dualling, thereby providing users with a motorway standard road for the whole journey between the two cities. This included part of the section dropped in 2013 as geotechnical problems had been solved. In addition, selected pavement strengthening works for existing lanes on other road sections were included to improve long-term sustainability and comply with new national standards. 22. The scope and outcomes for the institutional strengthening component were significantly expanded with Additional Financing. Capital investments were introduced to establish regional motorway maintenance units (RMUs) with the construction of offices and depots, and the purchase of specialist road maintenance equipment. Additional technical assistance further supported motorway management and maintenance system reform and the implementation of the planned capital investment in the RMUs. Project management support was also enhanced to cater to the increased implementation activity and update the road database established under the Second Highway Project. 23. Additional Financing was selected as the most appropriate vehicle to finance the new activities as they built on the original project activities and were consistent with the project PDO. An IBRD loan of US$140.00 million was agreed to finance 90 percent of the additional project cost. The agreement also extended the project closing date to December 31, 2019 to implement the additional activities. In parallel, the closing date of the original loan and credit was similarly extended. 24. Following the project restructuring in 2013, the PDO was adjusted to include the new M3 Yenikend-Shorsulu road section: “to contribute to a more efficient and safer Baku-Shamakhi and Yenikend-Shorsulu roads and higher quality road services as part of the general network upgrading to motorway standard, and to improve the management of the nascent motorway network”. No further changes were made to the PDO during project 10Of the 25 km, about 16 km were through a landslide prone area on the approach to Shamakhi. This section returned to the project in 2016 to complete the Baku-Shamakhi corridor. The remaining length was a bypass road around the centre of Shamakhi city. This section had geotechnical and land acquisition challenges and became a low priority for the Government. Page 6 of 58 The World Bank Third Highway Project (P118023) implementation. 25. The revised compound PDO statement amends the unpacked outcomes to be used in the assessment of the project. (i) a more efficient Baku-Shamakhi road. (ii) a more efficient Yenikend-Shorsulu road (new outcome). (iii) a safer Baku-Shamakhi road. (iv) a safer Yenikend-Shorsulu road (new outcome). (v) higher quality road services as part of the general network upgrading to motorway standard. (vi) improved management of the nascent motorway network. Revised PDO Indicators 26. PDO Indicators were revised in 2013 to reflect the amended PDO and inclusion of the M3, and in 2016 to better capture the institutional development activities which were expanded at that time. A table showing the development of outcomes and indicators over time is included in Annex 6B. Intermediate Results Indicators were also amended in 2013 and 2016 to better record the progress of the modified project. Revised Components 27. Restructuring in 2013 increased the scope of Component 1: Motorway Improvement. 48 km of new motorway construction was added with the inclusion of the Yenikend-Shorsulu section of the M3 Alat to Astara motorway. In parallel, the length of the M4 Baku-Shamakhi road being upgraded was reduced from 100 km to 77 km due to complex geotechnical issues at the western end of the road. The description of Component 1 was amended to reflect the widened scope and its cost increased by about US$20 million. 28. The 2016 Additional Financing enabled the scope of all three components to be increased . In Component 1, the M4 Baku-Shamakhi road upgrading was lengthened to 94 km. Component 2 saw the additional support for establishing Regional Maintenance Units (RMUs) and a Motorway Management Unit (MMU), construction of RMU offices and road maintenance depots, and the procurement of road maintenance equipment. Component 3 increased institutional support to update the road administration’s road database, undertake a road user satisfaction survey, and provide further project monitoring, evaluation, and audits. In 2020, component costs were further revised to reflect the actual costs. No change was made to component activities at that time. Details of all project components and cost changes are tabulated in Annex 6C together with a timeline. Other Changes 29. Loan Closing Date Extension. By the end of 2019, the only incomplete task was the construction of regional maintenance facilities. Therefore, an extension of the closing date to 30 June 2020 was agreed to allow completion of building work and to continue supporting RMUs operations. 30. Amendment to the Loan Agreement. In 2020, the Bank agreed to a Government request to cancel a portion of loan IBRD-86020 and extend the project by nine months to March 31, 2021. The amendment cancelled US$64 million, savings that mainly accrued from the devaluation of the local currency between 2015 and 2017. Project resources were refocused on preparing a new Bank investment project supporting regional development in the southern part of Azerbaijan, including the rehabilitation of the Salyan-Bilasuvar road. Project funds financed consultancy services to develop technical, environmental, and social documents necessary for project preparation. As a result, the new investment operation was successfully prepared and approved by the Bank Page 7 of 58 The World Bank Third Highway Project (P118023) Board11 on May 20, 2021 with the Bank financing equal to the amount cancelled from the Third Highway Project loan. Rationale for Changes and Their Implication on the Original Theory of Change 31. The first restructuring in 2013 was undertaken as part of a strategic reassessment of the highway portfolio (which at that time included Second and Third Highway Projects) for improving its performance, fixing issues identified at early stages of project implementation and boosting slow disbursements. The restructuring successfully met its objectives and became an important milestone in improving implementation progress and putting the project on track. Its implication on the Theory of Change was a modified PDO statement and derived Outcomes, and adjustment of the Input Activities, Outputs and PDO Indicators, which were duly reflected in the Results Framework. Intermediate Results Indicators were also adjusted. 32. The rationale for the second restructuring in 2016 used the Additional Financing instrument for scaling up the project to enhance its development effectiveness. Restructuring supported the road agency with the operationalization of the motorway maintenance reform model, identified in earlier stages of the project, and upgrading two further sections of M4 Baku-Shamakhi road. The implications for the Theory of Change were modifications to Activities and Outputs. While PDO did not change, the PDO Indicators and Intermediate Results Indicators were adjusted and incorporated in the Results Frameworks. 33. The two relatively small restructurings were undertaken in the final stage of the project when the key original project activities were completed. The first one was prepared to allow completion of the contract to construct RMU facilities so the last missing element of the new motorway maintenance mechanism could be put into operation. The second restructuring accelerated the preparation of a new investment project to boost regional development and connectivity amid the COVID-19 crisis. These restructurings did not have a direct impact on the Theory of Change. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 34. The Country Partnership Framework (CPF) for Azerbaijan period FY16-FY2012 was current when the operation ended in March 2021 as its validity was extended. This CPF was built on the Bank’s Systematic Country Diagnostic (2015)13, which identified improving connectivity as a priority and considered “the development of main transport corridors, along with relevant logistical services, will improve connectivity and open the way for producers to reach out to local and international markets in a faster and cheaper way.” This priority was encompassed in the current CPF under the second of the two Focus Areas: Economic Competitiveness. As part of this focus area, the CPF program supported connectivity improvement through investments in main highways. In addition, it sought to ensure the investments’ sustainability by introducing modern operation and maintenance (O&M) management practices. Although prepared five years earlier, the Third Highway Project’s 11 Regional Connectivity and Development Project (P174379) 12 World Bank- Country Partnership Framework for Azerbaijan- period FY16-FY20 June 3, 2015. As of November 2021, a Systematic Country Diagnostic is underway which will inform the new CPF. 13 The Systematic Country Diagnostic for Azerbaijan submitted to the Board on June 3, 2015 Page 8 of 58 The World Bank Third Highway Project (P118023) PDO shares common objectives and uses similar language as the SCD and CFP14. 35. Specific alignment of the PDO to the CPF is found with the CPF Objective 2.1, “development of the main transport network for enhanced international and domestic connectivity ”. With this the CPF recognizes the importance of main transport corridors in allowing producers to reach both local and international markets and that the corridors serve as a platform for turning the country into a key logistics hub between East and West. With both project roads being part of the primary road network and the Baku-Shamakhi road also being a component of the East-West corridor, the project’s objectives are very consistent with Objective 2.1 of the current CPF. One of the CPF Objective Indicators, “reduced road user costs on the M4 Baku-Shamakhi and M3 Alat-Yenikend highways”, reflects this close alignment. 36. The project’s activities to raise the quality of road services and improve motorway network management are also consistent with CPF Objective 2.1. In particular, the project supported the establishment of the road maintenance units along the M2, M3, and M4 corridors and explicitly addressed the CPF aim to introduce modern O&M practices. This project activity also fulfils Milestone 1 of the CPF Supplementary Progress Indicators for Objective 2.1, “modern road maintenance system established in one-two selected highways corridors”. Relevance Rating 37. The overall relevance of the Project Development Objectives is rated High, considering the broad convergence of the PDO with the current Country Partnership Framework and the specific alignment with the CPF Objective 2.1. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 38. The project efficacy is assessed by examining evidence of achievement of each of the six expected outcomes derived from the revised PDO: (i) a more efficient Baku-Shamakhi road. (ii) a more efficient Yenikend-Shorsulu road (new outcome). (iii) a safer Baku-Shamakhi road. (iv) a safer Yenikend-Shorsulu road (new outcome). (v) higher quality road services as part of the general network upgrading to motorway standard. (vi) improved management of the nascent motorway network. The primary tool used to assess these outcomes is the Results Framework, which compares the expected outcomes (targets) with actual results. However, other quantitative or qualitative evidence is used in support where available. Outcome (i). A more efficient Baku-Shamakhi road. 39. The upgrading of the 94 km of M4 Magistral (M) highway from Baku to Shamakhi (km 13.3 to km 107) 14The project also aligned well at Entry with the then current Country Partnership Strategy (CPS) FY07-10. In particular CPS FY07-10 Pillar (ii) which emphasizes “significant investment in infrastructure and transit corridors will be needed to develop Azerbaijan’s non-oil growth potential.” The project also aligned with CPS for FY11-14 which was under preparation at the time at Entry. However, this is not considered in determining the Relevance Rating. Page 9 of 58 The World Bank Third Highway Project (P118023) linking the capital city of Baku with central Azerbaijan achieved the project’s objectives of improved regional connectivity. Under the Third Highway Project, the highway was widened from an existing single carriageway road to a modern dual carriageway motorway. To do so, slopes were stabilized in areas prone to landslides, footbridges, culverts, and underpasses built, and safe U-turn arrangements were installed to improve local access. Upgrading delivered to road users the benefits expected with dualling. Capacity is significantly increased, driver frustration is reduced as overtaking is easier, and the average journey time is reduced. Passenger car speeds increased by almost 40 percent compared to the pre-project times, which translates into a substantial reduction of travel time along the Baku-Shamakhi corridor. The project has led to an average reduction in journey time of 19 minutes for road users traversing the length of the upgraded sections of M4 road. Consequently, the reduction in the road user cost achieved for the road is 30 percent, exceeding by 50% the PDO Indicator target of 20 percent15, thereby, inter alia, helping to improve the competitiveness of businesses in the region. The improved speeds and reduced user costs were calculated by HDM-4 modeling of the road before and after upgrading. Both models kept input prices constant (i.e. as per baseline) so the benefits attributable to the project were not masked by cost changes in fuel, labor, parts etc. M4 Baku to Shamakhi Landside adjacent to original road Slope stabilization Completed motorway with U-turn 40. Intermediate Results Indicators also evidence the achievement of this outcome. The “Number of km upgraded for Baku-Shamakhi road” target of 94 km was met, as was the length of “Roads constructed non-rural” with 144 km compared to a target of 142 km, a total to which the M4 contributed 65 percent. As an important indicator of the improved M4 road efficiencies, the road user satisfaction surveys among the M4 road users showed a significant increase in the level of satisfaction from the pre-project 17 percent to the post-project 73 percent. The findings of the road user survey are discussed further at the end of this section. Outcome (ii). A more efficient Yenikend-Shorsulu road. 41. The construction of a new 48 km section16 of the M3 highway between Yenikend to Shorsulu, linking Azerbaijan’s northern and central regions with the Iranian border in the South, achieved the project’s objectives of improved international and domestic connectivity. Under the Third Highway Project, a modern dual carriageway highway was completed along a new alignment, complete with grade-separated interchanges and rest areas, providing road users with a quicker and safer alternative to the existing single carriageway road. The section is an integral part of the 243 km M3 North-South corridor highway completed over the past decade with support from the World Bank and the Asian Development Bank. Following the project, the average travel time along the M3 from Yenikend to Shorsulu has reduced from 52 minutes to 34 minutes. Shorter travel times between Yenikend-Shorsulu have contributed to the reduction in road user costs by 23 percent against the PDO 15M&E HDM-4 analysis of road user costs by the road administration, as reflected in the Results Framework (Annex 1). 1648.2 km of motorway was constructed under the project plus associated slip roads, access roads, and interchange links. The project enabled a further 1.4 km of motorway, including a new bridge over the river Kur, constructed as part of the Second Highway Project, to be brought into use and incorporated into the Yenikend-Shorsulu section of the M3. Page 10 of 58 The World Bank Third Highway Project (P118023) Indicator target of 20 percent, as assessed by HDM-4 modeling. As with the M4, Intermediate Results Indicators also evidence the achievement of this outcome. The “Number of km upgraded for Yenikend to Shorsulu road” target was met, and the road section contributed to meeting the target for “Roads constructed non-rural”. The M3 section also contributed to the target for Intermediate Indicator “Improvement in road user satisfaction in the Project Area” being met. The road user satisfaction surveys among M3 road users have demonstrated a surge in the satisfaction among the project area residents and travelers from the pre-project 22 percent to the post- project 89 percent. M3 Yenikend to Shorsulu Old M3 road in Salyan Culvert construction Completed motorway Outcome (iii). A safer Baku-Shamakhi road. 42. With the dualling of the M4, the project’s objective of greater safety was clearly achieved. Accident data collected by the road administration showed a substantial reduction in the number of people injured or killed (per 100 million vehicle-km), of 71 percent17 against the target of 30 percent from the baseline. This reduction is mainly attributable to project activities as the reduction is substantially higher than the national trend, which has seen fatal road accidents reduce by only about a quarter18 over the past decade. As an illustration, eighty-nine fewer people died on the Baku-Shamakhi road during the past five years, as sections started to be completed under the project, than in the preceding five years. 43. The fall in the injuries and fatalities arising from traffic accidents along the project road is largely attributable to applying lessons learned and international best practices when designing the M4 upgrade. Dual carriageway roads are inherently safer than single carriageway roads as the traffic is segregated by the separation of carriageways and a median safety barrier. This arrangement reduces the number and severity of collisions, as head-on accidents are largely prevented despite faster speeds. The M4 was designed with the safety in mind to manage the risks associated with the hazardous mountainous terrain conditions. Road alignment has been improved, and where it could not, a reduced speed limit was imposed. Both metallic and concrete safety barriers have been used extensively in the median and at the roadside. Hazardous left turns (turns across the opposite carriageway) have been restricted and replaced with well-detailed U-turns with lanes dedicated to turning traffic. In urban areas, footbridges have been constructed, and sidewalks installed to separate pedestrians and vehicles. To further reduce hazards, formal road-safety audits were introduced as a requirement for the project as such audits are not yet institutionalized and routinely used for the state budget financed projects in the country. These audits were undertaken and acted on, which contributed to fulfilling two Intermediate Result Indicators, “Safety audits on detailed design” and “Safety audits on construction completion”. Road safety improvements for M4 17 Five-year average since sections of the upgraded road opened. Accident rates are lower for more recent years as upgrading of the final sections was completed. 18 https://www.stat.gov.az. Road fatalities per 100,000 population fell 21% from 2009 to 2019 and 32% if measured from 2010 to 2020. Page 11 of 58 The World Bank Third Highway Project (P118023) section km 0 to km 15 envisaged by the original project were implemented directly by the road administration with the government financing. The socio-economic outcomes of road safety improvements are discussed in detail in Section E. OTHER OUTCOMES AND IMPACTS. Outcome (iv). A safer Yenikend-Shorsulu road. 44. As with the M4, the reduction in the number of fatalities and injuries along the Yenikend-Shorsulu corridor was very substantial and much higher than the national average. All road safety features described in the paragraph 43 with regard to M4 road also apply to M3 road with one difference that the prevailing flat terrain and new alignment of M3 road allowed its design to even higher safety standards. As a result, a 76 percent reduction in fatalities and injuries was recorded compared to the target of 30 percent when the rate along the original M3 road was compared to the rate along the completed new motorway. As mentioned above, M3 road safety benefited from the project delivering an entirely new road on a different alignment than the original road. The new alignment bypasses Salyan city and nearby villages, so local road users (typically slower and stopping frequently) are separated from long-distance faster through traffic using the new road, thereby reducing the risk of collisions. Risks to pedestrians are also minimized as the design of the road ensured that the new road traverses mostly sparsely populated areas, and pedestrians and animals are not permitted on the highway with the entire road alignment fenced from both sides. Overall, the whole M3 Alat Astara highway, including Yenikend- Shorsulu section, is designed and built to the highest international motorway standards. All junctions are grade- separated, no turning is permitted, access or egress is not possible other than the junctions, and use is restricted to suitable vehicles. The project-designed rest areas along the road have also contributed to the higher safety standards of M3 motorway as a factor reducing driver fatigue. Finally, similar to M4, the road safety audits undertaken for M3 at both the design and completion stages, helped to further improve and finetune its original safety characteristics. Outcome (v). Higher quality road services as part of the general network upgrading to motorway standard. 45. The project supported activities aimed at improving the quality of road services. To achieve this outcome, the Third Highway Project had to have a demonstratable impact on the overall quality of the whole primary road (motorway) network. The methodology adopted by the Results Framework to assess this outcome was to measure the physical condition of all of Azerbaijan’s primary roads and classify them by condition from “very poor” to “very good”19. This logic was that roads in better condition provide a better service in terms of operating costs, safety, capacity, and maintenance costs. Surveys measured the condition of the primary road network before the project roads were completed and afterwards. A baseline survey identified that about half the 1600 km of M roads measured were in a “good” or “fair” condition, the remaining being “poor” or “very poor”. By 2018 (the date of the last complete survey and after M3 and M4 completion), the percentage of the primary road network in at least “fair” condition had increased to 90 percent, exceeding the PDO Indicator target of 70 percent. Over 75 percent of these roads, including the two project roads, fell into the highest category, “very good”, with an IRI of less than 2.5 mm/m. 46. The project’s direct contribution to improving the quality of road services was upgrading two sections of the primary road network to a motorway standard, specifically 144 km of the Baku-Shamakhi and Yenikend- Shorsulu roads. While the results indicator records network-wide improvements, the direct contribution made by the project to the quality of road services can be disaggregated from the road condition survey data. This analysis shows the project financed roads alone directly contributed about a ten percent improvement to the overall quality of the primary road network. The achievement of higher quality road services is also partially 19 RONET Road Condition Classification. Page 12 of 58 The World Bank Third Highway Project (P118023) indicated by Intermediate Results Indicators “Roads constructed non-rural” and “Improvement in road user satisfaction”. Overall, road user satisfaction was recorded as 79 percent at the end of the project, strongly indicating higher quality road services have been achieved. Outcome (vi). Improved management of the nascent motorway network 47. The project supported the road administration with reforms intended to improve the quality of motorway management, in particular maintenance practices. During the preliminary stages of the project, institutional development activities focused on studies to identify the reform options for the management and maintenance of motorways and support making decisions on selecting policy alternatives. Following successful stakeholder engagement and decision making, the project was restructured to support the design and implementation of substantial reforms to the structure of road maintenance within the road administration. These reforms removed magistral (motorway) road maintenance from local units to seven newly established regional motorway maintenance units responsible to a central motorway management unit. The project heavily supported the establishment of the new units with substantial technical assistance to support planning, capacity building, training, and reporting. In parallel, offices and depots were constructed in nine locations for the three priority RMUs, and 118 pieces of equipment were procured, including buses, light trucks pickups, multipurpose road repair vehicles, road sweepers, grass cutting tractors, and safety sign trailers. The final project action supported the introduction of performance-based service-level contracts between RMUs and the MMU. 48. Since early 2019, three priority RMUs have been operational and maintaining 774 km of highway under service level agreements with the MMU. Four more RMUs are operating and transitioning to SLA. Management structures are now in place within the road administration for maintenance planning and budgeting. The outcome of this activity is captured by a PDO indicator measuring the length of road being maintained under service level parameters, with a target of 500 km. Further evidence that the project improved management of the motorway network is by Intermediate Results Indicators: “Design of motorway guideline”, “Development of motorway and maintenance strategy”, “Annual maintenance plans developed by MMU using road database”, and “Rolling 2 year budget prepared by MMU for RMUs”. RMU Depot, Offices and Equipment RMU maintenance workshop RMU maintenance offices Winter maintenance equipment 49. The project-led introduction of service level agreements to regulate RMU performance is understood to be the first time this approach has been adopted in Azerbaijan’s public sector and aligns well with recent changes to the way the Government intends to manage state-owned enterprises 20. In 2020, the Government established the Azerbaijan Investment Holding to manage state-owned enterprises21 and improve their economic efficiency and transparency. Its charter includes directives to define and monitor performance indicators to evaluate their operation efficiency, stimulate public-private partnership, and take measures to optimize the 20 Presential Decree establishing the Azerbaijan Investment Holding, August 2020. 21 Not including the road administration (SAAAR) Page 13 of 58 The World Bank Third Highway Project (P118023) expenses of state entities and ensure financial transparency. The lessons learned reforming road maintenance represent a valuable resource for other state-owned enterprises as they move to modernize their services. 50. The project supported updating and improving the sustainability of the road administration’s national road database, a vital tool for network planning, management, and motorway maintenance reforms. Supported by the Second Highway Project, the road administration conducted a comprehensive road data collection program and established a national road database. The Third Highway Project sought to build on this and, from 2017 onward, supported data collection activities in a way that maximized the road administration’s own data collection capacity and reduced its reliance on external specialist contractors. The outcome was a self- sustaining specialist unit within the road administration capable of undertaking surveys and processing collected road network data. Additional Evidence of Achievement 51. With the support of the project, the road administration sought to engage beneficiaries with the commissioning of its first road user satisfaction surveys. The views of 800 residents and road users were sought along the Baku-Shamakhi and Yenikend-Shorsulu corridors, both before and after upgrading. Topics included road condition, sidewalk condition, signage, road markings, drainage condition, pedestrian safety, and the availability of bus stops and pedestrian crossings. In addition, data were disaggregated to identify gender gaps in the responses. Undertaking the surveys contributed to improving the management of the network by providing stakeholder feedback on the quality of its service provided. A summary of survey results is included in Annex 6D. Justification of Overall Efficacy Rating 52. The project satisfactorily achieved the intended outcomes expressed in the PDO and fulfilled the development objectives. The new and upgraded roads delivered by the project will contribute to the economic development of several poorer regions in Azerbaijan and have already yielded direct benefits such as lower user costs and accident reduction. In addition, the project’s institutional development activities have ensured the road administration can sustainably manage the country’s primary road network. Further improvements to the quality of the whole primary road network are expected to be realized due to the project-led reforms of maintenance practices, in addition to the ten percent quality improvement already delivered by the new and upgraded roads. Accordingly, as the achievement of the outcomes is attributable to project actions, the overall efficacy of the project is rated as Substantial. C. EFFICIENCY Assessment of Efficiency and Rating Design Efficiency. 53. The original project design was straightforward. The project was focused at upgrading 100 km of the M4 highway using traditional works and supervision contracts with the remainder providing institutional strengthening for the road administration. At the time of project preparation, a detailed technical design22 was ready for the first section (km 15 to km 45) of civil works and was ongoing for the remaining sections of the M4. An economic analysis had confirmed the road’s viability, implementation arrangements were in place, and appropriate safeguard actions were undertaken. 22 Design was financed under the Second Highway Project. Page 14 of 58 The World Bank Third Highway Project (P118023) 54. The factors which led to the later restructurings, such as the inclusion of the M3 motorway and the devaluation of the local currency, were not reasonably foreseeable when the project was designed, so they do not represent design weaknesses. Implementation delays arising from geotechnical challenges on sections of the M4 may have been mitigated during project design had the complexity of the problem been fully understood at the time, but the extent of the landslide problem was only fully realized after extreme weather (precipitation and freezing) shortly after the project started. However, construction of this section would still have been delayed until the completion of an extensive geotechnical investigation and technical design. Implementation Efficiency. 55. The earlier stages of the project saw less than optimal implementation efficiency, evidenced by slower than anticipated disbursement rates. Reasons for this were the technical designs and procurement taking longer than anticipated and poor progress by some works contractors. For example, the first works contract for M4 was awarded six months after project effectiveness despite its readiness at the project approval23 and work on the subsequent sections (km 45 to 67 and km 67 to 91) took a further two years to commence. Later in the project, the devaluation of the local currency and a banking sector crisis weakened contractors’ financial capacity and led to further delays. Nevertheless, despite difficulties, the challenges were satisfactorily overcome, and the project was fully implemented. Overall, all anticipated outcomes and key project activities were fully delivered. 56. An early-stage implementation decision was necessary regarding the MoT proposal to realign sections of the M4 road away from its original alignment and introduce additional bridges and viaducts. However, analysis of the proposal and constructive consultations between the Bank team and Government led to an eventual consensus about adhering to the original alignment of the road. This decision helped avoid adopting uneconomic solutions and maintain the project’s cost efficiency at a satisfactory level. 57. The upgrading of the M4 highway was constructed for US$2.47 million per km 24, substantially less than the US$2.96 million per km anticipated at Entry25. Efficiencies were achieved through effective engineering design, before the award of works contracts, that avoided complex and expensive technical solutions in the landslide areas and sections with steep inclines. Similarly, the completion of 48 km of the new M3 motorway was achieved for US$2.67 million per km, which compares favorably with the US$3.12 million per km estimated in the October 2013 Restructuring Paper. 58. Notwithstanding the engineering efficiency achieved, substantial additional unintended savings arose with the devaluation of the local currency by more than a half against the US dollar between 2015 and 2017 . Consequently, the dollar equivalent of payments for ongoing works contracts, which were denominated in local currency, fell substantially. 59. The savings made during project implementation allowed US$64 million of loan funds to be cancelled during the project restructuring in June 2020. Several proposals made to use these funds (such as constructing additional interchanges on the M4 highway and periodic maintenance on other network sections) were carefully reviewed and analyzed by the Bank team and the Government counterparts. As a result, the proposals were not accepted for efficiency reasons, and a decision was made to save these funds for other development needs of the country. 23 Project Appraisal Document (PAD) April 2010. 24 US$ 231 million for 94 km excluding VAT 25 US$ 296 million for 100 km excluding VAT - Project Appraisal Document (PAD) April 2010. Page 15 of 58 The World Bank Third Highway Project (P118023) 60. The actual project duration was substantially longer than planned at Entry, but the additional time also enabled significant enhancement of development effectiveness through expanded outcomes, such as the M3 Yenikend-Shorsulu road and motorway RMUs and MMUs. The project roads were completed and delivered benefits to the regions before the end of 2019, while the last year was used for the future project preparation. Consequently, the extended duration does not represent a significant implementation inefficiency, but one that may have been avoidable if the Government’s request had come earlier. Economic Efficiency. 61. The economic re-evaluation of the investment demonstrates that the project remains economically justified, with the weighted average EIRR of 13.2 percent and NPV of US$119.1 million, despite the external economic shocks. Some 1.5 million people living in the project affected areas are already benefiting and further efficiency are expected as the economy strengthens in the relatively poor regions 26 targeted by the project. The economic re-evaluation was conducted for the whole M4 highway between Baku and Shamakhi (94 km) and separately for M3 between Yenikend and Shorsulu (50 km). The methodology adopted was consistent with that used at appraisal (2010), restructuring (2013) and additional financing (2016). The ex-post economic analysis was carried out by comparing with- and without-project scenarios. Re-evaluation was conducted using the Highway Development Model (HDM-4) with actual project cost and construction period applied. The traffic was re- projected from measured 2019 traffic levels using economic projections made by the World Bank and ADB for Azerbaijan. Details of the economic analysis are included in Annex 4. 62. Sensitivity analysis indicates that the investment in the two motorways is robust and will remain economically viable if faced with adverse changes in critical factors such as traffic growth or maintenance costs. Considered on its own, the dualling of the M4 Baku-Shamakhi road from km 13 to km 107 delivers an EIRR of 13.2 percent and a positive NPV of US$105.7 million, broadly in line with the returns expected at Entry. For the M3 Yenikend-Shorsulu motorway, the EIRR is the same at 13.2 percent, with a lower NPV of US$13.4 million. While satisfactory, the lower than expected returns are a function of low traffic growth along the corridor as the economy contracted in 2015-2016 due to the external economic shocks and its subsequent slow recovery, compounded by the COVID-19 pandemic crisis27. The economic re-evaluation also uses a conservative traffic growth rate of 2.7% per year based on the expected long-term GDP growth. Currently, the economy is rebounding faster than expected (4.8% economic growth during January-September 2021), so traffic on the national road network will likely increase quicker than expected. The investment should additionally benefit from international initiatives such as International North-South Transport Corridor (INSTC), promoting transportation between Northern Europe and India. TABLE 6. Summary of Economic Evaluation Results at Entry and Completion Road Section Total NPV (US$) EIRR % At Entry28 M4 Baku-Shamakhi road (km 15 - km 116 in 2010) 33.6 million 14.5 M4 Baku-Shamakhi road (km 13 - km 107 in 2016) 128.9 million 13.5 M3 Yenikend-Shorsulu road (2013) 103.5 million 17.3 At Completion M4 Baku-Shamakhi road (km 13 - km 107) 105.7 million 13.2 M3 Yenikend-Shorsulu road 13.1 million 13.2 26See Annex 6E for regional population and relative income data. 27https://data.worldbank.org/country/azerbaijan 28As stated in the PAD 2010, Restructuring Paper 2013, and AF Project Paper 2016. NPVs are not directly comparable as the 2010 and 2013 analysis used a cost of capital of 12%, while analysis for AF in 2016 used 6%. Page 16 of 58 The World Bank Third Highway Project (P118023) Component Costs 63. Changes in component costs reflected restructuring and additional financing that widened the project’s scope during implementation. Detail of entry and outcome component costs are compared in TABLE 2 of the preceding Section A. CONTEXT AT APPRAISAL. However, actual component costs of each sub-component were generally less than at appraisal. 94 km of M4 highway was upgraded for US$232 million compared to the 2010 PAD estimate of US$296 million (for 100 km). Likewise, the M3 sub-component added in 2013 cost US$129 million, US$24 million less than estimated in the Restructuring Paper. The lower outcome costs are mainly attributable to the devaluation of the local currency mid-way through the implementation period. Administrative Costs 64. Annual administrative costs were closely aligned with the original estimates. At close in 2021, following restructuring and up-scaling, the yearly average supervision cost was US$121,422 29, comparing well with the US$120,000 per year estimated at entry. The higher total administrative costs incurred proportionally reflected the upscaling of the project and extended implementation period and did not impact the project's overall efficiency. Efficiency Rating 65. The efficiency of most aspects of the operation was consistent with what is expected in the sector. However, investment returns and the rate of progress of some civil works were below expectation, although largely due to external factors. In addition, extending the closing date at the end of the project to allow time for future project preparation may have been avoidable. Overall, the project efficiency almost met expectations, but the above-mentioned shortcomings, particularly related to delays in technical design, procurement, and implementation of contracts, require the assigned rating for efficiency to be “Modest ”. D. JUSTIFICATION OF OVERALL OUTCOME RATING 66. The overall project outcome rated as “Moderately Satisfactory” is based on the project ratings of “High” relevance of the PDO, “Substantial” efficacy, and “Modest” efficiency of the project . 67. A Split Rating is not necessary for the assessment of outcome ratings. The PDO statement was revised during the first restructuring in 2013 when a 48 km long section of the M3 motorway was added, and a 25 km section of the M4 dropped. At the second restructuring and Additional Financing in 2016, about 16 km of the M4 was added back. Therefore, the project’s scope was expanded and became more ambitious, while the revised PDO, outcomes and components remained consistent with the originals. E. OTHER OUTCOMES AND IMPACTS Gender 68. As part of the road user satisfaction surveys, gender-disaggregated public opinion and preferences were sought along the project corridors. The requirement for the data to be gender-disaggregated had been introduced as part of the 2016 Additional Financing restructuring and reflected in the revised Results Framework. The project was also screened for Gender-Based Violence (GBV) risks and assigned a Low GBV risk rating during the implementation. GBV retrofitting activities consistent with the project’s risk level were then undertaken between December 2018 and April 2019. 29 For the period FY11 to FY20, see Annex 2. Page 17 of 58 The World Bank Third Highway Project (P118023) 69. The project’s most significant impact on the lives of women and girls is brought by improved access to work, markets, healthcare and education. The new and upgraded project roads reduce the time local residents spend travelling to school, work, markets and medical facilities, thereby making these services more accessible. These improvements are reflected in the gender-disaggregated results of the road user satisfaction survey 30, which identified that the overall satisfaction of female respondents with the road service was 72 percent at the end of the project, up from a pre-completion baseline of less than ten percent. 70. The project financed M3 highway was designed to include several rest areas to improve the quality of services to road users. The rest areas are currently established with private investments on the infrastructure developed under the project with services including food and beverage places, markets, and petrol stations. The employment and income opportunities disproportionately benefit women in the region because domestically produced agriculture-based products are mostly traded by women31. The rest areas also provide basic restrooms with proper hygiene and lighting. Female road users particularly benefit from access to safe and clean lavatories. Institutional Strengthening 71. Institutional Strengthening was at the core of Component 2, improving road operations, maintenance, and effective asset management. Through the project activities (provision of technical assistance, training, offices, depots, and essential maintenance equipment), a Motorways Management Unit and Regional Motorway Maintenance Units were established, and modern maintenance practices were introduced. These enhanced institutional mechanisms for operation and maintenance of the primary road network were an important outcome of the policy dialogue between the Bank and the Government that has been ongoing since the Bank’s engagement in the transport sector of Azerbaijan in the early 2000s. 72. Introducing the concepts of service level agreements and client-provider relationships within a state agency was a significant step towards modernizing public sector service delivery. It provides a model for the management of the road administration to replicate for other agency activities and informs wider Government concepts that, though widely used elsewhere, are not yet common in the country’s public sector. On a practical level, the activity improved the capacity of the road administration to plan maintenance, develop budgets, collect data, set expectations, and monitor performance. 73. Component 3 financed project management costs, including advisory services, updating the road database, road user satisfaction surveys, audits, and monitoring and evaluation. The activities and practices adopted during the project implementation have also strengthened the capacity of the state road administration in planning, safeguards, procurement, design, the management of construction contracts, and financial management. For example, a sub-component helped update the road database for the country’s road network and maximize the capacity of the road agency’s in-house data collection unit. At the outset of this activity, most data collection was undertaken by specialist international contractors. However, during the last round of data collection under the project, the agency’s staff collected and processed data with occasional support from international consultants. Another example was the undertaking of road user satisfaction surveys which was a notable development in citizen engagement for the road administration. Mobilizing Private Sector Financing 30 A summary of the Road User Satisfaction Survey results is included in Annex 6D. 31 Project Appraisal Document for the Regional Connectivity and Development Project, April 2021. Page 18 of 58 The World Bank Third Highway Project (P118023) 74. Mobilizing Private Sector Financing was not part of the project’s design, but the Third Highway Project paved the way for engagement of the private sector in road maintenance. In particular, the project-led introduction of a client/service-provider model regulated by service level agreements provides a robust framework for private-public partnerships. Poverty Reduction and Shared Prosperity 75. The project substantially improved the capacity and reduced operating costs along the two major transport corridors of regional, national and international importance. These improvements to connectivity are expected to yield direct benefits to the diversification of the currently oil-based economy, such as increased transit, improved access from rural areas to the key markets in the capital city of Baku, access to international markets, notably in Iran and Georgia, and development of tourism. Equally significant are achievable improvements to the quality and accessibility of social services, such as health and education, because of better transport links. 76. The number of beneficiaries substantially increased during the implementation of the Third Highway Project. The project initially focused on improving the East-West corridor and promoting tourism and industrial production potential in the Mountainous Shirvan Economic Region. However, the inclusion of the Yenikend to Shorsulu section of the M3 enabled the project to support regional development in the Lankaran-Astara economic region and parts of the Aran economic region, all lagging in terms of relative income levels among the country regions. The cumulative number of beneficiaries in the project impacted regions exceeds 1.5 million people. Population and relative income data are reproduced in Annex 6E. 77. The major seven civil works contracts for the Baku-Shamakhi and Yenikend-Shorsulu roads have generated around 25,000 person-months of employment, with a substantial portion of the workforce being drawn from local towns and villages. Very few foreign staff were employed, even by the large Turkish contractors engaged for the Baku-Shamakhi road. The project's smaller but sustainable impact was the upskilling of road administration maintenance staff now working for the RMUs. Management and workers were trained in modern, safe, motorway maintenance techniques, including the use of specialist equipment, also provided by the project. Road Safety and Accident Reduction 78. Road accidents impact poverty, shared prosperity, and individual lives. In Azerbaijan, the socio-economic cost of road crashes was conservatively estimated by a World Bank study32 at AZN1.6 billion in 2019, equivalent to 2.0% of GDP. 821 people died in these road crashes. The study also calculated the cost33 of each fatality and each serious injury as US$346,000 and US$70,000, respectively. Therefore, the over 70 percent reduction in fatalities and injuries rates seen on the M3 and M4 roads (compared to about a quarter seen nationally 34) represent substantial social and economic benefits derived directly from the project’s road investments. 79. A comparison of fatality and injury rates before and after the project indicates annual socio-economic benefits of US$$7.0 million per year from accident reduction on the M4 and US$5.7 million per year for the M3. About two-thirds of these benefits derive directly from upgrading the project roads, with the remainder 32 Socio-Economic Costs and Human Impacts of Road Accidents in Azerbaijan, World Bank Group, June 2021. 33 Table 23 of the Socio-Economic Costs and Human Impacts of Road Accidents in Azerbaijan Study. The amounts shown are the costs associated with the death or serious injury of a person. They do not include the accompanying property damage or administrative costs. 34 See Section IIB, outcomes (iii) and (iv) Page 19 of 58 The World Bank Third Highway Project (P118023) accruing from the national trend. Similar benefits should be realized each year going forward, provided the accident rate remains low. The study also showed that the impact of accidents on quality of life is felt stronger by B40 households, so road accident reduction along the project roads is expected to make an ongoing contribution to poverty reduction. Climate and Environmental Outcomes 80. The project has introduced a methodology for estimating the project's impact on Carbon Dioxide (CO2) emissions35 during the preparations for Additional Financing in 2016. The analysis considered the traffic volume, traffic composition, travel speed, and travel distance over a twenty-year period, and provided stakeholders with, what was at the time, a novel data set to contribute to broader discussions on mitigating the impact of transport on GHG emissions. The same methodology was used for the post-completion estimation of the project’s impact on CO2 emissions (see Annex 4). The methodology and data obtained during the exercise are expected to inform the government policies on reducing CO2 emissions, as envisioned by the recently approved Government strategy document “Azerbaijan 2030: National Priorities for Socio-Economic Development”. 81. The above key government strategy emphasizes the priority use of environmentally friendly vehicles. This move towards electric vehicles and more efficient combustion engines will likely reduce the emission levels generated on the project roads over time. The project roads are currently in good condition, and maintenance systems have been established to ensure sustainability. Maintaining the roads in good condition beneficially impacts emissions and extends the periods between the carbon-intensive major periodic maintenance. 82. The project contributed to significantly improving the resilience of the country’s road network to climatic events. Both project roads are strategic links in the primary road network and were constructed to international standards to ensure resilience and durability. For example, roadside drainage, culverts, and bridges all have the capacity to handle abnormal but increasingly frequent rainfalls and local flooding. Additionally, the new M3 highway is constructed to a level sufficient to remain operational in the event of wide-scale flooding of the River Kur floodplain and delta, through which it passes. In 2010 widespread flooding badly disrupted transport links in the region, which remains vulnerable to climate change impacts36. Both project roads are also covered by Regional Maintenance Units established with the project support. The better management, training and equipment now available help to ensure the network remains operational despite the extremes in temperature, rainfall and snowfall. 83. Well designed and implemented civil works reduced the risk of landslide damaging the M4 Baku- Shamakhi road. The road passes through hilly areas with slopes prone to landslides induced by rainfall and earthquakes. Traffic restriction and partial closures were not uncommon and significant as the road is the only route between Baku and Shamakhi suitable for heavy goods vehicles. The project supported extensive geotechnical investigations and specialist design to remove or mitigate landslide risk to the upgraded road, thereby safeguarding a strategic network link Other Unintended Outcomes and Impacts 84. Upgrading of M3 road to motorway standards proved very timely as the importance of the road has increased since the start of the project. Thus, while the original idea was mainly to improve domestic and regional connectivity by upgrading the M3, in more recent years the road became an important element of the 35 Additional Financing Project Paper, 2016. 36 Project Appraisal Document for the Regional Connectivity and Development Project, April 2021. Page 20 of 58 The World Bank Third Highway Project (P118023) development of the global International North-South Transport Corridor (INSTC) initiative to facilitate transportation between Northern Europe and India. 85. The existing bridge over the River Kur was fully rehabilitated and strengthened to maintain accessibility for local and non-motorway traffic. While the M3 motorway uses a new Kur River crossing, the original bridge remains a critical link for residents and businesses in the Salyan region. Initially, only relatively minor repairs were planned, but the bridge's condition deteriorated, necessitating substantial repairs and upgrading of its weight capacity. This work was undertaken as part of the project, thereby maintaining the integrity of the local road network and its integration with the new motorway. 86. New project support. Following the completion of the M3 motorway, the Government sought to further develop connectivity along the Salyan-Bilasuvar area in the south of the country. With the World Bank’s concurrence, a new Bank financed project, the Regional Connectivity and Development Project (RCDP), was prepared and approved. About 250,000 people are expected to benefit from the upgraded safe, efficient, and climate-resilient connectivity and improved market accessibility in the predominantly rural area, in addition to improved economic opportunities provided by the new project. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 87. Project preparation drew on experience gained during two previous Azerbaijan highway projects and incorporated lessons from the ongoing Second Highway Project37. These included ensuring the readiness of civil works at the start of the project. For the Third Highway Project, a feasibility study was undertaken in advance, an economic evaluation report was prepared using specialist consultants, design documents for the first 30 km long section of M4 were ready, procurement advanced, and no land acquisition was necessary at that stage. 88. The design of the Third Highway Project benefited from an established long-term relationship between the Government and the Bank in the road sector. As a result, government stakeholders and the implementing authority were familiar with the Bank financed operations, and an experienced Project Implementation Unit (PIU) was in place managing the Second Highway Project. Knowledge of the implementing authority’s strengths and weaknesses helped the preparation team include appropriate strengthening to enable the PIU to simultaneously manage the Second and Third Highway Projects. Six operation-specific risk areas were also identified during design and mitigated them appropriately. Annex 6F contains a tabulates risks and mitigation at Appraisal. B. KEY FACTORS DURING IMPLEMENTATION 89. The socio-economic context underwent several changes during the lifetime of the project which had impacts on its implementation. Midway through the project, the country experienced a dramatic economic and banking crisis, currency volatility, a down-turn in Government revenues, and institutional changes. Project implementation also overlapped with the global pandemic and the country’s involvement in a major military conflict. These challenges required effective management of the project by the road administration and the Bank team. 90. Despite the project’s preparedness and advanced procurement, the implementation of the civil works 37 World Bank Project Appraisal Document, April 2010. Page 21 of 58 The World Bank Third Highway Project (P118023) over the initial years was below target. The contractor selected for the first M4 section, which had previously performed well under the Second Highway Project, failed to make adequate progress due to the emerging internal management and solvency issues. Progress was also affected by unaccustomed interventions from the Ministry of Emergency Situations, which gained new technical and safety oversight powers during the project implementation. The PIU and Bank Task Team closely monitored the situation and engaged with the contractor’s management to try and improve progress. This dialogue was ultimately unsuccessful, so the road administration terminated the contract and selected another firm to complete the work. The original contractor eventually stopped trading. 91. The detailed design and procurement of civil works for the remaining three sections of the M4 road were delayed by a discussion of uneconomic technical solutions initiated by counterparts after implementation had commenced. Some stakeholders preferred an option with a straighter road alignment in hilly sections. However, this solution required very costly structures and extensive land acquisition, an outcome specifically avoided when the project was prepared. The problem was resolved when the Bank raised the issue with the Government expressing concerns about a potentially large cost overrun and associated delays in project implementation. 92. The detailed design of the westerly sections of the M4, km 92 to km 106, was further hampered by complex geotechnical issues and the need to mitigate landslide risks . The design of this road section was ongoing at the start of the project by consultants financed by the Second Highway Project. However, as site investigations and preliminary design progressed, it became apparent that the complexity of the geotechnical conditions and landslide risks required resources beyond those the design consultants could provide. Consequently, an extensive and complex geotechnical investigation was commissioned from a specialist geotechnical contractor and a new international consulting firm appointed to undertake the design, both financed by the Third Highway Project. However, due to low disbursement associated with the landslide mitigation and the previously mentioned discussion of uneconomic technical solutions, the project performance rating was downgraded to Unsatisfactory in December 2012. 93. The Institutional development component of the project was successfully implemented in the early years and yielded good results. The Government and road administration agreed to follow recommendations arising from the technical assistance provided to modernize motorway management and maintenance practices. The recommendations included establishing a motorway management unit (MMU) and dedicated regional motorway maintenance units (RMUs). This Governmental agreement was a significant development in ensuring the sustainability of infrastructure investments. 94. Successful restructuring of the project in 2013 resolved implementation issues with civil works in both the Third and parallel Second Highway projects and enabled motorway maintenance reforms to move to the next stage. Restructuring excluded the western end of the M4 Baku-Shamakhi road due to the need for lengthy geotechnical investigations. It replaced it with the two sections of M3 highway, which were ready for implementation and previously were parts of the Second Highway Project. The restructuring also allowed for the adjustment of the institutional development component to align with the Government’s motorway maintenance modernization plan. 95. Devaluation of the national currency by more than a half between 2015 and 2017 and economic downturn severely impacted civil works contractors. The devaluation led to steep increases in both foreign and domestic costs, which was problematic because the contracts were denominated only in local currency and did not include price adjustment provisions38. The crisis also hit the banking sector, so the lack of credit availability further 38The price adjustment provisions were not used at that period per recommendation of the Ministry of Finance to minimize the price escalation under the contracts. Page 22 of 58 The World Bank Third Highway Project (P118023) aggravated cash flow problems being experienced by most contractors. The implementing agency, in consultation with the Bank, designed measures to support heavily affected contractors. One helpful action was to modify contract conditions to allow retention money to be replaced with a bank security, thereby improving cash flow with a cash injection. A second measure was to directly compensate for the effects of devaluation on the foreign cost component of a contractor’s costs. While the second measure, in particular, required careful monitoring by the road agency, the measures helped mitigate implementation delays and led to the eventual completion of works. 96. The Government’s request for Additional Financing in 2016 allowed the final stage of motorway maintenance reforms to be supported with specific measures, including the provision of depots, offices, and equipment along with technical assistance and capacity building. It also provided for two missing links on the M4 Baku-Shamakhi road to be reconstructed. 97. The COVID-19 pandemic and the recent military conflict were two major external events that impacted the project’s completion activities in 2020. Despite the severity of the events, their impact on the project was limited as most activities were complete. Nevertheless, restrictions on movement between the regions during the pandemic and the conflict slowed the completion of the maintenance facilities as the supply of labor and materials became constrained, and more stringent and safer working practices were introduced. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 98. The Project Appraisal Document provided the operation’s theory of change sufficiently clearly. It spelled out the activities, outputs, outcomes and identified indicators in the Results Framework to assess the achievements of the PDOs. The PDO indicators and the intermediate results indicators were designed to measure and relate the project’s progress towards the achievement of the PDO. 99. Three separate indicators were listed in the PAD to measure outcomes, although unpacking the PDO suggests four intended outcomes (efficiency, safety, quality, and improved management). The first two indicators directly measured the first two outcomes, leaving the third PDO indicator to rather broadly capture the network-wide quality and management improvement aspects. The indicator, which measured the network condition, did not differentiate between improvements gained through capital investment (road construction) or improvements due to better-managed maintenance. To address this, a fourth PDO indicator was introduced during the 2016 restructuring to precisely capture improved network management and reflect the increased scope of this component. 100. The indicators used to assess safety measured the reduction of personal injuries along the Baku-Shamakhi and Yenikend-Shorsulu roads for a baseline at the start of the project. These indicators could have been more rigorously designed to overcome some potential weaknesses. Firstly, the indicators would have been more precise if they had explicitly mentioned fatalities as well as injuries. Secondly, the indicators did not disaggregate benefits attributable to the project from broader national road safety trends. Thirdly, the accident patterns usually develop over several years after opening or improvement with considerable year-on-year variability, especially when considering individual road sections. These shortcomings were largely addressed during implementation as the M&E methodology utilized the combined fatality and injury data and used the multi-year average numbers. Page 23 of 58 The World Bank Third Highway Project (P118023) 101. The quality of the roads was assessed with a PDO indicator that measures the quality of the entire primary road network, not just the impact from project activities. The use of this “Core Indicator” indicator accurately captures the 35 percent increase in the total of M roads in good and fair condition. However, the indicator does not disaggregate the about ten percent increase directly attributable to the project. M&E Implementation 102. The PIU was responsible for collecting project information from other units and supervising consultants . Given that the PIU was overseeing two large World Bank projects, its capacity was somewhat constrained, so a full-time Monitoring and Evaluation Specialist was hired after project restructuring. The strengthened PIU team had the necessary knowledge and skills to collect and analyze data for the remainder of the project. Supervision consultants and a technical support consultant also supported the PIU with collecting and reporting M&E data. Semi-annual progress reports, prepared by the PIU, provided information on progress, payments, procurement, and safeguard compliance, thereby allowing the achievement of the PDO and implementation progress to be accurately evaluated. 103. The Bank’s supervision team ensured M&E was undertaken and reported continually to improve the quality and relevance of the project. This was particularly necessary to ensure that the restructuring and additional financing activities were well designed, reflected current circumstances, and addressed performance issues flagged up during monitoring and evaluation. One improvement that promoted citizen engagement and complemented M&E arrangements was introducing road user satisfaction surveys at the 2016 restructuring. As discussed earlier, the road administration engaged citizens in project areas to assess the public’s perception of road use. The survey, designed and managed by international consultants, studied whether the project improved road users’ satisfaction with the services and informed the project monitoring indicators. 104. One incomplete Intermediate Indicator was dropped during project restructuring – “Percentage of safety audits conducted on road accidents recorded during the construction stage of Baku-Shamakhi”. The road administration had difficulty obtaining detailed audit data due to weaknesses in the traffic police’s crash data system at the time. The Bank’s supervising team identified this issue and amended the Intermediate Indicators appropriately. 105. During the project implementation, the Bank team employed a tool to monitor the physical progress of various civil work activities through the innovative Project Assessment Tool (PAT). The exercise involved collecting structured information during the civil works site visits and visualizing the data against various project performance indicators. The analysis was based on measurable and comparable data, which was used to benchmark the performance of contractors and supervision engineers. The tool was very efficient in tracking progress and measuring the effectiveness of actions and a direct method to advise the road agency on performance. An example of PAT outputs is included in Annex 6G. M&E Utilization 106. Throughout project implementation, all indicators included in the Results Framework were regularly updated and reported in the six-monthly Implementation Status and Results (ISR) reports . Lack of implementation progress at the early stages of the project was evidenced by the reported indicators and was followed by the design of remedial actions, which led to project restructuring in 2013. The restructuring actions were also reflected in the revised Results Framework. During the additional financing in 2016, the project results framework was adjusted again to keep the project monitoring and evaluation system relevant and up to date. Page 24 of 58 The World Bank Third Highway Project (P118023) Justification of Overall Rating of Quality of M&E 107. During project implementation, the road agency consistently performed monitoring and evaluation, supplemented by the Bank implementation support missions and monitoring by the Bank’s local team. Despite some initial constraints, project monitoring and evaluation arrangements overall remained satisfactory for the project's duration. The M&E system was sufficient to assess the achievement of the objectives despite some relatively minor shortcomings with the original indicators which were overcome during implementation. Based on the above review, the quality of the M&E system is rated “Substantial”. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental Safeguards 108. The project was assigned environmental category “A” and involved implementing large and medium scale civil works. Site-specific EIAs and EMPs were undertaken for all road works contracts and updated at different stages during the implementation. The PIU ensured environmental compliance procedures were enforced and site staff included an experienced safeguards specialist to oversee environmental and social issues. Both contractors and consultants were contractually required to employ appropriately experienced staff and monitor and manage environmental and social safeguards at the site level. This multi-layer approach was complemented by oversight from Bank specialists and physical environmental inspections. The only significant issue to impact the environmental safeguard compliance ratings were delays in finalizing the EIA and EMP for some sections of the Baku-Shamakhi road. Overall, the system of environmental management functioned properly and managed the entire range of environmental impacts. Social Safeguards 109. The original project triggered OP 4.12 (Involuntary Resettlement) to treat the anticipated involuntary resettlement impacts for the original road sections. Similar impacts were also managed when new road sections were introduced following the restructuring. The land acquisition process was guided by the national land acquisition law prepared under the World Bank financed Second Highway Project and approved by Parliament. The road administration prepared a robust Resettlement Policy Framework (RPF) and Resettlement Action Plans (RAP) based on World Bank Policy standards, guidelines, and relevant local legislation, which were duly disclosed nationally and by the World Bank. The RAP detailed land acquisition and compensation entitlements for the users of the project affected land parcels as well as specific terms and conditions. 110. Overall, the project managed impacts on 441 affected people, including fourteen cases of economic displacement. Compensation payments were provided from state funds and administered by the road administration’s specialist department. Before works commenced, the PIU ensured that owners and users of the impacted land were fully compensated in accordance with the RAPs. However, the process was lengthy, particularly when landowners could not be traced, and a few cases were resolved only after being litigated. Slow progress with land acquisition in 2014 led the Bank to briefly downgrade the Involuntary Resettlement and Overall Safeguards ratings to Moderately Unsatisfactory. The ratings were upgraded at the next ISR review as outstanding legal matters were being resolved. 111. The project involved an operational grievance redress mechanism , run by contractors and monitored by supervision consultants under the oversight of PIU. Resettlement and land acquisition related grievances were also received directly by the land acquisition unit of the road administration. The contractors’ and supervision Page 25 of 58 The World Bank Third Highway Project (P118023) engineers’ teams included grievance focal persons who maintained grievance logbooks to record and process complaints. Most cases (such as accidental damages to irrigation or drainage pipes, ensuring continued access to land plots and dust levels) were resolved on-site by the contractors, with the remainder referred to and addressed by the PIU. Financial Management 112. The project benefited from having financial management arrangements already in place for earlier Bank financed projects. However, these arrangements were strengthened by hiring an additional dedicated financial management specialist for the PIU. Further improvements to financial management were made following recommendations made during the project's mid-term review in November 2013 when the PIU activated the contract management module of the 1C accounting software. This allowed the PIU to regularly monitor contract execution, develop project budgets by applying revised budget procedures, and enact the practice of financing invoices from DA or project account. The project financial audit reports were consistently submitted on time during the project implementation, and all payments under the project have been completed with no debt remaining. Financial Management was rated as Satisfactory in every ISR report since 2014 and the borrower complied with all the FM legal covenants39. Procurement 113. The procurement of Goods and Services for the project was undertaken following the World Bank’s Guidelines on Procurement of Goods, Works and non-consulting Services, January 2011 (revised July 2014) and Guidelines on Selection and Employment of Consultants under IBRD Loans and IDA Credits & Grants by World Bank Borrowers, January 2011 (revised July 2014). World Bank standard procurement documents were always used, and all procurement notices and requests were appropriately published in local and foreign media, the World Bank’s external website, UNDB online, and the implementing authority’s website (during the last years of the project). Increasingly, the Client Connection (CC) System of World Bank was used for advertisement. Throughout the project, participation in bids and proposal requests was good. 114. Procurement activities were managed and undertaken by the PIU, staffed by experienced procurement specialists. The evaluation of bids and proposals was undertaken by a tender commission established by the head of the road administration. This committee consisted of representatives of the road administration, including the PIU, plus external stakeholders from the Ministry of Finance and Ministry of Economy. In some cases, evaluation procedures took longer than planned due to the mandated size of the tender commission. The PIU reported evaluation results on standard World Bank forms and, in total, 47 procurement procedures led to the award of contracts for works, goods or services. C. BANK PERFORMANCE Quality at Entry 115. The Third Highway Project was built on the previous road projects in Azerbaijan and utilized existing implementation arrangements, with a PIU already operational within the road administration . Financial management, procurement, and safeguarding arrangements were in place, but some strengthening of implementation arrangements was later necessary. Design documents for the first major civil works contract were ready, with due environmental and social safeguard considerations complete. Risks to the project had been assessed and mitigated (see Annex 6F) during the project preparation phase and M&E arrangements were in 39 Aide-Memoire- World Bank Project Implementation Support Mission May 15-19, 2017 Page 26 of 58 The World Bank Third Highway Project (P118023) place. 116. Project design reflected lessons learned from earlier and ongoing transport projects. For example, the Baku-Shamakhi road upgrading was planned to minimize the need for land acquisition, a cause of delays in the Second Highway Project and the first section was designed without the need for acquisition despite the road being widened to a dual carriageway. Project preparation was also careful to ensure the project’s specifications aligned with local capacity and its scope and size matched to the ability and track record of the road administration. 117. The project was designed to be closely aligned with the Government’s long-term objectives to improve the living standards of the low-income population through non-oil sector growth and strengthen Azerbaijan’s resilience to external shocks. The project was also designed to help improve the sustainability of the substantial capital investments being made in the road sector. While the institutional development activity was not particularly ambitious at Entry and not directly reflected in the PDO Indicators, these issues were addressed during the later restructuring when the Government’s approach to maintenance reform firmed up. Quality of Supervision 118. The Bank had a strong team predominantly based in the country for the duration of the project. The permanent presence of the TTL (or co-TTL), along with procurement, financial, environmental, and social specialists, helped the supervision team to closely support the Government and road administration with implementation and quickly react to changing circumstances and events. In addition, for much of the period, the team was supplemented with the CO-based highway engineer. 119. The proactive management of the Bank task team allowed making appropriate decisions on project restructuring and additional financing for ensuring the required developmental impact . Restructuring and adjustments enabled the project to respond to evolving realities, realign the Bank’s road sector portfolio, and eventually achieve its objectives. Delivered outcomes were also broader than envisaged at Entry, such as completing the Yenikend-Shorsulu motorway and establishing regional maintenance units. 120. The Bank team undertook about two implementation support missions per year, 23 in total during project implementation. The observations and the issues relating to the progress obtained from the field visits were then discussed with the Government’s counterpart agencies, and the required steps and improvements were agreed upon to achieve the relevant development objectives. The component for institutional development of the road agency in motorway operation and maintenance required particularly close supervision and guidance to the client. This task was successfully achieved, and service level principles were introduced to help ensure the sustainability of the road network assets. 121. The project’s ISRs highlighted issues as they arose and reflected the same in the assigned ratings. For example, during the earlier stages of the project, some of the project ratings for the Development Objectives (DO) and the Implementation Progress (IP) were at Moderately Unsatisfactory and briefly Unsatisfactory levels. However, ratings improved after the critical project challenges were addressed and put on the proper development track. These changes reflected the team’s effort to candidly report the actual project situation as it was happening during implementation. Justification of Overall Rating of Bank Performance 122. The Bank performance during both project preparation and implementation was without significant Page 27 of 58 The World Bank Third Highway Project (P118023) shortcomings. Overall, the Bank’s team proactively addressed the emerging challenges during the project implementation and ensured successful delivery of the project activities and outcomes. Therefore , the overall rating for the Bank’s performance is considered “Satisfactory”. D. RISK TO DEVELOPMENT OUTCOME 123. At completion, the risk to the sustainability of the achieved development outcomes is low, but not insignificant. The institutional development component facilitated comprehensive and sustainable maintenance arrangements for the primary road network, including the two project roads. The maintenance units have been resourced with facilities, equipment, and trained staff. However, the implementation of performance-based road maintenance practices remains in its early stages, so the long-term adoption of this approach will depend on ongoing benefits being perceived and realized by the road agency and wider Government. 124. To ensure the sustainability of the primary road network, the MMU and RMUs must be adequately funded through annual budget allocations. Continuous financial support will be required for the reforms to be rolled out to the entire road network. Furthermore, given the significant financing needs for the post-conflict reconstruction, the state budget for road maintenance might be constrained as priorities shift to rehabilitating infrastructure in the reacquired territories. 125. During implementation, the length of M4 upgraded under the project was adjusted with the section km 13 to 15 added and section km 107 to km 116 removed. The action optimized the investment without putting the development objectives at risk, as the project upgraded the entire length of the single carriageway road between Baku and Shamakhi to a dual carriageway motorway. The removed section was a bypass planned for Shamakhi City, which had less strategic importance than the other sections, and did not go ahead due to developing Government priorities, including ensuring sufficient financing to complete the key M3 motorway. V. LESSONS AND RECOMMENDATIONS 126. Proactive supervision and implementation support by the Bank’s team and skillful use of the restructuring tool can make a big difference. The design of restructuring in 2013 undertaken with a portfolio review helped to overcome sectoral challenges existing at that time with two ongoing highway projects. The projects both faced challenges, which were quite severe in the case of the Second Highway project. Having two parallel projects with broadly similar PDO enabled a complex but creative simultaneous restructuring to optimize their scopes and reassign activities to the most appropriate project. This action significantly improved the performance of both projects while ensuring the eventual achievement of their development objectives. 127. The benefits of safe design are realized very quickly after the completion of road construction. Death and injury rates declined substantially along both project corridors as soon as new road sections opened to traffic. This trend was because the accident risks associated with the higher speeds were more than offset by the road safety features, particularly separating traffic with median barriers, improved junctions, and provisions for pedestrians in urban areas. The project’s support for good design practices through the use of experienced international consultants and road safety audits during both the design and construction phases significantly contributed to this outcome. 128. The road maintenance reforms help demonstrate that the reform of state-owned entities and the introduction of modern management approaches through Bank-supported projects is feasible but takes time and requires flexibility. At Entry, the institutional development objectives of the Third Highway Project were Page 28 of 58 The World Bank Third Highway Project (P118023) modest. However, after providing options and engaging in a long-running dialogue, the Government adopted recommendations on the modernization of motorway management and maintenance practices. This decision allowed the project to be restructured to provide extensive support for establishing new maintenance units and improving management practices. The full cycle of maintenance and management reforms were supported by the project from initiation and design, to implementation. 129. Related to the above, the actions to strengthen and reform institutions must be made in the context of the institution’s capacity for change. The project’s support for performance-based contracting concepts without private sector involvement was a cautious but necessary approach because the private sector involvement was considered too ambitious at the time. Ultimately, this pragmatism allowed the successful adoption of client- provider relationships governed by service level agreements - a significant advance in modernizing public sector service delivery. At the same time, the model provides a platform for introduction of the private sector engagement in the road maintenance. The model can also be replicated during the implementation of ongoing state-owned enterprise reforms recently launched by the Government. 130. To safeguard successful early disbursement, more than one high value activity should ideally be ready to commence at Effectiveness or shortly afterwards. The Third Highway Project became Effective with the design and procurement ready for one 30 km section of the Baku-Shamakhi road and other sections under advanced preparation. However, disbursement was initially poor because a contractor struggled to progress the work and then external challenges delayed the completion of designs and the procurement of subsequent sections. 131. Implementing agencies should strongly encourage all bidders for works contracts to reflect their foreign currency costs in payment schedules, regardless of historical exchange rate performance. The sudden devaluation of the local currency, during project implementation, created cashflow problems for several contractors as they had priced the work in local currency but had costs in foreign currencies. While the standard World Bank bidding documents used for the project allowed bidders to include foreign currencies, none did as the local currency had previously been very stable. Consequently, contractors were over-exposed to the effects of currency fluctuation and devaluations. 132. Complex technical challenges should be recognized early and addressed using appropriate specialist firms. During early implementation, site investigation and engineering study revealed ground conditions to be far more complex than originally expected. These conditions contributed to a high landslide risk along part of the M4 corridor. Solutions proposed included a substantial realignment of the road with many heavy civil engineering structures, but these were uneconomic. Restructuring allowed the project to support the appointment of a specialist geotechnical contractor and designer find an economic technical solution. This approach took time, but subsequently allowed the road section to be successfully completed under the project. . Page 29 of 58 The World Bank Third Highway Project (P118023) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: To contribute to a more efficient and safer Baku-Shamakhi and Yenikend-Shorsulu roads Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Efficiency: Reduction in road Percentage 100.00 80.00 77.00 user costs from Yenikend to Shorsulu 15-Jan-2014 31-Dec-2019 31-Mar-2021 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Efficiency: Reduction in road Percentage 100.00 80.00 70.00 user costs from Baku to Shamakhi 25-May-2010 31-Dec-2019 31-Mar-2021 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 30 of 58 The World Bank Third Highway Project (P118023) Safety: Reduction of personal Percentage 100.00 70.00 29.00 injury per 100-million vehicle- km along the Baku-Shamakhi 25-May-2010 31-Dec-2019 31-Mar-2021 road. Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Safety: Reduction of personal Percentage 100.00 70.00 24.00 injury per 100-million vehicle- km along the Yenikend- 15-Jan-2014 31-Dec-2019 31-Mar-2021 Shorsulu road. Comments (achievements against targets): Objective/Outcome: To contribute to higher quality road services as part of the network upgrading to motorway standard Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Quality:Percentage of M Percentage 55.00 70.00 90.00 roads in good and fair condition as a share of total 15-Jan-2014 31-Dec-2019 31-Mar-2021 M roads of the network Comments (achievements against targets): Page 31 of 58 The World Bank Third Highway Project (P118023) Objective/Outcome: To improve the management of the nascent motorway network Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion RMUs to maintain 500km of Kilometers 0.00 500.00 774.00 M roads under service level parameters 01-May-2016 31-Dec-2019 31-Mar-2021 Comments (achievements against targets): A.2 Intermediate Results Indicators Component: Motorway Improvement Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of km upgraded for Kilometers 0.00 94.00 94.00 Baku-Shamakhi road 15-Jan-2014 31-Dec-2019 31-Mar-2021 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of km upgraded for Kilometers 0.00 48.00 50.00 Page 32 of 58 The World Bank Third Highway Project (P118023) Yenikend-Shorsulu road 15-Jan-2014 31-Dec-2017 31-Mar-2021 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Roads constructed, non-rural Kilometers 0.00 142.00 144.00 25-May-2010 31-Dec-2019 31-Mar-2021 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Safety audits on detailed Text No Yes Yes design. 25-May-2010 31-Dec-2018 31-Mar-2021 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Safety audits on construction Text No Yes Yes completion. 25-May-2010 31-Dec-2019 31-Mar-2021 Page 33 of 58 The World Bank Third Highway Project (P118023) Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Improvement in Road Users Percentage 21.00 75.00 79.00 Satisfaction in Project Area with disaggregated gender 01-May-2016 31-Dec-2019 31-Mar-2021 information Comments (achievements against targets): Component: Institutional Development and Support for Motorway Maintenance Reform Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Design of a Motorway Text No Yes Yes Guidelines. 25-May-2010 30-Sep-2016 31-Mar-2021 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Development of motorways Text No Yes Yes Page 34 of 58 The World Bank Third Highway Project (P118023) management and 15-Jan-2014 30-Sep-2016 31-Mar-2021 maintenance strategy Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Annual maintenance plan Kilometers 0.00 500.00 774.00 developed by MMU using road data base. 01-May-2016 31-Dec-2019 31-Mar-2021 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Rolling 2 year budget Text No Yes Yes prepared by MMU for RMUs 01-May-2016 31-Dec-2019 31-Mar-2021 Comments (achievements against targets): Page 35 of 58 The World Bank Third Highway Project (P118023) B. KEY OUTPUTS BY COMPONENT Objective/Outcome 1 - A more efficient Baku-Shamakhi road. Outcome Indicators 1. Efficiency: Reduction in road user costs from Baku-Shamakhi. 1. Number of km upgraded from Baku-Shamakhi road. Intermediate Results Indicators 2. Roads constructed, non-rural. 1. 94 km of the M4 Baku-Shamakhi road upgraded from a single carriage road Key Outputs by Component to a dual carriageway highway. (linked to the achievement of the 2. 30 percent reduction in road-user costs. Objective/Outcome 1) 3. An average reduction in journey time of 19 minutes for drivers and passengers traversing the length of the project road. Objective/Outcome 2 - A more efficient Yenikend-Shorsulu road. Outcome Indicators 1. Efficiency: Reduction in road user costs from Yenikend to Shorsulu. 1. Number of km upgraded from Yenikend to Shorsulu. Intermediate Results Indicators 2. Roads constructed, non-rural. 1. A 50 km long section of the M3 Alat Astara motorway between Yenikend Key Outputs by Component and Shorsulu. (linked to the achievement of the 2. 23 percent reduction in road-user costs. Objective/Outcome 2) 3. The average journey time from Yenikend to Shorsulu reduced by 18 minutes to 34 minutes. Objective/Outcome 3 - A safer Baku-Shamakhi road. 1. Safety: Reduction of personal injury per 100-million vehicle-km along the Outcome Indicators Baku-Shamakhi road. 1. Safety audits on detailed design. Intermediate Results Indicators 2. Safety audits on construction completion. 1. The accident rate on the Baku-Shamakhi section of the M4 highway fell by Key Outputs by Component an average of 71 percent compared to pre-project levels. (linked to the achievement of the 2. Safety audits undertaken at design and completion stages to international Objective/Outcome 3) standards. Objective/Outcome 4 - A safer Yenikend-Shorsulu road. 1. Safety: Reduction of personal injury per 100-million vehicle-km along the Outcome Indicators Yenikend-Shorsulu road. 1. Safety audits on detailed design. Intermediate Results Indicators 2. Safety audits on construction completion. Page 36 of 58 The World Bank Third Highway Project (P118023) 1. A 76 percent reduction in the accident rate recorded along the new Key Outputs by Component motorway section compared to the rate pre-project along the original road. (linked to the achievement of the 2. Safety audits undertaken at design and completion stages to international Objective/Outcome 4) standards. Objective/Outcome 5 - Higher quality road services as part of the general network upgrading to motorway standard. 1. Quality: Percentage of M roads in good and fair condition as a share of total Outcome Indicators M roads of the network. 1. Roads constructed, non-rural. Intermediate Results Indicators 2. Improvement in Road Users Satisfaction in Project Area with disaggregated gender information 1. 90 percent of the Magistral (M) road network is, at least, in a fair condition, of which over 75 percent were measured as being in a very good condition. Key Outputs by Component 2. 144 km of the M3 and M4 roads upgraded to modern dual-carriageway (linked to the achievement of the highways by the project. Objective/Outcome 5) 3. Overall satisfaction amongst travelers and residence in the two project areas rose from 21 percent to 79 percent. However, satisfaction amongst women was nine percent less than for men. Objective/Outcome 6 - Improved management of the nascent motorway network. Outcome Indicators 1. RMUs to maintain 500 km of M roads under service level parameters. 1. Design of Motorway Guidelines. 2. Development of motorways management and maintenance strategy. Intermediate Results Indicators 3. Annual maintenance plan developed by MMU using road data base. 4. Rolling 2 year budget prepared by MMU for RMUs 1. 774 km of Magistral (M) roads operated and maintained under Service Key Outputs by Component Level Agreements between the MMU and three RMUs. (linked to the achievement of the 2. Management structures are in place within the road administration for Objective/Outcome 6) maintenance planning and budgeting. Page 37 of 58 The World Bank Third Highway Project (P118023) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Nijat Valiyev Task Team Leader(s) Tanvir Hossain Procurement Specialist(s) Tural Jamalov Financial Management Specialist Rahmoune Essalhi Procurement Team Gulana Enar Hajiyeva Environmental Specialist Vusala Asadova Procurement Team Sophia V. Georgieva Social Specialist Ioannis Dimitropoulos Team Member B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY10 40.022 273,064.80 FY11 0 0.00 Total 40.02 273,064.80 Supervision/ICR FY10 0 18.95 FY11 46.285 141,241.30 FY12 36.294 105,986.37 FY13 42.602 112,540.95 FY14 45.348 135,922.01 FY15 43.609 137,159.96 Page 38 of 58 The World Bank Third Highway Project (P118023) FY16 42.803 114,140.64 FY17 62.885 103,378.57 FY18 47.724 132,512.53 FY19 31.908 101,180.53 FY20 35.974 130,157.30 Total 435.43 1,214,239.11 Page 39 of 58 The World Bank Third Highway Project (P118023) ANNEX 3. PROJECT COST BY COMPONENT Amount at Actual at Percentage of Components Approval* Project Closing Approval (US$ (US$ million) (US$ million) million) Component 1: Motorway Improvement, including upgrading a 94 km section of M4 Baku-Shamakhi and 48 419.40 374.43 89% km section of M3 Yenikend-Shorsulu roads into a four- lane motorway. Component 2: Support for Motorway Maintenance Reform (management and financing Azerbaijan 25.00 15.60 62% Motorways; institutional development of the ARS) Component 3: Project Management and Institutional Support to ARS including technical assistance and 4.00 3.27 82% financing additional costs for project implementation. Contingency 28.70 0 0 Front End Fee 0.78 0.78 Total 477.88 394.08 82% * Amount at Approval is the approved component cost after restructuring in 2013 and additional financing in 2016 Page 40 of 58 The World Bank Third Highway Project (P118023) ANNEX 4. EFFICIENCY ANALYSIS A. Methodology 1. The project was to finance the widening and upgrading of Baku-Shamakhi section of M4 motorway and Yenikend-Shorsulu section of M3 motorway under the Third Azerbaijan Highway Project (AHP3) approved in 2010, AHP3 restructuring (2013), and AHP3 additional financing (2016). 2. The economic re-evaluation was conducted for the overall motorway section of M4 Baku-Shamakhi from km 13 to km 107 (94 km) and the overall M3 Yenikend-Shorsulu (50 km), and for five sections of M4 consistent with the appraisal reports (the additional financing to AHP3). It was carried out by comparing with- and without- project scenarios. The base case reflects expected conditions in the absence of the project, with maintenance sufficient to hold the motorways in their present condition. 3. The re-evaluation was conducted by using the Highway Development Model (HDM-4) version 2.11. It predicts the average travel speed, pavement International Roughness Index (IRI), agency costs, and road user costs and calculates the economic viability of motorways. All costs and benefits are expressed in constant mid- 2020 prices. 4. The analysis was conducted using the border price numeraire. A conversion factor of 0.85 was applied to calculate the economic price of the capital and maintenance costs like that used at appraisal. The net present value (NPV) at a 12 percent discount factor and the economic internal rate of return (EIRR) were calculated to evaluate the economic viability of the road sections after completion financed under the original financing and a 6 percent for road sections under the additional financing. The km 15 to km 45 section was financed under the original and additional financing, and therefore a 9 percent weighted average discount factor was used to calculate the NPV for this section. 5. The economic re-evaluation applied the actual project cost and construction period and the updated traffic forecast.40 The traffic was re-projected with the actual 2019 traffic as the base. B. Project scope, cost, and construction period 6. The project under the original and additional financing was to upgrade 94 km of the existing two lane M4 Baku-Shamakhi road. The existing 2-lane M4 was already improved under the World Bank-financed Second Highway Project which closed in 2015 and hence AHP3 was to widen the motorway into a 4-lane motorway. 7. Through restructuring of the original project, one road section, M4 (km 92 – km 116), was dropped and the associated fund diverted to construct a new 48 km 4-lane Yenikend-Shorsulu section of the M3 Alat-Astara motorway parallel to the existing road. There has been no substantial difference in the project road length during appraisal and after project completion. 8. The project involves capital expenditures followed by routine maintenance and was implemented over 40 The project was appraised in 2010 (original Azerbaijan Highway Project or AHP3), 2013 (AHP3 restructuring), and 2016 (additional financing to AHP3) Page 41 of 58 The World Bank Third Highway Project (P118023) various construction periods for different motorway sections as reported by the Borrower’s ICR. The design and supervision costs were included in the project (civil works) cost for the economic analysis purposes. 9. The roadwork costs at appraisal and the actual expenditures after completion are presented in Table 4-1. Table 4-1: Civil Works Costs at Completion* Motorway section Cost Cost per kilometer (US$ million) (US$ million) M4 Baku-Shamakhi Section 1: km 13.3-15 11.67 6.5 Section 2: km 15-45 63.64 2.1 Section 3: km 45-67 57.16 2.5 Section 4: km 67-91 55.32 2.3 Section 5: km 91-107 68.70 4.3 Subtotal M4 256.49 2.7 M3 Yenikend-Shorsulu 120.68 2.2 * The cost includes taxes, excludes design and supervision costs 10. Road maintenance strategy is as follows and assumed to be identical for all project roads: Table 4-2: Maintenance Strategy Intervention Unit Cost (financial cost) Routine maintenance US$4,720/km Pothole patching US$16.5/m2 Crack sealing US$5.3/m2 Edge break repair US$14.2/m2 50 mm overlay (IRI>6) US$36.5/m2 Source: Consultant estimates 11. The construction period at appraisal was estimated for 4 years (2011-2014 for the original project and 2016- 2019 for the additional financing). During restructuring, the original project was expected to finish by 2016. The economic re-evaluation has incorporated the actual implementation period for different road sections. C. Project Benefits 12. The project’s economic benefits are vehicle operating cost (VOC), time value, and accident cost savings. The VOC-international roughness index (IRI) equations of HDM-4 were used to calculate VOC. Without project, the pavement roughness was estimated at IRI 2 for M4 following the completion of the existing 2-lane M4 upgrade under Azerbaijan Highway Project 2 and IRI 6 for M3 and with the project, the IRI is assumed to be 2.0 for both motorways. 13. The value of travel time savings was estimated at US$7.5 per hour for working time and US$2.25 per hour for non-working time over the forecast period under the assumptions the working time includes 33 percent of non-wage costs and the non-working time cost was estimated at 30 percent of the working time value. 14. As per the appraisal report, traffic accidents were expected to reduce by 20 percent. The re-evaluation used the average accident level over 2016-2018 period as the basis. Under the without-project scenario, the accident Page 42 of 58 The World Bank Third Highway Project (P118023) and fatality rates were taken from the actual traffic accident data available in the project area. The values of US$204,780 for fatality, US$27,657 serious injury, and US$817 damage accidents were used to value benefits from traffic accident savings suggested at appraisal. D. Vehicle Characteristics 15. The following vehicle characteristics at appraisal were used in the economic re-evaluation. Table 4-3: Vehicle characteristics input for HDM-4 3-axle 4+axle 2-axle Vehicle costs (in US$) Car Minibus LGV Bus Rigid articulated GV HGV GV New vehicle cost 21,603 24,916 18,264 155,820 32,474 54,500 283,073 Cost of new tire 86 99 348 175 200 300 375 Cost per hour of Repair/ Maintenance 24.74 24.74 24.74 24.74 24.74 24.74 24.74 Cost per hour of Vehicle Crew 0 25.11 20.92 25.11 20.92 25.11 25.11 Passenger in work time delay cost per hour 7.50 7.50 7.50 7.50 7.50 7.50 7.50 Passenger in non-work time delay cost per 2.25 2.25 2.25 2.25 2.25 2.25 2.25 hour Annual interest (%) 12 12 12 12 12 12 12 Source: Project Appraisal Document, Azerbaijan Third Highway Project, The World Bank, 2010. E. Traffic Forecasting 16. Traffic forecast was carried out for 20 years after project completion. At appraisal, the normal traffic was projected to grow following the demand elasticity of 1.2 and 1.0 with respect to the economic growth for passenger and freight traffic, respectively. Generated traffic of 10 percent was added to the normal traffic following the anticipated lower transport costs and shorter travel times after project completion. 17. M4 Baku-Shamakhi. The base traffic data for M4 at appraisal (2010) was estimated by extrapolating the 2005 traffic data to result in about 15,000 vehicles per day (vpd) in 2009, which was higher than the observed 13,000 vpd average traffic for the entire motorway section. The Restructuring Paper has adjusted the traffic forecast based on the actual traffic and the traffic growth rate for the forecasting. 18. This ICR used the actual 2019 traffic of 12,668 vpd average for the entire M4 section as the base and the updated traffic growth considering the recent World Bank’s GDP growth forecast for Azerbaijan. This has resulted in the forecasted traffic (average for the entire project road) of about 16,700 vpd in 2030 and 22,600 vpd in 2040. 19. M3 Yenikend-Shorsulu. M3 average daily traffic has increased from 7,720 vpd in 2010 to reach 8,522 vpd in 2019. After project completion, 8,738 vpd traffic was recorded. The new road traffic was projected to reach approximately 15,000 vpd in 2040. F. Findings 20. The upgrading of the M4 Baku-Shamakhi highway from km 13 to km 107 (Shamakhi) is economically justified with an average EIRR of 13.2 percent and a NPV of US$105.7 million. The rate of return for the four Page 43 of 58 The World Bank Third Highway Project (P118023) individual sections from km 13 to km 91 are in a range from 12.2 percent to 34.0 percent, with the short section from km 13 to km 15 showing the highest EIRR due to very high traffic. Under the without project scenario, the traffic already reached capacity in 2020 resulting in high road user costs. The km 67 to km 91 section, when considered on its own, is justified from an economic perspective with EIRR of 12.2 percent but sensitive to traffic being lower than predicted or higher maintenance cost. If considered in isolation, the final section, km 91 to km 107, has a low EIRR of 4.8 percent due to the high cost of construction in landslide affected areas, but not to the extent that it compromises the economic justification of the whole M4 project. Table 4-4: Economic internal rates of return of M4 Baku-Shamakhi EIRR (%) Section NPV* Base case Traffic No Maintenance cost No accident (US$ million) 10% lower generated increase by 10% reduction traffic km 13-15 52.6 34.0 29.5 33.8 31.8 32.9 km 15-45 44.5 16.9 15.3 16.6 15.8 16.1 km 45-67 13.5 15.6 14.0 15.4 14.5 14.8 km 67-91 0.7 12.2 10.7 11.9 11.2 11.7 km 91-107 -5.6 4.8 3.9 4.6 4.0 4.3 *Note: The net present value (NPV) at a 12% discount factor and the economic internal rate of return (EIRR) were calculated to evaluate the economic viability of the road sections after completion financed under the original financing and a 6% for road sections under the additional financing. The km 15-45 section was financed under the original and additional financing, and therefore a 9% weighted average discount factor was used to calculate the NPV for this section. 21. M3 Yenikend-Shorsulu. The economic re-evaluation estimated 13.2 percent EIRR and US$13.4 million NPV for the entire Yenikend-Shorsulu section demonstrating the project is economically viable. In addition, the sensitivity tests undertaken to assess the robustness of the project demonstrated that the project road remains economically justified under different scenarios. Table 4-5: Economic Re-evaluation of M3 Yenikend-Shorsulu Scenario EIRR (%) Base case 13.2 Traffic decrease by 10% 12.1 No generated traffic 12.8 Maintenance cost increase by 10% 12.6 No reduction of traffic accidents 12.8 22. Overall evaluation. The economic re-evaluation of the overall investment suggests the project is economically justified with the weighted average EIRR of 13.2 percent and NPV of US$119.1 million. The sensitivity tests also demonstrate the project remains viable economically with respect to changes in critical factors, including lower projected traffic, higher maintenance costs, and no reduction in traffic accidents. It should be noted, however, that the 16 km long M4 section km 91 to km 107 is not economically feasible if considered in isolation from the rest of the project. Page 44 of 58 The World Bank Third Highway Project (P118023) Table 4-6: Economic Re-evaluation of the Overall Project EIRR (%) No 10% increase in 10% lower generated maintenance No accident Threshold Road section Cost ($ million) NPV ($ million) Base case traffic traffic cost reduction (%) M4 256.49 105.7 13.1 11.6 12.9 12.1 12.5 9.4 M3 120.68 13.4 13.2 12.1 12.8 12.6 12.8 12.0 Overall 377.17 119.1 13.2 11.8 12.9 12.3 12.6 10.2 23. Carbon dioxide (CO2) Emission. CO2 emissions are estimated using HDM-4 software primarily influenced by the traffic volume, traffic composition, travel speed, and travel distance. HDM-4 predicted CO2 emissions under with and without project scenario over the operation period and the result is summarized in Table 7 showing the CO2 emissions with project is estimated 17 percent higher than that of without project due to higher travel speeds. Table 4-7: Carbon Dioxide Emission Prediction (ton) Motorway Without project With project M4 Baku-Shamakhi 2,873,741 3,341,989 M3 Yenikend-Shorsulu 2,162,688 2,560,833 Total 5,036,429 5,902,842 Page 45 of 58 The World Bank Third Highway Project (P118023) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Page 46 of 58 The World Bank Third Highway Project (P118023) Unofficial translation Ministry of Finance of the Republic of Azerbaijan November 25, 2021 To: Mr. Rufat Mammadov Cheif of Office of the Cabinet of Ministers of the Republic of Azerbaijan Ref: No. 38262 dated November 20, 2021 Implementation Completion Report for the Third Highway Project, Azerbaijan Dear Mr. Rufat Mammadov, The letter submitted by the World Bank and the attached report on the Implementation Completion Report for the Third Highway Project implemented with the World Bank’s co-financing has been duly reviewed by the Ministry of Finance of the Republic of Azerbaijan and considering that the Bank has assessed the issues related to implementation and financing of the project as satisfactory, we hereby state that we have no comments on the report in general. Meanwhile, regarding the World Bank's remarks on some delays in completion of the road maintenance facilities under the project for a number of reasons (such as some delays in mobilization of the contractor and land acquision documentation, closure of borders and interruptions in the manufacturing of the ordered goods due to the Covid-19 pandemic), please note that, considering that the delays in the implementation of some of the investment projects co- financed by IFIs lead to additional unnecessary costs and dissatisfaction of donors, the Ministry of Finance has sent at various times notification letters to the implementing agencies to speed up the construction works and strengthen the necessary supervision measures. Deputy Minister Azer Bayramov Page 47 of 58 The World Bank Third Highway Project (P118023) Page 48 of 58 The World Bank Third Highway Project (P118023) Page 49 of 58 The World Bank Third Highway Project (P118023) ANNEX 6. SUPPORTING DOCUMENTS Contents A. Project Map B. The development of outcomes and indicators C. Component Costs D. Road User Satisfaction Surveys Findings E. Population and relative income data F. Risk and mitigation at Appraisal G. Project Assessment Tool (PAT) Page 50 of 58 The World Bank Third Highway Project (P118023) A. Project Map Page 51 of 58 The World Bank Third Highway Project (P118023) B. The development of outcomes and indicators TABLE 6B-1. Revision to Outcomes and PDO Indicators End Target Outcome Indicators Baseline Re- Additional Original structuring Financing (2010) (2013) (2016) Efficiency: Reduction in road (i) a more efficient Baku- user costs from Baku to 100% 80% 80% 80% Shamakhi road. Shamakhi. Efficiency: Reduction in road (ii) a more efficient Yenikend- user costs from Yenikend to 100% -- 80% 80% Shorsulu road. Shorsulu. (iii) a safer Baku-Shamakhi road. Safety: Reduction of personal injuries along the Baku- 100% 70% 70% 70% Shamakhi road. (iv) a safer Yenikend-Shorsulu Safety: Reduction of personal road. injuries along the Yenikend- 100% -- 70% 70% Shorsulu road. (v) higher quality road services Quality: Percentage of M Roads as part of the general network in Good and Fair Condition as a 55% 75% 90% 90% upgrading to motorway share of total M Roads of the standard. network. (vi) improved management of RMUs to maintain 500 km of M the nascent motorway network. roads under service level 0 km -- -- 500 km parameters. Page 52 of 58 The World Bank Third Highway Project (P118023) C. Component Costs TABLE 6C-1. Changes to Project Components and Costs during Implementation US$ million excluding VAT 2016 2013 2020 Project Cost by Component At Entry Additional Restructuring Amendment Financing Component 1: (at Entry) Upgrading a 100 km section of existing 296.00 two lane M4 Baku-Shamakhi road into a four-lane motorway. Component 1 (Revised on 2013): Upgrading a 77 km section of M4 Baku-Shamakhi and 48 km section of M3 Yenikend-Shorsulu 316.00 roads into a four-lane motorway. Component 1 (Revised on 2016): Upgrading a 94 km section of M4 Baku-Shamakhi and 48 km section of M3 Yenikend-Shorsulu 419.40 369.60 roads into a four-lane motorway. Component 2: Institutional development (management and financing Azerbaijan’s motorways; institutional development of 5.00 5.00 the ARS). Component 2 (Revised 2016): Institutional Development and Support for Motorway Maintenance Reform (management and 25.00 19.35 financing Azerbaijan’s motorways; institutional development of the ARS). Component 3: Project Management including technical assistance and financing additional costs for project 1.00 1.00 implementation. Component 3 (Revised 2016): Project Management and Institutional Support including technical assistance and 4.00 5.32 financing additional costs for project implementation. Contingency 28.70 Total for Components 302.00 322.00 477.10 394.27 Front-end Fee 0.43 0.43 0.78 0.78 Total Cost 302.43 322.78 477.88 395.05 CHART 6C-1. Project Cost Change Timeline Total Project Cost (US$ million) against years of implementation 600.00 BANK GoA VAT 400.00 US$ milion 200.00 0.00 10 11 12 13 14 15 Year 16 17 18 19 20 21 Page 53 of 58 The World Bank Third Highway Project (P118023) D. Road User Satisfaction Surveys Findings Undertaking the road user satisfaction surveys evidenced improvements in quality, safety, and overall quality of road services. This citizen engagement also contributed to improving the management of the network by providing stakeholder feedback on the quality of service provided. Selected findings from the Road User Satisfaction Survey of Motorways M3 and M4 - May 2019. Azerbaijan Third Highway Project are tabulated bellow. TABLE 6D-1. Road User Satisfaction at completion Overall Satisfaction* 79% Satisfaction with M3 89% Satisfaction with M4 73% Satisfaction of female respondents 72% Satisfaction of male respondents 81% Satisfaction of residents 77% Satisfaction of travelers 80% * up from a baseline of 19% Page 54 of 58 The World Bank Third Highway Project (P118023) E. Population and relative income data TABLE 6E-1. Beneficiaries in the Project Impacted Regions Relative Income Level42 Region Population (2020) 41 (ranked, 2020) Mountainous Shirvan Economic Region 322,200 7th out of 11 economic regions Lankaran-Astara economic region 946,700 10th out of 11 economic regions Bilasuvar, Salyan and Neftchala rayons 288,900 11th out of 11 economic regions43 of the Aran Economic Region All directly impacted regions 1,557,800 41 The State Statistical Committee of the Republic of Azerbaijan, table 1.15. Territories, number, and density of population by towns and regions of the Republic of Azerbaijan, 2020. 42 The State Statistical Committee of the Republic of Azerbaijan, table 5. Income by economic regions in 2020. 43 Based on income for the whole Aran region Page 55 of 58 The World Bank Third Highway Project (P118023) F. Risk and mitigation at Appraisal TABLE 6F-1. Operation-specific risk areas and mitigation at Appraisal Rating after Risk Explanation mitigation (i) Technical Design The technical challenge of designing a four-lane road largely to fit within Moderate the existing corridor, especially in the mountainous section. (ii) Implementation Capacity Stretched implementation capacity of the ARS and slow disbursement of Moderate and Sustainability the Second Highway Project. (iii) Financial Management Financial Management weakness during the Second Highway Project. Moderate (iv) Governance and Limited coordination between public agencies, weak country systems, Moderate Procurement capacity constraints, and corruption risk with large civil works contracts. (v) Social and Environmental The Third Highway Project is environmental category A, land acquisition is Moderate Safeguards required and has previously been the source of delays. (vi) Sustainability Sustainability and the realization of full economic benefits require the Moderate ARS to modernizes maintenance practices. Page 56 of 58 The World Bank Third Highway Project (P118023) G. Project Assessment Tool (PAT) The PAT is a structured approach to monitoring the performance and progress of civil works contracts. Data is usually collected by the Bank’s team during site visits, often as part of the implementation support mission. The information is then presented graphically to inform both the Bank’s team and the Borrower. During the Third Highway Project, the output graphics were included in Implementation Support Mission Aide Memoires on occasions. In the above example, the brown line represents the situation during the previous mission, while the blue line shows the current situation. The sudden worsening of the contractor’s performance, particularly in the Rate of Progress quadrant, is clearly visible, thus allowing early remedies to be sought. Another PAT module enables the Bank’s team to undertake a technical review of civil works quickly and systematically. Again, a structured approach is used, this time presented in tabular form as more detailed comments are necessary (see overleaf). Page 57 of 58 The World Bank Third Highway Project (P118023) MODULE 3: TECHNICAL REVIEW Contract name: Severity (N-A, or 1 to 5 with 5 most severe) Lot: Urgency (1 to 5 with 5 most urgent) Main responsibility (contractor/supervision) Item Comments 1. Design compliance 1.1. Large adjustment to design 2 1 Sup A variation to increase bridge design loads and amend the embankment construction to take into account actual ground conditions has been approved and implemented. 1.2. Amendment observed not 1 1 approved 1.3. Large change in BOQ 2 1 Some changes in quantities arose from the variation. quantities 2. Material compliance 2.1. Earthwork 1 1 2.2. Drainage 1 1 2.3. Pavement 1 1 2.4. Bridges 1 1 2.5. Others 1 1 3. Construction compliance 3.1. Earthwork 1 1 3.2. Drainage 1 1 3.3. Pavement 2 2 Con The Engineer has notified the Contractor of some tolerance non-conformities that require rectifying during the Defects Notification Period. 3.4. Bridges 1 1 3.5. Others 1 1 4. Sustainability 4.1. Issues related to The embankment variation addressed constructability sustainability and long term settlement problems 4.2. Best practice identified Page 58 of 58