Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) Report Number: ICRR0022356 1. Project Data Project ID Project Name P130413 EAST-WEST HIGHWAY 4 Country Practice Area(Lead) Georgia Transport L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-82630,IDA-52450 28-Feb-2018 70,761,647.37 Bank Approval Date Closing Date (Actual) 09-May-2013 31-Dec-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 75,000,000.00 0.00 Revised Commitment 72,023,064.56 0.00 Actual 70,761,647.37 0.00 Prepared by Reviewed by ICR Review Coordinator Group William B. Denning Christopher David Victoria Alexeeva IEGSD (Unit 4) Nelson 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The objectives of the Project are to: (i) contribute to the gradual reduction of road transport costs and to improve road safety along the section upgraded under the Project; and (ii) to strengthen the capacity of the RD [Roads Department] and the Ministry of Regional Development and Infrastructure to plan and manage the road network and improve road traffic safety. (Loan Agreement, Schedule 1) Page 1 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components Component 1: Improvement and asset management of the East-West Highway (EWH, within the area of this project it is also designated as E60, or as the Tbilisi-Senaki-Leselidze Highway) (Estimated cost: US$74.5 million, Actual US$70.8 million) (ICR para 11). (a) Upgrade the existing EWH by constructing a 2-lane dual carriageway from Agara to Zemo Osiauri; with 12 km of concrete pavement, including two interchanges, two bridges, several overpasses and underpasses, and riverbank protection. Note that Zemo Osiauri is the highway interchange located on the outskirts of Khashuri to serve this town and that the two names appear to be used interchangeably in some parts of the ICR. (Estimated cost: US$55.0 million, Actual US$53.2 million). (b) Maintain the EWH existing 2-lane dual carriageway between Natakhtari and Ruisi (67 km) by financing a multi-year maintenance contract. The maintenance program is based on a preparatory study under Component 3a. (Estimated cost: US$8.0 million, Actual US$0.0 million). (c) Complete civil works to improve road safety and access roads on the existing EWH between Natakhtari and Ruisi (67 km) and between Ruisi and Chumateleti (44 km). The identification and design of the road safety measures and access roads are carried out under Component 3b. (Estimated cost: US$6.0 million, Actual US$5.9 million). (d) Implement environmental improvements along the EWH between Natakhtari and Ruisi (67 km). This includes addressing adverse environmental impacts of civil works in this section (Component 1c). The identification and design of the measures are based on an environmental audit in Component 3b. (Estimated cost: US$1.5 million, Actual US$0.0 million). (e) Construction supervision and quality assurance services. (Estimated cost: US$4.0 million, Actual US$3.7 million). Component 2: Institutional strengthening (Estimated cost: US$2.5 million, Actual US$0.66 million) (ICR para 11). (a) Review and update road sector strategy in two key areas: (i) institutional arrangements; and (ii) financing. (Estimated cost: US$0.5 million, Actual US$0.1 million). (b) Improve road safety management capacity of the Ministry of Regional Development and Infrastructure (MRDI). Provide technical assistance for capacity building in road safety policy issues and management and coordination, including: (i) assessment of the existing national traffic safety strategy and action plan for 2010- 2013; and (ii) development of a new strategy and plan for 2014-2017. (Estimated cost: US$0.2 million, Actual US$0.0 million). Page 2 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) (c) Improve the operating environment for the local construction industry. Promote improved professional practices within the industry through training needs assessment and manpower development. (Estimated cost: US$0.2 million, Actual US$0.0 million). (d) Improve manpower planning and development in MRDI including: (i) training needs assessment and preparation of manpower development plan; (ii) development and implementation of a training plan; and (iii) regular learning events to promote Ministry knowledge of best practices in the road sector. (Estimated cost: US$0.2 million, Actual US$0.0 million). (e) Improve organizational efficiency and manpower planning and development measures in the Roads Department (RD) in the MRDI. Provide technical assistance to: (i) improve RD’s internal business processes, and national design standards; (ii) improve multi-year planning and development; (iii) develop and implement a communication strategy; and (iv) other studies to be determined. For manpower planning and development, the activities include assess training needs and prepare a manpower development plan; implement strategic incremental training with particular focus on FIDIC Rules, and have regular learning events to promote knowledge of best practices in road management and promote their adoption. (Estimated cost: US$0.9 million, Actual US$0.4 million). (f) Prepare a strategic roadmap for the development and implementation of Intelligent Transport Systems (ITS) along the EWH corridor, from Tbilisi to the Turkish Border, including a long-term vision and a comprehensive action plan for the deployment of ITS. The study includes exploring possible roles for the private sector in the implementation of the proposed ITS, and training activities. (Estimated Cost: US$0.5 million, actual cost US$0.2 million). Component 3: Preparation of designs and supporting studies for future projects for the development of the EWH (Estimated cost: US$10.8 million, Actual US$21.65 million) (ICR para 11). (a) Study to define the scope of maintenance interventions along some upgraded sections of the EWH 2- lane dual carriageway from Natakhtari to Ruisi, Component 1b (67 km). This subcomponent includes the review and update of maintenance guidelines used by the RD. (b) Road safety improvement studies and designs to identify needed road safety interventions and access improvements along the existing road between Natakhtari and Chumateleti (111 km) under Component 1c including: (i) road safety audits along the road section; and (ii) detailed designs of the proposed measures. (c) Update feasibility studies and undertake the preliminary engineering design of a new Rikoti Tunnel and 60 km of highway between Chumateleti and west to Argveta. The study will use a 'corridor improvement approach' and identify physical measures to improve connectivity between the EWH and adjacent local road network. The studies will support subsequent phases of the EWH program, including the East-West Highway Corridor Improvement Project, which will be financed by the World Bank. (d) Detailed design of the new tunnel at Rikoti, preparation of bidding documents, and independent review of all technical and safety provisions. The design will include the design of access roads to nearby population centers. This activity was separated from Component 3c, above, to allow flexibility in the possible financing arrangements for the tunnel. Component 4: Project Management (Estimated cost: US$1.4 million, Actual US$1.7 million) (ICR para 11). Page 3 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) (a) Implementation support to RD; including provision of goods and services. (b) Financial audits and project monitoring and evaluation services provided by the Transport Reform and Rehabilitation Centre (TRRC). The TRRC is based in Tbilisi and is financed by contributions from many international development finance institutions. It has been renamed the Eurasian Transport Corridor Investment Centre. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost. At appraisal the total cost of the project was estimated at US$93.75 million (PAD, para 26). At closing, the actual total cost of the project was US$92.311 million (ICR, page 2). At closing the actual costs of components 1b, 1d, 2c, and 2d were zero. This is because they were either restructured or moved to other projects. Details are provided under restructuring below. The cost of Component 2 was estimated at appraisal as US$2.5 million and on completion had cost US$0.66 million. Component 2a (road strategy) was completed for 20% of its estimated cost. Component 2b (road safety strategy) was completed at zero cost. Components 2e and 2f had actual costs of less than half their estimated costs. Component 2e was a pool allocation for training activities that was 44% utilized. For 2f, there was a slight cost overestimation at project preparation stage because the activity is not a common one, with no similar assignments conducted in the past in Georgia, and so it was not a straight forward exercise to estimate a cost with adequate level of accuracy. The cost of Component 3 was estimated at appraisal as US$10.8 million and on completion had cost US$21.7 million. This was because while the four original sub-components completed at less than estimated cost, an additional five design and feasibility studies were included. Financing. At appraisal, the amount of Bank financing for this project was estimated as: US$38.0 million, IBRD Specific Investment Loan, and US$37.0 million, IDA Credit (PAD, para 26) for a total commitment of US$75.0 million. At completion IBRD funds disbursed were US$35.023 million, and IDA funds disbursed were US$35.739 million (ICR, page 2) for a total disbursement of US$70.762 million. Borrower Contribution. At appraisal, the Borrower was estimated to provide US$18.74 million, or of 20% of the project total (PAD, para 26). At completion, the Borrower had disbursed US$21.549 million, or of 23% of the project total (ICR, page 2). Dates. The project was approved on 09-May-2013 and became effective on 05-Aug-2013. It underwent a midterm review on 20-Apr-2016. The original closing date of 28-Feb-2018 was extended twice; to 28-Feb- 2019 and then to 31-Dec-2019. Restructuring. The project was restructured twice. Both restructurings did not change the PDO (ICR para 15 to 24). The first restructuring in December 2017, was a Level 2 restructuring to extend the closing date by one year, from 28 February 2018 to 28 February 2019; to revise the technical assistance components to take Page 4 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) advantage of a new funding opportunity and use project funds for additional studies; and to update other TA components. In 2016 the Bank obtained a Global Infrastructure Facility (GIF) Trust Fund funding (US$ 500,000) to undertake a new Advisory Services & Analytics (ASA) project, the Private Sector Participation and Sustainability Options for the East-West Highway (P161225). The activities of this ASA superseded Components 3a (study on maintenance interventions) and 1b (putting in place a maintenance contract from Natakhtari to Ruisi). These sub-components were cancelled, as the former became an activity in the ASA, while the latter would potentially become obsolete, depending on the recommendations of the ASA (ICR para 19). The savings were allocated to carry out feasibility studies and detailed engineering designs for a set of three priority road sections north and east of Tbilisi (ICR para 16 and Annex 5, Additional Studies Introduced Under Component 3). The savings also paid to install highway lighting, to improve road safety, along the Ruisi-Agara-Agara Bypass (E60, km95+km114) and the EWH (Tbilisi-Senaki-Leselidze, 115+km 121.3). Component 1d (environmental improvement measures) was dropped as a stand-alone item and was merged into the key civil works contract (Component 1c). Component 2d (undertaking a training needs assessment and preparation of manpower development plan for the MRDI) was replaced with a similar activity, which is a "Capacity Gap Analysis" study. Component 2c (Construction Industry Review Update) was cancelled by the Borrower due to: i) internal challenges with the Contractors Association, which was a key stakeholder in the activity; and ii) a study with similar objectives examining the industry was delivered under another project (Secondary & Local Roads Project-II) as a priority activity to enable introduction of Output-&-Performance-Based-Road-Contracts (ICR para 24). The second restructuring in February 2019, was a Level 2 restructuring to extend the closing date from 28 February 2019 to 31 December 2019 to accommodate the completion of all ongoing small-scale road safety improvement works, feasibility and design studies, and the capacity building activities. Two outstanding deliverables, added in the first restructuring, the Capacity Gap Analysis study and the feasibility studies and detailed engineering designs for three priority road sections north and east of Tbilisi, remained unfinished at project completion. They were transferred to the East-West Highway Corridor Improvement Project (EWHCIP, P160152) because: (i) this project had already been extended twice, (ii) all major civil works contracts had been completed by July 2017, and (iii) only these minor activities were unfinished and could feasibly be completed under the EWHCIP (ICR para 23). Overall, the changes during the project were not in conflict with the project's theory of change. 3. Relevance of Objectives Rationale Country Context Page 5 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) Georgia's economy has recovered strongly since the severe double shocks of the financial crisis and conflict of 2008. The country's economic growth rebounded strongly to 7.2 percent in 2011 and 6.4 percent in 2012, mainly due to an increase in exports and tourism and the continued high levels of public investment. The public investment of the Government of Georgia (GoG) was mainly focused on infrastructure, with emphasis on improving main road corridors and local connections. (ICR para 1). Georgia's location is a cross-roads of East-West geographic (Black Sea and Caspian Sea) and socio- economic (Europe and Central Asia) linkages, and North-South geographic (the Greater and the Lesser Caucasus Mountains) and socio-economic (Russia and Turkey) linkages. Historically and presently it is well placed to be an important hub for the transit of goods and people. Within this context the GoG embarked on the development of the East-West Highway, a transformational initiative for the Georgian economy. The Highway is part of the Caucasus Transit Corridor (CTC), which is a key transit route between Western Europe and Central Asia for transportation of oil and gas as well as dry cargo (ICR para 3). Alignment with Strategy and Rationale for Bank Involvement As indicated in the PAD (para 15), the proposed Project was in line with investment priorities identified in the 2010-2013 Country Partnership Strategy (CPS), including two strategic pillars: (a) meeting post-conflict and vulnerability needs, and (b) strengthening competitiveness for postcrisis growth; to which the Project contributes. In addition, the 2014-2017 Country Partnership Strategy (CPS) identified two strategic pillars of relevance: (i) strengthening public service delivery to promote inclusive growth; and (ii) enabling private sector led job creation through improved competitiveness. The ICR indicates, "This Project approved in 2009 is fully in line with the strategic directions, as it contributes to both strategic pillars of the 2014-2017 CPF and the 2018-2022 Focus Area, by providing infrastructure to facilitate transit and growth and strengthening the capacity and accountability of RD in road infrastructure management, as well as supporting regional integration." (ICR para 26). The current Country Partnership Framework (CPF) (FY19-FY22) was published in April 2018 and includes three Focus Areas: (1) Enhance Inclusive Growth and Competitiveness, (2) Invest in Human capital, and (3) Build Resilience. Focus Area 1 includes four objectives: (i) Support agricultural modernization and access to markets, (ii) Improve connectivity and integration, (iii) Diversify sources of finance and strengthen innovation capacity, and (iv) Increase economic participation in the regions. Clearly transportation services are a major part of Objective 2. Transport also plays a key role in agricultural and regional economic development. Sectoral and Institutional Context A policy-based, efficient, and long-term strategic investment was the main priority of the government. To lay out infrastructure investment plans, the MRDI developed several Action Plans over the years that allocated considerable resources to improvement of roads. The funding for the EWH corridor continued to be a key priority in all the Action Plans. In addition, the government placed maintenance of the road infrastructure amongst the highest priorities. To enable this planned growth in infrastructure spending, the MRDI pays close attention to building institutional capacity within the ministry and within the RD (ICR para 27). Innovation was also a priority in the FY14–FY17 CPS and has been applied to this project. The introduction of output and performance based contracts (OPRCs) in the transport sector has proven highly effective and will be expanded during the CPF period. Piloting this innovative contracting approach required a Page 6 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) comprehensive and in-depth assessment of: (i) the potential road network area, (ii) public and private sector capacity, (iii) macroeconomic factors, and (iv) the time required for adjustments during implementation. As the government already intends to generalize the use of OPRCs across Georgia starting in 2019, and it is engaging with other donors to seek their financial support, the Bank will provide technical support in evaluating existing OPRC models." (CPF para 40). Previous Sector Experience The project built extensively on five previous transport projects and drew heavily on extensive ASA (CPF page 46). The ICR (para 7) notes that the PDOs are similar to those used in the earlier East West Highway projects supporting the improvements to the E60 highway since they contribute to the achievement of the overall objective of improving the E60 East West Highway Corridor westward from Tbilisi. At closing, the Project remained highly relevant to the country’s current development priorities and Bank’s engagement. The relevance of the PDO for this project is High. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To contribute to the gradual reduction of road transport costs and to improve road safety along the section upgraded under the project. Rationale Theory of Change The ICR developed a theory of change (ICR para 5), which was consistent with the project description and results framework in the PAD. The theory of change of this objective is: Upgrading the capacity of the highway and ensuring the new and existing sections of highway are well operated and maintained will lead to reduced road transport costs and improved road safety. Good operations and maintenance will be supported by ensuring construction, maintenance, and design activities are informed by up-to-date road safety considerations. These benefits would be felt primarily along the road sections actively constructed, maintained, or designed by the project, and contribute to increasing connectivity from the Black Sea to Azerbaijan. Outputs Page 7 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) The project completed all outputs associated with PDO 1. Four Intermediate Results Indicators (IRIs) were associated with PDO 1 in the PAD (Annex 1, Intermediate Result 1, page 25). Three of these were re- classified in the ICR to be associated with PDO 2 (para 35). The original classification in the PAD more specifically and precisely matches the IRIs to the PDOs and will be used here. 1. Roads Constructed, non-rural (Component 1a, km) - baseline: 0, target: 12, achieved: 12. 2. Multi-year maintenance contract for section of E60 between Natakhtari and report Ruisi completed (Component 1b, text) - baseline: No, target: Yes, replaced by other TA and GoG funding (see below). 3. Length of East West Highway corridor from Tbilisi to Sarpi with IR Index >6 (km) - baseline: 12.4, target: 12.4, slightly not achieved: 16.2 km in ~375 km 4. Length of E60 between Natakhtari and Ruisi and between Agara and Chumateleti with road safety improvement measures (Component 1c, km) - baseline: 0, target: 100, achieved: 100. Two related Components, 1b (maintenance of the existing EWH between Natakhtari and Ruisi), and 3a (define the scope of the maintenance interventions), were cancelled as they were replaced by new TA in September 2016, the Private Sector Participation and Sustainability Options for the East-West Highway project (P161225, Advisory Services & Analytics), financed by the GIF and administered by the World Bank. The TA covered a wider scope, by examining overall operational options for the highway, including maintenance and finance. The TA was completed in December 2018 and concluded that tolling for the EWH would be a financially viable option for operating and maintaining the highway and presented options to implement such arrangements. The GoG is reviewing the options and scenarios presented by the TA. In the meantime, a three-year maintenance contract was signed in 2016 for the Natakhtari - Ruisi sections and another contract of two years period was signed in 2019, funded by the GoG (ICR para 33, and P161225 Activity Completion Summary). At the first restructuring, Component 1d (environmental improvement measures) was merged into the civil works contract of Component 1c and implemented as part of that activity (ICR para 15). The IR Index (International Roughness Index of road quality, a lower value is smoother) measure of highway condition between Tbilisi and Sarpi at the border with Turkey (approximately 375 km) is based on a geography outside of the activities in Component 1. It may serve as a proxy for overall road maintenance activity in the EWH corridor. The usefulness of measures that extend beyond the project as indicators is discussed below in Section 9 M&E Design. Outcomes Project outcomes for PDO 1 were measured by five PDO Indicators. Three achieved, one partially achieved, and one substantially exceeded its targets (ICR para 30): 1. Reduce travel time from Agara to Zemo Osiaur (minutes) - baseline: 10, target: 7, achieved: 7. 2. Reduce vehicle operating costs from Agara to Zemo Osiaur for cars (US$/km) - baseline: 0.20, target: 0.18, achieved: 0.18. Page 8 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) 3. Reduce vehicle operating costs from Agara to Zemo Osiaur for trucks (US$/km) - baseline: 0.76, target: 0.72, achieved: 0.72. 4. Reduce fatality rates along the Tbilisi-Senaki section of the EWH (per 100 mil. vehicle-km) - baseline: 11.6, target: 8.5, achieved: 3.0. 5. Increase in asset value of the EWH Tbilisi-Sarpi (GEL million), - baseline, N/A, target: 2,300, partially (70%) achieved: 1,599. The reduction in travel time, automobile operating costs, truck operating costs and the greater than expected reduction in road fatalities was the direct result of the outputs completed under Component 1a and Component 1c. This achievement was supported by the road maintenance activities that were derived from Component 1b and 3a. Like the highway condition IR Index above, the measure of reduced fatality rates between Tbilisi-Senaki (approximately 275 km) is based on a geography, including but beyond the construction of Component 1a (12 km) and even outside of Component 1c (111 km). Improved road safety in this larger geography can be attributed to the cumulative effect of road safety improvements from earlier initiatives, such as improved police capacity and increased public awareness achieved under previous Bank projects. A useful addition to the evidence for this objective would be information illustrating the improvements in these issues. The asset value indicator (#5 above) was underachieved due to the delay in: (1) completion of Samtredia- Grigoleti road segment in the western part of the country under the EIB (ICR, para 30, Annex 5), and (2) upgrading of Zemo Osiauri-Chumateleti Lot 2 section, which was terminated in December 2018 and is currently under bidding under a different project (EWHCIP) (ICR para 30). The usefulness of this “asset value”, essentially the book value of recently made investments, as an indicator is discussed under Section 9 below. Rating The Efficacy of the achievement of PDO 1 is rated Substantial. Rating Substantial OBJECTIVE 2 Objective To strengthen the capacity of the RD and MRDI to plan and manage the road network and improve road traffic safety. Rationale Theory of Change The theory of change of this objective is: Having updated strategy support (road sector strategy, road safety strategy and action plan, ITS strategic roadmap) that has been developed "in-house" with TA, strengthened Page 9 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) government institutional capacity (human resources planning and development in MRDI and RD), and local construction industry support, would improve road safety management capacity, government organizational efficiency, and overall capacity of the RD and the MRDI. The end result is a road network that has been well planned and managed, and with improved road safety. Outputs The project completed most of the outputs associated with PDO 2. Five IRIs were associated with PDO 2 in the PAD (Annex 1, Intermediate Result 2, page 26). The ICR re-classified an additional three IRIs from PDO 1 to PDO2 (ICR para 35). The original classification in the PAD more specifically and precisely matches the IRIs to the PDOs and will be used here. 1. Updated road safety strategy and action plan developed and adopted by decision MRDI (Component 2b, text) - baseline: No, target: Yes; achieved: Yes. 2. Training needs assessment and manpower development report plan for MRDI completed (Component 2d, text) - baseline: No, target: Yes. Assigned as "Gap Analysis" to a subsequent project, EWHCIP. 3. Training needs assessment and manpower development report plan for RD completed (Component 2e, text) - baseline: No, target: Yes. Assigned as "Gap Analysis" to a subsequent project, EWHCIP. 4. Chumateleti to Argveta feasibility and preliminary engineering study completed (Component 3c, text) - baseline: No, target: Yes, achieved: Yes. 5. ITS strategic roadmap for East West Highway corridor developed (Component 2f, text), - baseline: No, target: Yes, achieved: Yes. Component 2a, the review and update of the road sector strategy was completed, but without social and gender dimensions . Component 1 (Improvement and Asset Management) was primarily concerned with achieving PDO 1 but provided ongoing work for the MRDI and RD in which to put into practice the strengthened planning and management capacity desired in PDO 2. Component 2 (Institutional Strengthening) was primarily concerned with achieving PDO 2. Component 3 (Preparation of designs and supporting studies for future projects) is similar to Component 1, in that it provides ongoing work for the MRDI and RD in which to put into practice the strengthened planning and management capacity desired in PDO 2. Components 2d for MRDI, and 2e for RD, were replaced with a “Capacity Gap Analysis" study. Stage 1 of Component 2d (“Review of Organization Structure”) was completed under the project, with the remaining deliverables being transferred to the EWHCIP (ICR paras 15, 23). The Bank team and the RD agreed to slightly widen the scope of the original activity (assess training needs) to include an assessment of RD capacities in its different functions. The preparation of the ToRs took some time resulting in a late start of the activities, which was already delayed to workload pressure on RD. Page 10 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) Therefore some deliverables under the sub-component were not completed by the time of the project close (ICR para 23). The ITS strategic roadmap for East West Highway corridor was completed by 31-Jul-2017. It defines the implementation of the ITS system along the EWH. It will guide the subsequent project (EWHCIP), which has a component for financing the installation of ITS along the corridor (including sections not financed by the Bank). The RD is considering merging this strategy with aspects of tolling the highway. Outcomes Project outcomes for PDO 2 were measured by one indicator: 1. Road sector strategy incorporating social and gender dimensions developed and adopted by RD and MRDI (Component 2a, text). The road sector strategy was developed and adopted by RD and MRDI, however, the strategy did not include social and gender dimensions. See the discussion under Section 10d below. The road safety strategy and action plan was completed 30-Oct-2015. The ITS strategic roadmap was completed 31-Jul-2017. The road sector strategy was completed 05-Nov-2018 (ICR Annex 1). PDO 2, Institutional Strengthening, was achieved in many respects, i.e., technical analysis and formal documents were delivered as planned and additional technical work was also completed; and so technical capacity was improved (ICR para 34). However, "soft capacity" (Components 2c, 2d, 2e) dealing with local professional practices, government manpower planning and development, and training were transferred to the EWHCIP or deferred to later work. This is an area that continues to need attention in the future (ICR para 43). Efficacy is also measured by timeliness. The transfer of activities to the EWHCIP means that these activities are not being completed within the planned time. In April 2018 the EWHCIP was provided with EUR16.9 million additional financing (P160152) and the project closing date was extended from 31-Dec-2020 to 31-Dec-2023. The EWHCIP Supervision Report of 28-Apr-2020 mentions one, two, and, two RD staff being trained in ITS & Road Strategy in each of 2017, 2018, and 2019 respectively. Positive results from the two “outside the immediate project area” indicators from PDO 1: (1) Length of East West Highway corridor from Tbilisi to Sarpi with IR Index >6 and, (2) Increase in asset value of the EWH, add to the understanding that the project had broadly positive outcomes. Rating PDO 2 was somewhat achieved. The Efficacy of the achievement of PDO 2 is rated Modest. Rating Modest Page 11 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale At closing, the targets for outcome indicators were mostly met. Road transport costs were reduced, road safety infrastructure was improved, and government capacity to plan and manage the road network and improve road safety was strengthened. Based on the evidence of substantial and modest outcomes achieved against the two project objectives, the overall efficacy of the project’s achievements is rated substantial, with moderate shortcomings. Overall Efficacy Rating Substantial 5. Efficiency Economic Analysis The economic analysis at appraisal was based on a conventional cost-benefit analysis (CBA) applied to Component 1a of the project (constructing a 2-lane dual carriageway from Agara to Zemo Osiauri), which comprised 62% of the total project cost. The CBA was based on: (i) actual traffic volumes; (ii) forecast traffic volumes; (iii) vehicle operating cost savings, travel time savings, and reduced road accident costs; and (iv) highway maintenance cost savings. The evaluation was done using the Highway Development and Management Model (HDM-4) to estimate road agency and road user’s costs for each project alternative over an evaluation period. The traffic forecasts were based on assumed growth rates derived from recent GDP growth, recent growth in car ownership, and an assumption for generated traffic. Capital investment costs were converted to economic costs by elimination of price contingency, taxes, custom duties, with a resulting overall economic cost of about 70% of the financial costs. The CBA used a discount rate of 12% and yielded an Economic Internal Rate of Return (EIRR) of 17.5 % and a Net Present Value (NPV) of US$26.7 million (PAD para 42, 43, Annex 4). The CBA was not applied to other minor civil works in the project (Component 1c). Minor civil works related to road safety and access to trunk roads typically have good economic returns. Components 2 and 3 were TA and were not included in the analysis of economic efficiency. No emissions analyses were conducted at appraisal. The analysis for the ICR was also based on a conventional CBA, also using the HDM-4 model, and adopting the same main inputs and assumptions used at appraisal (road and vehicle fleet characteristics, evaluation period, discount rate), but now considering the actual project costs, actual duration and completion of the road works, and the actual roughness of the road measured from 2013 to 2019. The method for deriving traffic forecasts not described in the ICR. The ICR did provide actual traffic data in the initial years (2013 to 2019) which had been a forecast in the PAD. This showed traffic 26% higher than the corresponding period in the PAD (ICR Annex 4, para 3). Page 12 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) The ex-post economic analysis yielded an EIRR of 19.9% and a NPV of US$37.7 million. The higher economic benefits are due to the higher traffic measured on the project road compared with the appraisal estimates and to reduced project costs. (ICR para 38, 39, Annex 4). Additional economic context is provided in the Country Partnership Framework for Georgia for the period FY19- FY22 (April 2018). Box 2 (page 24) indicates, "The Impact of East-West Highway Corridor Investments. The Government of Georgia has prioritized infrastructure investments to improve connectivity (local and global). During the past two decades, about USD 5 billion have been committed to develop the East West Highway Corridor (EWHC) with support from several IFIs. The economy wide direct and indirect benefits of such investments in the EWHC have been found to be significant. Indirect benefits have been assessed using a computable general equilibrium model calibrated for Georgia. The transmission channels modeled are the reduction in transportation costs - reduction in vehicle operating costs and time savings - resulting from the construction and upgrading investments along the EWHC. ... The analysis revealed that indirect benefits from the EWHC investment program have an overall positive impact on key macroeconomic and welfare variables over the medium and long term. Real GDP was assessed to increase by 1.5 percent over a medium-term horizon and 4.2 percent over a long-term horizon. ... Infrastructure development contributed to growth in welfare of all categories of households, although the lowest two income quintiles would gain relatively less than other household groups. On average, rural households are to gain more than urban households." Administrative and Operational Efficiency There were delays in completion of some activities that led to an extension of the project closing date by about two years. A few analytical activities were not completed by project close and deferred to a follow-up project. The ex-post EIRR estimate of the road works (56% of the project as completed) was satisfactory (19.9%). Based on this and the above discussion, the efficiency of the project is rated Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 62.00 Appraisal  17.50  Not Applicable 56.00 ICR Estimate  19.90  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Page 13 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) Relevance of the objectives is rated high. There were moderate shortcomings in the operation’s achievement of its objectives, particularly in institutional capacity building; but only minor shortcomings in its efficiency. The overall outcome of the project is rated Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Government Commitment and Local Capacity There is little risk that commitment and local capacity will be eroded. The infrastructure built and highway management, road safety, and organizational concepts and practices advanced by the project have been well received by the GoG and its agencies. The GoG has secured financing to complete the full corridor from Tbilisi to Kobuleti (40 km from the border with Turkey), with six major contracts being currently under implementation; and has so far obtained financing for three out of five priority road projects to the east of Tbilisi (ICR para 90, Annex 5). Local institutions improved their management and practices by developing strategic policies, manpower planning, technical training, and implementing maintenance and operating practices across the whole EWH corridor. The institutional capacity measures are expected to continue to achieve their objectives, supported by continued TA activity from the Bank (ICR para 92). The Bank has worked through two TA projects, (1) East-West Highway O&M, and (2) Private Sector Participation and Sustainability Options for the East-West Highway, to assess the options available for maintaining the highway in a sustainable manner, including the engagement of the private sector. The second TA (2018) concluded that tolling would be a financially viable option for operating and maintaining the highway and presented options to implement such arrangements. The GoG is reviewing the options presented. (ICR para 33). Technology Risk The RD has developed a strategic roadmap for deployment of ITS along the EWH (31-Jul-2017) and analyzed possible private sector engagement in delivering ITS services on the highway (ICR para 45). Future activities in the EWH corridor could include: (1) the implementation of ITS, and (2) the adoption of innovative procurement of road maintenance. Both types of initiative have been advanced by this project and related Bank work. Both areas are being supported by the GoG for development and implementation. The risk that technological and operational innovation will not be supported is assessed as low. 8. Assessment of Bank Performance a. Quality-at-Entry Page 14 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) Project preparation work began in FY13, leading to appraisal later in FY13 (03-Apr-2013) and utilizing 51 weeks of staff time. The project was prepared meeting all Bank requirements, including attention to the strategic context of the project and its relevance to Government development plans. Project design benefitted from lessons learnt under the Third East-West Highway Improvement project (TEWHIP). Environmental, social, and fiduciary aspects were adequately assessed at preparation stages. Risks were properly identified and mitigation measures were adequately developed during project preparation. Extensive public consultations were held during RAP preparation (ICR para 77, PAD para 57). A Grievance Redress Mechanism had been established under previous East-West Highway projects and was maintained for this project. The implementation arrangements were well detailed, including the role of the different entities (RD, and MRDI) in the supervision of different activities. The support to improve the local construction industry relied on the engagement of the Contractors Association, yet no assessment of the Association was included in the PAD. The institutional weakness of the Association led to delays in starting this component and ultimately cancelling it. Three areas of concern at entry were raised in the ICR: (1) the choice of financing instrument, (2) the implementation capacity of the RD, and (3) the complexity of ensuring consistency between preparation activities undertaken at different times and by different donors. First, while using a sequence of Investment Project Financing (IPF) loans has produced satisfactory results, the use of a programmatic approach would have enabled the Bank to engage deeper, and over longer period, in the institutional and governance aspects in the sector. It would also offer the desired continuity in implementing the physical infrastructure (ICR para 49). Second, the total number of activities under each component challenged RD’s implementation capacity. The project included measures to improve the capacities of RD. While the project was well designed by having an integrated view of road management beyond the physical infrastructure, the list of activities introduced became too many to be implemented in a timely manner. Third, the use of one project to prepare feasibility studies and designs for future road investments by other IFIs may mean detailed project designs are prepared by one donor under their preparation framework, in advance of and in isolation from environmental impact assessments (EIAs) done by another donor under their preparation framework. The risk associated with complex arrangements materialized to some extent in this project when a stakeholder complained to the Bank about a situation that arose under another IFI. The Bank team and Georgia Country Office adequately handled the complaint advising RD on the corrective action and including the other IFI. A further area of concern relates to program and donor coordination. The risk of poor coordination was assessed as low in the PAD because there is good inter-donor and donor-Government coordination and the Bank is the only IFI assisting the government to upgrade the EWH east of the Rikoti Tunnel (PAD page 56). However, the RD workload beyond the project segment presented a challenge. Overall the project was well prepared. The discussion above highlights complexities in multi-staged project delivery with multiple donors. The Bank performance in ensuring quality at entry is rated Moderately Satisfactory. Page 15 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The project was managed by three different Bank Task Team Leaders (TTLs) over its implementation. There was a Co-TTL based in Georgia who left the unit shortly before project close but after delivering the key components. The transition between TTLs was handled effectively, with each incoming TTL first assuming the role of co-TTL prior to taking leadership of the project. Regular supervision missions and technical visits, including site visits, were conducted throughout the project implementation. Team composition reflected the project supervision needs, including transport experts, engineers, financial, and procurement and safeguards specialists, both from Washington, D.C. and from the field. Mission aide memoires and implementation status and results reports (ISRs) were well-written, reflecting progress, delays, and challenges and including ratings on the project implementation objectives and implementation progress. The Bank supervision was proactive in raising implementation flags to RD’s management (ICR para 82, 85). The procurement workload challenges of the RD described above in Quality of Preparation had their corresponding challenges during implementation (ICR para 52, 55, 56). Close support from the Bank team helped facilitate and support the RD in addressing challenges that emerged on the procurement, safeguards, and technical sides. The Bank teams exercised flexibility in adjusting and modifying activities of the project in a manner that best served the GoG in achieving its development objective (ICR para 83). RD and MRDI continued to value the Bank’s role as an advisor on the strategic aspects such as maintenance, and institutional development. Five priority activities were added to Component 3 during the project. The Bank team worked closely with RD to review and scrutinize the quality of the corresponding feasibility and design studies. This helped other IFIs to trust the quality of the studies, particularly for projects that they would be financing. On the other hand, these studies did not benefit from detailed guidance by the Bank on documents review and clearance. This led to challenges in document review, delays in providing Bank clearances, and delayed delivery of these studies. This delayed the overall timeline for implementation of the civil works of these projects, which were to be financed by other IFIs. Based on the above, the Bank performance in supervising the project is rated Moderately Satisfactory. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization Page 16 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) a. M&E Design The five PDO Indicators for PDO 1 were well specified with comprehensive measurement of road transport costs for both people and freight; and measuring the most important outcome of road safety, fatalities. Two indicators were "beyond project", see comment below. Relevant IRIs for PDO 1 were defined and provided an understanding of the achievement of the individual project components. One IRI was "beyond project". The PDO 2 indicator was based on the completion of a national road sector strategy. It was complemented by five IRIs which provided more detail on institutional strengthening, but were also based only on the completion of documents. As the ICR indicates, the activities under Components 2 and 3 were not as well represented by the IRIs and hence to PDO 2 (ICR para 57). The IRIs could have been defined with clearer, measurable expected results. This would have enabled better assessment of the value added by these components and their contribution to the sectoral development objectives (ICR para 80). The underlying issue is that completion of documents may not be the best measure of capacity building. The PDO2 indicator and the five IRIs were all not linked to a specific project geography. This is appropriate given the overall goal of national capacity building. The M&E Design includes two measures or approaches perhaps described as innovative. The first innovation, the indicator, Increase in Asset Value of the EWH corridor, is the cumulative book value of recently made investments. It is not an "asset value" that takes into account depreciation or its economic counterpart, deterioration; nor does it take into account the economic benefit value of its usefulness depending on level of demand. It is an engineering spending value that, despite its name, should not be confused with a financial nor an economic value. It does not refer to changes in the value of adjacent land arising from the transport improvement. Nevertheless it can be a useful measure, as intended, of government and IFI commitment to the corridor as a whole. The PAD, in an Attachment to Annex 1, Results Framework and Monitoring, devotes one and a half pages to describing this indicator and its calculation. The second innovation is the three measures that extend beyond the geography of the project's civil works. These are: (1) Reduced fatality rates between Tbilisi to Senaki, (2) Increase in asset value of the EWH corridor, Tbilisi to Sarpi, and (3) Length of the EWH corridor with IRI greater than 6, from Tbilisi to Sarpi. Despite being "beyond project" and therefore beyond immediate project control, these measures are important because, as the PAD describes (para 20), "it will be possible to measure success in the development and management of the E60 corridor as a whole." Since the PDO 2 is about national capacity to manage the "road network", these corridor measures are appropriate. b. M&E Implementation The M&E arrangements were clearly defined with the RD responsible for preparing progress reports. The indicators identified for assessing progress toward achieving the project outcome and PDO were suitable and easily measurable. All indicators were retained unchanged over the entire duration of the project, although some target dates were modified (ICR para 59). The RD was responsible for the collection of project’s performance indicator data and analysis of the results. A specific Monitoring Unit were established within RD. Progress towards the achievement of the final targets was monitored through regular project progress reports by RD and Bank’s ISRs. Most indicators were regularly updated without major issues. In early 2016, as the capacity of RD, including the Page 17 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) Monitoring Unit, were strengthened by filling vacant positions which in turn improved the data collection and reporting process. Hence, the project's Monitoring and Evaluation was rated Satisfactory in the last ISR (ICR para 60). c. M&E Utilization The M&E indicators were used by both RD and the Bank during supervision activities to gauge and assess performance and to prepare response measures to ensure the project was on track towards achieving its development objectives (ICR para 61). M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project triggered three safeguard policies: OP4.01 Environmental Assessment, OP4.11 Physical Cultural Resources, and OP 4.12 Involuntary Resettlement. It was classified as environmental category A due to the complexity of the expected environmental and social impacts. The Borrower carried out a full-scale Environmental Impact Assessment (EIA) of upgrading EHW between Agara and Zemo Osiauri. Environmental performance was satisfactory throughout the project life due to experience gained by the RD in undertaking previous EWHIPs as well as good corporate environmental standards of the works provider and strong implementation support by the Bank team. RD’s inhouse institutional capacity for environmental management had a few deficiencies, such as shortage of dedicated core staff; and insufficient integration of environmental and social inputs into procurement, contract management and reporting lines. However, supplementing the environmental and social team of the RD with external individual consultants and using services of a technical supervision company allowed the RD to minimize the existing gaps (ICR para 63). One environmental challenge concerned gaps in the national regulatory framework for the extraction of construction materials which meant that the RD experienced difficulty in enforcing mitigation measures beyond the scope of the national framework despite these measures being prescribed by the environmental management plan of the EIA. Social safeguards The Bank’s OP 4.12 on Involuntary Resettlement was triggered because the construction from Agara to Zemo Osiauri required land acquisition, causing both physical and economic displacement. Social safeguard compliance was mainly rated as satisfactory and social impact and risk related to land acquisition and resettlement was assessed as moderate. The Resettlement Policy Framework (RPF) was disclosed in February 2012 and the Resettlement Action Plan (RAP) in March 2013. The Project required the acquisition Page 18 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) of 110 ha (571 parcels) of which 60% was state land and 40% private land. The resettlement process was completed in February 2018 (ICR para 67). A total of 1,162 persons in 272 households were affected. Five households were resettled; four businesses stopped operation; one construction material processing plant and 20 reservoirs were relocated (within the existing area of the company); and 20 ancillary and 45 minor structures were impacted. No complaints related to land acquisition and resettlement were reported. However, some 50 complaints related to indirect negative impact caused by highway civil works were filed by the residents of the Agarebi village. These were mainly verbal complaints registered by both the RD and the contractor. The Agarebi village case took a long time to be resolved and the complaint was extensively covered and reported in the missions' Aide Memoires starting in May 2017. Several public consultation meetings were held with affected locals and local officials to agree on appropriate corrective and compensation measures to resolve the complaints (ICR para 68). b. Fiduciary Compliance Financial Management (FM) FM performance under the Project was reported to be satisfactory throughout the project implementation. All project related interim unaudited project financial statements were acceptable and submitted on time. Annual audited project financial statements were acceptable and submitted on time. The audit of project financial statements did not reveal major weaknesses and/or errors. Throughout the project implementation, the implementing agency maintained adequate financial management arrangements including staffing, project planning and budgeting, accounting, funds flow, reporting, internal controls and external audits (ICR para 73). The TRRC assisted the RD in implementation by processing disbursement, financial auditing and related activities. Procurement The overall procurement risk was moderate throughout the life of the project and procurement was completed without major issues. Procurement of goods, works, non-consulting services, and consultants was carried out in accordance with relevant Bank Guidelines and the Loan Agreement. Component 2c (Construction Industry Review Update) was cancelled by the Borrower due to: i) internal challenges with the Contractors Association, which was a key stakeholder in the activity; and ii) a study with similar objectives examining the industry was delivered under another project (Secondary & Local Roads Project-II) as a priority activity to enable introduction of Output-&-Performance-Based-Road- Contracts (ICR para 24). c. Unintended impacts (Positive or Negative) The key civil works contract involved a local contractor from Georgia (the Blacksea Group) who was engaged as a subcontractor to the key contractor (COPRI). The engagement of the local contractor (one of the largest in Georgia) presented an opportunity for the local construction industry to develop expertise and Page 19 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) experience in developing major highways. This will, in the long run, together with other factors, contribute to building the resilience and the competitiveness of the local firms in the field of Highway construction. This should assist in generating more sustainable jobs in construction and produce incremental reductions in highway construction costs (ICR para 46). d. Other Gender The project design did not have a distinct activity targeted at improving gender mainstreaming, however the updating of the road sector strategy was intended to include analysis of social and gender dimensions. The RD and the Bank agreed to remove this analysis because the sector strategy was focused at high level, institutional and maintenance arrangements and sector financing; and would not provide the most suitable entry point into the discussion of social and gender aspects. Instead this would be incorporated into other ongoing projects such as the Secondary Roads Asset Management project, The ICR does not indicate if any mention of social and gender considerations was made in the road sector strategy, such as: a high-level introduction to the scale and importance of the issues, guidance on how these issues might figure in more detailed studies or projects, or initiating data collection activities to support this analysis in the future. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Moderately Satisfactory Modest Efficacy of PDO 2. Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The lessons are drawn with some adaptation of language. 1. The overlap of design and planning from one section of road to another, and from one project to another, can speed implementation of a whole strategic road corridor but comes at a cost. Such approach under this project required an extraordinary effort, EIA coordination, multi- donor coordination, and multi-project phasing. The adoption of a programmatic approach could have deepened the level of the Bank’s engagement. This is particularly the case with governance interventions such as road safety and institutional development, where many stakeholders are involved, and interventions are required over a long period of time. The programmatic approach Page 20 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) would also be suitable for the phased approach followed in developing the highway (earlier phases include construction of specific sections, while preparing studies for the subsequent sections). 2. Capacity assessment needs to take into account sector-level, not just immediate project considerations. The risk of poor program and donor coordination causing a problem for the RD was assessed as low under this project because the Bank was the only IFI assisting the government in the road segment of this project and because there is a good inter-donor and donor-government coordination. However, the long list of activities that needed to be procured and managed in the country as a whole, and involving other donors (EIB, ADB, AIIB), led to the RD's capacity being stretched and affected the timeliness of delivering the project. 3. A wide range of components and responsible government agencies can reduce the level of diligence at preparation. For example, the readiness of the Contractors Association to support delivery was not assessed as adequately as aspects related to the larger components. The large number of activities placed pressure on RD’s teams, particularly when several new projects were being prepared. Projects should be prepared with streamlined implementation arrangements in mind and should be designed with more attention to executing agencies’ capacities, not only to implement the subject project, but also the pipeline of projects that will be prepared and implemented in parallel. The study on assessing the capacity of RD should have taken place during project implementation. 4. In M&E design, the outcome indicator “increase in asset value of a road corridor” may not convey meaningful information. It is the cumulative book value of recently made investments. It is not a financial "asset value" nor a measure of economic benefit. It does not measure the change in the value of adjacent land arising from the transport improvement. If it is to be useful as a measure of government and IFI commitment to the corridor as a whole, it should be re-defined in more formal financial terms or economic terms, or as the economic benefit of higher land value. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR was well written; it provided a good summary of the project and its implementation. It was difficult to get a good understanding of the geography of the project. At the overall country level the concept of the east-west highway was clear and a map of the country was provided. However the geography of specific road sections, interchanges, intersections, bypasses; either under construction, being maintained, or being studied was difficult to sort out. There seemed to be inconsistent naming, or perhaps two different names for locations that were essentially the same place. More than half the place names or locations used in the report were not identified on the map. An expanded local map, or diagram with all place names, to show the project geography would have been helpful. Page 21 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EAST-WEST HIGHWAY 4 (P130413) Estimated and actual costs should have been provided for the individual sub-components of Component 3. There were various changes and additions to the advisory work and studies undertaken as part of the project. It would have been useful if Annex 5, Supporting Information, could have provided a complete list of all studies, reports, assessments, outputs prepared and the estimated and actual costs of each. In Efficiency, the ICR did not discuss aspects of design and implementation that either contributed to or reduced efficiency, including delays in implementation of key activities, procurement issues, cost overruns, etc. The lessons were presented in the form of findings specific to the project, rather than for general application to similar operations in the sector. a. Quality of ICR Rating Substantial Page 22 of 22