Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) Report Number : ICRR0020619 1. Project Data Project ID Project Name P112523 EW HIGHWAY IMP 3 Country Practice Area(Lead) Additional Financing Georgia Transport & ICT P128863 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-77410,IDA-51080 30-Jun-2013 184,120,000.00 Bank Approval Date Closing Date (Actual) 10-Sep-2009 30-Jun-2016 IBRD/IDA (USD) Grants (USD) Original Commitment 147,000,000.00 0.00 Revised Commitment 190,000,000.00 0.00 Actual 182,501,654.70 0.00 Prepared by Reviewed by ICR Review Coordinator Group Kavita Mathur John R. Eriksson Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives The project development objectives were: (a) to contribute to the gradual reduction of road transport costs and improve access, ease of transit, and road safety along the central part of Georgia’s East-West corridor; and (b) strengthen the capacity of the relevant government entities to plan and manage the road network and to improve traffic safety (Loan Agreement page 6 and Project Appraisal Document page 6). Revised PDO: Only a minor change in the text of the second sub-objective was made on June 26, 2012. The sub-objective was changed to "strengthen the capacity of the relevant government entities to plan and manage the road network and to improve road safety." Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components The project consisted of three components: (a) civil works, (b) institutional strengthening, and (c) project management and implementation. Component A: Sveneti-Ruisi Road Link (15km) (appraisal cost US$180.8 million including contingencies; actual cost US$220.3 million). Upgrading the existing E60 East-West Highway from Sveneti to Ruisi to a dual carriageway road including: (i) partial rehabilitation of the existing road; (ii) realignment and construction of new carriageways; (iii) surface drainage, culverts and underpasses; (iv) bridges, including road approaches, particularly a bridge of about 877 meters long over the Liakhvi River and an existing railway line; (v) two tunnels of about 800 meters long; (vi) safety barriers, road signs and marking; (vii) demolition of existing structures and relocation of various utilities; (viii) construction supervision services; and (ix) design services for future investments in the road sector. Component B: Institutional Strengthening (appraisal cost US$2.7 million including contingencies; actual cost US$10.28 million). The specific activities to be undertaken were: (a) Strengthening the capacity of the roads department to improve its operational effectiveness through: (i) carrying out a functional analysis for establishing the appropriate organizational structure for the roads department to meet its current and anticipated future needs; (ii) improving the capacity of the roads department to plan, design, manage and maintain the road network; and (iii) strengthening the roads department capacity in environmental monitoring, through provision of goods, consultants’ services and training. (b) Developing a framework for the introduction of performance based contracts (PBC) for main roads’ maintenance, through provision of consultants’ services. (c) Improving road safety along the entire E60 East-West Highway corridor by preparing and implementing a corridor road safety management plan to cover engineering, enforcement, emergency response and publicity campaigns, through the provision of goods, consultants’ services, and training. (d) Strengthening the curriculum and training at the technical university including provision of training for lecturers, twinning with overseas universities, as well as provision of consultants’ services and goods. Component C. Project Implementation (US$0.5 million; actual cost US$1.24 million). Institutional support to the roads department and the Transport Reform and Rehabilitation Center (TRRC) related to project Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) implementation, audits and project monitoring and evaluation. Revised Components The Additional Financing (AF) would scale-up the project by financing the following activities (restructuring paper page 5): Component A, Ruisi-Agara Road Link (US$51.1 million). Upgrading of the Ruisi-Agara road section and the construction of the Agara bypass (19 km in total) to a dual carriageway standard, and consultancy services for a technical quality review (US$51.1 million equivalent. Component B. Institutional Strengthening (US$1.16 million) •Employment by the roads department of a Road Safety Adviser, and financing of the purchase/development of a road crash database (US$0.50 million equivalent). • Additional staff for the Foreign Project Unit (FPU) comprising a Contract Manager, an Environmental Specialist, and a Monitoring and Reporting Specialist consultants; and procurement of vehicles and office equipment (US$0.66 million equivalent). Component C. Project Management (US$0.42 million). • Transport Reform and Rehabilitation Centre (TRRC) salaries and operating costs, and additional project financial audits. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Costs: The actual project cost (including the Additional Financing) is US$232 million compared to the appraisal estimate of U$184 million. Financing: The original loan amount was US$147 million. The project received Additional Financing credit for the amount US$43 million equivalent to scale up some of the activities. The final loan amount (after restructurings) is US$187.8 million equivalent (including IBRD loan of US$ 147 million equivalent and IDA credit of XDR 27.7 million, i.e. US$ 40.8 million equivalent as of December 5, 2016). The IBRD loan was fully disbursed. At project close, the total disbursed amount of the IDA credit was XDR 23.8 million, i.e. US$ 35.5 million equivalent. The ICR reports (para 51) that US$ 5.3 million equivalent will have to be cancelled as these funds were not committed. Borrower Contribution: The original Borrower contribution was US$37.12 million. At the time of Additional Financing, the Government committed additional US$10.75 million – making the total Borrower commitment US$47.87 million. The actual Borrower contribution is US$49.7 million. Dates: The project was granted an extension of 2 years on June 26, 2012 to complete the civil works Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) activities of the parent project and scale up the project scope to include Ruisi-Agara link and Agara bypass. On February 25, 2015 the project was further extended by twelve months to complete the outstanding civil works and three remaining road safety activities. In total, the project was extended by three years and closing date was extended from June 30, 2013 to June 30, 2016. The project was restructured twice, first on June 26, 2012 to complete the civil works activities of the parent project and scale up the project scope to include the following road section (Ruisi-Agara bypass road link). The second restructuring (June 25, 2015) was to complete the outstanding civil works and three remaining road safety activities. 3. Relevance of Objectives & Design a. Relevance of Objectives At appraisal, the project objectives were substantially relevant to the second strategic objective of the Country Partnership Strategy (CPS Report No. 48918-GE FY 10-13) which aimed at strengthening competitiveness for post-crisis growth. Reduction in transport time and costs along the key transit corridor and improving road safety were the key outcomes associated with the results area 3 - upgrade the transport corridor and increase connectivity (CPS page 22). At project closing, the project objectives remained substantially relevant to the Bank’s strategy for Georgia. The current CPS (Report No. 85251-GE FY 14-17) identifies two strategic pillars: (i) enabling private sector led job creation through improved competitiveness; and (ii) strengthening public service delivery to promote inclusive growth. The project objectives are aligned with both pillars as they contribute by providing infrastructure to facilitate transit and growth, and strengthening the capacity and accountability of Roads Department in road infrastructure management. The project objectives remained substantially relevant to the country’s current development priorities. Georgia is strengthening connectivity to the global economy by developing the strategic East-West Highway corridor between the Black Sea ports and the Caspian Sea. This project would allow Georgia to lower transit costs and become more competitive as an international transport route. Rating Substantial b. Relevance of Design This project was the third East - West Highway Improvement Project in Georgia. The first East-West Highway Improvement Project was in 2006 (US$19 million) which also introduced a road safety program. In 2007 the Bank financed the US$35 million Second East-West Highway Improvement Project and the Additional Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) Financing of US$20 million in 2008. There were other donors including European Investment Bank (EIB), the Asian Development Bank (ADB), and the Japan International Cooperation Agency (JICA), financing separate road projects aiming at upgrading the E60 East-West Highway in Georgia. The design included a clear and focused statement of objective. There was a logical causal link between the project activities, their outputs and the intended outcomes. Component A - the upgrading of the Sveneti-Ruisi highway section and the Ruisi-Agara bypass highway section (under AF) would contribute to the achievement of the first sub-objective and its constituent elements - the reduction in road transport costs, improved access, ease of transit and road safety. However, to ensure local connectivity (access), the project could have included construction of local roads in the vicinity of the highway. Institutional strengthening activities under component B would support the second sub-objective by strengthening the capacity of the Roads Department and relevant government entities to plan and manage the road network and to improve traffic safety. In order to minimize the risk of tension among communities directly or indirectly impacted by the project, the preparation indeed involved some formal consultations and social preparatory work. Rating Substantial 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Contribute to the gradual reduction of road transport costs and improve access, ease of transit, and road safety along the central part of Georgia’s East-West corridor. Rationale Outputs The project, in total upgraded 34 km of which 15 km upgraded for the Sveneti-Ruisi section and 19 km upgraded or built for the Ruisi-Agara section. The works included: (i) Partial rehabilitation of the existing road; (ii) Realignment and construction of new carriageways; (iii) Construction of surface drainage, culverts and underpasses; (iv) Construction of bridges, including approach roads, particularly a bridge of about 877 meters long over the Liakhvi River and an existing railway line; (v) Construction of two tunnels of about 800 meters long; (vi) Installation of safety barriers, road signs and markings (Annex 2). Outcomes Reduction of transport costs: The project investments reduced transportation costs on the project corridor: • Vehicle operating cost (cars) from Sveneti to Ruisi was reduced from 0.2 US$/km (baseline) to 0.17 Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) (target was 0.18). • Vehicle operating cost (trucks) from Sveneti to Ruisi was reduced from 0.76 US$/km (baseline) to 0.71 (target was 0.0.72). • Vehicle operating cost (cars) from Sveneti to Agara Bypass was reduced from 0.2 US$/km (baseline) to 0.18 (target was 0.18). • Vehicle operating cost (trucks) from Sveneti to Agara Bypass was reduced from 0.76 US$/km (baseline) to 0.72 (target was 0.72). Improved access: No information provided in the ICR. Improved ease of transit: The project investments improved the “ease of transit’ by reducing travel time. All targets but one related to transit time and vehicle operating costs are achieved: • Travel time from Sveneti to Ruisi decreased from 12 minutes (baseline) to 8.1 minutes, exceeding the target of 9 minutes. • Travel time from Sveneti to Agara bypass through Ruisi was reduced from 27 minutes to 21 minutes (target was 20 minutes). The target was not completely achieved as local traffic and low speed vehicles were using the highway prior to the opening of all local parallel roads. Furthermore, the speed limit on the Ruisi-Agara section was reduced from 110 km/h to 90km/h as low-speed vehicles were using the highway prior to the opening of local parallel roads. . Improved road safety: The implementation of road safety improvement works along the completed sections of the highway contributed to a reduction in road accident fatality. The road fatality rate per 10,000 vehicles on E60 corridor from Senaki to Tbilisi was reduced by 43%, exceeding the final target of 30%. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Strengthen the capacity of the relevant government entities to plan and manage the road network and to improve traffic safety. At restructuring “improving traffic safety was changed to “improving road safety”. No split rating will be carried out as the original and revised PDOs used for the AF were very similar. Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) Rationale Outputs • The Roads Department developed a new framework for the “Design and Implementation of Concessions” (contracts) for the E60 Performance-Based Road Maintenance. This included contract formats, quality assurance and partnering arrangements (ICR para 63). • The staff from the Roads Department (RD) received training and attended seminars on topics such as procurement of goods and works, financial management, contract management and disbursement monitoring. Senior environmental specialists attended overseas trainings on the Bank’s safeguard policies (ICR para 64). Functional analysis to establish the appropriate organizational structure for RD was not carried out as the government was planning a larger institutional reform at the ministry level (ICR para 64). The project team informed IEG that this was done by the ministry. Outputs for improving capacity for road safety. • Engineering improvements to improve road safety were completed. • Road safety audit manual was prepared. • Road safety audits were done for the entire Senaki to Tbilisi corridor. • A road crash database was developed for the entire country and was piloted in three regions as well as the capital Tbilisi. The new database has more details required to determine the cause of a crash and develop adequate mitigation measures. However, since the database was piloted in limited areas and could not be expanded to other areas due to the lack of equipment for all police officers, the police officers are continuing the old method of paper-based data collection (ICR para 68). • A road safety awareness campaign along the highway was completed. The campaigns material (brochures, videos) was made available on RD’s website. RD is also planning to distribute brochures/booklets to targeted audiences (at bus stops, gas stations, etc.). • A study tour to Finland to learn about road crash statistics analysis and road safety best practices was conducted. • The corridor road safety management plan was not prepared. • The strengthening of the curriculum and training at the Georgian Technical University (GTU) was not achieved due to a lack of timely collaboration between RD and GTU. Outcomes The project supported a couple of innovative technologies for slope stabilizing and greening, which enabled the exposure of RD to new environmental management practices in road projects, and built the necessary capacity in the country during implementation. Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) The Roads Department is planning to pilot performance based road maintenance on a secondary road network under the Bank financed “Second Secondary and Local Roads Project”. The learning from this pilot will help RD to better manage and maintain the road network and use this on the entire road network (ICR para 63). Significant accomplishment under road safety was that the road safety audits were institutionalized by the RD. Rating Substantial PHREVDELTBL PHREVISEDTBL 5. Efficiency Economic Efficiency: At appraisal, a cost-benefit analysis was conducted for Sveneti-Ruisi road section. The analysis was based on actual traffic volumes and forecasts, vehicle operating costs (VOC) and travel time- savings for users, and benefits derived from: (i) avoidance of traffic slowdown in adjacent sections of the road as traffic traveling on a four-lane highway has to enter a narrower than two-lane road; and (ii) reduction in accidents in a segment identified as one of the worst safety black spots on the network. An ex-post economic analysis was done using the appraisal methodology. The upgrading of Sveneti to Ruisi section (original project) yielded an Economic Rate of Return (ERR) of 20.9 percent, which is higher than the ex-ante estimate of 16.3 percent. The ex-post ERR for Ruisi to Agara section (AF) is 24.4 percent compared to the ex-ante estimate of 21.6 percent. Overall, the ex-post ERR is 21.7 percent, slightly higher than the ex-ante estimate of 17.5 percent. This is due to lower actual construction costs and higher actual traffic growth rates. Operational and Administrative Efficiency: There were no cost overruns and the original project components were implemented at or below costs and within the project period. The components that were added at restructuring were completed with a one-year extension of the closing date. There were some procurement delays due to staff departures at the Roads Department. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  17.50 Not Applicable 0 ICR Estimate  21.70 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of objectives and design is rated substantial. Efficacy of each of the two sub-objectives and efficiency are rated substantial. The project outcome is satisfactory. a. Outcome Rating Satisfactory 7. Rationale for Risk to Development Outcome Rating The risk that the development outcomes will not be sustained is modest as the project contributed to improvement in roads management and maintenance practice. The project supported the development of a framework to institutionalize performance based contracts for road maintenance in Georgia and the first OPRC is being piloted in Kakheti under the Second Secondary Roads Project. Also, ongoing highway projects are supported the development of relevant strategies to further enhance operations and management such as road sector financing strategy, updating the National Road Safety Strategy, and strengthening Roads Department capacity through targeted training (ICR para 75). a. Risk to Development Outcome Rating Modest 8. Assessment of Bank Performance a. Quality-at-Entry The Bank has been engaged in improving Georgia’s major East West transit corridor - the E60 highway since 2006 when the first East West Highway Improvement Project was approved. A year later the second project was approved. The lessons from these two projects were reflected in the project design. These Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) included: funding for the preparation of feasibility studies and detailed designs; hiring of a “technical auditing team” for supervision; preparation and implementation of road safety plan; upfront clarification of the roles and responsibilities of the Roads Department and Transport Reform and Rehabilitation Center (PAD page 8). During project preparation, the capacity of the Implementing Agency was considered adequate with the staff having relevant experience (ICR para 26). Key stakeholders including the general public were consulted through participatory discussions (ICR para 28). A number of risks such as sector specific risk, technical design risk, implementation capacity risk; and procurement risk were identified at appraisal and the overall implementation risk was assessed as significant. The PAD included appropriate mitigation measures and based on these measures the overall risk rating was assessed as moderate (PAD pages 12-14). The relevant safeguards policies were triggered. M&E design had a few weaknesses (see section 10a below). Overall, the components design was moderately satisfactory for the following reasons: (i) the project did not include a mechanism to ensure the collaboration between multiple stakeholders (Police, Ministry of Internal Affairs, emergency services, Georgian Technical University, etc.) responsible for road safety activities; (ii) the engineering designs were not systematically complete at the end of the project preparation phase; and (iii) lack of focus on local connectivity through the provision of access roads for local residents in the vicinity of the Ruisi-Agara highway. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Supervision missions and technical visits (including site visits) were conducted regularly. On an average two supervision missions were conducted per year. The mid-term review was carried out in November- December 2015. The missions were staffed with professionals such as transport experts, engineers, financial experts, and procurement and safeguards specialists (para 79). Implementation Status and Results Report were candid and reported delays and challenges faced by the project. The Environmental Assessment Safeguards was rated "moderately unsatisfactory" in June 28, 2014 ISR. The rating was upgraded to "moderately satisfactory" in December 8, 2014 ISR.Supervision was pro-active and the Additional Financing decision was made on time. The Bank team mobilized relevant internal or external expertise when necessary. The Bank closely worked with the government and the implementing agency to resolve the contractual issue when the contractor discontinued implementation for six months. In addition, the Bank facilitated training of the project manager in International Federation of Consulting Engineers (FIDIC) standards (ICR para 81). The ICR reports (para 80) that the project was a major scale-up from the previous project, which were in the range of 30-50 million USD, while this Project was closer to 200 million due to the larger size of the civil works. Moreover, this project comprised major structures work with two 800 meter long tunnels and several bridges, sometimes using new technologies (such as an 1100 m pushed bridge, which was the first in the Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) region of this type). This required constant attention from the supervision team. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. Assessment of Borrower Performance a. Government Performance During preparation, the government was committed to the project, as reflected in counterpart financing of 20% of the total project cost. The Government also prepared the detailed design and launched the tender for original works components of the road prior to project appraisal (ICR para 28). During implementation, the government provided counterpart funds on time. However, the government was not able to quickly appointment the Chairman of the Roads Department (the position remained vacant for nine months) and replacement of procurement specialist. Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance The Roads Department (RD) was the implementing agency. It faced implementation issues during project implementation: • Three civil works contracts on the Sveneti – Ruisi section experienced delays due to poor quality of construction, supervision issues (late mobilization of the supervision consultant) and initial poor compliance with environmental guidelines/Environmental Management Plan (EMP) requirements (ICR para 33). The compliance with the EMP improved and was eventually moderately satisfactory (ICR para 33 and 83). The Environmental Assessment Safeguards was rated "moderately unsatisfactory" in June 28, 2014 ISR. The rating was upgraded to "moderately satisfactory" in December 8, 2014 ISR. •On the Ruisi – Agara bypass section, initially the implementation was slow as the contractors unilaterally decided to stop the works for several weeks halfway through implementation. RD did not undertake effective monitoring and did not apply effective sanctions towards the contractor during the period when construction was stopped; (ICR para 83). • A large sub-section of the 19 km Ruisi – Agara section had to be redesigned delaying the commencement of the works for that section. Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) Despite these challenges, the RDs technical and project management capacity steadily improved during implementation. In the end, the overall technical quality of civil works completed under the project was considered good (ICR para 34). The Foreign Projects Unit (FPU) within the Roads Department was responsible for the collection of project’s performance indicator data and analysis of the results. FPU also suffered from staff departures in 2015. In 2016, vacant position were filled and FPU capacity was strengthened. This improved FPUs data collection and reporting. Implementing Agency Performance Rating Moderately Satisfactory Overall Borrower Performance Rating Moderately Satisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design The project included four outcome indicators: (i) Transit time from Sveneti to Ruisi (minutes); (ii) Vehicle operating cost from Sveneti to Ruisi (cars in US$/km); (iii) Vehicle operating cost from Sveneti to Ruisi (trucks in US$/km); and (iv) Percentage reduction in road fatalities on E60 East-West Highway from Senaki to Tbilisi. Targets were set at appraisal. These indicators were appropriate in measuring the three parts of the first PDO, namely the reduction in road transport costs improvement in ease of transit and road safety. However, there were no indicators to measure improved access. Regarding the second PDO, there were no indicators to measure the strengthening of capacity of relevant government entities in planning and managing the road network. During the first restructuring, the results framework was revised to take into account the upgrading of the Ruisi-Agara road section and the construction of the Agara Bypass. b. M&E Implementation The Monitoring Unit within the Foreign Project Unit (FPU) of the Roads Department (RD) was responsible for the collection of project performance indicator data and analysis of the results. The FPU and its Monitoring Unit suffered from staff departures in 2015. However, the vacant positions were filled quickly. Progress reports were submitted to the Bank and by RD regularly and most indicators were regularly updated without major issues (ICR para 38). c. M&E Utilization There is no information in the ICR on M&E utilization. Owing to satisfactory design and regular monitoring of the outcome indicators for the first PDO, M&E is rated Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) substantial. M&E Quality Rating Substantial 11. Other Issues a. Safeguards The Project was classified as Category A as it financed the widening of the existing carriageway from a two- lane into four-lane and the construction of a four-lane section in a new alignment, construction of bridges and construction of a new double-tube tunnel all having significant, complex and irreversible environmental impacts. Following safeguards policies were triggered: Environmental Assessment (OP/BP 4.01), Physical Cultural Resources (OP/BP 4.11), and Involuntary Resettlement (OP/BP 4.12). There was no change to the environmental category under the AF - and no new safeguard policies were triggered. Environmental Safeguards Environmental Assessment (OP/BP 4.01). A draft Environmental Impact Assessment (EIA) was prepared for the Sveneti-Ruisi section and a draft Environmental Management Plan (EMP) prepared, and both were disclosed at the Infoshop and in-country. The final EIA/EMP was disclosed prior to negotiations (PAD pare 68). The Environmental and Social Impact Assessments (ESIAs) report for works between Ruisi and Gomi covering the AF was disclosed at the Infoshop and in-country on February 2012. In 2010, a supplemental ESIA was carried out to cover additional works required for the construction of a two-lane bypass road and a bridge over Liakhvi River (ICR para 41). The improvements of highway sections under the project and AF were also covered by the Regional Environmental Assessment and the Environmental Management Framework developed in 2009 for a larger section of East West Highway between Sveneti and Rikoti Tunnel (ICR para 41). During implementation, weaknesses in country systems were detected: (a) weak national regulations for materials; (b) issuance of licenses for extraction by the national authority without preliminary environmental assessment: and (c) lack of specificity in environmental requirements for license holders. The ICR reports (para 42) that the weakness of the country systems was partially offset by mandatory application of safeguard documents prepared during appraisal of both the parent project and the AF. The main environmental issue related to works in the waterway i.e. the construction of bridges over Liakhvi River implied diversion of the stream and installation of bridge piles in the river bed (ICR para 42). The ICR reports that mitigation measures were not applied - cleaning of the river bed from piles of excess material and construction waste was delayed for months (ICR para 42). The ICR reports (para 42) that environmental performance was once rated moderately unsatisfactory (in Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) 2009), and thereafter rated either moderately satisfactory or satisfactory. Social Safeguards Physical Cultural Resources (OP/BP 4.11). The PAD indicated (para 74) that although there is no apriori evidence of archaeological sites on the proposed alignment, the triggering of physical and cultural resources safeguard was considered to be prudent because of archeological finds under the first and second East West Highway Improvement Projects. The EMP included chance find procedures outlining the actions to be taken in the event archaeological relics were found.The ICR reports (para 42) that chance finds were encountered at one of the allocated sites. The find of a major graveyard at the beginning of the AF was treated thoroughly. Involuntary Resettlement (OP/BP 4.12). At appraisal, the Resettlement Policy Framework (RPF) was prepared and disclosed. This was updated for AF and disclosed on February 7, 2012. A Resettlement Action Plan (RAP) for the Ruisi-Agara bypass road section was prepared and disclosed in country on February 9, 2012 and at the InfoShop on February 21, 2012. The ICR reports (para 45) that in total 2393 people (617 households) were affected and 736 land plots (land area of 1.318.448m²) had been acquired. By project closing (June 30, 2016), resettlement activities had not completed as a paved road was completed in November 2016 and a gravel one is expected to be completed in April 2017. The ICR reports (para 46) that the compliance with the Involuntary Resettlement safeguards was mainly satisfactory; however, the compliance was downgraded to moderately satisfactory towards the end of the project due inadequate capacity to manage social safeguards, delays in updating of the RAP (at the time of AF), and lack of reporting and inconsistencies in reported data. b. Fiduciary Compliance Procurement: The ICR reports (paras 47 to 48) that procurement was carried out in accordance with the Bank’s procurement guidelines. Most contracts were subject to the Bank’s prior review and procurement of civil works was carried out without major issues. The project supported the improvement of procurement arrangements and training activities for the Roads Department (RD). In 2015, the procurement capacity was negatively affected due to staff departures in the Roads Department (RD). The RD was unable to attract/maintain qualified staff essential to improve and streamline procurement and the contract management process. Financial Management (FM): The ICR reports (para 50) that the FM rating was satisfactory. TRRC was adequately staffed and the audit reports were timely and acceptable to the Bank. The Roads Department was in compliance with the audit covenant. Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) c. Unintended impacts (Positive or Negative) To improve the safety of local farm-produce sellers along the roadbed, the project facilitated consultation between the Roads Department and the Gori municipal government to construct proper and safe markets at the Gori intersection of the East-West Highway. The construction of two markets on both sides of the Gori intersection were completed (ICR para 74). d. Other --- 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory --- Risk to Development Modest Modest --- Outcome Moderately Bank Performance Moderately Satisfactory --- Satisfactory Moderately Borrower Performance Moderately Satisfactory --- Satisfactory Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons The ICR (pages 22-23) identified a number of lesson of which three have broad applicability: • To prepare good engineering design, there is a need to ensure that qualified engineers (international or local) are hired by the Implementing Agency during the preparation of road designs. These engineers should supervise the preparation and/or review the draft designs. • There is a need to re-assess risks and put in place adequate mitigation measures to address project safeguards management capacity issues. The project experience showed that land acquisition and Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review EW HIGHWAY IMP 3 (P112523) resettlement procedures took longer than expected. Therefore, the safeguards documents need to be periodically revised/updated, approved, disclosed and correctly implemented. • Since the implementation of road safety requires a number of stakeholders, a clear framework for multi- sectoral coordination needs to be established during project preparation. 14. Assessment Recommended? No 15. Comments on Quality of ICR The ICR provides a comprehensive overview of project preparation and implementation. Its evidence is robust, drawing from a variety of sources of data. It contains all necessary data and analysis. It is results oriented, is internally consistent, and provides lessons that are based on evidence and analysis. a. Quality of ICR Rating Substantial Page 16 of 16