ICRR 12435 Report Number : ICRR12435 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 08/14/2006 PROJ ID :P008593 Appraisal Actual Project Name :Roads 2 Project Project Costs 540 473.4 US$M ) (US$M) Country :Poland Loan /Credit (US$M) Loan/ US$M ) 300 300 Sector (s):Board: ): TR - Roads and US$M ) Cofinancing (US$M) 0 0 highways (96%), Central government administration (4%) L/C Number :L4236 FY ) Board Approval (FY) 98 Partners involved : Closing Date 06/30/2003 12/31/2006 Evaluator : Panel Reviewer : Division Manager : Division : Kavita Mathur Roy Gilbert Alain A. Barbu IEGSG 2. Project Objectives and Components a. Objectives The main objective of the project was to support and speed up the modernization of the road sector along the lines of the environmental and traffic safety policy initiated by the Government under the Roads Project (Loan 3564-POL). Specifically, the project would: (i) improve the priority national road network sections with traffic volumes of at least 6,000 vehicle/day; (ii) reconstruct and repair high priority flood-damaged roads and bridges; (iii) study and solve the rutting situation on the network; (iv) promote the competitive capacity of domestic contractors through improvement of their technical, managerial, and financial capacity; (v) improve the road safety situation; (vi) adjust road user charges for different vehicle types to cover the economic costs incurred by each type of vehicle, and to achieve a well balanced combined transport framework; and (vii) advise on the operation, management, and administration of road safety and its coordination with the National Police. b. Components (or Key Conditions in the case of Adjustment Loans ): The main components of the project were: A. Civil Works (estimated cost US$440.4 million, actual cost US$447.3 million) - rehabilitation, upgrading, realignment and resurfacing of priority roads identified under the Project Implementation Plan (PIP); reconstruction, upgrading of bridges and construction of bypasses identified under the PIP; and emergency reconstruction and repair works of roads and bridges damaged as a result of flooding in the year 1997. B. Goods (estimated cost US$25.5 million, actual cost US$15.9 million) - provision of equipment, materials and vehicles for road maintenance and road safety. C. Technical Assistance (TA) and Training (estimated cost US$1.7 million, actual cost US$10.2 million) - provision of TA and training to the Ministry of Transport and Maritime Economy (MTME), General Directorate of Public Roads (GDDP), and Regional Administration of GDDP (DODPs) for (a) road maintenance; (b) road construction and design methods; (c) road safety program; and (d) modernization of selected road construction and road engineering firms. Under this category, the project was also to finance the Road User Charges Study. Based on a formal request from the Ministry of Infrastructure (MOI), the Legal Agreement was amended in December 2004 and a new expenditure category “Provision of technical assistance to the Railway Sector Reform� was created to carry out three specific studies for the Polish National Railways (PKP). US$2.8 million was allocated from project savings to finance these studies. c. Comments on Project Cost, Financing, Borrower Contribution, and Dates The project's closing date was extended twice. First, it was extended by 18 months due to: (a) the introduction of administrative reform in 1998, which caused considerable delays in the processing of contracts (on average many contracts were finalized 12-18 months later than originally estimated at appraisal); and (b) because of the delays in local authority approvals, land acquisition, design documentation by local consultants, and in the preparation and presentation of Environmental Impact Assessments (EIAs) for the new bypass and bridge contracts. Second, it was extended for two more years to complete the studies for the reform of the railway sector. The ICR notes that the local costs are presented as net and are exclusive of land acquisition costs, design and documentation costs, supervision costs and VAT. If these were included, the overall actual cost of the project would exceed the original estimate of US$540 million. It is not clear from the ICR if the appraisal costs had all the same elements included as the ICR costs. The loan was fully disbursed. 3. Relevance of Objectives & Design : The project was consistent with the Bank's Country Assistance Strategy (CAS - April 14,1997) for Poland, which focused on the removal of infrastructure bottlenecks, on the creation of better conditions for private sector participation in infrastructure, and on infrastructure modernization, all part of the Borrower's strategy for Poland's successful integration in the European Union (EU). The project reflected the government's priority for the roads sector and was based on GDDP's 1996-2000 Road Investment Program. The project as designed was expected to continue to pursue the policy initiatives of the first roads project. However, the project objectives were not clearly defined and performance indicators for evaluation of impacts or measuring outcomes were not developed at appraisal. Some of the project objectives, for example "reconstruct and repair high priority flood-damaged roads and bridges" are really project components. The project design was relevant as it not only included rehabilitation and upgrading but reconstruction of critical flood damaged roads and road safety initiatives. The risk factors were adequately identified and adequate measures were included in the project to reduce these risks. The project's quality at entry was rated satisfactory by the Quality Assurance Group (QAG). QAG noted that the project design was sound not only for technical aspects, but also concerning institution building and road safety. 4. Achievement of Objectives (Efficacy) : Improve the priority national road network ( Substantially Achieved). Physical targets for rehabilitation, upgrading, realignment, and resurfacing of priority road sections were met, 535 km of road works were carried out under the project compared to appraisal estimate of 522 km. Appraisal targets for reconstruction and upgrading of two bridges were met. Targets were exceeded for construction of new major bridges and bypasses. Four bridges were constructed compared to three estimated at appraisal and 13 bypasses were completed compared to the appraisal target of 12. The project contributed to the improvement of the main national road network. The condition of network in good condition improved from 24% in 1997 to 46% in 2004. Since no targets were set at appraisal, comparisons between actuals and appraisal targets cannot be made. Reconstruct and repair high priority flood-damaged roads and bridges ( Substantially Achieved). Reconstruction of 392 km of flood damaged roads and bridges were completed against the appraisal target of 300 km. Flood preventive measures such as construction of retaining walls, flood prevention bunds, raising of alignments, and improvement of drainage systems were also completed. Flood alert early warning systems were also put in place. Solve the rutting situation on the network (Modestly Achieved). The capacity of GDDKiA (General Directorate of National Roads and Motorway, former GDDP) to plan its annual rehabilitation was strengthened through the adoption of the computerized Highway Design and Maintenance Standards Model-4 (HDM-4). The model is only partly operational at both the national and regional levels. Promote the competitive capacity of domestic contractors (Modestly Achieved, given insufficient evidence to the contrary ). All contracts for reconstruction and repairing of flood damaged roads and bridges were carried out by Polish contractors. For non-emergency civil works, 47 out of 70 contracts were undertaken by Polish contractors. The sharp fall in International Competitive Bidding (ICB) contracts, from US$466.6 million at appraisal to US$304.4 million at completion may be partially responsible for the polish firms taking over most of the contracts. The ICR does not provide evidence regarding whether there was any bidding by foreign firms. Improve the road safety (Substantially Achieved). As mentioned earlier, no targets for reduction in fatalities or road crashes were set at appraisal. The ICR reports that number of road crashes declined from 66,586 in 1997 to 51,069 in 2004. For the same period, the number of fatalities has decreased from 7,310 to 5,712. A number of road safety initiatives were undertaken. These included: (a) treatment of hazardous locations; (b) black spots elimination (seven out of eleven contracts were completed); (c) installation of cats eyes and rumble strips; (d) adoption of National Road Safety Program in 2001; and (e) coordination between the National Road Safety Council (NRSC), the national traffic police and GDDKiA.. Adjust road user charges (Modestly Achieved, given insufficient evidence to the contrary). A study was carried out in 2003 to review all fees, taxes and charges paid by the road users and the cost that each class of vehicle imposes on the road network (wear and tear costs). The results and findings from the study were used to determine the Government’s road funding policy and the GDDKiA's future investment plans. However, the ICR does not provide actual evidence about the adjustment of road user charges. 5. Efficiency : For bridge and new bypass construction, the average internal rate of return (IRR) was 40% compared to 25% at appraisal. The ex-post IRR for modernization/rehabilitation of roads was 34%, compared to 25% identified in the SAR. The ICR does not provide explanation as to why the IRRs were much higher at completion compared with those at appraisal. The bypass have achieved traffic diversions between 70 and 90 percent of total traffic. 6. M&E Design, Implementation, & Utilization: The project design did not include output and performance monitoring indicators. Annex 3.10 of the ICR included progress reports for project components. Performance monitoring indicators along with baseline data are an important means of assessing project achievements and outcome. During implementation project outcomes and impact were monitored and the ICR lists them in the Annex1. However, since no targets were set at appraisal, comparisons between actuals and appraisal targets cannot be made. The other shortcoming regarding M&E is that the project did not result in capacity building for specific project monitoring. This was mainly becasue the Government outsourced monitoring activities to consultants. 7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative): Safeguards - The ICR notes that none of the schemes undertaken created any adverse environmental impacts during construction or after. Regarding involuntary resettlement, the ICR does not provide evidence that the involuntary resettlement carried out under the project complied with the Bank's OD 4.30 policy requirements. The Borrower's ICR reports that there were a number of protests by local residents on a number of GDDKiA construction sites. How the public concerns were resolved is unclear. Unintended Impacts - None 8. Ratings : ICR ICR Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Unsatisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness . 9. Lessons: Investment projects can provide a good opportunity for initiating policy dialogues on reforms. Flexibility in implementation matters allows the Borrower to adopt new approaches on procurement and become more efficient. Contracts for works should have a realistic "time duration" to discipline contractors and to reduce situations where advance payments are cashed and no or little on-site progress is noted. While strengthening the capacity of local contracting industry, the Bank should also assist in preparation of standardized bidding documents. This will reduce the ambiguities in the contract conditions and enable better transparency in the selection. 10. Assessment Recommended? Yes No 11. Comments on Quality of ICR: The quality of the ICR is rated unsatisfactory for the following reasons: 1. As a self evaluation, the ICR misses out stating what the project objectives were which are important for evaluation of the achievement of project outcome. A reader of the ICR cannot know what the purpose of the project was. The Region later acknowledged what was an editing oversight and drew IEG's attention to the fact that the Borrower's ICR does include the project objective statement (Annex 8 p.32). But to guide the reader through the subsequent evaluation, it is very important to have the project objective statement up-front, as per the ICR guidelines. 2. Before reporting what it was, the ICR simply states that the Development Objective (DO) was fully met, without saying how or giving specific evidence of actual results in terms of the modernization, and environmental and safety improvements sought by the DO. The ICR did address the results of the project's seven supporting objectives, however. 3. The ICR notes that there were no counterpart funding problems, even though the Government of Poland's contribution fell from US$240 million to US$173.4 million. However, it provides no explanation. According to later comments by the Region (not available in the ICR) counterpart funding needs had been overestimated at appraisal and that some elements of counterpart funding--incl. VAT on construction--were not incorporated into the completion estimates. 4. The ICR does not provide explanation as to why the IRRs were much higher at completion compared with those at appraisal. The Region subsequently informed IEG that such an explanation had not been possible because the SAR estimate had been made on a project-by-project basis, with no detail on the parameters taken into consideration in the computation of the project benefits. The ICR could have been forthcoming about this limitation. 5. The ICR does not report on safeguard compliance (the project was assigned Environment Category "A"), in particular, how the environmental issues and negative environmental impacts were followed up during implementation. Also there is lack of evidence that the involuntary resettlement carried out under the project complied with the Bank's OD 4.30 policy requirements. The Region later agreed that the involuntary resettlement was amicable and consistent with Polish legislation and that the ICR should have mentioned that involuntary resettlements were carried out in compliance with Bank policy. 6. There is no information on the three studies for the Polish National Railways, but the ICR does, however, that this information will be forthcoming later in a separate implementation completion memorandum. The ICR does not explain why the cost of TA and training component increased from US$1.7 million at appraisal to US$10.2 million at completion. Subsequent comments from the Region recognize this oversight, and inform that the increase was the result of the Borrower's request to mitigate problems of a staff shortage in GDDKiA for project supervision. 7. The ICR notes that the local costs are presented as net and are exclusive of land acquisition costs, design and documentation costs, supervision costs and VAT and that if these were included, the overall actual cost of the project would exceed the original estimate of US$540 million. It is not clear from the ICR if the appraisal costs had all the same elements included as the ICR costs.