ICRR 14318 Report Number : ICRR14318 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 06/11/2014 Country : Pakistan Project ID : P010556 Appraisal Actual Project Name : Highways US$M ): Project Costs (US$M): 261 576 Rehabilitation L/C Number : C3846; L7212; L7341; Loan/ US$M ): Loan /Credit (US$M): 200 514 L7376 Sector Board : Transport US$M ): Cofinancing (US$M): 0 0 Cofinanciers : Board Approval Date : 12/23/2003 Closing Date : 06/30/2009 06/30/2013 Sector (s): Roads and highways (92%); Central government administration (6%); Forestry (2%) Theme (s): Infrastructure services for private sector development (40% - P); Injuries and non-communicable diseases (20% - S); Public expenditure; financial management and procurement (20% - S); Administrative and civil service reform (20% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Kavita Mathur Victoria Alexeeva Christopher David IEGPS1 Nelson 2. Project Objectives and Components: a. Objectives: The objective of the project was to assist with the sustainable delivery of a productive and efficient national highway network, thereby contributing to lower transportation costs (Project Appraisal Document p. 3 and Development Credit Agreement p. 14). b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: The project had following two components: Network Conservation (appraisal cost US$159.3 million, actual cost US$433.2 million). This consisted of civil works along the National Highways N-5 and M-9: a. Rehabilitation and Improvement of about 550 km of highway; b. Resurfacing and Strengthening of 306 km of highway; c. Safety Improvement works at about 15 - 20 locations; d. Resettlement & Land Acquisition; e. Relocation of utilities; f. Afforestation (tree planting) along the project corridors for a length of about 2,500 avenue km; and g. Consultancy Services for design, contract administration and construction supervision. Policy Support and Institutional Development (appraisal cost US$14.9 million, actual cost US$14.7 million). This included technical assistance, training, equipment support and incremental operating costs to: a. help implement improved sub-sectoral policies (Medium Term Budget Framework (MTBF) to improve targeting of public sector resources; and operationalization of the Road Maintenance Account (RMA) to provide stable and secure funding for maintenance). b. strengthen NHA institutional capacity; improve performance and efficiency through Corporate Strengthening Program; improve road asset management; improve road safety practices; enhance environmental management and resettlement practices; sector studies and future project preparation. First Additional Financing Component : Reconstruction and Rehabilitation of Earthquake Damaged Roads (appraisal cost US$0.0 million, actual cost US$90.8 million) included rehabilitation of 128km of highways. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Original Financing: The project was approved on December 23, 2003 with a Credit amount of US$150 million and a Loan amount of US$50. Additional Financing: The project received three additional financing: First, an Additional Financing loan of US$100 million was approved on December 1, 2005 for reconstruction and rehabilitation of three strategic roads of national importance that were damaged by the earthquake of October 8, 2005. Second, an Additional Financing loan of US$ 65 million was approved on March 29, 2006 to help finance the cost overrun resulting from significantly higher than expected bid prices for the second phase of civil works contracts. Third, an Additional Financing credit amount of US$130 million was approved on September 28, 2010 to finance cost overruns and repair damage caused by the floods in 2010. Borrower Contribution: The actual Borrower contribution was US$62.2 million compared to the appraisal estimate of US$31.4 million. Dates: The original closing date of the project was June 30, 2009. The project was extend four times: 1. In March 2009, the closing date of original financing and the first and second Additional Financing was extended by 18 months to December 31, 2010. 2. In August 2010, the closing date for the original Financing and the first and second Additional Financing was extended by five and half months to June 15, 2011. 3. In April 2011, the closing dates of the original financing and the first and second Additional Financing were extended to June 30, 2012 to match the closing date of the third Additional Financing. 4. In June 2012, the project was restructured to extend the closing date of the third Additional Financing by a period of twelve months to June 30, 2013. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Rated high. The project development objectives remained highly relevant throughout the life cycle of the project. The project objectives were consistent with the focus on enhancing the efficiency of the national highway network in the Government of Pakistan's multiyear National Highway Improvement Program adopted in 2002. The objectives were in line with the first and second pillars of the Country Assistance Strategy (CAS) at appraisal for the period FY03-06. These were: (i) strengthen macroeconomic stability and government effectiveness, (ii) improve the business environment for growth. The physical works financed by the Project were aimed at lowering transportation costs and improving traffic safety on the national highway network. The project objectives were consistent with the strategic objective of the FY06-09 CAS to remove infrastructure bottlenecks, support sustained economic growth and improve competitiveness. The project objectives were fully aligned with the second pillar of the current FY10-14 Country Partnership Strategy (CPS) - improving infrastructure to promote growth. Under Pillar 2, the current CPS seeks to improve efficiency and reliability of the transport and logistics network. b. Relevance of Design: Rated substantial. The statement of development objectives was clear. The project results framework indicated a clear causal chain between the activities financed by the project and the outputs and outcomes related to the attainment of the development objectives. For example, with regard to physical investments in road network conservation (Component 1), the activities were expected to reduce vehicle operating costs, travel time and number of fatalities from road accidents on the project corridor. These, accompanied by policy support and institutional development, were intended to ensure sustainable delivery of a productive and efficient national highway network. 4. Achievement of Objectives (Efficacy): To assist with the sustainable delivery of a productive and efficient national highway network, thereby contributing to lower transportation costs: rated substantial. Outputs The project met all its physical targets – 1,035 km (100%) of the project length was reopened to traffic a few months prior to project closing. Appropriate engineering measures were implemented under the project to improve safety of road sections. Safety works were included in the main civil works contracts, i.e., grade separated interchanges, vehicular underpasses, pedestrian under- and over-passes. The following institutional development outputs were delivered: A Road Asset Management System (RAMS) was developed. The ICR reports that it will be managed by staff comprising: a full-time Director, Deputy Director (Bridges), four Assistant Directors (Engineering), one Assistant Director (Computer Programming), a Data Base Officer, and eight Enumerators. The Road Maintenance Account (RMA), which was set up prior to Board approval to finance Annual Maintenance Programs, became operational on August 21, 2003. RMA is financed through road user charges. At project closing, the RMA revenues were sufficient to finance about 80% of the maintenance needs. The Axle Load Management Program was not initiated as planned. Only the Effective Weight Control Management Study was completed. A Human Resource Development Strategy and Action Plan was prepared and implemented. This defined the NHA overall human resource framework and priorities. Based on the plan, key staff was recruited. Also, a Training Needs Assessment was carried out and a special 2-year Catch-up Training Program was implemented. An Action plan to improve financial management and internal controls of the National Highway Authority (NHA) was implemented. The ICR reports that at project closing, NHA financial statements were current and showed decline in the number of qualifications (the ICR does not provide figures). The Ministry of Finance and NHA adopted a mutually agreed Medium-Term Budget Framework (FY 03/04 - FY 07/08) for construction, rehabilitation and maintenance. However, this was not updated for later years. A Business Process Re-engineering (BPR) study was undertaken but the study was not implemented. The study aimed to computerize NHA’s core functions for enhanced performance and corporate governance. A state-of-the-art laboratory was set up at the Highway Research and Training Center for developing improved pavement design procedures for Pakistan. A National Highway Safety Council Secretariat was set up and staffed. The Secretariat developed a National Road Safety Plan. Road safety audits were carried out. Safety Awareness Campaigns were conducted in nineteen cities along the project roads. Driver and school children education pilots were also launched. Outcomes Average network-level roughness was reduced from 5.4 International Roughness Index (IRI) at appraisal to 4.5 IRI at the end of the project (against the target of 4.4 IRI). The national highway network in poor condition was reduced from 45% (baseline value in 2003) to 35% at project closure (2013) meeting the appraisal target. This is based on the annual data collected for the Road Asset Management System (RAMS) covering about 11,000 km of the national highway network. The vehicle operating cost for road users decreased by about 5.0% meeting the target, as measured by the reduction in average network-level roughness and poor condition of the road network. Travel time on 1,700 km of Peshawar- Karachi section of National Highway (N-5) was reduced from 47 hours in 2003 to 39 hours in 2013, exceeding the appraisal target of 42 hours. Improvement in the road safety along the National Highway (N-5) corridor - road fatalities on 1,700 km of Peshawar-Karachi section of National Highway N-5 was reduced from 109 fatalities/100 km in 2003 to 96 fatalities/100 km in 2009 and 39 fatalities/100 km in 2013. Routine and periodic maintenance funding increased from 25% of the maintenance need to 88% but was short of the target of 100%. The share of total expenditure on network conservation at the end of the project was about 66% compared to 38% at appraisal. 5. Efficiency: Rated modest. The Economic Internal Rate of Return (EIRR) at closure is estimated to be 26%, showing a decrease compared to 33% at appraisal. The ex-post economic analysis followed the same methodology adopted at appraisal. It was carried out using the Highway Development and Management Model (HDM-4), taking into account the estimated economic cost, current road condition, traffic volume and its projected growth. The impact of higher prices on the rates of return was moderated by the significant growth in traffic volumes, reduction in maintenance costs and traffic delays. The implementation period was about 10 years for the project, which was originally scheduled for 5 and 1/2 years. Delays were to a large extent attributed to shortages in supply of bitumen and adverse security situation in the country but also due to non-payment of GOP counterpart funds and cash flow problems, delays in finalization of designs and initially weak contract management. The project final cost was about twice of the original estimate, which seems high even after controlling for price increases in key inputs. The project's initial cost of US$361 million (original US$261 million and AF of US$100 million for the earthquake damaged roads in 2005) increased to US$576 million at project closure. In addition to increase in prices of key construction materials, several technical design (and re-design) changes due to early mistakes that needed to be remedied during implementation seem to have affected the costs (ICR, p. 43, 47). However, the elevated costs at the end did not secure the quality of the works, as evident in localized distress and early need for repairs (ICR, p. 46-47, 51). The latter may also indicate shortcomings in the quality and performance of the contractors and materials. ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 33% 83% ICR estimate Yes 26% 68% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The objectives were highly relevant to the country's development priorities. Relevance of project design was substantial. The project assisted in the sustainable delivery of a productive and efficient national highway network in a substantial way, thus contributing to lower transportation costs. Efficiency is assessed as modest. a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The GOP and NHA continue the National Highway Improvement Program using a phased approach to improve about 11,000 km of the national network with the GOP’s own, Roads Maintenance Account (RMA), and multi/bi-lateral donor resources. The RMA revenues were sufficient to finance about 80% of the network maintenance needs at project closure. The planned axle load management program was however not initiated as planned. While the overall condition was judged to be in a satisfactory condition, some localized distress was evident; heavy axle loads continue to be a concern for the highway network condition (p. 47, 53). While the Bank financing of the project was provided by GOP to NHA as a grant, the Government’s resolution of the existing debt stock of NHA and future grant financing was delayed and remains outstanding (the legal covenant was waived in May 2012 through project restructuring). NHA does not have a revenue base to re-pay the loans to GOP that finances the NHA’s capital development program. NHA's own revenue (primarily tolls) is barely sufficient to cover the costs of operations and maintenance of the existing road network (p. 22, 52). a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: The early stage of project preparation took a programmatic approach to addressing the rehabilitation and resurfacing needs of the entire NHA network. However, the results of the network level economic prioritization showed that the network needs were dominated by the requirements of National Highway N-5 because of the high traffic levels along this north-south corridor. The project's physical works were therefore restricted to the rehabilitation and/or improvement of specific sections of National Highway N-5 only. Safeguards identification was satisfactory. The key lessons from the previous highway projects were incorporated in the project design. At appraisal, the project risk rating was assessed as substantial. However, major unforeseen risks included: (i) enormous increase in the road construction costs driven by sharp oil price increases in the world market; (ii) shortages in supply of bitumen; and (iii) adverse security situation. The Borrower’s ICR assesses the institutional development program as ‘ambitious and unrealistic’; the project included multifaceted policy reforms, institutional strengthening, and capacity building measures which were to be implemented with the limited institutional capacity for which it was designed (p.42). At effectiveness, the detailed engineering designs and environmental and social safeguards instruments for phase-1 civil work contracts were complete. NHA had received the bids for 8 phase-1 contract packages. NHA had also received the technical and financial proposals for construction supervision consultant services. at -Entry Rating : Quality -at- Satisfactory b. Quality of supervision: The ICR reports (p. 19) that the project supervision was facilitated by adequate numbers of staff with requisite skills and expertise in technical and safeguards functions. In addition, the Bank mobilized services of reputed international experts in the areas of road asset management, road safety, human resource development and business process re-engineering. The project team provided advice and proposed solutions to remove implementation bottlenecks such as introduction of price adjustment clause in phase-I contracts, increase in price adjustment ceiling in phase-II contracts, additional advances to contractors secured through Joint Escrow Accounts, arrangements of a special quota for bitumen from refineries for the project, reduction in duties and taxes on imported bitumen. The Bank team was flexible and additional financing was secured in the aftermath of natural disasters such as earthquake and floods. The ICR reports (p 20) that the key implementation issues were regularly and candidly brought to the attention of the sector and country management through the Implementation Status Reports (ISRs) and the guidance received was promptly acted upon. Despite ‘proactive and intensive’ supervision, as described by the ICR p.20, the axle load management program had not been initiated as planned by project closure with the extended implementation period of almost 10 years. Technical inputs provided by the Bank may not have been sufficient or adequate to progress on this program as envisaged. Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Government of Pakistan helped facilitate project implementation through a number of actions that included: (i) amending the NHA Act to restore central review and approval of all NHA programs and projects (exceeding Rs 100 million) through the Central Development Working Party/Executive Committee of the National Economic Council; (ii) on-lending World Bank financing to NHA as a grant; and (iii) reducing the Custom Duty on imported bitumen from 25% to 5% and abolished the Federal Duty of Rs 2,000 per ton to bring price of imported bitumen at par with the locally produced bitumen. While the Government provided adequate counter-part funds during the original five year project implementation period, there were some delays starting in 2008 when macro-economic condition of the country deteriorated resulting in budgetary cuts. The Government however did not follow through with its commitment to implement a road sector reform related to the NHA’s financial standing. The legal covenant was waived in May 2012, and the Government’s resolution of this issue remains outstanding (p.22). There were also issues with land acquisition under the earthquake damaged road component (1AF) that was affected by utilization of funds by the Azad Jammu and Kashmir Government for other purposes rather than disbursement to affected persons (p.49). Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance: The project was implemented by NHA and the Project Management Unit. NHA regularly submitted monthly progress reports and quarterly financial management reports to the Bank. The NHA staff had to go through a steep learning curve with new processes for managing fiduciary and safeguards aspects as well as outline-based design-build, lump-sum contracts. The Implementation Progress and Output Indicators remained on track and IP was rated ‘Satisfactory’ almost throughout project implementation. About 1,035 km (100%) of the project length was reopened to traffic a few months prior to the project extended closing date. While NHA was proactive in implementation and committed to project completion (p.7-8), the initially weak contract management (p.44-47) and design changes contributed to delays and cost overruns in civil works contracts. Also, land acquisition under the earthquake damaged road component (1AF) was affected by delay in releasing of funds by NHA despite the fact that the land acquisition was financed by the Bank (p.49). Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The project design included the following outcome indicators with targets: (a) reduction in vehicle operating costs due to better quality of the road network (the International Roughness Index (IRI) to be reduced from 5.4 to 4.4 and network in poor condition reduced from 49% to 35%); (b) reduction in travel time on N-5 by 10%; and (c) reduction in road fatalities along the project corridor by 10%. The NHA was responsible for the overall monitoring of the project. The PAD included baseline data and targets. The output indicators included rehabilitation of 550 km of highways, resurfacing of 306 km, and safety improvement works at 15-20 locations. b. M&E Implementation: During implementation, data on network-level road condition and roughness were collected every year. Travel time data were collected twice - at mid-term and at the end of the project. c. M&E Utilization: The ICR reports (p. 10) that the outcome indicators were used for broad decision making and resource allocation purposes. M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: The project was assigned Environment Category "B" at appraisal and this categorization was retained under the three packages of Additional Financing as the scope of work was not altered. Two safeguards policies were triggered: Environmental Assessment (OP 4.01) and Involuntary Resettlement (OP/BP 4.12). Environmental Safeguards: An Environmental Impact Assessment (EIA) was carried out at appraisal for both phase-1 and phase-II of the Project. Negative impacts of the project were mostly related to the construction stage only and included soil erosion, air quality deterioration, improper disposal of soil, contamination of soil and water, loss of vegetation, disruption of traffic and increase in greenhouse gasses emissions due mainly to the increased volume of traffic. The EIA included measures to mitigate these impacts. The ICR reports that the implementation of the Environmental Management Plan (EMP) faced some problems early on due to capacity limitations. The project team clarified that as this was the first project undertaken by NHA which contained environmental and social safeguards, capacity building of civil works contractors was required. The civil works contractors hired the services of Environmental Engineers and Environmental Consulting Firms from the market who provided training to the contractor's staff and labor. The biggest challenge was ensuring that the contractor's staff and labor used personal protection equipment at active work sites, which was new to them. The project addressed the institutional capacity by nominating environment focal persons in the Project Management Unit (PMU) as well as with the contractors. The field staff and contractors were provided training on EMP implementation. Soil, air and water testing was carried out in the project impact zone to monitor any potential impacts. The project conducted a biannual third party audit to assess EMP implementation. The NHA set up an Environmental Management Unit to support environmental safeguards for both Bank and non-Bank funded projects. To mitigate greenhouse gas emissions due to increased traffic, about 837,000 trees were planted along the project corridor on a length of about 2,500 roadside kilometers. For the earthquake managed roads in the mountainous region, the project introduced soil bio-engineering measures to stabilize mountain slopes and employed innovative techniques that included terracing slopes with plantation (trees, shrubs and grasses) and engineering measures (cutoff drains, retaining walls). The ICR reports that the project complied with the Environmental Safeguards. Social Safeguards: At the time of appraisal in 2003, land acquisition and displacement triggered by this project envisaged about 6 hectares of land and impact on about 350 project affected persons (PAPs). Social Impact Assessments (SIAs) were carried out for both phase-I and phase-II of the project and Resettlement Policy Framework (RPF) was prepared. The ICR reports that the land acquisition was carried out according to the Land Acquisition Act. Also, livelihoods of PAPs were preserved by resettling them as quickly as possible at a nearby location within the Right of Way. Livelihood assistance was provided when the relocation was delayed. However, the management of social safeguards of the earthquake damaged roads was challenging. It involved acquisition of about 1,007 Kanals of land (valued at US$ 1.1 million). Of which, US$0.78 million i.e. 72% compensation was paid to the PAPs. The balance amount (of about US$0.32 million) is with the Land Revenue Departments and was not paid because: (a) the persons died during the earthquake; (b) had left the area and are not traceable; (c) disputed claims in the courts; and (d) shamlat land cases (i.e. multiple joint land owners with entitlement amounts less than US$ 100 who are unwilling to open a Bank account). The ICR reports (p. 10) that the court cases are unlikely to be resolved in the near future. However, the compensation will eventually be paid upon settlement of court cases since the compensation amounts are already deposited and held in escrow accounts with the Land Revenue Departments. The project team further elaborated that the disputed claims were US$ 0.11 million: out of the total compensation amount, the proportion was 11.5%, and out of the balance amount of US$ 0.32 million, the disputed claims proportion was 36%. The ICR (page 49) also reports that the Azad Jammu and Kashmir Government used funds for other purposes rather than disbursement to affected people. b. Fiduciary Compliance: Financial Management: An accounting unit with staff and funding was set-up at the Project Management Unit for financial management of the projects. The financial statements and audit reports were timely and unqualified. The financial management of the project was satisfactory (ICR page 13). To strengthen financial management capacity of NHA, a Financial Management Improvement Plan was developed. This introduced International Accounting Standards and commercial auditors. The financial statements of the NHA and Road Maintenance Account (RMA) are current and annually audited by the commercial auditor. A Standard Operating Procedure (SOP) was issued for processing payments to contractors and suppliers. The SOP laid down in detail the steps to be followed for verification and checking of bills. Procurement: The ICR reports (page 12) that the procurement was carried out in accordance with the Bank guidelines and was satisfactory. c. Unintended Impacts (positive or negative): None. d. Other: The project innovations related to structural (avalanche galleries) and soil bioengineering applications for slope stabilization in mountainous terrain were transferred to Timor Leste under a grant from the South-South Exchange Facility. 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Efficiency is assessed as modest Satisfactory due to significant cost overruns. Risk to Development Moderate Significant Due to NHA’s unsustainable Outcome : financial standing and a lack of progress on axle load management. Bank Performance : Satisfactory Moderately Technical inputs provided by the Satisfactory Bank may not have been sufficient or adequate to progress on the axle load management program. Borrower Performance : Satisfactory Moderately Treatment of the existing stock of Satisfactory NHA debt has not been resolved (legal covenant was waived); slow progress on axle load control over network. Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: Adapted from the ICR: Implementing a sector policy reform through in an investment operation is difficult. Under this project, the legal covenant regarding NHA's financial standing was waived and the government’s resolution of NHA’s debt remains outstanding. The time gap between the project preparation and implementation must be avoided to reduce the likelihood of financing gaps and bid prices. The engineer's estimates must be updated at the market value at the time of appraisal and adequate premium be allowed to cover for the procurement delays. Unanticipated price hikes due to market dynamics particularly in developing countries like Pakistan must be expected and adequate allocation for price contingencies must be built into the budget. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The quality of the ICR is satisfactory. The ICR is internally consistent and well written. The analysis in the Borrower's ICR is candid and insightful. There are some inconsistencies between the ICR and Borrower’s ICR; for example, while the ICR on p.8 ascertains that � NHA, contractors and construction supervision consultant were committed to the project and ensured that the works were of good quality�, the Borrower’s ICR p.51 talks about “average quality of supervision staff on site�, and “not satisfactory “ performance of the Lead Material Engineer and Lead Asphalt Technologist “as is evident from the failure of pavement due to inconsistent quality control�. a.Quality of ICR Rating : Satisfactory