Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NP: Bridges Program Support (P125495) Report Number : ICRR0021734 1. Project Data Project ID Project Name P125495 NP: Bridges Program Support Country Practice Area(Lead) Nepal Transport L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) 55,083,471.66 Bank Approval Date Closing Date (Actual) 28-Jun-2012 IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 0.00 Revised Commitment 0.00 0.00 Actual 0.00 0.00 Prepared by Reviewed by ICR Review Coordinator Group Elisabeth Goller Christopher David Christopher David Nelson IEGSD (Unit 4) Nelson 2. Project Objectives and Components a. Objectives According to the Financing Agreement (Schedule 1, page 4), the objective of this Program for Results (PforR) was to provide safe, reliable and cost-effective bridges on the Recipient's Strategic Roads Network (SRN). The PAD (para 32) pointed out that in the Government’s 2004 Bridge Policy and Strategy “safety” referred to designs that adequately addressed standard loading requirements, user safety and climate and seismic resilience; “reliability” referred to uninterrupted access through adequate maintenance and emergency Page 1 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NP: Bridges Program Support (P125495) measures; and “cost-effectiveness” referred to the use of sound bridge management and commercial practices in the sector. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Components The SRN Bridge Program, supported by this operation, was part of the Department of Road’s (DOR) overall road and bridge infrastructure program set forth in the Recipient's 2004 Bridge Policy and Strategy for the SRN. The Program encompassed the following expenditure areas: Planning, technical design, and quality control of bridge construction and maintenance, which included activities to plan, prepare, supervise, and monitor the civil works, including social and environmental aspects. It comprised funding for consulting services to prepare detailed engineering designs, supervision of bridge construction works, quality assurance and monitoring, data collection linked to bridge conditions and maintenance of the bridge management system (BMS), auditing and verification, training, and miscellaneous activities. Major and minor maintenance of existing bridge assets, which was expected to focus on approximately 98 bridges, totaling about 6,225 m, that urgently needed major maintenance to prevent impending failures. Many of these bridges were considered structurally unsound and therefore unsafe. The operation was also expected to cover major maintenance of 233 bridges, totaling 10,900 m in length, and minor maintenance of 95 bridges, totaling 3,500 m in length. These bridges were in a relatively stable condition. New bridge construction, to construct approximately 121 new bridges for a total length of about 6,000 m. This total included approximately 95 bridges, for a length of 5,000 m, of the DOR’s existing backlog of bridge construction. In addition, a total of 26 new bridges, for about 1,000 m, was expected to be constructed. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates Program Cost: At appraisal, the estimated cost of the Program was US$147.6 million over five years. The actual cost was US$97.3 million, which is 66% of the appraisal estimate. The estimated cost at appraisal was distributed as follows among the three expenditure areas: • SRN bridged maintenance: US$58.5 (40% of the total program cost) • SRN bridge construction: US$81.7 (55% of the total program cost) • SRN bridge feasibility, design and quality management: US$7.4 (5% of the total program cost) • The ICR did not provide the distribution of the total actual cost among the different expenditure areas at program closure. Page 2 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NP: Bridges Program Support (P125495) Financing: The financing was expected to come from an IDA credit for an amount of SDR38.7 million, equivalent to US$60 million (para 26 PAD). Due to the depreciation of the SDR, the actual Bank-financing was US$54.3 million (para 45 ICR), corresponding to 91% of the appraisal amount. Recipient Contribution: The Government contribution estimated at appraisal was US$87.60 million (para 26 PAD). The actual contribution was US$43 million (para 45 ICR), which is 49% of the appraisal amount. Dates and Revisions: The credit was approved on June 28, 2012, became effective on December 20, 2012, and closed on July 15, 2017. The operation was restructured on January 13, 2017 to revise the program end targets for the first and the second DLIs and to reallocate the credit proceeds between them accordingly. This restructuring was necessary in response to the mid-term review (MTR) in June 2015, it had become clear that adjustments would be required to DLI-1 on major maintenance. During the initial phase of implementation, the DOR struggled to follow the new concept of the planned maintenance system introduced by the SRN Bridge Program. Local consulting firms and contractors hired to carry out the work were also not familiar with the new system, which became a major cause for delays in the completion of the engineering designs and the construction works under DLI-1. This in turn created a series of setbacks towards achieving the DLI-1 targets. The Ministry of Finance (MOF) did not want to request the restructuring immediately after the MTR and advised the DOR to assess the situation further. This led to a delay in receiving the Government’s restructuring request. At the time of restructuring, the SRN Bridge Program had already achieved the DLI- 2 target on minor maintenance and further minor maintenance of bridges was needed (page 7 December 2016 Restructuring Paper). Therefore, the DLI-1 and the DLI-2 end targets were revised to become 14,125 m (originally 17,125 m) and 10,000 m (originally 3,500 m), respectively. At program completion, the revised DLI-2 target was exceeded by 7% (10,729 m instead of 10,000 m), the revised DLI-1 target was exceeded by 14% (16,215 m instead of 14,215 m), and the original DLI-1 target was also nearly achieved (16,136 m instead of 17125 m). Since no changes took place to the program objectives and outcome targets, and the program scope was also not substantially altered, no split evaluation was undertaken. 3. Relevance of Objectives & Design a. Relevance of Objectives At appraisal, the proportion of the SRN in poor condition had increased from 18% in 2008 to 22% in 2010, and much of the existing bridge stock was over 35–40 years old and in desperate need of rehabilitation and maintenance. Further, there were still many gaps on the SRN caused by lack of bridges. This contributed to a substantial lack of physical access to economic centers and social services, particularly in remote areas. Nepal’s topography and geology also complicated efforts to provide adequate transport infrastructure, and bridges are key in mountainous areas. The lack of physical access was considered one of the root causes of the conflict and instability in the country. Roughly one-fourth of district headquarters did not have an all-weather road connection. This lack of connectivity had adversely impacted the ability of people, particularly in the severely affected rural areas, to participate in economic activities and access quality health care and education services (para 2 ICR). Page 3 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NP: Bridges Program Support (P125495) The Government’s 2007–2016 Priority Investment Plan (PIP) for the road sector, which prioritized investments to reach target accessibility levels by 2016, had not focused on bridges (para 6 ICR), but the DOR had developed the 2004 Bridge Policy and Strategy. With the support of the Bank-financed Road Sector Development Project (RSDP), the DOR had also improved its capacity and tools to efficiently manage its bridge assets. In addition, the Government had shown its commitment to the bridge sector by increasing budget allocations, especially for major bridge maintenance works. In fiscal year (FY) 2011- 2012, the total budget for SRN bridges was approximately US$10 million. The Government agreed to increase it to US$18.5 million in the following FY. Nevertheless, these resources were insufficient to meet the total needs in the sector, which were estimated at US$328 million (ICR para 18). Support to the SRN Bridge Program was consistent with the FY2012-2013 Interim Strategy Note (ISN) for Nepal, particularly the first pillar that related to enhancing connectivity and productivity for growth. Poor road and bridge quality was considered a key constraint to the provision of universal physical access. The ISN explicitly mentioned supporting bridge-related interventions in Nepal to extend year-round accessibility and improve the quality of existing bridges (para 10 ICR). Work on bridges continued to be a priority under Pillar 1: Increasing Economic Growth and Competitiveness in the FY2014-2018 Country Partnership Framework (CPF) for Nepal. Under Outcome 1.2: Improved Transportation Connectivity, internally and with India, the CPF referred to trail bridges and bridges on local roads, capacity strengthening for transport sector institutions and improvement in maintenance, safety and quality of transport infrastructure and services. Bridges are also relevant under Pillar 2: Increasing Inclusive Growth and Opportunities for Shared Prosperity since it put emphasis on inclusive growth, strengthened governance, institutions for improved service delivery and accessibility to all citizens. The Government showed its continued commitment to the Bridge Program through a request for a follow-up operation, which was approved by the Bank’s Board on September 24, 2018. The PforR was a logical continuation of the Bank’s effort to support the country in the management of bridge assets through the RSDP. Given this support and the relatively low capacity in the country, the not overly ambitious objective, which was outcome-oriented and causally linked to the program activities, was appropriate. Rating High b. Relevance of Design --- Rating High 4. Achievement of Objectives (Efficacy) Page 4 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NP: Bridges Program Support (P125495) PHEFFICACYTBL Objective 1 Objective Provide safe bridges on the Recipient's SRN Rationale As mentioned in section 2.a above, safe bridges under this Program were to be understood as bridges which adequately addressed standard loading requirements, user safety, and climate and seismic resilience. The objective was to be measured through the percentage of bridges rated as structurally unsafe and hence requiring urgent maintenance. During the implementation of the Program, the percentage of unsafe bridges on the SRN was reduced from 6% in 2012 to 1% in 2017, which was in line with the program end target. The Program also introduced new methods for bridge inspection and testing, such as the use of bridge inspection vehicles and nondestructive testing equipment (para 40 ICR). A total of 85 bridges were made structurally safe through urgent major maintenance carried out under the PforR (para 70 ICR). All but one of the 14 actions of the Program Action Plan (PAP) were satisfactorily completed (para 44 ICR). Several of them had likely contributed to this result by improving institutional capacity, systems and procedures. These included (i) the allocation of at least 5% of the civil work’s budget for feasibility studies, design, supervision, quality control, management of environmental and social impacts, etc., (ii) the implementation of annual training plans, and (iii) the implementation of a Quality Assurance Plan (QAP) for civil works. However, as seen in relation to Objective 3 below, there were shortcomings in civil works quality and this issue constituted a constant worry during implementation. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Provide reliable bridges on the Recipient's SRN Rationale According to the PAD (para 32), reliable bridges were meant to be bridges for which adequate maintenance was likely to ensure uninterrupted access. This objective was measured by the percentage of bridges on the SRN in good or fair condition, which was increased from 53% in 2012 to 81% at program completion. This was 8% more than the program end target. The condition of 481 bridges out of a total of 1,709 bridges on the SRN was improved to a good or fair condition under the PforR. When the credit closed, these bridges were under routine maintenance or required only minor maintenance. This meant that 105 bridges more than the program end target of 376 bridges were improved to a good or fair condition (para 67 ICR). In addition to the physical works, it is likely that other factors have contributed to this result. These included (i) enhanced maintenance capacity, systems and procedures derived from the PAP actions mentioned under Page 5 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NP: Bridges Program Support (P125495) Objective 1 above as well as (ii) an updated bridge management system (BMS) with data on all bridges on the SRN, (iii) its use to prioritize the annual programs for bridge maintenance and construction, and (iv) the disclosure of such programs. The three latter actions were also part of the PAP. Rating High PHREVDELTBL PHEFFICACYTBL Objective 3 Objective Provide cost-effective bridges on the Recipient's SRN Rationale Cost-effectiveness referred to the use of sound bridge management and commercial practices within the sector (para 32 PAD). The related PDO indicator looked at the strengthened performance management in the bridge sector and was to be measured by the percentage of the bridge works completed on planned schedule. At the time of appraisal, this was negligible. When the credit closed, the number of bridge works completed on schedule was 63% or 13% more than the 50% program end target. For these bridges no claims for delay penalties or price adjustments were made during PforR implementation (para 75 ICR). Even if the ICR did not specifically elaborate on it, it is important to emphasize that all activities were carried out and the outputs and outcomes were achieved with 66% of the budget estimated at appraisal. The ICR (para 75) pointed out that the design and build contract introduced under this PforR decreased the cost of bridge construction and reduced the construction duration. Moreover, the new construction technologies and practices used, such as carbon fibers and precast concrete, were much more cost-effective than the traditional practices because they enabled the contractor to work without traffic disruptions. The following actions of the PAP might also have contributed to the timely completion of bridge works: (i) the formation of project management teams for contract implementation with technical, social, and environmental personnel, (ii) the appointment of procurement unit chiefs in each procuring entity with clear job descriptions and training on procurement, (iii) the implementation of the annual training plans for all the SRN bridge-related functions, (iv) the provision of regular funds among others for works supervision, and (v) the addressing of outstanding audit observations by the audit committee of DOR. Further, the ICR mentioned (para 74) that cost-effectiveness was achieved in particular through competitive tendering, professional specifications, costs that were in line with costs recorded for similar public-sector projects, and generally sound arrangements for management and financing of operations and maintenance. Actions of the PAP that might have had a positive impact on cost-effectiveness of bridges included (i) the use of the updated and fully operational BMS, which took budget constraints into account, to prioritize the annual bridge maintenance and construction works, (ii) the publication of these annual work programs, (iii) improved feedback from users through the grievance redress mechanism (GRM) satisfactorily operationalized under this operation, and (iv) the implementation of a QAP. With respect to quality of civil works, the PforR instrument contributed to enhance quality assurance through the QAP and an independent verification mechanism by entities, such as the National Vigilance Center Page 6 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NP: Bridges Program Support (P125495) (NVC) and the Commission for the Investigation of Abuse of Authority (CIAA). However, poor construction quality was a constant issue throughout implementation, which was caused by the lack of DOR supervision staff and their limited construction supervision experience of new bridges. Four new bridges were found of inadequate quality and were not recommended for disbursement (paras 68, 86 and 100.d ICR). Finally, the PforR helped strengthen the Government’s audit capacity and its’ systematic audit approach. The fiduciary and environmental and social guidelines developed under this operation were mainstreamed in the DoR’s operations (para 99.i ICR). Rating Substantial PHREVDELTBL PHREVISEDTBL 5. Efficiency --- Efficiency Rating --- a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Point value (%) *Coverage/Scope (%) Available? 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The PDO was highly relevant for the country and in the light of the Bank’s Country Partnership Framework at program end. The DLIs were clearly defined, easily measurable, and fully aligned with the PDO. The PDO was achieved with minor shortcomings in terms of cost-effectiveness given quality issues in bridge construction. Page 7 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NP: Bridges Program Support (P125495) a. Outcome Rating Satisfactory 7. Rationale for Risk to Development Outcome Rating The ICR (para 86) flagged two main risks to the development outcomes, which had already been identified at appraisal. The risk rating was elevated from modest to substantial possibly because the risks subsisted despite the mitigation actions during implementation, but the ICR did not elaborate on it. These risks are related to (i) the sustainability of the SRN Bridge Program funding and (ii) the poor bridge construction quality caused by inadequate supervision by DOR staff. The recently approved follow-up operation to this PforR will focus, among others, on civil works quality. By providing financing, it will also contribute to the funding sustainability of the SRN Bridge Program at least in the short to medium run. Therefore, the current risk to development outcome is considered low. a. Risk to Development Outcome Rating --- 8. Assessment of Bank Performance a. Quality-at-Entry The task team focused on a development problem of strategic relevance for the country. They thoroughly evaluated the pros and contras of the PforR approach for this operation and concluded that it was a more efficient and cost-effective instrument than investment project financing. The team’s judgment at appraisal was that the risks related to a PforR were manageable and that the capacity was adequate. The soundness of these judgements was confirmed by the overall successful implementation and the results of this operation (para 87 ICR). The team carried out three thorough assessments of the technical, social and environmental, and integrated fiduciary aspects. These assessments provided the basis for the design of the program scope, the interventions types, the PAP, and the risk identification and mitigation measures. The operation’s risks were adequately assessed, and overall the risks during implementation proved manageable. In terms of work quality, however, the risk mitigation measures were insufficient. The design of the PforR was relatively simple and straightforward, which seems adequate for the level of development in the country. The operation had only six DLIs. The DLIs were clearly defined, verifiable, and attributable to the SRN Bridge Program. They also had an agreed verification protocol. As seen in section 9.a below, the overall M&E design was also adequate. Page 8 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NP: Bridges Program Support (P125495) The program scope focused on critical major and minor maintenance needed to sustain the existing stock of SRN bridges, thus preventing the Program from making a tradeoff between completing its backlog of bridges under construction and undertaking the needed critical maintenance (para 21, 36 and 88 ICR). The implementation arrangements were relatively simple and proved mostly effective. Quality-at-Entry Rating Satisfactory b. Quality of supervision The Bank maintained co-task team leaders throughout program implementation. The budget for implementation support was adequate and well-used according to the ICR. The task team was qualified and multidisciplinary, and there was fair continuity. Regular site visits were undertaken to assess progress over the life of the operation (para 96 ICR). As part of the implementation support, the task team provided assistance in different areas of program implementation and the Government gave positive feedback on this assistance and acknowledged the value of the Bank’s focus on results (para 93 ICR). The task team’s support related to (i) preparing and adopting modern bridge technologies, including the increased use of prestressed concrete technology and network arch bridges, (ii) systematically reviewing results and DLIs monitoring and verification reports and risk assessments, and (iii) tracking compliance with the Government’s fiduciary, environmental and social systems, and safeguards. The task team consistently raised issues and proposed solutions to overcome problems, such as lack of quality in bridge construction and inadequate operational health and safety (OHS) on construction sites (para 95 ICR). In hindsight, however, this support should have been stronger right from the start of the implementation. One area, in which the Bank provided strong support, was the introduction of citizen engagement under the PforR. Several innovative approaches to enhance citizen engagement were used, including (i) the design and operationalization of the GRM, (ii) the provision of a sign board in all construction sites with contact details of the responsible official(s) to be contacted in case of grievances, and (iii) information to the general public about the Program using local FM radio stations (para 94 ICR). The task team also played an important role in supporting the Government in its remarkable turnaround of the operation, which happened after the PforR had lost about two years of implementation time. This implementation delay was due to (i) initial teething problems, (ii) a sudden change of the SRN Bridge Program management, who had limited interest in the operation, (iii) post-earthquake uncertainties, and (iv) unofficial trade blockades along the Nepal-India borders for over six months (para 97 ICR). Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. Assessment of Borrower Performance Page 9 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NP: Bridges Program Support (P125495) a. Government Performance --- Government Performance Rating --- b. Implementing Agency Performance --- Implementing Agency Performance Rating --- Overall Borrower Performance Rating --- 10. M&E Design, Implementation, & Utilization a. M&E Design The Results Framework for this operation was well designed. The PDO was simple and clearly defined. The PDO indicators had clear baselines and were adequate to measure the PDO’s achievement. The PDO indicators were also easy to measure, and the necessary tools to do so existed. Similarly, the DLIs were clearly defined and easily measurable. They were fully aligned with the PDO. One of the DLIs was also a PDO indicator, while the others were intermediate results indicators. The protocol to measure the achievement of the DLIs and the methodology for monitoring and verification were clear too. b. M&E Implementation The BMS was the primary basis for monitoring results under the SRN Bridge Program. The BMS together with a geographic information system and spatial data provided data on (i) the physical bridge characteristics and recent works; (ii) network information, such as potential detour time if a bridge was out of service; and (iii) other variables, such as the population served by a bridge. To control the quality of the BMS data, the DOR developed a hierarchy of checks and approvals that aligned with the SRN Bridge Program’s implementation structure (para 51 ICR). These included a verification process by the National Planning Commission (NPC). The results were uploaded on the DOR’s website to provide the public with an opportunity to lodge complaints related to the improvement and maintenance of bridges (para 47 ICR). An annual report was disclosed with the status of complaints, actions on fraud and corruption, and those related to social and environmental issues (para 49 ICR). The DOR submitted the financial management reports on time. However, there were delays in the submission of some financial audit reports (para 56 ICR). The ICR did not mention the submission of program implementation progress and procurement reports. Page 10 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NP: Bridges Program Support (P125495) For the impact evaluation study of the SRN Bridge Program only the baseline data was collected due to a lack of funds (para 52 ICR). c. M&E Utilization The ICR did not specifically comment on M&E utilization, but from the 2016 Restructuring Paper it is clear that during the MTR the program monitoring data helped identify shortcomings in the compliance with the intermediate indicator targets on major maintenance and take the decision to restructure the operation (see section 2.e above). The M&E data was also the basis to disburse against the DLIs. M&E Quality Rating Substantial 11. Other Issues a. Safeguards At appraisal, an environment and social systems assessment was carried out. Given the SRN Bridge Program’s scope, coverage, and bridge size, the adverse environmental issues and impacts related to program implementation were limited in nature and in their geographic extent. Investments in environmentally sensitive areas, such as Government listed national parks and conservation areas, were excluded from the operation (para 58 ICR). There were limited adverse impacts from land acquisition and resettlement under the SRN Bridge Program, and the adverse social impacts were temporary. This was because the area of impact was mostly confined within bridge worksites (para 59 ICR). The environmental and social safeguards performance of the SRN Bridge Program was moderately satisfactory. The DOR implemented all three PAP environmental and social actions, including updating the DOR’s Environmental and Social Management Framework (ESMF) to cater to bridge-related environmental and social issues. All other agreed actions were also complied with, including the recruitment of two OHS specialists and the preparation of a safeguards implementation summary note. Safeguard teams at the central and field levels carried out supervision and compliance monitoring of the ongoing bridge works. There were no unresolved social or environmental issues when the credit closed. However, there were shortcoming in (i) the process of timely integration of safeguards assessments and inputs into engineering designs and bidding documents, particularly in the case of design and build contracts and (ii) OHS compliance, especially in the early years of implementation, which led to two fatalities (para 60 ICR). b. Fiduciary Compliance Page 11 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NP: Bridges Program Support (P125495) At appraisal, an integrated fiduciary assessment was carried out. It assessed the fiduciary arrangements relevant to the SRN Bridge Program to determine whether they provided a reasonable assurance that the SRN Bridge Program funds would have been used for their intended purpose. Based on the assessment, it was concluded that the overall fiduciary framework for the SRN Bridge Program was adequate (para 53 ICR). The Government had a well-developed budget process, a Medium-Term Expenditure Framework and sectoral business plans, but there were inadequacies in their implementation. To address the fiduciary risks from these shortcomings, the MOF provided an indicative resource envelope for three years at a time for the SRN Bridge Program against which annual expenditure plans were prepared. The MOF also informed the DOR of the annual allocations of budgetary resources at the start of each fiscal year. This was to ensure that the DOR’s divisional offices had time to prepare the annual operating plans based on a firm assurance of resource availability (para 54 ICR). The NVC carried out integrated technical audits that examined the compliance of the SRN Bridge Program with the technical, procurement, and social and environmental standards and guidelines. The DOR referred all reliable allegations/complaints regarding corrupt practices to the Ministry of Physical Planning, Works and Transport Management (MoPPWTM). If specific instances of fraud and corruption were identified, investigations were conducted through the CIAA in accordance with Nepal’s laws and regulations (para 55 ICR). The ICR (para 56) mentioned the fiduciary performance was moderately satisfactory. The complete credit amount of SDR 38.7 million was disbursed. As seen above in section 9.b above, the DOR submitted the financial management reports on time. However, there were delays in conducting the internal financial audits, and some audit reports were submitted with delays. The ICR did not specifically comment on the procurement performance under this operation, except for mentioning the DOR had adequate capacity to manage procurement processes efficiently (para 99.h ICR). c. Unintended impacts (Positive or Negative) --- d. Other --- 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Quality of ICR Substantial --- Quality of M&E Substantial --- Bank Performance Satisfactory --- Outcome Satisfactory --- Page 12 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NP: Bridges Program Support (P125495) Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons The following are the key lessons from the ICR (paras 99 and 100) with some adjustment in language and content by the ICR reviewer: For the successful use of the PforR instrument, the Government vision, the timing, and the availability of adequate planning and monitoring tools matter. In this case, the PforR instrument was effective in supporting the DOR’s bridge program, which had already been part of the Government’s vision and plan to improve Nepal’s vital bridge sector. However, this program was not effective due to several issues, including a non-transparent bridge prioritization process and poor construction quality. The RSDP had helped the DOR develop the BMS, which could be used for program planning from the beginning. It was also used for program monitoring. Therefore, it was an opportune moment for the Bank to support the DOR’s bridge program. Even if the implementation agency has extensive World Bank experience, comprehensive training at the beginning of program implementation is important if the PforR is a new instrument for the agency. The DOR had extensive experience in working with the Bank, and it was considered capable of managing fiduciary, social and environmental, and technical aspects of the SRN Bridge Program. However, the PforR was a new instrument with different processes and mechanisms and it took time to adapt to it. The DOR’s procurement capacity contributed to the efficient program implementation, but there were issues in managing works quality effectively given the poor performance of the field officials and weak control structures. The SRN Bridge Program initially also suffered delays due to the client’s limited understanding of the PforR requirements, especially for financial management, budgeting, managing social and environmental risks, and planning of the Program interventions. Providing comprehensive training from the outset could have avoided problems and saved time. Elevating OHS standards to internationally accepted level is challenging. The SRN Bridge Program had been leading the effort to make construction sites safer. By program completion, it had made substantial progress in this regard by enforcing OHS measures on all sites and providing safety equipment to construction workers. However, the main challenge was to elevate Nepalese standards to internationally accepted/Bank standards. Fines and penalties for OHS breaches need to correspond to the expected level of performance and risk associated with the implementation of works. Inadequate fines for OHS breaches can result in inadequate performance by contractors and passing on of responsibilities. Other measures need to be properly enforced from the beginning too, such as the submission and approval of the qualifications of OHS staff and the organizational chart and ensuring that sufficient OHS officers are on-site to ensure a proactive rather than reactive approach to OHS compliance. Lack of bridge work construction quality can jeopardize the program results. Poor construction quality had been identified as a risk at appraisal and was a constant issue throughout implementation. The main reason for the poor construction quality was the lack of resources in terms of DOR supervision staff and their limited experience to supervise the construction of new bridges. The PforR instrument contributed to enhance the quality assurance mechanisms by introducing an independent verification mechanism and establishing the necessary procedures. A QAP was also introduced as one of the actions of the PAP. However, this was not Page 13 of 14 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NP: Bridges Program Support (P125495) sufficient. The issue could have been mitigated by hiring qualified consulting firms to undertake the construction supervision and provide capacity building in quality control and assurance to the DOR staff. 14. Assessment Recommended? Yes Please explain Yes, since this was the first PforR in the transport sector. 15. Comments on Quality of ICR The ICR is reasonably comprehensive and concise. Some sections, such as the parts on Risk to Development Outcome or M&E have some information gaps. The ICR is internally coherent and largely well-written. However, some information is found throughout the report rather in the pertinent section, and there are several repetitions. The quality of the evidence is adequate. It is presented in a concise way in the main text and comprehensively in the annexes. When presenting the results, strong focus was placed on the achievement of indicator targets and the compliance with actions in the PAP. The presentation of the causal links between activities, outcomes and the PDO is rather concise and has some limitations. The ICR is not fully in compliance with OPCS guidelines, but the shortcomings are minor. For instance, the ICR did not comment on the relevance of the DLIs, but it contains a more general part on DLIs in paras 30 to 32. Similarly, it did also not specifically report on the utilization of the M&E system. The ICR contains a wealth of experiences that are useful for similar operations, which is particularly valuable since this was the first transport PforR. However, several of these experiences are presented as facts and findings rather than lessons learned. a. Quality of ICR Rating Substantial Page 14 of 14