ICRR 14603 Report Number : ICRR14603 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 03/23/2015 Country : China Project ID : P082993 Appraisal Actual Project Name : Pcb Management And US$M ): Project Costs (US$M): 35.1 32.8 Disposal Demonstration Project L/C Number : Loan/ US$M): Loan /Credit (US$M): 18.34 17.65 Sector Board : Environment Cofinancing (US$M): US$M ): Cofinanciers : Board Approval Date : 12/15/2005 Closing Date : 06/30/2010 12/31/2012 Sector (s): Other industry (57%); Solid waste management (33%); Sub-national government administration (6%); Central government administration (4%) Theme (s): Pollution management and environmental health (50% - P); Other environment and natural resources management (25% - S); Environmental policies and institutions (25% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Fang Xu Roy Gilbert Christopher David IEGPS1 Nelson 2. Project Objectives and Components: a. Objectives: “To assist the Recipient in mitigating the impact of [polychlorinated biphenyls] (PCB) threats to human health and the environment by establishing a cost effective framework for cleaning up PCBs .� (Grant Agreement (TF-056008) Schedule 2, p.15, March 16, 2006). The Project Appraisal Document (PAD) uses a different formulation, namely : “to identify and demonstrate environmentally-sound and cost-effective policies, procedures and techniques for safely managing and disposing of China’s unique temporarily stored PCBs and associated PCB -contaminated wastes (e.g., PCB-contaminated soils and water).� (PAD, p.11) As a Global Environment Facility (GEF) operation, this project also had a GEF Global Environmental Objective (GEO), namely “to help China eliminate PCBs, one of the 12 Persistent Organic Pollutants [POPS] covered by the Stockholm Convention on POPS, which are harmful to human health and to the environment .� (PAD, Section B.2, Technical Annex 3, p.62). However, per IEG practice, the current review does not include an assessment of the GEF ’ s GEO, and is instead based upon the objective as stated in this operation ’s legal agreement, the GEF Trust Fund Grant Agreement: namely "to assist the Recipient in mitigating the impact of [polychlorinated biphenyls] (PCB) threats to human health and the environment ." It does not include the latter part of the objective statement "by establishing a cost effective framework for cleaning up PCBs � since this is a means of achieving the objective, not the objective itself. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: The project had six components, as follows : 1. Institutional Strengthening (Appraisal Estimate: US$1.945 million; actual cost: US$1.404 million): to strengthen Chinese institutions for PCB management and disposal in a sustainable manner by supporting : (a) an inception workshop for the project; (b) capacity building at the national level by strengthening the PCB project team and establishing a national PCB expert group in China ’s Convention Implementation Office (CIO) under the State Environmental Protection Administration of the Foreign Economic Cooperation Office (SEPA/FECO); (c) establishing a local project implementation unit (PIU) in Zhejiang Province; (d) providing PCB management training to centrally-based (that is, CIO) and provincial authorities and technical personnel; (e) supporting a study tour abroad to countries with more modern PCB disposal technologies, and (e) undertaking public awareness activities . 2. Development of a Policy Framework for PCB Management and Disposal (Appraisal Estimate: US$0.810 million; actual cost: US$0.409 million): to develop and improve the legal and regulatory framework for the safe management of PCBs in Zhejiang Province. In addition, relevant policies and regulations at the national and the local levels governing PCB management and disposal would be reviewed, revised, and supplemented, as necessary, to help China meet its obligations under the Stockholm Convention on Persistent Organic Pollutants (POPs). 3. PCB Management in the Zhejiang Province (Appraisal Estimate: US$15.016 million; actual cost: US$15.735 million): 56 sites with highly contaminated PCB wastes (greater than 500 parts per million (ppm)) would be stored temporarily in Zhejiang until transported to Shenyang for final destruction; other PCB contaminated soil and wastes (between 50 and 500 ppm) were to be treated by thermal desorption in Zhejiang . Thermal desorption is the only in-situ process that allows at least some preservation of the original properties of treated soils, thereby making possible the subsequent return of de -contaminated soils to their original locations . The cleaned-up PCB sites were to be monitored and managed until final remediation brought PCB levels down below the national standard of 50 parts per million (ppm). (China Regulation GB13015-91) This component consisted of eight activities : (i) feasibility studies on technical issues; (ii) identification of the exact location of PCB sites using Geo -radar (ground penetrating radar) technology; (iii) clean-up of PCB storage sites using strict remediation protocols; (iv) a feasibility study of using a fixed storage facility versus a mobile one would be conducted; (v) PCB wastes with concentrations from 50 to 500 ppm would be treated using a mobile thermal desorption system; (vi) PCB site cleanup and soil decontamination would be supervised by independent consultants; (vii) verification and monitoring of PCBs sites would be conducted during and after clean-up activities; and (viii) testing and decontamination of an estimated 78 large, still-operational PCB transformers imported during the 1980s. 4. Disposal of Highly -Contaminated PCB Wastes in Liaoning Province (Appraisal Estimate: US$13.609 million; actual cost: US$12.455 million): provide funding for the final destruction of highly contaminated PCB wastes (with PCB concentration greater than 500 ppm) collected from storage sites in Zhejiang Province, using an existing but not fully completed ro tary kiln incinerator in Shenyang (Liaoning Province) that would meet the requirements of the Stockholm Convention. 5. Project Monitoring and Evaluation (Appraisal Estimate: US$0.127 million; actual cost: US$0.034 million): specific activities were to include: (a) integration of PCB information in the persistent organic pesticides (POP) management information system (POPS-MIS) for the project; (b) a workshop on the monitoring system designed for PCB management in Zhejiang Province; and (c) three annual workshops to review project progress . 6. Design of a National Replication Program (Appraisal Estimate: US$3.290 million; actual cost: US$2.804 million): to develop a program to disseminate the experiences gained in Zhejiang Province to the rest of China and the world . This was to include: (a) a national workshop for review experiences gained and lessons learned from the demonstration project; (b) development of a national replication program; (c) a national workshop to discuss the draft replication program, after which the replication program will be finalized; and (d) an international workshop to disseminate the information and knowledge gained . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost : Total project costs were estimated at appraisal to be US$ 35.097 million; actual project costs reached US$32.843 million, or 93.6 percent of the amount estimated at appraisal . The lower than expected project costs and financing required was due to the decreased number of sites and actual contamination levels actually requiring remediation, as revealed in subsequent detailed investigations during project implementation that could only be determined once clean-up efforts had commenced. An initial joint Chinese-Italian study tentatively identified (56) sites potentially contaminated by PCBs, but further investigation using geo -radar (ground penetrating radar) technology revealed that only 35 sites were possibly contaminated . Of these, contamination was below levels posing any health risks at 15 sites, and another four sites were inaccessible due to existing roads and buildings . Those sites continue to be monitored. Thus, only 16 sites required remediation. See Table 2-4 in Annex 2 of the ICR for more details (p. 33). Project Financing : GEF project financing, all in the form of a grant, was estimated at appraisal to be US$ 19.34 million, while actual GEF project financing was US$ 17.65 million. In addition, an Italian project to develop a PCB inventory methodology and PCB disposal strategy and other co -financing support provided by the Japanese and American governments contributed US$ 1.355 million out of an appraisal estimate of US$ 2.024 to the project. Borrower Contribution : The borrower’s estimated contribution at appraisal from both the national and provincial governments was US$14.733 million. The actual contribution was US$ 13.838 million (nearly 94 percent) of the estimate due to the decreased number of sites actually contaminated by PCBs (mentioned above). This supported national and provincial Project team operations, the development of a sound framework of policies, regulations and standards, as well as the design of a national replication program and its initial operation . Dates : The project became effective on June 29, 2006. The project was restructured four times (the first with the Regional Vice President’s approval and the last three with the approval of the Country Director ), and extended three times for a total of 2½ years on the following dates for the following reasons : • On Nov. 21, 2006 the project was restructured to change the disbursement modality from a Statement of Expenses (SOE) mode to the Financial Monitoring Report (FMR) mode as part of the Bank’s move toward the FMR approach. This project restructuring was approved by the Acting Country Director of the Bank, but there was no extension of the project at that time. • On June 28, 2010 the project was extended six months until December 31, 2010 to allow sufficient time to resolve key bottlenecks requiring Government approvals related to additional activities of transporting highly -contaminated PCB wastes from Zhejiang to Shenyang, and obtaining licenses from the Ministry of Environment to operate the thermal desorption unit (TDU) for treatment of low-level PCB-contaminated soils in-situ and the Shenyang incinerator for highly-contaminated PCB wastes from remote sites . • On December 29, 2010 the project was extended by one year to December 31, 2011 in order to reallocate US$2,266,000 or 12% percent of Grant proceeds from Works to Goods, and to replace the media -specific technical standards for PCBs with two new intermediate results indicators (the issuance of national regulations on PCB management and disposal, and national PCB pollution control standards ). In addition, the number of PCB storage sites requiring clean-up (estimated at 56) was revised to encompass ‘all’ temporary PCB storage sites, albeit without attaching a specific number to them . Finally, US$985,000 was earmarked to decontaminate 10 large, operating PCB electrical transformers installed before 1980 and test additional large transformers . These funds were reallocated to transport highly-contaminated PCB wastes from Zhejiang Province to the Shenyang incinerator . • Finally, on December 13, 2011 the project was extended again for one year until December 31, 2012 in order to allow the clean-up of remaining sites, and to modify intermediate results targets to reflect the volume of tons of PCB-containing capacitors, highly -contaminated PCB waste, and low-contaminated soils to be remediated . This project restructuring was approved by the Regional Vice -President for the Bank since the cumulative extension of closing dates exceeded 2 years. At Project completion, 96% of project funds had been disbursed . Six months later at the end of disbursement period, US$ 689,722 was cancelled on June 13, 2012. 3. Relevance of Objectives & Design: a. Relevance of Objectives: Rating : Substantial . The project’s objective was substantially relevant to the Country Partnership Strategy (CPS) for 2006-2010, in effect at the initiation of the project. It remains relevant to the current CPS 2013-2016, which explicitly maintains the previous CPS’s focus on this issue under outcome 1.6, “Demonstrating Pollution Management Measures .� This outcome seeks to continue supporting efforts to reduce hazardous waste, specifically by reducing the presence of persistent organic pollutants (POPs) in the environment. The project’s objective also serves the Government ’s priority for observing international environmental agreements, such as the Stockholm Convention on POPs, by reducing exposure to PCB contamination, failing to do which can result in damage to human health and the environment . b. Relevance of Design: Rating : Substantial The project’s design was well-aligned to achieve its objective since the logical link between inputs, activities and components, all responded to the major challenges (“main sector issues�) posed by PCB contamination. These were enumerated in the PAD (pp.8-10) as: (a) inadequate and incomplete baseline data in China on PCB contamination; (b) weak PCB management and disposal capacity; (c) significant health and environmental risks from PCB contaminated soils and water; (d) inadequate regulatory policies and standards along with a lack of credible enforcement; and (e) insufficient public awareness and education about the risks of PCBs . Even though the Results Indicators in the Results Framework were tied to each component (instead to the two aspects of the PDO), these components were logically linked in a causal chain to the PDO (although they were stated differently in the Results Framework of the PAD than they were in the GEF Grant Agreement ). The linkage between the activities, outputs and outcomes was clearly and persuasively reflected in the project ’s design and structure, which mirrored the main sector issues that needed to be addressed effectively in order to reduce exposure to POPs. Exogenous factors and other parallel, on -going work in China at that time addressing issues surrounding many types of POPs, including PCBs, were taken into consideration and incorporated into the design of this project . Funding levels were all well-estimated and most of the delays and other problems encountered during implementation were due more to flaws in preparation or lack of experience among the implementing partners than they were to poor project design . 4. Achievement of Objectives (Efficacy): Rating : Substantial “to assist the Recipient in mitigating the impact of PCB threats to human health and the environment � Evidence of the achievement of this objective includes the following : • The PCB baseline inventory identified by a predecessor Italian project included 43 PCB-contaminated sites with an additional 18 sites suspected of containing PCBs . However, of these 61 total sites suspected of requiring remediation at appraisal, only 35 sites were later identified as actually being contaminated using more accurate field visits and ground penetrating radar technology . And of these 35 sites, only 16 sites eventually required remediation since they were contaminated and accessible; the other 19 did not need to be addressed either due to having low levels of PCB present, or because it was not feasible to access those sites as they had subsequently been buried under new roads or buildings. At the 19 sites that did require remediation, there were three principle sources of PCB risk to human health and the environment identified in the PAD and ICR that were to be mitigated by project interventions . The actions taken at these sites and the resulting mitigation of PCB contamination included the following : • The original targets for decontaminating 10 large transformers and 250 tons of PCB-containing oils found inside transformers to be incinerated were both adjusted and then later dropped altogether since no large transformers remained in-service by the time the project started, and because PCB oils had already leaked out of capacitors and mixed with surrounding soils, making measurement and treatment of PCB oils impractical . The targets were formally revised and dropped in amendments to the GEF Grant Agreement in December 2010 and 2011. • At appraisal, roughly 2,000 tons of highly-contaminated soils (with concentrations exceeding 500 ppm of PCB) in Zhejiang province were to be transported and incinerated at the Shenyang facility . This was revised in the December 2010 project restructuring to reflect the reality that the capacitors were badly degraded and broken in most cases, and varied in size . The unit of measurement was changed from the number of capacitors to tons of capacitor-containing hazardous waste material to 1,200 tons. By the closing of the project on December 31, 2012 over 1,040 tons had been treated (87% of target). An additional 420.8 tons were treated during the six -month period following project closure using counterpart funds only . • 20,000 tons of less contaminated soils (between 50 to 500 ppm of PCB) were to be treated using a mobile thermal desorption unit (TDU) to reduce PCB concentration levels below the national and Stockholm Convention standard of 50 ppm. This target was reduced to 10,000 tons in the December 2011 restructuring after more accurate field testing revealed much lower amounts . By the close of the project, 8,572 tons had been treated by the TDU, and an additional 2,856 tons were treated in the six months after project closure using counterpart funding only . In addition, there were a number of project activities and outputs that resulted in outcomes that contributed to establishing a cost-effective framework for the continued clean -up of remaining PCBs in China in the future . These included: (i) upgrading the existing in-country rotary kiln incinerator in Shenyang to meet Stockholm Convention standards for pre-treatment of wastes, and controlling and continuously monitoring dioxin and furan emissions; (ii) extensive training of technical staff at both the national and provincial levels to strengthen internal capacities to address remaining risks from PCBs; (iii) increased monitoring capacity to prevent or reduce risks from future accidental releases of PCBs in Zhejiang Province; (iv) a national strategy for PCB reduction and disposal as well as regulations and pollution control technical standards governing PCB waste management and disposal practices; and (v) the scaling-up of PCB management activities in 12 other provinces. The new strategy, regulations, and standards served to inform the design of the most recent national Five -Year Plan with regard to POPs, and have allowed China to substantially meet its commitments under the Stockholm Convention by directly supporting its National Implementation Plan (NIP) priorities. 5. Efficiency: Efficiency is rated modest . There was no ex-ante or ex-post economic or financial cost-benefit analysis conducted for the Project. This was due to what the ICR (p.37) described as a “lack of reliable baseline data and difficulties associated in measuring the impact of the overall environmental, health and related socio -economic impacts under a ‘without project’ scenario.� Instead, an incremental cost analysis was conducted at project appraisal and ICR stage to understand the additional cost associated with the project, the incremental cost was defined as “a measure of the future economic burden on the country that would result from its choosing the GEF Alternative in preference to the course of action that would have been sufficient in the national interest � (GEF / C.7 / Inf. 5, para. 2.1). From the definition, one can tell that the incremental cost is not a measure of cost effectiveness or efficiency . For this specific project, the incremental cost was calculated as US$ 31.5 million. The ICR (p.17) stated that the incremental cost analysis demonstrated that the Project, compared to the “without project� scenario, would be an “efficient means� to assist the government of China fulfill its obligation under the Stockholm Convention to better manage its PCB, � However, this conclusion was not substantiated by data or experience . Both the estimated cost of thermal desorption treatment (US$270/ton) to treat low-concentration levels of PCB-contaminated wastes and soils turned out to be 70 percent higher than projected (US$460/ton), and estimated costs of incineration at appraisal (US$2,000/ton), also turned out to be much higher (87%) than projected with a weighted average cost of US$ 3,723/ton. Part of these cost increases, measured in US dollars, reflects the US dollar devaluation against the local currency during project implementation . It was argued (ICR, p. 17) that, “Assessing the average cost/ton [of PCB] clean-up is therefore, not a good indicator of efficiency . Comparison with PCB project results in other countries also does not yield useful comparative data .� Both of these assertions were unconvincing, and no other persuasive measure to assess efficiency was provided . For example, it was stated that the estimated average cost per ton of clean -up was based on “European/North American data,� but the PAD (p.21) made it clear that this was not the estimated cost of incineration, but rather an alternative to the selected one that was rejected because it “entails unacceptable risks and expenses [of trans-oceanic transport].� The project’s execution was extended three times for a total of 30 months (lasting 6.7 years instead of 4.3 years before closing). One of these delays involved the licensing of a single source contractor, the Hangzhou Dadi Environmental Protection Co. Ltd. (DADI), selected to recover PCB wastes, treat contaminated soils and decontaminate PCB-contaminated transformers based on previous experience . However, technical requirements under the “Measures for Administration of Hazardous Waste (HW) Operation Permits � (2004) “appear not to have been fully understood at preparation, � creating persistent implementation delays and requiring extensive inter-ministerial consultations that impeded progress on issuance of the license for five years until 2010. Similarly, the Shenyang Center also faced delays in obtaining a license to operate, but this issue was more easily resolved . Finally, there were also problems with the initial site location of the thermal desorption unit (TDU): (i) due to local government and civil opposition, (ii) the TDU procurement process had to be re -bid, which delayed implementation by many months and, then (iii) persistent technical problems with the TDU that resulted in delays in obtaining completion of the required Proof of Performance test . All of these problems “plagued aspects of its [the project’s] implementation� (ICR, p.15). ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: The Relevance of Objectives was assessed as substantial because the PDO was consistent with the Country Partnership Strategy of 2013-2016, as well as China’s international legal obligations as a signatory to the Stockholm Convention on Persistent Organic Pollutants (POPs). The Relevance of Design was also assessed as substantial since the resources, structure, and activities were consistent with achieving the desired outputs, outcomes and project objective. Efficacy was rated as substantial since the project helped China make significant progress to mitigate the impact of PCB threats to human health and the environment . However, Efficiency was assessed as modest due to unnecessary delays caused by poor execution, and higher than expected costs to identify, transport, treat and dispose of PCV-contaminated wastes and soils in an environmentally safe manner . Taken altogether, these ratings result in an overall Outcome rating of moderately satisfactory . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The project helped develop China ’s national and provincial technical and administrative capacities as well as inter-governmental coordination, created new partnerships with scientific and academic knowledge centers and project partners, and enhanced public awareness and support for actions that need to be taken to effectively control the release of such substances in the future . At the time of project completion, results and lessons learned from the demonstration project were being applied to scale -up PCB management activities in 12 other PCB-target provinces, and had informed the design of the nation ’s latest Five-Year POPs Plan, which emphasized decreasing the use of hazardous, persistent pollutants and strengthening economic and environmental governance for their effective management. However, the risk to consolidating the gains already made remains significant given that the national replication program supported by a sustainable financing strategy, whose effective adoption are both key to the ongoing sustainability of PCB management efforts in China, have not yet been operationalized . One of the chief reasons for this lack of energetic replication was the project ’s inability to demonstrate the economic or technological viability of the remediation technologies employed (incineration of highly-contaminated PCB wastes to international standards and the use of thermal desorption for low -concentration wastes). The Shenyang rotary kiln incinerator was upgraded to Stockholm Convention standards, but operating costs turned out to be 87% higher than estimated. Given uncertainty over what emission standard would eventually apply in China for dioxin and furan, this raised concerns that the stricter emission standards which the Shenyang incinerator was designed to reach made it less cost-competitive compared to other similar incineration facilities in China . In addition, the thermal desorption unit (TDU) proved to be “unstable and prone to frequent break -downs� (ICR, p. 37). For example, it took 25 trial runs for the TDU to pass the Proof of Performance test required to ensure that the process was reaching the specifications required under the Stockholm Convention . The costs for the TDU were also nearly twice that at appraisal, and resulted in the Bank’s rejection of the first bidding results . The ensuing negotiations and re -bidding process resulted in a three-year delay before the TDU was finally installed in September 2010. a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: On the positive side, the Bank identified the key issues and constraints to achieving the ultimate goal of helping China deal with the risks posed to the environment and human health by the presence of inadequately stored PCB wastes, and wisely chose to adopt a demonstration project approach in one province (Zhejiang Province). Based on previous experience, the Bank also chose to take a comprehensive approach to this serious problem by incorporating legal and regulatory reforms, providing extensive training at both the central and provincial government levels, and by conducting public outreach and awareness -building efforts. Two issues affected the project implementation . One involved using a single source contracting mechanism for Project contractors: the Hangzhou Dadi Environmental Protection Co . Ltd. (DADI) to install and operate the thermal desorption unit (TDU) for low-concentration PCB wastes, and the Shenyang Center to incinerate high-concentration wastes. It was noted that at the project preparation stage, the State Environmental Protection Administration had confirmed that Dadi would be the 2nd waste management company in China to be granted an official license for handling PCB wastes, excluding final disposal . However, during the implementation, there was a lack of clarity surrounding the licensing requirements for DADI that proved to be very detrimental to project implementation. The second issue involved inaccurate estimates regarding the location and ease of accessing, type and quantity of temporary storage sites of PCB -waste materials and soils contaminated by leaking PCB ‘coffins. Initial estimates placed these quantities at much higher levels and concentrations than what was subsequently verified. Consequently, the project had to rescale the remediation facilities and amend targets in the Grant Agreement during project implementation . The project team clarified that the nature of ‘storage’ of PCB capacitors in China was unique (e.g., compared to other countries with ongoing similar interventions (e.g., Moldova and Belarus), and the estimates were based on 20-30 year old records from the power companies, as well as interviews with power companies' employees . This review took note that the above issues could not easily be foreseen at the project preparation stage. Nevertheless, given the vital role these entities were to play in implementation, the Bank should have paid greater attention to the potential risks . at -Entry Rating : Quality -at- Moderately Satisfactory b. Quality of supervision: Problems encountered during implementation were directly attributable to contracting oversights and imprecise estimates made during preparation and appraisal of the project . Thus, the subsequent repercussions of these decisions and actions later during implementation will not be repeated here and ‘double-counted’ in assessing the quality of supervision . On the positive side, the Bank team provided Government and other Project counterparts with consistent, timely and sound policy and technical advice, including guidance on financial management, procurement, and environmental and social safeguard issues through regular supervision missions . The Bank was able to develop more accurate estimates on the location, extent, and feasibility of accessing PCB -contaminated sites once project implementation began by using geo -radar (ground penetrating radar) technology. Accordingly, it made refinements to the Results Framework targets in two Level 2 restructuring on 12/27/2010 and 12/13/2011. However, there was a lapse in project supervision by the Bank for an 18-month period until mid-2010, during which time no ISR ratings or aide memoires were filed (ICR, p.20). The reason for such lapse was not provided by the ICR though. The ICR did mention that in response to concerns raised by past supervision missions, as well as by the Mid-Term Review Report, Bank management initiated a new approach involving a change of TTL and more intensive project implementation support and supervision which consequently improved the results reporting and supervision proficiency . The project team adjusted the project result framework twice through the project restructuring. However, the two outcome indicators which were not directly related to the project ’s main objective (details of the two outcome indicators are explained in Section 10) were not revised. Quality of Supervision Rating : Moderately Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Government contributed US$13.838 million or 94 percent of its estimated counterpart financial commitment of US$14.733 million. It also demonstrated its commitment to the project by ensuring the coordination of various government entities at the central level and between the central and provincial level to implement the project. The Government agencies also developed a comprehensive policy framework which has eight technical standards and guidelines and six policy regulations to cover PCBs among many other types of hazardous chemicals/substances. The whole framework was completed four months after project completion in April 2013. However, the prolonged process associated with issuing a license to contractors to handle, transport, and treat PCB wastes revealed that the communication between government agencies should be improved, the policy and regulation in this area should be clarified . Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance: The Implementing Agencies were the Ministry of Environmental Protection (MEP) and the Environmental Protection Department (EPD) in Zhejiang Province. MEP was responsible for coordinating and managing project activities and progress at the central government level; EPD was responsible for managing project activities at the provincial level and provided technically qualified staff to ensure implementation of project activities through environmental protection bureaus in another 12 provinces, and helped develop provincial policies, regulations, and standards. MEP established a PCB Project team in Beijing within the Convention Implementation Office of the Foreign Economic Cooperation Office (FECO/CIO) to manage and monitor all central level Project operations, as the FECO/CIO was familiar with Bank operations and procedures, given their previous experience managing the Montreal Protocol Multilateral Fund-funded projects, as well as other GEF -funded initiatives. The staff of Project Implementation Unit (PIU) did not have experience working with Bank operations, though Zhejiang Province did have previous experiences with the Bank . It took time for the PIU to become familiar with the Bank’s policies and procedure so as to manage the project in a more effective way . It did provide financing resources and technically qualified staff to ensure implementation success and the PIU performed a good job in developing the provincial policy, regulations and standards required to sustain the eventual elimination of PCBs . Implementing Agency Performance Rating : Moderately Satisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The project monitoring and evaluation (M&E) system aimed to serve the dual functions of tracking progress toward achieving the project objectives, and gathering the information necessary to prepare the national replication program (PAD, p.26). The project’s development objective and indicators were clearly stated in the Results Framework . However, the framework was poorly designed because the project ’s two outcome Indicators were not directly related to the project ’ s main objective of mitigating risks to the environment and human health from PCBs, and demonstrating their cost-effectiveness. Rather, they measured the extent to which lessons learned were disseminated, and national PCB policy/regulations were updated (‘refined’) within a broader regulation governing hazardous chemical substances . The M&E framework also contained 23 intermediate outcome indicators, several of them, including establishing the national guidelines and regulations for PCB cleanup, were relevant to measuring the achievement of the project development objective, but the majority of the so called intermediate outcome indicators were really output indicators measuring only the delivery of the project components . b. M&E Implementation: M&E reporting on activities and results was performed by CIO /FECO and the Zhejiang PIU through semi -annual project progress reports. Standard monitoring and reporting formats were designed by the PCB project team in the Convention Implementation Office (CIO/FECO). Key baseline information was collected from an Italian project that had conducted an inventory of all PCB capacitors and transformers buried in temporary PCB storage sites in Zhejiang province. However, site specific data was not introduced into the national POPs -MIS until 2010 since the system was delayed and not put into use until then . Project activities and outputs, such as capacity -building training for provincial and central government agency staff, number of PCB policies, regulations and standards developed, and public outreach and awareness -building materials, were systematically measured by the project . Several targets were revised downward because of the decrease in the actual number of sites requiring clean -up. The ICR reports that data collected appeared to be accurate and of good quality, beneficiaries were involved, and had sufficient stakeholder involvement and ‘buy-in’ to be sustainable in future PCB clean -up actions. c. M&E Utilization: No information was provided in the ICR regarding the extent to which M&E data was used by project managers to inform the strategic direction of the project, the allocation of resources, or alter tactical decisions mid-stream in project implementation, or if changes in the project ’s direction or outcomes could be attributed to feedback from the M&E system. M&E Quality Rating : Modest 11. Other Issues a. Safeguards: The project complied with all Bank environmental and social Safeguard Policies, and set up an adequate monitoring and tracking system, which the Bank approved . This operation was classified as a Category A project . It triggered the Bank’s Environmental Assessment Safeguard Policy (OP 4.01) due to potential environmental impacts from remediating highly toxic chemicals at contaminated storage sites. Two environmental assessments (EAs) were initially prepared, followed by three additional EAs during implementation, and an environmental management plan was put into place . Pursuant to Project Grant Agreement, the Project Implementation Unit in Zhejiang province and the project contractors (DADI and Shenyang Center) implemented the Project’s Environmental Management Plan (EMP) in accordance with sound environmental practices and standards . Due diligence protocols were developed and focused technical training programmed to adequately apply international standards and requirements for storing, transporting, and disposing of PCB-contaminated wastes and soils with the intent of minimizing undue risks to the environment and people living in surrounding communities. Continuous monitoring is performed for the four contaminated sites that could not be cleaned up because they are either under existing buildings or busy roads, The Bank’s Social Safeguard Policy on Involuntary Resettlement (OP 4.12) was also triggered, and the project ’s social impacts were assessed and its mitigation measures outlined in a Resettlement Policy Framework . The project established communication channels with local residents, who were involved in all potentially controversial aspects of the project, such as the location of the thermal desorption unit, which had to be relocated as a result of local political and community opposition. The social impacts of the project were minimal with only two of the 35 sites to be cleaned up triggering the involuntary resettlement policy . In these isolated cases, the cost of land affected was fully remunerated, affected enterprises, infrastructure and public facilities were fully compensated, and the livelihoods of those affected were restored . Based on this evidence, implementation of the Project ’s resettlement mitigation plan is deemed satisfactory. b. Fiduciary Compliance: The Project created the required special project account and prepared financial statements in a timely manner, thereby implementing the project’s financial management system in accordance with Bank regulations and Grant Agreement requirements. The change from a Statement of Expenses (SOE) disbursement modality to the Financial Monitoring Report (FMR) system was new to project implementing partners, which slowed disbursement as a result . All of the project annual audit reports were submitted to the Bank with unqualified opinions . Procurement plans were developed and shared with the Bank team for review with Bank no-objections provided in a timely manner. These procurement plans were disclosed publicly in accordance with Bank requirements . Ex-post procurement reviews conducted by the Bank found that the procurement of works and goods, as well as the selection of consultants, was carried out in a satisfactory manner in accordance with the Bank ’s Procurement and Consultants Guidelines. However, the Bank did reject the initial award of the contract for the thermal desorption unit made by the implementing partners due to concerns regarding the financial and technical viability of the supplier selected (DADI). This decision delayed implementation by more than one year, and significantly raised the total cost of equipment procured. c. Unintended Impacts (positive or negative): No significant unintended impacts were encountered during project implementation . d. Other: None. 12. Ratings : 12. ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Satisfactory Satisfactory Risk to Development Moderate Significant The National Replication Program and Outcome : a sustainable financing strategy have not yet been operationalized . Bank Performance : Moderately Moderately Satisfactory Satisfactory Borrower Performance : Moderately Moderately Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: Several lessons were drawn by the ICR, centering on how to better design, implement and utilize the M&E of the demonstration project modality. The key lessons were summarized below : • Design: set more conservative target for a demonstration project . For this specific project case, the number of sites contaminated, the extent of site specific contamination and the level of clean -up required, is either difficult to determine or simply not known. During implementation, the number of sites to be cleaned had to be adjusted downwards because of various reasons . The design should avoid ambitious targets, which can require time-consuming restructuring during implementation . • Implementation: In projects where legal dispensation is required to carry out work (licensing/permitting), it is vital that the relevant legal requirements be clearly understood and the centers of authorization engaged from the outset of a project. In this specific case, the project implementation delay could have been avoided if the licensing authorities had been more familiar with the project . • Utilization: when practices/methods used in the demonstration project proved to be effective, it is key to work out a clear plan for replicating the practice /method at the national level, and the national government should ensure the implementation of such plan, so as to maximize the benefit of the demonstration project . For this specific case, after the pilot project proved to be effective in cleaning up PCB, the national government could follow up with a comprehensive and properly funded national replication PCB cleanup plan to benefit from the project investment . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR was well-organized and covered all of the required fields. It documented the implementation challenges encountered, and the reasons for the ensuing setbacks and delays. However, a few project achievements were characterized by the report in a more output-oriented than results-based manner. This was the case of not discussing the reduction in risk achieved as a result of addressing the three principle sources of PCB contamination . At times, some information was presented in confusing ways at different points in the text, requiring IEG to make additional inquiries. These covered the status of large transformers or the PCB-containing oils inside transformers, the confusion over the number of sites identified by the original inventory study. Overall, the quality of this ICR is assessed as being Satisfactory albeit with the above mentioned shortcomings . a.Quality of ICR Rating : Satisfactory