ICRR 14004 Report Number : ICRR14004 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 06/21/2013 Country : East Asia and Pacific Project ID : P079610 Appraisal Actual Project Name : Livestock Waste US$M ): Project Costs (US$M): 24.71 25.84 Management In East Asia Project L/C Number : Loan/ US$M): Loan /Credit (US$M): 0.0 0.0 Sector Board : Agriculture and Rural Cofinancing (US$M): US$M ): 8.2 9.0 Development Cofinanciers : GEF, Global Methane Board Approval Date : 03/31/2006 Initiative, FAO, Project Closing Date : 12/31/2010 12/31/2011 Preparation Facility Sector (s): Animal production (69%); Central government administration (20%); Sub-national government administration (10%); Agricultural extension and research (1%) Theme (s): Pollution management and environmental health (40% - P); Rural policies and institutions (40% - P); Rural services and infrastructure (20% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Hassan Wally Robert Mark Lacey Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: According to the Project Appraisal Document (PAD, p. 73) the Project Development Objective was : "to reduce the major negative environmental and health impacts of rapidly increasing concentrated livestock production on the water bodies of and thus on the people of East Asia ." The Global Environment Objective (PAD, p. 73) was: "to reduce livestock-induced, land-based pollution and environmental degradation of the South China Sea ." The GEF Trust Fund Grant Agreements for China, Thailand and Vietnam all stated that the project objective was : "to reduce: (i) the negative environmental and health impacts of livestock-induced pollution on water bodies; and (ii) the environmental degradation of the South China Sea." In this review IEG will use the objective as stated in the GEF Trust Fund Agreements . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: 1. Livestock Waste Management Technology Demonstration (Appraisal Cost : US$14 14 .2 million, actual cost US$ 14. US$1717 .89 million ). This includes two sub-components: US$ 17. (a) Technology Demonstration : to demonstrate technically, geographically, economically and institutionally workable solutions to reduce regionally critical livestock waste pollution caused by industrial or concentrated livestock production. The livestock waste management strategies would focus on reducing excess nutrients (nitrates and phosphates in particular ) and human health risks. The methods to be used would include : (a) reducing through better feeding practices the volume of nutrient emission; (b) returning nutrients to the crop cycle locally or in other areas once processed and packaged; (c) converting the nutrients to plant -available forms; (d) destroying the nutrients; and (e) taking measures to minimize potential human health risks associated with livestock waste management practices . The actual physical demonstration of improved livestock waste management would be carried out on farms or in villages (in Vietnam) proposed on a yearly basis by each participating country according to agreed selection criteria . (b) Training and Extension : would finance consultant services and training, and finance goods and civil works (through sub-grants) related to the development and construction of cost -effective and replicable livestock waste management systems and facilities and the implementation of effective waste management approaches in areas with a high concentration of intensive pig farms . The demonstration activities would be supported by training and extension to provide: (a) farmers with the essential skills and technical support needed to improve their on -farm manure management practices and (b) training for capacity building. Activities would include training of livestock extension agents, farmer associations and planning officers, study tours for participating farmers on demonstration farms, preparation of training manuals, collaboration with livestock extension projects, etc . Details are specified in the master capacity building development plan prepared by each participating country . 78million ). This 2. Policy and Replication Strategy Development (Appraisal cost US$ 4.4 million, actual cost US$ 2.78million includes two sub-components: (a) Policy Development and Testing : would finance consultant services, training and workshops to support the establishment of a policy and regulatory framework for environmentally sustainable development of livestock production in each country that would induce further policy reforms and encourage farmers to adopt improved manure management practices. This would be achieved through the development and testing of a replication strategy and other policy measures in each country . Replication potential of alternative livestock waste management technologies as related to farm scale, affordability, operational capacity, material availability, reduction of public health risks and compatibility with the waste handling methods of the local farm communities would be assessed to achieve widespread replication of the tested manure management practices . Specific activities may vary among participating countries, but the focus would remain on addressing waste management through: (a) development and introduction of codes of practice; and (b) development and implementation of policy measures to direct the geographic focus of future intensive livestock production . Both would be coordinated with respective national legislation programs and tested in synergy with the Livestock Waste Management Technology Demonstration and Project Management and Monitoring components . Other activities would include: (a) the review and revision of existing regulations; and (b) the development and introduction of livestock waste recycling and discharge standards . Specific policy packages would be tested in sub -national jurisdictions and testing experience would be incorporated in the finalized respective Replication Strategy . (b) Awareness Raising : would support activities to raise awareness on development, testing and implementation of the replication strategy focusing on policy measures, environment and public health issues associated with inadequate manure management . 3. Project Management and Monitoring (Appraisal cost US$ 3.9 million, actual cost US$ 2.96 million ). This includes two sub-components: (a) Project Management : would finance consultant services, training, office equipment and incremental operating costs to support efficient project management . The project would support the establishment and operations of a national (provincial in China) Project Management Office (PMO) in each participating country as the secretariat of, and reporting directly to, the respective National Steering Committee . The PMO, comprising a project director supported by competent staff and based on existing administrative structure and physically located within the main implementing agency of each participating country, would be responsible for day -to-day project administration. Institutional capacity, monitoring and evaluation skills of the implementing agencies at local levels would also be strengthened . (b) Monitoring and Evaluation : would support effective project monitoring and evaluation of the social, economic, environmental, human health risks and other changes brought about by the project, and the dissemination of project outcomes within the respective participating country . Monitoring on human health risks associated with the project would focus on measures taken to minimize potential transmission of pathogens, antibiotics and their resistant strains from livestock to humans . Specific activities would be detailed in project monitoring and evaluation plans developed by each participating country . 4. Regional Support Services (Appraisal cost US$ 1.5 million, actual cost US$ 1.51 million ). This includes two sub-components: (a) Capacity Building Support : would finance consultant services, training, workshop, office equipment and incremental operating costs to provide : (a) capacity building support to strengthen the participating countries ’ institutional capacity in project implementation; and (b) regional coordination and facilitation support to ensure regional coordination and achieve cross -country synergies and regional replication . Also, would respond to the participating countries’ need for an easily accessible source of support for capacity building, including support for: (a) decision tools development; (b) evaluation of project activities and outcomes; and (c) development of training modules and packages . (b) Coordination and Facilitation : would focus on regional coordination, facilitation amongst the three participating countries and the dissemination of project outcomes, decision support tools, technical guidelines and standards within the three participating countries and to other countries bordering the South China Sea . d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost . According to the PAD (Annex 5) the total project cost at appraisal was estimated to be US$ 24 million, of which US$7.5 million was for activities in China, US$7.51 million for activities in each of Thailand and Vietnam, FAO would receive US$1.0 million and US$0.70 million for the project preparation facility . Financing . According to the PAD (p. 5) the project would be financed by a stand -alone US$ 7.0 million GEF Grant and FAO would contribute US$0.5 million of counterpart funds. The actual amount disbursed was US$ 6.86 million (project portal). The project also received US$ 0.91 from the Global Methane Initiative (GMI) and FAO contributed US$0.53 million (ICR, Annex 1). Recipient Contribution . At appraisal the Recipients were expected to contribute US$ 16.51 million of counterpart funds. The ICR (Annex 1) reported that the borrower governments contributed a total of US$ 10.11 million of counterpart funding while the private sector (pig farmers financed by the project ) contributed US$6.67 million. Dates . The project was scheduled to close on 12/31/2010. The closing date was extended by 6 months for both Vietnam and FAO and by 12 months for Thailand. The ICR (p. 3) reported that slower than expected implementation was the main reason for the extensions granted for Vietnam and Thailand . Delays in Thailand stemmed from the distraction of the project implementation agency due to participation in the preparation of other projects; delays in approving counterpart funding by the Thai government and social unrest in April and May 2010. Delays in Vietnam stemmed from a change in project areas; major restructuring within the implementing agency in 2008; severe storms in 2009 and 2010; and weak capacity of the Project Management Office in first years of the project ’s implementation. The closing date was extended for FAO to accommodate the organization of the final project completion workshop and last Regional Coordination Group meeting in June 2011, once all three countries generated sufficient implementation results to share with each other . 3. Relevance of Objectives & Design: a. Relevance of Objectives: High. At closure, the objectives remain relevant as the project addresses the environmentally unsustainable and High geographically-concentrated intensive livestock production in China, Thailand and Vietnam, which represents a significant and rapidly growing source of land based pollution of the South China Sea . The project objectives remain in line with the Bank's China Partnership Strategy (FY13-FY16) which calls among other things for supporting greener growth and advancing mutually beneficial relations with the world . Objectives also remain in line with the Bank's 2010 Interim Strategy Note for Thailand, which called among other things for supporting environmentally and socially sustainable growth and the Bank's Vietnam Country Partnership Strategy (FY12-FY16), which calls among other things for environmental and natural resource sustainability . The project objectives were also relevant to the Bank ’s Country Assistance Strategies (CASs) at appraisal in China, Thailand and Vietnam, reflecting the need for rapid economic growth that is environmentally sustainable . In the Bank's 2002 China (CAS) protecting the environment is an overarching objective for support by the Bank to sustain rural income growth while maintaining the natural resource base . The CAS also aimed to assist the government in poverty reduction and support investments in environmentally sustainable agricultural and livestock development . The Bank's 2003 Thailand CAS aimed to support the government by complementing its development partnership at the country level with work on regional and global public goods . The Bank's 2002 Vietnam CAS and 2004 CAS progress report called among other things for an equitable, socially inclusive and sustainable pattern of growth . The project’s regional approach is relevant to the two regional GEF Action Programs by ensuring that the region ’s three most important countries, in terms of livestock production and waste pollution, are all involved while emphasizing their common interest in protecting the ecosystems of the South China Sea and promoting important cross-country synergies; and replicating experience from the project demonstration throughout the region (PAD, p. 5). The objectives were also relevant to the strategic priorities of the GEF ’s International Waters (IW) Focal Area, in particular IW-1 which sought to catalyze financial resource mobilization for the implementation of reforms and which stressed reduction measures through elaboration of Trans -boundary Diagnostic Analysis and Strategic Action Plans, as well as IW-3, which promoted innovative demonstration of barrier removal to sustainable industrial livestock management (ICR, p. 1). The project objectives are also in line with the GEF contaminant -based operational program and with the objectives and eligibility criteria of the proposed GEF -World Bank Strategic Partnership Investment Fund for Land-based Pollution Reduction in the Large Marine Ecosystems of East Asia . The project would also contribute to the objective of the GEF Focal Area of climate change and OP 2 Costal, Marine, and Freshwater Ecosystems (ICR, p. 2). b. Relevance of Design: Substantial . Overall, project components described intended activities and inputs and the results chain was logical and relevant. To achieve the development objectives, design featured a comprehensive approach that would address waste management through technological solutions, policy development and enforcement and regional synergy among participating countries . Design called for promotion and demonstration of effective and affordable livestock waste management technologies selectively in each of the three participating countries . Such approach could be expected to increase the likelihood of replicating technologies in project countries . Design also emphasized the importance of providing financial incentives and technical support to encourage livestock farms to pilot sound waste management technologies . Design could have benefited from a mechanism that linked project demonstrated technologies to policy development and enforcement . To maximize contribution to the GEF objectives, design called for a regional approach that included the three most important countries in terms of livestock production and waste pollution to act together under a common interest geared towards protecting the ecosystems of the South China Sea . Such an approach would increase cross country learning and knowledge sharing through FAO and GEF networks, and would also facilitate knowledge sharing between countries and positively contribute to the project's impact in the region . The project could have benefited from higher levels of financing given the scope of activities and the regional nature of the project. 4. Achievement of Objectives (Efficacy): o reduce the negative environmental impacts of livestock -induced pollution on water bodies ." Substantial . (a)" to The project supported the establishment of a policy and regulatory framework for environmentally sustainable development of livestock production in each country . Also, spatial distribution plans for livestock production and nutrient management plans were developed and implemented in selected areas resulting in improved capacity in all three countries. The introduction of cost efficient livestock waste management technology packages is an important step towards starting a process to reduce pollution on water bodies . Outputs : The project introduced ten livestock waste management packages that were demonstrated on 56 farms and one village. As part of the project replication efforts, a total of 679 farms, with an estimated total of 1.9 standing pig populations, requested intervention to apply the project promoted technologies . Technological packages covered 262,000 (285% of appraisal target), 286, 000 (141% of appraisal target) and 105,000 (86% of appraisal target) standing pig populations in China (Guandong province), Thailand and Vietnam, respectively . Overall appraisal target value was surpassed by 57%. The project promoted sound livestock waste management practices through public awareness campaigns that called for treatment of wastes from all livestock populations on participating farms and prohibited sale of fresh waste; solid waste to be composted if not treated by anaerobic digestion facilities and all treated solid and liquid wastes to be disposed properly to avoid secondary pollution . The project disseminated 9,100 implementation manuals, 134,100 project brochures and 1100 copies of DVDs and VCDs (ICR, p. 25). The project supported the establishment of a policy and regulatory framework for environmentally sustainable development of livestock production in each country . Replication strategy reports were submitted by the three countries according to their respective closing dates . Three spatial distribution plans were developed and implemented resulting in improved capacity in all three countries . Fifty eight nutrient management plans were prepared and integrated in farm specific mini project implementation plan for all participating farms . By project completion 55% of standing pig population in both China (Guandong province) and Vietnam(appraisal targets: 50%) and 40% in Thailand (appraisal target: 40%) were implementing improved (level 1 and level 2) livestock waste management practices compared to a baseline of 35% , 25% and 35% in China (Guandong province), Thailand and Vietnam, respectively . A total of 16,000 ton of dried digested pig manure was sold during the implementation period . It is estimated that 13,500 ton of dried digested pig manure would be sold annually beginning in 2012. Anaerobic digestion resulted in the production of 10.1 million cubic meters of biogas during project implementation with an estimated annual production of 5 million cubic meters after 2012. A total of 46.4 million kilowatt-hour of biogas fueled electricity were generated during implementation . It is estimated that 27.6 million kilowatt-hour of biogas fueled electricity would be generated annually as of 2012. It is estimated that waste management technologies promoted by the project contributed to the reduction of greenhouse gas emissions by 219, 200 ton of carbon dioxide during implementation of the project with an expected annual reduction of 149,500 ton of carbon dioxide starting 2012 (ICR, p.12). The project organized 114 workshops attended by 809 persons, trained 3,100 person-days and 3600 person-days for staff and farmers, respectively, also 422 person participated in 36 domestic study groups and 111 person participated in 11 international study groups (ICR, p. 24). The ICR (p. x) reported that the project met its targets on number of staff trained and number of workshop and study tours carried out . The project delivered 3 out of 4 decision support tools Outcomes : The analysis of treated waste discharge in farms using project supported waste management technologies showed significant reduction in pollutants and biological waste . The analysis of treated waste water at discharge points showed that anaerobic digestion technological packages removed on average 89% of phosphorus, 87% of nitrogen and 95% of biological oxygen demand from treated waste in participating farms and the quality of treated livestock waste on such farms were in line with the national discharge standards of the respective countries. to reduce negative health impacts of livestock -induced pollution on water bodies ." Modest . (b) "to Outputs : Most of the outputs related to objective (a) are also pertinent to this objective . Outcomes : The ICR (p. 12) reported that anecdotal evidence from surveys and data from participating countries showed that as a result of project activities E. coli was reduced by 90% in participating farms in China and Thailand . In Vietnam, swine related human infections showed a reduction as a result of the project implementation, intestinal disease were reduced from 53% to 12%, surface water caused allergy from 35% to 6%, eye infections went down from 24% to 12% and respiratory infections were reduced from 18% to 6%. However, while the project promoted sound livestock waste management practices that would probably carry some health benefits, especially to communities in the vicinity of participating farms, the evidence provided is only “anecdotal.� Moreover, it is unclear if the improvements were a result of the project rather than of other health programs in the areas concerned . The project had no impact on the use of antibiotics and other residues in animal feed . Unregulated use of antibiotics is therefore continuing . This could lead to the rise of resistant strains of bacteria which would represent a serious health challenge in the case of zoonotic diseases . In a subsequent communication the project team explained that given the size of the project they preferred to focus resources on livestock waste management especially that animal feed and antibiotics have a different set of stake holders (animal feed producers, pharmaceutical companies, veterinary service ..etc) . to reduce the environmental degradation of the South China Sea ." Substantial . (c)"to Anaerobic digestion treatments prevented a total of 1,500 tons of phosphorus, 2,600 tons of nitrogen, 17,200 tons of biological oxygen demand, and 45,400 ton of chemical oxygen demand from reaching the South China Sea (ICR, p. 23). There were no baseline values or targets . 5. Efficiency: The PAD (Annex 7) included an analysis of cost -effectiveness of two selected waste management approaches (waste treatment-level 1 compared to waste treatment and recycling -level 2). Financial analysis revealed that adoption of improved manure management systems (level 2) as proposed under the project generally adds 4-6 percent, or between US$1.00 and US$1.25, to the direct costs of raising a fat pig, costs of using improved manure management as proposed under the project would correspond to between 3% and 5% of gross sales. The analysis concluded that the costs of manure management as proposed under the project would be affordable for medium - and large-sized industrial pig farms which are the targeted farms under this project . Smaller farms in villages could also afford the technologies provided that they are successful in defraying part of the costs through manure sales, fish production or similar means under the support of the project . The ex post financial analysis adopted the same methodology, but included carbon emission reduction as an important benefit that emerged during implementation . Overall, the project financially subsidized the farm investments for waste management technological upgrades in the order of 60%. Participating farms actual contributions ranged from 26% in Vietnam, 36% in China to 54% in Thailand (ICR, p. 31). The analysis showed that the ten anaerobic digestion packages demonstrated by the project were cost effective in reducing pollution discharge, although the weighted average unit investment cost for the ten packages was higher than the benchmark options estimated at appraisal. This was due to higher market prices for inputs during implementation and higher initial costs because the project promoted technological packages that included additional features making them more efficient and safer to operate, and Clean Development Mechanism (CDM) compliance in the case of Thailand . A cash flow analysis on typical farms participating in the project showed that the Financial Internal Rate of Return (FIRR) on full costing was 8.7%, 14.7% and 8.2% for the Chinese, Thai and Vietnamese farms, respectively . The higher FIRR for the Thai farms was directly due to their expected certified emission reduction (CER) revenues and their additional sales income from dried digested sludge . However, in a subsequent communication the project team revealed that the carbon finance operation linked to the Thailand farms is being cancelled due to slow Clean Development Mechanism processing and the certified emission reduction revenues will not be achieved . The analysis concluded that the project presented an attractive option to all participating farms in converting their livestock waste management facilities to more sustainable modes of operation . However, interviews with non-project farms confirmed expectations at appraisal that subsidies would still be needed for future livestock waste management investments. The calculated full-cost basis FIRRs for China and Vietnam were barely above the opportunity cost of capital of 8.0%. Thailand had a higher FIRR driven by certified emission reduction revenues and sales of digested sludge. The ICR (p. 33) noted that such FIRRs were acceptable for the participating countries and that cash revenues were adequate to cover all costs of the anaerobic digestion technological packages and partially or even fully recover their investments costs . There are some concerns that cast doubt on efficiency . First, there is no economic, only financial, analysis, Second, it is not clear from the ICR where the pay off for the farmers comes from (the private benefit as opposed to the public one of reducing pollution). In a subsequent communication, the project team explained that the private financial returns were derived from biogas, biogas -generated electricity, sale of dried digested sludge and pig manure, saved labor costs, and increased fish production (in Vietnam). Third, there seems to be a disconnect between the discussion of the ex ante analysis – which focuses on least cost – and the ex post analysis which focuses on financial rates of return to farmers. Fourth, the financial returns in two of the three countries were fairly modest and only a little higher than the (lower than usual) assumed opportunity cost of capital of 8.0%. Fifth, there were concerns on administrative efficiency in Vietnam where two cases of procurement collusion were identified and reported to the Bank's Institutional Integrity Unit (INT), while in Thailand fiscal management and audit reports were submitted with delays (see section 11 b). In both countries, the project closed later than the appraisal date . In view of these concerns, efficiency is rated modest . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 8% 60% ICR estimate Yes 8% 60% * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Relevance of objectives is rated high and relevance of design is rated substantial . The objectives addressed the environmentally unsustainable and geographically -concentrated intensive livestock production in China, Thailand and Vietnam, and also contributed to the strategic priorities of the GEF . Design featured a comprehensive approach that aimed to address waste management through technological solutions, policy development and enforcement and regional synergy among participating countries . Efficacy is rated substantial for two objectives and modest for the other. Efficiency is rated modest . Overall outcome is assessed as moderately satisfactory . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: There is a concern regarding the commitment of the three participating countries to mobilize the incentives needed for scaling up the project. The ICR (p. 17) noted that the three countries have developed actions to implement replication strategies to scale up activities . Also, there is considerable risk associated with pig farming in general, including price fluctuations and disease outbreaks, which might discourage farmers from committing investments in waste management facilities especially with lax pollution enforcement due to the strong governmental interests in ensuring a sufficient and affordable supply of pig products . In addition, disruption of pig farming operations in farms equipped with biogas producing waste management facilities could lead to a decrease or even halt biogas production. This would negatively impact communities relying on such supplies as their sole source of energy . Also, any mismanagement of waste treatment facilities especially during maintenance time could potentially lead to pollution of nearby water bodies . The ICR (p. 17) noted that sufficient mitigation measures to avert environmental risk have been included into the operation manuals of waste treatment facilities . a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: The Bank ensured that the project design was technically sound through relying on a group of experienced international and national experts . The design also benefitted from input by FAO, by all participating countries, and from the experience and lessons learned from other initiatives and programs funded by the Bank . Most notable of these lessons was to emphasize the importance of on -the-ground demonstrations of innovative, cost-effective live stock waste management technology options by private livestock producers, complemented by country specific replication strategies to promote the broader adoption of such technologies . The project was strategically relevant to the three participating countries and closely aligned to their government priorities as it addressed an existing environmental concern of a regional nature . The project design correctly identified technical solutions, policy instruments and capacity and awareness as three key challenges that needed to be addressed to successfully manage animal waste in the three participating countries . The ICR (p. 6) highlighted that quality at entry was rated by Bank ’s Quality Assurance Group as satisfactory on all dimensions except for fiduciary aspects which was rated moderately satisfactory due to certain inadequacies in financial management . Design also included a set of outcome indicators that are consistent with the expected outcomes . Attribution, however, is unclear given the broad -ranging development objectives . Design also lacked indicators to assess the impact of the project activities on pollution levels on water bodies and at the South China Sea . at -Entry Rating : Quality -at- Moderately Satisfactory b. Quality of supervision: The Bank conducted seven supervision missions over the 63 month implementation period. According to the ICR, missions consisted of qualified staff from both the Bank's headquarters and country offices . The ICR reports that the missions ensured that implementation progress was on track, Bank policies were followed, legal covenants were observed, technical support was provided and concerns were timely addressed . In January 2009, two cases of collusion in procurement were identified in Vietnam and reported to the Bank's Internal integrity unit (INT). The Vietnam Project Management Office, with the Bank's close supervision, undertook the necessary corrective actions to sanction the firms involved and to ensure that the two contracts of concern were rebid. Fiduciary compliance was effectively managed through country based Bank specialists . Also, safeguard policies in all three countries were supervised by the Bank's country office - based environmental and social development specialists, and this, according to the ICR, ensured cross -country consistency on safeguard supervision. The ICR (p. 18) highlighted that the Bank awarded a 2012 East Asia and the Pacific Vice presidency Unit Team award to the project for its successful implementation and significant developmental impacts. Quality of Supervision Rating : Satisfactory Overall Bank Performance Rating : Moderately Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: Governments of the three participating countries provided strong political support and financial commitment to the project. The three governments contributed higher levels of counterpart funding than had been originally estimated at appraisal. Governments were also responsive and supportive to the project activities and actively coordinated project preparation and implementation in their respective countries . Government Performance Rating Satisfactory b. Implementing Agency Performance: The project was implemented by four implementing agencies : three country Project Management Offices and the Regional Facilitation Office of FAO. Implementing agencies contributed to project preparation and implementation, delivering counterpart funds committed by the three countries and FAO, as well as establishing functional institutional arrangements and mobilizing technical support needed for project implementation . The Bangkok based Regional Facilitation Office of FAO provide support to the three country implementing agencies and contributed to capacity building, coordination, and knowledge dissemination . The ICR (p. 19) reports that the three implementing agencies were responsive to issues raised by supervision missions and closely followed the Bank's recommendations to resolve issues in a timely manner . Implementing agencies also carried out M&E activities according to the agreed M&E plans . However, the lack of experience with Bank projects and organizational adjustments within the three country implementing agencies contributed to a slow project implementation start (ICR, p. 19). Implementation pace improved through training and study tours and hands -on support provided by Bank supervision missions . The project management office in Vietnam undertook the necessary steps to resolve two cases of procurement collusion reported in January 2009, which were referred to INT, and adopted a procurement action plan in October 2009. Implementing Agency Performance Rating : Satisfactory Overall Borrower Performance Rating : Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: A monitoring and evaluation plan was prepared by each participating country, reviewed by the Bank, and included in the country-specific project implementation plan . The plans included process indicators, stress reduction indicators and environmental indicators relevant to International Waters projects (PAD, p. 11). The plans would monitor nitrate, phosphate, biological oxygen demand, chemical oxygen demand, E. coli levels at discharge outlets for individual farms and at critical downstream locations, number of standing pig population covered by farms adopting sound waste management systems and the extent of awareness and regional exchange of information on pollution threats and health problems from livestock waste . M&E design appeared realistic and to reflect appropriately the impact of the project activities . However, design lacked indicators to assess the impact of the project activities on pollution levels at the South China Sea . National Project Management Offices (PMOs) were responsible for overseeing all M&E activities while monitoring assignments for sampling, analysis, and reporting on livestock waste reduction were contracted out to independent professional firms or institutions (ICR, p. 8). b. M&E Implementation: In each participating country the PMO had designated M&E staff members assigned at both the local and national levels. Specialized training was provided and funds were budgeted for the implementation of M&E plans . Semi-annual progress reports included a summary of monitoring results and by project completion 52 environmental monitoring reports were submitted. Each country submitted a final aggregated M&E report as part of its ICR package . The ICR (p. 8) highlighted that semi-annual reports, environmental monitoring reports and specific studies provided reliable information and data on project implementation . In a subsequent communication the project team explained that monitoring nitrate, phosphate, biological oxygen demand, chemical oxygen demand and E. coli levels would not be done at downstream locations as "(a) it would be extremely challenging to develop a causal relationship between monitoring results at downstream locations and pollution discharge from project farms as there are many other pollution sources contributing to downstream; and (b) the project's limited scope and resources will not be able to support such a thorough pollution investigations ." c. M&E Utilization: M&E reports and results were a useful tool to evaluate the performance of the project in comparison to the baseline and assess the achievement of project impacts . In Guandong, China, M&E results were used for academic research by South China Agricultural University as well as for decision making purposes with respect to the development of livestock waste management policies at the county, municipal, and provincial government levels . The project promoted wide dissemination of its M&E practices to support adoption of new waste management technologies in additional livestock farms in the three participating countries . The project also facilitated the collection of post project data through the mobilization of a grant from the Global Methane Initiative . This would allow environmental monitoring according to the same protocol specified in the PAD at a selected number of participating farms for one additional year in Thailand and Vietnam and for two years in China . M&E Quality Rating : Substantial 11. Other Issues a. Safeguards: The project was assessed as a Category "B". According to the PAD (p. 17), three safeguards policies were triggered: Environmental Assessment (OP 4.01), Involuntary Resettlement (OP 4.12) and Indigenous Peoples (OP 4.10). Environment The ICR reports that environmental assessment and management plans were prepared for each of the three participating countries in accordance with each country's and the Bank's safeguard policies and requirements . Implementing agencies in each country included independent environmental management teams to supervise and monitor the implementation of the environmental management plan . According to the ICR (p. 9), project implementation was fully in compliance with the Bank's environmental safeguard policies with no environmental issues reported during the course of the project's implementation . This was also confirmed by internal and external environmental monitoring results that showed that adverse environmental impacts resulting from project construction were mitigated and controlled to acceptable levels . Involuntary Resettlement The PAD (p. 17) reports that respective Resettlement Policy Frameworks were prepared and approved by each of the participating countries in compliance with the requirements of OP 4.12. The ICR (p. 9) states that the Frameworks and related guidelines were applied during implementation and that “it was concluded that implementation was in compliance with the Bank’s social safeguards policies .� However, no information is provided on the number of people or households that were resettled, the adequacy of compensation amounts and procedures, or how much land acquisition took place. In a subsequent communication the project team explained that all physical construction under the project was carried out within the existing boundaries of the participating farms . This was confirmed by the three participating countries and the Bank's supervision missions . Therefore, no resettlement nor any land acquisition took place under the project. In addition, there was no minority village around the participating farms . Indigenous Peoples Similarly, with regard to indigenous people, no information is provided in the ICR on the extent of the issue, nor of the measures taken to address it, but the ICR (p. 9) highlighted that the project implementation was in compliance with the Bank's social safeguard policies . In a subsequent communication, the project team explained tha t "no indigenous people were identified in China and Vietnam. In Thailand, Ethnic Minority Development Plans were developed for project farms with minority and Myamar workers to improve their living and working conditions . Participating farms took various measures to : (i) support Myanmar workers obtaining their Thai work permits; (ii) improve condition of these workers' life and work conditions such as providing service of tap water, electricity, play ground, TV etc ., and reduce odor/dust in work place; (iii) conduct training program for them to adapt to the new facilities financed by the project ." b. Fiduciary Compliance: Procurement . According to the ICR, the Bank's procurement guidelines were closely followed . However, delayed initiation of the project contributed to slow procurement in the early years of implementation . Two cases of procurement collusion were identified in Vietnam and reported to the Bank's Institutional Integrity Unit (INT). The ICR reports that corrective actions were carried out and that both cases were resolved in 2010. The ICR (p. 9) noted that the procurement capacity of the three PMOs improved significantly as a result of the project ’s implementation. Financial Management . The ICR (p. 9) reported that financial management was rated satisfactory for China and FAO, and moderately satisfactory for Thailand and Vietnam . The capacity of the financial management staff in Vietnam was persistently weak with respect to financial reporting, internal control system management and monitoring of counterpart funding . Several annual financial audit reports suffered from inconsistencies and errors and had to be returned . In addition, on some supervision missions, ineligible expenses were identified . In Thailand, fiscal management and audit reports were submitted with delays. No information is provided in the ICR concerning external audits of project funds – whether these were carried out in a satisfactory and timely manner, whether the auditor ’s opinions were qualified, and if so whether the qualifications were addressed . In a subsequent communication, the project team explained that "annual external audits of project funds, except that the Bank allowed a combined audit for the first two years of project implementation as there were very limited physical construction and expenditure, were carried out by each participating country and auditing reports were reviewed by the Bank's project FMS for each country . All auditors' opinions were unqualified . The annual audit for China (Guangdong) was conducted in a satisfactory and timely manner and all the auditor's opinions were unqualified. For Vietnam, the Project audit reports were timely submitted to the Bank . All auditor opinions on audited financial statements are unqualified . The audited reports for fiscal year FY 2009, FY2010 and last audit period in FY 2011 had been returned due to some errors and inconsistencies noted in the reports, both due to neglect of the auditor and weak capacity of the PMO staff . These reports later had been re -submitted and accepted by the Bank . For Thailand, almost all audit reports were submitted with delay ." c. Unintended Impacts (positive or negative): Positive . During project implementation in Thailand, the project integrated with two clean development mechanism projects, Thailand Livestock Waste Management Program (P107716) and Thailand Small scale Livestock Waste Management Program (P112092). This integration enabled 16 out the project's 20 participating farms to generate up to a total of 148,000 tons of carbon dioxide equivalent emission reductions and an expected revenue of about US$1.9 million over the next five years through Clean Development Mechanism . This also served as an incentive for other eligible farms to seek project support . The project- supported farms were also able to produce and provide or sell digested liquid and sludge waste as a safe and inexpensive good quality fertilizer and soil conditioner. Such sales provided extra revenue for project farms in Thailand . d. Other: Another added benefit is biogas production and generation of electricity . A total of 46.4 million kilowatt-hour of biogas fueled electricity were generated during implementation . It is estimated that 27.6 million kilowatt-hour of biogas fueled electricity would be generated annually as of 2012. 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Of the three development objectives, Satisfactory two were rated substantial. The third objective was rated modest because, inter alia, the project had no impact on the use of antibiotics and other residues in animal feed. Efficiency is rated modest. Risk to Development Negligible to Low Significant There is considerable risk associate Outcome : with pig farming in general, including price fluctuations and disease outbreaks, which might discourage farmers from committing investments in waste management facilities. In addition, disruption of pig farming operations in farms equipped with biogas producing waste management facilities could lead to a decrease or even halt biogas production which would negatively impact communities relying on such supplies as their sole source of energy. Bank Performance : Satisfactory Moderately There were moderate shortcomings in Satisfactory Quality at Entry. Borrower Performance : Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The following lessons are taken from the ICR with some adaptation of language : The appropriate technology depends on actual country situations . A number of livestock waste management technologies exist, ranging from simple to complex, with various cost implications that range from moderate to high. The project experience demonstrates that the appropriate technology varies according to country specific conditions. These conditions determine acceptance, feasibility, efficiency and sustainability of project investments. Appropriate livestock waste management technologies adapt their operating complexity to country conditions and are compatible with the various waste -handling practices and methods of livestock farms. Such compatibility would also ensure cost -effectiveness, offer financial returns, and enable local development and servicing. A comprehensive approach can promote long -term sustainability . The project featured a comprehensive approach that integrated a suite of actions ranging from demonstration of technical solutions, policy development and improvement in enforcement, capacity building and awareness -raising, to regional coordination and dissemination . Successful demonstration of technical solutions provided concrete waste management practices that are financially viable, environmentally sound, socially beneficial and locally accessible, show casing what could be studied and further replicated in the countries . Such demonstrations also formed the basis for development and implementation of country specific policy actions such as the Replication Strategies and improvement in environmental enforcement . The needs and results of such demonstration then helped to refine capacity building activities to match actual needs, as did increased awareness among all stakeholders, which improved acceptance of project interventions and replication actions. Strong commitments - from government partners (for compliance, enforcement, and provision of incentives ) and from key stakeholders (for full involvement in project preparation and implementation ) – are needed to ensure ownership and sustainability . The project succeeded in gaining strong commitments from all stakeholders through the integration of project interventions with the governments ’ mainstreamed programs; implementation based on existing institutional mechanisms; and identification and involvement of key local stakeholders such as, village committees and the women ’s federation in China, pig cooperatives in Vietnam, and local administrations, communities, and nongovernmental organizations in Thailand, on project design and implementation. 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR provided thorough coverage of the project activities . It included relevant lessons that reflected the project experience. However, due to the absence of relevant environmental indicators, the ICR did not fully reflect the impact of the project on the achievement of the GEO . The ICR also provided only a brief summary describing the project components. a.Quality of ICR Rating : Satisfactory