Document of The World Bank Report No: ICR2682 IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF55745, TF55763, TF57678, TF90543, TF57778, TF56693) ON A GRANT FROM THE GLOBAL ENVIRONMENT FACILITY TRUST FUND IN THE AMOUNT OF US$2.62 MILLION TO THE FEDERAL DEMOCRATIC REPUBLIC OF ETHIOPIA US$2.55 MILLION TO THE REPUBLIC OF MALI US$4.0 MILLION TO THE KINGDOM OF MOROCCO US$1.7 MILLION TO THE REPUBLIC OF SOUTH AFRICA US$6.87 MILLION TO THE UNITED REPUBLIC OF TANZANIA US$4.0 MILLION TO THE REPUBLIC OF TUNISIA FOR THE AFRICA STOCKPILES PROGRAM – PROJECT 1 Environmental and Natural Resources Management Unit Sustainable Development Department Africa Regional Integration Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective October 10, 2013) Currency Unit = Ethiopian Birr (ETB) ETB 18.94 = US$1 Currency Unit = West African CFA Franc (XOF) XOF 484.91 = US$1 Currency Unit = Moroccan Dirham (MAD) MAD 8.29 = US$1 Currency Unit = South African Rand (ZAR) ZAR 9.96 = US$1 Currency Unit = Tanzanian Shilling (TZS) TZS 1608.00 = US$1 Currency Unit = Tunisian Dinar (TND) TND 0.61 = US$1 Vice President: Makhtar Diop Regional Integration Director: Colin Bruce Sector Manager: Magda Lovei Project Team Leader: Dinesh Aryal ICR Team Leader: Dinesh Aryal EXECUTIVE SUMMARY The Africa Stockpiles Program – Project 1 (ASP-1) was designed to address the issue of obsolete pesticides by developing and implementing disposal and prevention activities through the establishment of a new partnership between African countries and their collaborating partners, including the World Bank, the United Nations Food and Agriculture Organization (FAO), the pesticide industry represented by CropLife International (CLI), the Pesticides Action Network (PAN) in the UK and Africa (PAN-UK and PAN-Africa, respectively) and the World Wide Fund for Nature (WWF). ASP-P1 was launched in September 2005 and was designed as an Adaptable Program Lending (APL) operation to be implemented over a 4-year period in seven countries: Ethiopia, Mali, Morocco, Nigeria, South Africa, Tanzania, and Tunisia. Program preparation took more than four years, which was mainly the result of a lengthy partnership-building process, caused by a complex envisaged implementation and partnership arrangement. Despite the lengthy preparation, the program’s design suffered from several shortcomings. Implementation was subsequently characterized by a challenging collaboration among partners, substantial implementation delays and challenges, and varying performance across the participating countries. At mid-term review, it was considered difficult to achieve the PDO within the originally set timeframe and budget in all ASP-P1 participating countries, and it was agreed to restructure the program. A Level 1 restructuring conducted for four ASP-P1 countries in 2011 eventually positively affected implementation and facilitated PDO achievement. The overall outcome of ASP-P1 is rated moderately unsatisfactory. However, despite the program’s complexity, challenges, and poor initial performance, mitigation efforts eventually resulted in complete disposal of inventoried obsolete pesticides in ASP-P1 Ethiopia, ASP-P1 Tanzania, and ASP-P1 Tunisia, and partial disposal in ASP-P1 South Africa, having significantly reduced risks to the environment and public health. In addition, measures to reduce future re-accumulations by establishing sustainable systems of pest and pesticide management were substantially strengthened across ASP-P1 participating countries. The elimination of obsolete pesticide stocks remains a development priority. Rural communities cannot hope to advance if the soil and water upon which their livelihoods and health depend are contaminated with pesticides, and urban populations cannot prosper if they are suffering from severe illnesses caused by pesticide poisoning. National development strategies of ASP-P1 participating countries confirm the importance of eliminating hazardous waste, as do World Bank priorities. The World Bank’s dual goals of ending extreme poverty and promoting shared prosperity are to be achieved in an environmentally sustainable manner by taking account of the impacts of climate change, environmental degradation, and ecosystem changes. New analytical work on hazardous wastes, POPs, and the clean agenda are expected in the future. ABBREVIATIONS AND ACRONYMS APL Adaptable Program Lending ASP-P1 Africa Stockpiles Program – Project 1 CCAME Crosscutting Management Entity CESA Country Environmental and Social Assessment CIDA Canadian International Development Agency CLI CropLife International EC European Commission FAO United Nations Food and Agriculture Organization FFEM French Global Environment Facility GEF Global Environment Facility GEO Global Environment Objective ICM Implementation Completion Memorandum IPM Integrated Pest Management IVM Integrated Vector Management MDTF Multi-Donor Trust Fund M&E Monitoring & Evaluation NEAP National Environment Action Plan NEPAD New Partnership for Africa’s Development NGO Non-Governmental Organization PAD Project Appraisal Document PAN Pesticides Action Network PCU Project Coordination Unit PDO Project Development Objective POP Persistent Organic Pollutants TSU Technical Support Unit UNEP United Nations Environment Program WHO World Health Organization WWF World Wildlife Fund TABLE OF CONTENTS 1. PROGRAM CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES, AND DESIGN ....... 2 1.1 CONTEXT AT APPRAISAL ................................................................................................................................................ 2 1.2. ORIGINAL GEO, PDO, AND INDICATORS ..................................................................................................................... 4 1.3 REVISED GEO, PDO, AND INDICATORS ........................................................................................................................ 5 1.4 MAIN BENEFICIARIES....................................................................................................................................................... 5 1.5 ORIGINAL COMPONENTS ................................................................................................................................................ 6 1.6 REVISED COMPONENTS ................................................................................................................................................... 7 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ......................................................................... 7 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY ................................................................................ 7 2.2 IMPLEMENTATION .......................................................................................................................................................... 10 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTILIZATION .......................... 12 2.4 SAFEGUARDS AND FIDUCIARY COMPLIANCE ......................................................................................................... 13 2.5 POST-COMPLETION OPERATION/NEXT PHASE ........................................................................................................ 14 3. ASSESSMENT OF OUTCOMES ........................................................................................................................................... 14 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION ........................................................................ 14 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE ..................................................................................... 15 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE ..................................................................................... 17 3.4 EFFICIENCY ...................................................................................................................................................................... 17 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING................................................................................................... 18 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS .............................................................................. 19 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS ...................... 19 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME ............................................................................................ 19 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE .................................................................... 20 5.1 WORLD BANK PERFORMANCE .................................................................................................................................... 20 5.2 BORROWER PERFORMANCE......................................................................................................................................... 21 6. LESSONS LEARNED .............................................................................................................................................................. 22 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS .................. 23 8. ANNEXES ................................................................................................................................................................................. 24 8.1 PROJECT COSTS AND FINANCING ............................................................................................................................... 24 8.2 OUTPUTS BY COMPONENT ........................................................................................................................................... 25 8.3 ECONOMIC AND FINANCIAL ANALYSIS .................................................................................................................... 30 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS ................................... 30 8.5 BENEFICIARY SURVEY RESULTS ................................................................................................................................ 30 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS.............................................................................................. 30 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR ................................................................... 30 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS ..................................................... 30 8.9 VERBATIM COMMENTS ON DRAFT ICR ..................................................................................................................... 33 TABLE OF TECHNICAL ANNEXES TECHNICAL ANNEX I: ASP-P1 ETHIOPIA .......................................................................................................................... 36 TECHNICAL ANNEX II: ASP-P1 MALI .................................................................................................................................. 77 TECHNICAL ANNEX III: ASP-P1 MOROCCO ................................................................................................................... 115 TECHNICAL ANNEX IV: ASP-P1 SOUTH AFRICA ........................................................................................................... 140 TECHNICAL ANNEX V: ASP-P1 TANZANIA ...................................................................................................................... 178 TECHNICAL ANNEX VI: ASP-P1 TUNISIA ......................................................................................................................... 220 ii A. Basic Information 1 Federal Democratic Republic of Ethiopia, Republic of Mali, Kingdom of Morocco, Country: Project Name: Africa Stockpiles Program - Project 1 Republic of South Africa, United Republic of Tanzania, Republic of Tunisia TF55763/TF55745 (South P075776 (South Africa/Tunisia), TF57678/ Africa/Tunisia), P103189 Project ID: L/C/TF Number(s): TF57778/TF56693 (Mali/Morocco/Tanzania) (Mali/Morocco/Tanzania), , P105711 (Ethiopia) TF90543 (Ethiopia) ICR Date: 06/28/2013 ICR Type: Core ICR Federal Democratic Republic of Ethiopia, Republic of Mali, Kingdom Lending APL Borrower: of Morocco, Republic of South Instrument: Africa, United Republic of Tanzania, Republic of Tunisia Original Total US$21.7 million Disbursed Amount: US$16.32 million Commitment: Revised Amount: - Environmental Category: A Global Focal Area: CHEM Implementing Agencies: Ethiopia: Crop Protection Department; Mali: Direction Nationale de l’Assainissement et du Contrôle des Pollutions et des Nuisances (DNACPN); Morocco: Direction de la Protection des Végétaux des Contrôles Techniques et de la Répression des Fraudes (DPVCTRF); South Africa: Department of Environmental Affairs and Tourism (DEAT); Tanzania: National Environment Management Council (NEMC); and Tunisia: Agence Nationale de Gestion des Déchets (ANGed) Co-financiers and Other External Partners: National Governments of ASP-P1 participating countries, Multi-Donor Trust Fund (EU, Denmark, Sweden, Switzerland), Bilateral Financing (Belgium, Finland, Japan, Canada, and the Netherlands), French Global Environment Facility (FFEM), UN Food and Agriculture Organization (FAO), CropLife International (CLI), and Development Grant Facility (DGF). 1 The information presented in the data sheet was inserted manually. Information provided in the system was incomplete resulting from coding issues experienced during implementation; these were not resolved prior project closing. iii B. Key Dates Process Date Process Original Date Revised / Actual Date(s) ASP: 09/30/2005 ASP: 11/21/2005 Ethiopia: 06/30/2007 Ethiopia: 07/13/2007 Mali: 02/12/2007 Mali: 12/26/2006 Concept 07/17/2002 Effectiveness: Morocco: 02/12/2007 Morocco: 02/05/2007 Review: South Africa: 09/30/2005 South Africa: 04/04/2006 Tanzania: 02/12/2007 Tanzania: 12/19/2006 Tunisia: 09/30/2005 Tunisia: 11/21/2005 ASP: 04/19/2004 ASP: NA Ethiopia: 04/19/2005 Ethiopia: 06/30/2011 Mali: 04/19/2004 Mali: NA Morocco: 04/19/2004 Morocco: NA Appraisal: Restructuring: South Africa: South Africa: 06/30/2011 04/19/2004 Tanzania: 04/19/2004 Tanzania: 06/30/2011 Tunisia: 04/19/2004 Tunisia: 06/30/2011 ASP: 09/08/2005 ASP: NA ASP: NA Ethiopia: 06/19/2007 Ethiopia: 06/30/2009 Ethiopia: 10/28/2009 Mali: 12/05/2006 Mali: 02/11/2009 Mali: 02/06/2009 Mid-term Approval: Morocco: 12/05/2006 Morocco: 11/30/2008 Morocco: 12/11/2009 Review: South Africa: South Africa: 03/2008 South Africa: 02/06/2009 09/08/2005 Tanzania: 12/05/2006 Tanzania: 08/31/2008 Tanzania: 10/16/2009 Tunisia: 09/08/2005 Tunisia: 03/2008 Tunisia: 10/31/2008 ASP: 06/30/2011 ASP: 05/31/2013 Ethiopia: 06/30/2011 Ethiopia: 12/31/2012; 05/31/2013. Mali: 08/31/2010 Mali: 12/31/2011; 06/30/2012; 12/31/2012. Morocco: 06/30/2010 Morocco: 06/30/2010 Closing: South Africa: 09/30/2009 South Africa: 04/30/2011; 06/30/2012. Tanzania: 06/30/2010 Tanzania: 12/31/2011; 12/31/2012; 05/31/2013. Tunisia: 09/30/2009 Tunisia: 07/31/2011; 06/30/2012. iv C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Global Environment Outcome: Not Applicable 2 World Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of World Bank and Borrower Performance World Bank Ratings Borrower Ratings Moderately Quality at Entry: Government: Satisfactory Unsatisfactory Moderately Implementing Quality of Supervision: Moderately Satisfactory Unsatisfactory Agency/Agencies: Overall World Bank Moderately Overall Borrower Moderately Satisfactory 3 Performance: Unsatisfactory Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project at Quality at Entry Yes Moderately Satisfactory 4 any time (Yes/No): (QEA): Problem Project at any time Quality of Supervision Yes Moderately Unsatisfactory 5 (Yes/No): (QSA): GEO rating before Moderately Closing/Inactive status: Unsatisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total World Bank financing) Sanitation 33.3 33 Health 33.3 33 General Agriculture 33.3 34 2 The risk to development outcome varies across ASP-P1 participating countries and has been assessed at the country-level. Please see Section 4 of the Technical Annexes for details. 3 Note that this rating represents an average rating of the participating ASP-P1 countries’ performances. See Section 5.2 for details. 4 A Quality at Entry Assessment for FY06-07 was undertaken in for P075776. The assessment evaluated ASP-P1 at the program- level by taking ASP-P1 projects in South Africa and Tunisia into account. 5 A Learning Review of Regional Projects for FY07-08 was undertaken for P103189. The review evaluated ASP-P1 at the program-level by taking ASP-P1 projects in Mali, Morocco, and Tanzania into account. v Theme Code (as % of total World Bank financing) Environmental policies and institutions 50 50 Pollution management and environmental health 50 50 E. World Bank Staff Positions At ICR At Approval South Africa /Tunisia Christiaan J. Poortman Gobind T. Nankani Vice President: Makhtar Diop Mali/Morocco/Tanzania Gobind T. Nankani Ethiopia Obiageli Katryn Ezekwesili Director Regional Colin Bruce Integration: South Africa/Tunisia Theodore O. Ahlers Country Director: - Mali/Morocco/Tanzania Mark D. Tomlinson Ethiopia Mark D. Tomlinson South Africa/Tunisia Vijay Jagannathan Sector Manager: Magda Lovei Mali/Morocco/Tanzania Mary Barton-Dock Ethiopia Mary Barton-Dock South Africa/Tunisia Steven P. Maber Project Team Dinesh Aryal Mali/Morocco/Tanzania Peter Kristensen/Denis Jordy Leader: Ethiopia Peter Kristensen/Denis Jordy ICR Team Dinesh Aryal Leader: ICR Main Author: Veruschka Schmidt F. Results Framework Analysis 6 Project Development Objective (PDO) - as presented in the PAD (Report No: 32232-MNA): Selected African countries eliminate inventoried publicly held obsolete pesticide stockpiles and associated waste, and implement measures to reduce and prevent future related risks. 7 Revised PDO - The ASP-P1 PDO at the program-level was not revised. However, four ASP-P1 country projects underwent design adjustments under a combined Level I restructuring in June 2011 (see Technical Annexes on ASP-P1 Ethiopia, South Africa, Tanzania, and Tunisia). Global Environment Objective (GEO)- as presented in the PAD (Report No: 32232-MNA): Reduction of the effects of Persistent Organic Pollutants (POPs) on the global environment. 6 Annex 3 in the PAD outlined a results framework with slightly differing indicators in comparison to the arrangements for results monitoring. This ICR draws its evaluation on the results framework. 7 Corresponding Legal Agreements with ASP-P1 participating countries are discussed in each Technical Annex. vi Revised GEO - The ASP-P1 GEO was not revised. (a) PDO Outcome Indicator(s) - as presented in the PAD (Report No: 32232-MNA) Original Target Actual Value Formally Values Achieved at Indicator Baseline Value Revised (from approval Completion or Target Values documents) Target Years Inventoried publicly held obsolete pesticide stockpiles in 6 ASP- Indicator 1: P1 countries eliminated Value (quantitative or qualitative) 0% 100% removed - 35% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 Target partly achieved (35%): Overall, 3,164 (35%) of the inventoried 8,949 tonnes of publicly held obsolete pesticides and Comments (incl. % achievement) associated waste were disposed of in four ASP-P1 countries (Ethiopia, South Africa, Tanzania, and Tunisia). ASP-P1 Mali and ASP-P1 Morocco closed without disposal activities having begun. Indicator 2: Quantity of new publicly held obsolete pesticide stockpiles Value (quantitative or qualitative) 0 0 stockpiles - N/A Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 Target not measurable: Inventories conducted during implementation revealed additional obsolete pesticides; however, it is difficult to Comments (incl. % achievement) assess whether additional stockpiles were accumulated during implementation or whether they resulted from inaccurate estimates used at appraisal. No further accumulation of publicly held obsolete pesticide Indicator 3: stocks 0 additional Value (quantitative or qualitative) 0 - N/A stockpiles Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 Indicator corresponds to PDO Outcome Indicator #2 and is therefore Comments (incl. % achievement) not taken into account. (b) GEO Indicator(s) – The GEO was measured by the PDO and associated indicators. (c) Results Indicator(s) - as presented in the PAD (Report No: 32232-MNA) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years vii An inventory database in place in 7 8 ASP-P1 countries, accepted Indicator 1: by the ASP partnership and being used and updated by government staff Value (quantitative or qualitative) 0% 100% - 83% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 Target partly achieved (83%): An inventory database is in place, accepted by the partnership, and being used by all participating ASP- Comments (incl. % achievement) P1 countries, except for ASP-P1 South Africa where it was considered cost-ineffective by the government. Indicator 2: In case of accident, emergency plans successfully implemented. Value (quantitative or qualitative) 0% 100% - N/A Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 No accidents were reported; therefore, this indicator is not taken into Comments (incl. % achievement) account. Disposal contract completed in 6 ASP countries in line with Indicator 3: national and international regulations Value (quantitative or qualitative) 0 100 - 50% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 Target partly achieved (50%): Disposal contracts, in line with national and international regulations, were completed in four ASP-P1 Comments (incl. % achievement) countries (Ethiopia, Mali, Tanzania, and Tunisia); but only implemented in three (Ethiopia, Tanzania, and Tunisia). Quantity of accumulation of new obsolete pesticide stockpiles Indicator 4: minimized 0 additional Value (quantitative or qualitative) 0 - N/A stockpiles Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 Indicator corresponds to PDO Outcome Indicator #2 and is therefore Comments (incl. % achievement) not taken into account. Government adopts of International Code of Conduct on pesticide Indicator 5: distribution and use Value (quantitative or qualitative) 0% 100% - 100% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 Indicator was achieved prior to appraisal and is therefore not taken into account. The International Code of Conduct on pesticide distribution Comments (incl. % achievement) and use was adopted by all ASP-P1 participating countries during a FAO council session in 2002. Technical Support Unit (TSU) fully funded and staffed by end of Indicator 6: year 1 8 Project activities for Nigeria, the seventh ASP-P1 country, were funded entirely from a Nigeria-specific, World Bank-managed Trust Fund provided by the Canadian International Development Agency. Project activities in Nigeria were therefore evaluated separately (see Implementation Completion Memorandum (ICM) – Nigeria, Africa Stockpiles Program, March 14, 2012). viii Value (quantitative or qualitative) 0% 100% - 100% Date Achieved 04/19/2004 End of year 1 - 04/01/2005 Target fully achieved (100%): The TSU was fully funded by two Comments (incl. % achievement) grants through FAO (Government of the Netherlands and GEF) and staffed by end of year 1. Indicator 7: Client satisfaction as determined by survey Value (quantitative or qualitative) 0% 85% - 0% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 Target not achieved (0%): A survey to determine client satisfaction Comments (incl. % achievement) was not conducted; therefore client satisfaction was not determined. Preparation activities for ASP-P2 countries complete – national Indicator 8: programs identified, selection conditions met, country counterparts in place and funding secured Value (quantitative or qualitative) 0% 100% - 50% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 Target partly achieved (50%): National programs were identified and selection conditions met. Country counterparts were not in place yet; Comments (incl. % achievement) and funding not secured as no support was obtained from the GEF Secretariat. Strategic studies delivered, e.g., alternative disposal technologies Indicator 9: and locust control issues assessed and adopted if ASP standards met Value (quantitative or qualitative) 0% 100% - 100% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 Target fully achieved (100%): A study on disposal technology options Comments (incl. % achievement) was completed by WWF in June 2008; a study on locust control issues was carried out by FAO as part of the Red Locust Control Program. NGO participation in project activities in all ASP-P1 countries Indicator 10: participating in at least two of the following: awareness raising, monitoring, capacity building and communication Value (quantitative or qualitative) 0% 100% - 100% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 Target fully achieved (100%): NGO participation in project activities Comments (incl. % achievement) was achieved in all participating ASP-P1 countries. Indicator 11: Communication strategy implemented Value (quantitative or qualitative) 0% 100% - 67% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 Target partly achieved (67%): Communication strategies were Comments (incl. % achievement) developed in collaboration with WWF, and implemented in all ASP-P1 participating countries, except for ASP-P1 South Africa and Morocco. Monitoring reports delivered with sufficient data on time to GEF, Indicator 12: donors and the World Bank ix Value (quantitative or qualitative) 0% 100% - 100% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 Target fully achieved (100%): Technical and financial reports were Comments (incl. % achievement) annually produced by the World Bank and shared with GEF and donors. ASP- Project Coordination Unit (PCU) moving to African Indicator 13: organization within ASP-P1 Value (quantitative or qualitative) 0% 100% - 0% Date Achieved 04/19/2004 06/30/2011 - 06/30/2013 Target not achieved (0%): An ASP- PCU was not moved to an African Comments (incl. % achievement) organization within ASP-P1. G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 10/31/2005 Satisfactory Satisfactory 0.00 2 06/30/2006 Satisfactory Satisfactory 0.50 3 12/27/2006 Satisfactory Satisfactory 0.57 4 06/28/2007 Moderately Satisfactory Moderately Satisfactory 0.58 5 12/18/2007 Moderately Satisfactory Moderately Satisfactory 0.59 6 05/30/2008 Moderately Satisfactory Moderately Satisfactory 1.38 7 12/18/2008 Moderately Satisfactory Moderately Satisfactory 1.65 8 03/25/2009 Moderately Satisfactory Moderately Satisfactory 2.03 9 12/29/2009 Moderately Unsatisfactory Moderately Unsatisfactory 2.56 10 06/29/2010 Moderately Unsatisfactory Moderately Unsatisfactory 2.66 11 03/27/2011 Moderately Satisfactory Moderately Satisfactory 3.66 12 12/25/2011 Satisfactory Satisfactory 3.66 13 06/29/2012 Moderately Unsatisfactory Moderately Unsatisfactory 5.37 06/30/2013 Final disbursement 16.32 H. Restructuring – Following World Bank Management’s decision, ASP-P1 at the program-level was not restructured. However, four ASP-P1 country projects underwent design adjustments under a combined Level I Board restructuring in June 2011 (see Technical Annexes on Ethiopia, South Africa, Tanzania, and Tunisia). I. Disbursement Profile – A disbursement profile is not available. 9 9 The World Bank’s budgeting system did not allow to capturing all ASP-P1 country-projects under one project number; therefore an aggregate disbursement profile is not available. Disbursement data presented in Section G was retrieved from the Implementation Status and Results Reports (ISRs). x 1. PROGRAM CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES, AND DESIGN 1.1 CONTEXT AT APPRAISAL 1. Country and sector background: During the second half of the 20th century, the African continent accumulated large quantities of obsolete pesticides. 10 The accumulation of such obsolete products occurred as a result of inadequate stock management, non-distribution to farmers, bans on several pesticides, and/or un-coordinated or inappropriate supply from donor agencies. Often in severely deteriorated condition, poorly stored and located close to urban and rural settlements and public infrastructure, including water supplies, these obsolete pesticide stockpiles represented a serious risk to human and livestock health, ground and surface water, land use, and the broader environment. In addition, some of these obsolete pesticide stocks included Persistent Organic Pollutants (POPs) 11, posing serious long-term danger to human health and the environment because of their mobility, toxicity, bioaccumulation potential, and persistence. 2. At appraisal of the Africa Stockpiles Program – Project 1 (ASP-P1), the amount of publicly held obsolete pesticides stockpiled across Africa was estimated at 50,000 tonnes. 12 Most African countries were lacking adequate technical, institutional, and financial capacity to properly manage the destruction and safeguarding of obsolete pesticide stocks, and clean contaminated sites. They also had limited capacity and means to implement sound prevention practices and develop the necessary policy and regulatory framework to avoid continuous accumulation. Although several development agencies and donors 13 had supported initial activities targeting disposal and prevention operations, resources remained limited because of countries’ pressing development needs in areas such as poverty alleviation, food security or other elements of sustainable development. In 2000, countries and the donor community began to realize that important efficiencies could be gained by promoting a strategic programmatic approach to obsolete pesticide elimination for Africa. Such an approach, together with the provision of funds specifically dedicated to pesticide/hazardous waste management and safe disposal of obsolete pesticides, was therefore sought. 3. ASP was designed to address the issue of obsolete pesticides by developing and implementing cost- efficient disposal and prevention activities through the establishment of a new partnership between African countries and their collaborating partners, including the World Bank, the United Nations Food and Agriculture Organization (FAO), the pesticide industry represented by CropLife International (CLI), the Pesticides Action Network (PAN) in the UK and Africa (PAN-UK and PAN- Africa, respectively) and the World Wide Fund for Nature (WWF). In order to remove all estimated 10 The ASP classified pesticides as obsolete when they were banned, had deteriorated, or had been damaged, had passed their expiration date, could not be used for another reason, or were no longer wanted by their owner. 11 POPs as addressed under the Stockholm Convention, and in this program are: Aldrin, Dieldrin, Endrin, Chlordane, Heptachlor, DDT (dichlorodiphenyltrichloroethane), Mirex, Hexachlorobenzene and Toxaphene. 12 In Africa, a total of 51,794 tonnes of obsolete pesticides had been identified by the Food and Agriculture Organization (FAO). This rough estimate did not include contaminated soils (except for contaminated soils in Mali and Botswana), and was intended to be refined through the country-specific detailed inventories planned under ASP. 13 For example FAO, Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ; previously called Gesellschaft für Technische Zusammenarbeit (GTZ)), Danish International Development Agency (DANIDA), United States Agency for International Development (USAID), Canadian International Development Agency (CIDA), Belgium, Finland, Japan, and the Netherlands. 2 obsolete pesticide stockpiles in Africa (50,000 tonnes), ASP was expected to require funding of US$200-250 million. 14 4. The program was planned to be implemented through a series of projects within a 10-15 year timeframe, with ASP-P1 representing the first. ASP-P1 was launched in September 2005 and was designed as an Adaptable Program Lending (APL) operation to be implemented over a 4-year period in seven countries: Ethiopia, Mali, Morocco, Nigeria, South Africa, Tanzania, and Tunisia. South Africa and Tunisia represented the first APL phase, Mali, Morocco and Tanzania the second, and Ethiopia the third. ASP-P1 Nigeria, was not considered part of the APL as it was entirely funded by the Canadian International Development Agency (CIDA). Project activities in Nigeria were therefore not reviewed as part of this Implementation Completion Report (ICR); however, their outcomes are summarized in Box 1.1 15 All seven countries had ratified the Stockholm Convention on POPs and the Basel Convention on Trans-boundary Shipment of Hazardous Waste. 16 During the first year of ASP- P1 operations, an additional eight countries were expected to be identified and the corresponding projects prepared for inclusion in a follow-on project called ASP-P2. 17 This ICR pertains to the first group of projects – ASP-P1. Box 1.1: ASP-P1 Nigeria – Project Outcome Trust Fund Amount: US$3.0 million provided by the Canadian International Development Agency (CIDA) PDO: To strengthen the recipient’s capacity for pesticide management and prevention of future accumulation of publicly held obsolete pesticides and associated waste. Overall outcome – Rating Moderately Unsatisfactory: ASP-P1 Nigeria was a pilot targeted at sensitizing stakeholders for and generating knowledge on obsolete pesticides and their impact on the environment and health. Despite operational difficulties encountered during implementation as a result of weak project management, the following outcomes were achieved: (i) a national inventory of obsolete pesticides was completed; (ii) a Country Environmental and Social Assessment (CESA) was prepared, found satisfactory by the World Bank, and publicly disclosed; (iii) improved draft legislation was endorsed by the Steering Committee; (iv) a prevention workshop was held; (v) ties with the private sector and NGOs were established; (vi) an emergency containment of two identified problem sites was carried out; and (vii) communication and awareness raising activities were carried out. 5. The primary financing instrument for ASP-P1 was a Global Environment Facility (GEF) grant of US$25 million, split between the two principal GEF Agencies; the World Bank (US$21.74 million) and FAO (US$3.26 million) towards a total program cost of US$60 million for the seven countries. 14 This figure was based on estimates of obsolete pesticides in Africa, taking into account that actual costs could rise significantly if additional stocks were discovered during country level inventories. Since no adequate hazardous waste destruction facilities existed in Africa, it was agreed that pesticide waste would be exported to approved facilities in developed countries at a cost of approximately US$3,500 per tonne, resulting in an estimated total of US$150-175 million. The total budget for prevention measures was estimated at US$50-75 million; thus the total ASP costs were expected to be in the range of US$200-250 million. 15 See Implementation Completion Memorandum (ICM) – Nigeria, Africa Stockpiles Program, March 14, 2012. 16 The Stockholm Convention on Persistent Organic Pollutants (POPs) is an international environmental treaty, signed in 2001 and effective since May 2004, aiming to eliminate or restrict the production and use of POPs. In implementing the Convention, governments take measures to eliminate or reduce the release of POPs into the environment. The Basel Convention on the Control of Trans-boundary Movements of Hazardous Wastes and Their Disposal is an international treaty that was designed to reduce the movements of hazardous waste between nations, and specifically to prevent transfer of hazardous waste from developed to less developed countries. 17 Pre-identified candidate countries for ASP-P2 were Benin, Botswana, Cameroon, Cote d’Ivoire, Ghana, Lesotho, Namibia, Niger, Rwanda, Senegal, Sudan, and Swaziland. 3 Additional funds were mobilized through CIDA, a Multi-Donor Trust Fund (MDTF), 18 and the French Global Environment Facility (FFEM). Fundraising was expected to be continued by the partnership throughout the life of the ASP, including from the partners themselves. 6. Rationale for World Bank assistance: The World Bank intended to support ASP-P1 based on its comparative advantage in having: (i) significant global experience in managing MDTFs and resource mobilization; and (ii) unique expertise in managing complex, and regional programs, establishing broad-based partnerships and leveraging additional financing. As a GEF Implementing Agency, the World Bank had a responsibility to implement activities toward achievement of GEF operational programs. 19 7. Higher-level objectives: At the national level, ASP-P1 intended to contribute to the participating countries’ development strategies in the areas of public health, poverty alleviation, environmental protection and agriculture, and to help them implement their respective National Implementation Plans (NIPs) for elimination of POPs. At the global level, ASP-P1 intended to support international efforts to eliminate POPs by contributing to the objectives of international agreements such as the Stockholm Convention on POPs, and the Basel, Bamako, Rotterdam and Biological Diversity Conventions. The program further intended to improve management of toxic chemicals, and promote alternatives to pesticide usage, including Integrated Pest Management (IPM) and Integrated Vector Management (IVM). Capacity building for pesticide management and disposal aimed at enhancing the overall capability of partner countries to manage other toxic chemicals and hazardous waste. 1.2. ORIGINAL GEO, PDO, AND INDICATORS 8. Global Environment Objective (GEO): Reduction of the effects of POPs on the Global Environment. 9. Program Development Objective (PDO): Selected African countries to eliminate inventoried publicly held obsolete pesticide stockpiles and associated waste; and implement measures to reduce and prevent future related risks.20 10. Indicators: The success of ASP-P1 in meeting its PDO was based on the following indicators: 18 The MDTF was funded by the European Commission (EC), the Swedish International Development Cooperation Agency (SIDA), the Swiss Agency for Development and Cooperation (SDC) and the Government of Denmark. 19 By implementing ASP-P1, together with FAO (which became a GEF Agency under the direct access to GEF financing in 2001), the World Bank intended to support the first large scale GEF-funded activity toward achieving objectives of the GEF Operational Program 14 (GEF OP14) for Reducing and Eliminating Releases of POPs, and support the role of GEF as a financing mechanism for the Stockholm Convention on POPs. 20 Corresponding Legal Agreements with ASP-P1 participating countries are discussed in each Technical Annex. 4 Table 1.1: ASP-P1 Original PDO Outcome and Results Indicators Original PDO Outcome Indicators as presented in the PAD 21 a. Inventoried publicly held obsolete pesticide stockpiles in 6 ASP-P1 countries eliminated. 22 b. Quantity of new publicly held obsolete pesticide stockpiles. c. No further accumulation of publicly held obsolete pesticide stocks. Original Results Indicators as presented in the PAD a. An inventory database in place in 7 ASP-P1 countries, accepted by the ASP partnership and being used and updated by government staff. b. In case of accident, emergency plans successfully implemented. c. Disposal contract completed in 6 ASP countries in line with national and international regulations. 23 d. Quantity of accumulation of new obsolete pesticide stockpiles minimized. e. Government adopts of International Code of Conduct on pesticide distribution and use. f. Technical Support Unit (TSU) fully funded and staffed by end of year 1. g. Client satisfaction as determined by survey. h. Preparation activities for ASP-P2 countries complete – national programs identified, selection conditions met, country counterparts in place and funding secured. i. Strategic studies delivered, e.g., alternative disposal technologies and locust control issues assessed and adopted if ASP standards met. j. NGO participation in project activities in all ASP-P1 countries participating in at least two of the following: awareness raising, monitoring, capacity building and communication. k. Communication strategy implemented. l. Monitoring reports delivered with sufficient data on time to GEF, donors and the World Bank. m. ASP- Project Coordination Unit (PCU) moving to African organization within ASP-P1. 1.3 REVISED GEO, PDO, AND INDICATORS 11. The ASP-P1 GEO, PDO, and indicators were not revised. 1.4 MAIN BENEFICIARIES 12. Local beneficiaries: Primary beneficiaries were intended to be poor and farming communities. The project aimed to improve their quality of life by reducing environmental health risks. (i) Poor communities: Since obsolete pesticide stockpiles were often located near poorer communities where storage conditions were sub-standard and awareness to the proper use of pesticides was low, the intended primary beneficiaries were the urban and rural poor. (ii) Farming communities: A wide range of pesticides had been used for pest and vector management in agricultural areas, with pesticide distributors and small-scale farmers often having low levels of awareness for exposure risks, little knowledge of safe pesticide use, inadequate storage facilities, and limited alternative pest control methods. 13. National beneficiaries: The governments, in particular the ministries of health, environment, and agriculture in the participating countries were intended to benefit from the program through reduced loss in capital, and increased pesticide management capacity. 21 The indicators presented in the results framework differ from those presented in the arrangements for results monitoring. This ICR draws its evaluation on the indicators presented in the results framework. 22 Elimination of obsolete pesticides was not targeted in ASP-P1 Nigeria. 23 Elimination of obsolete pesticides was not targeted in ASP-P1 Nigeria. 5 14. Global beneficiaries: The global communities of people, species, and ecosystems were considered as global beneficiaries. Taking into account that obsolete pesticides are often characterized by high persistence in the environment, 24 the negative environmental impact in the form of damaged ecosystems had been considered serious. Extensive cleanup and contamination-prevention activities were, therefore, expected to generate substantial global environmental health benefits. 1.5 ORIGINAL COMPONENTS 15. In support of its PDO/GEO, ASP-P1 had four components: (1) country operations; (2) technical support; (3) crosscutting activities; and (4) project coordination. • Component 1 - Country Operations (GEF: US$21.74 million; Total: US$51.37 million): This was the program’s principal component, encompassing a full range of disposal, cleanup and prevention activities to achieve the PDOs in the participating countries. Sub-components were: (i) cleanup and disposal; (ii) prevention of pesticide accumulation; (iii) capacity building; and (iv) project management and monitoring. Country-specific details on these activities are outlined in the respective annexes. • Component 2 - Technical Support (GEF: US$3.26 million; Total: US$4.31 million): This component was intended to provide highly technical and specialized expertise required for implementation, supervision and monitoring of country-level activities outlined under the first component, including: technical advice requested by the countries for the design and delivery of training; production of technical guidelines for cleanup and prevention operations; assistance in managing technical aspects of procurement and supervision of specialized contractors; advice on linking countries with other specialized agencies and organizations; enhancement of health and safety; assessment of laboratory capacities; and oversight of Monitoring and Evaluation (M&E). The technical support was also intended to include advice on disposal options, targeted to specific country and site conditions. FAO was expected to be responsible for hosting a Technical Support Unit (TSU) to coordinate delivery of technical services envisaged under this component and to tap into pertinent expertise of other ASP stakeholders. 25 • Component 3 - Crosscutting Activities (Total: US$2.44 million): This component was expected to deliver knowledge management, awareness-raising, strategic studies and outreach services to ASP-P1 through a coordinated multi-country, multi-partner approach. The key sub-components included outreach to promote ratification of conventions relevant to the PDO, building Non- Governmental Organization (NGO) capacity, knowledge management, and communication. The strategic studies sub-component was intended to include an evaluation of alternative technologies for the disposal of obsolete pesticides. In addition, these studies were expected to support better understanding of environmental health risks associated with the specific pesticides, and to contribute to the prioritization and decision-making process within each country. WWF and PAN-UK were to host the Crosscutting Activities Management Entity (CCAME) responsible for delivery of support services under this component. 24 For example, DDT persistence in soil ranges from 22 to 30 years, Toxaphene up to 14 years, and Dieldrin up to 7 years. 25 E.g., United Nations Environment Program (UNEP), World Health Organization (WHO), Basel Convention Regional Centers, NGOs, and the private sector (represented by CLI). 6 • Component 4 – Project Coordination (Total US$2.44 million): This component aimed to coordinate the implementation and monitoring of ASP-P1 across the various partners. In addition to overall coordination, key activities included developing and ensuring implementation of a fund-raising strategy, acting as a Secretariat to the ASP Steering Committee (ASP-SC), maintaining awareness at the policy level about the project, reporting to donors, preparing work plans, progress reports and financial reports, coordinating independent project monitoring, and coordinating the design of follow-on projects. It was intended that initially, the World Bank would host a Project Coordination Unit (PCU), which during the implementation phase of ASP- P1 would be transferred to an appropriate African organization (e.g., the New Partnership for Africa’s Development (NEPAD)). 26 1.6 REVISED COMPONENTS 16. The ASP-P1 components were not formally revised through a World Bank-approved restructuring. However, four ASP-P1 country projects (Ethiopia, South Africa, Tanzania and Tunisia) underwent design adjustments, including revision of components. The associated revisions made are outlined in the country-specific Technical Annexes. 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY 17. ASP-P1 was prepared between 2001 and 2005. It took more than four years from the time of GEF eligibility in July 2001, and over three years from concept note review in July 2002 to the program’s approval by the Board in September 2005. The long preparation time resulted from a lengthy partnership-building process at the program-level caused by a complex envisaged implementation and partnership arrangement. Associated discussions included defining each partner’s role, clarifying the World Bank’s legal, fiduciary and operational requirements for a high-risk regional program, and avoiding conflicts of interest among contracting partners. Particularly challenging was the collaboration between FAO and the World Bank as a result of competing interests in accessing GEF funding resources. 27 While collaboration between CLI and the World Bank was adequate, it was also characterized by lengthy negotiations on finding modalities for channeling industry funds to the projects. A mutually endorsed funding agreement was not finalized prior to ASP-P1 Board approval in 2005; thus, a definition of CLI’s role and envisaged funding commitments was not included in the PAD or the Operational Manual. 18. Soundness of background analysis: ASP-P1 preparation was comprehensive and built on an extensive consultation process carried out during preparation comprising: (i) a body of analytical work on pesticide disposal and prevention, which informed project design; (ii) various workshops 26 NEPAD is an economic development program of the African Union, and aims to provide an overarching vision and policy framework for accelerating economic co-operation and integration among African countries. 27 Originally, FAO was envisaged to be a technical support agency to the program. However, FAO was approved as a GEF Agency in 2001, and therefore was able to access GEF funds directly without going through a GEF Implementing Agency as previously required. FAO’s role and responsibilities in ASP were therefore changed, and caused confusion among partners. 7 focusing on sharing experience and lessons learned, 28 which highlighted several crucial areas that were incorporated into design; and (iii) a country workshop focusing on gathering feedback from ASP-P1 participating countries. 29 ASP-P1 participating countries were selected, appropriately, on the basis of FAO guidelines such as country ownership and commitment, a well-identified lead agency, and stakeholder participation and national buy-in. Substantial background analysis was also undertaken at the country-level with close collaboration between governments and partners, as was evident through joint pre-appraisal missions to all participating ASP-P1 countries, and pledged financial contributions. 19. However, while key preparatory outputs such as the PAD and the ASP-P1 generic Operational Manual provided adequate and comprehensive information for implementing program-level activities, country-specific information was lacking (see Section 2.1 of the individual Technical Annexes). Despite extensive consultations and preliminary inventories conducted by FAO, the nature and size of the obsolete pesticide stocks scattered across ASP-P1 participating countries were only broadly estimated during preparation; the discovery of higher or lower amounts of stocks in participating countries during implementation was therefore a possibility, which the program’s plans did not reflect sufficiently. It was further assumed that no new stocks of obsolete pesticides would be accumulated during program implementation and thereafter, which was unrealistic in view of food security concerns and intensified agriculture. The STAP Roster Technical Review of ASP-P1 rightly pointed to these risks and suggested “to investigate whether the conditions that had led to the initial stockpiling are still applicable, and if changed, how this would affect preventative measures to be promoted by ASP-P1”. 30 20. Assessment of project design: The program’s design suffered from several weaknesses: • The program design gave insufficient attention to specifying and defining the partnership arrangements, and the roles and responsibilities of each partner involved. ASP-P1 was a complex multi-regional, multi-country program involving a variety of partners and heavily relying on efficient and effective collaboration. With the exception of a set of ‘Partnership Principles for Implementation of ASP’ articulated in Annex 1 of the ASP-P1 PAD, no formal documents defined the partners’ respective roles and their collaboration in detail. Further, NGO participation in ASP-P1 was configured and intended to be managed at the program-level, whereas the collaboration arrangements at the country-level were unclear. An independent evaluation of the program’s design and initial implementation, finalized in June 2009, suggested that unclear definition of the roles and working relationships among the institutional partners, particularly those of the World Bank and the TSU, negatively affected the program’s efficiency and effectiveness. 31 • The uniform approach ASP-P1 used to the project implementation in participating countries did not take individual country contexts sufficiently into account. The PDO, associated indicators, and 28 In addition to a series of informal partnership meetings, a lessons learned exercise at the First Continental Conference for Africa on the management of hazardous waste (Rabat, January 2001), and a lessons learned workshop for disposal contractors (March 2002), were conducted. 29 ASP-P1 country workshop (April 2002) 30 See Annex 15 in South Africa and Tunisia: Africa Stockpile Program – Project 1: PAD (Report No: 32232-MNA), August 19, 2005 31 For more detail see Independent Evaluation of Design and Initial Implementation of Africa Stockpiles Program – Evaluation Report, May 2010, COWI – Document no. 69292-A-1 8 components were pre-configured at the program-level and, as such, applied to all ASP-P1 participating countries. Country-specific environments were not sufficiently taken into account, and during implementation, it became clear that the program approach was not suitable for all ASP-P1 participating countries. While the PAD noted a need for flexible implementation at the country-level, discussions between the World Bank and ASP-P1 participating countries on how to revise the design, together with internal World Bank discussions on how to process these restructurings, were lengthy. • The design of the PDO, associated indicators, and components was overly ambitious. Given the lack of accurate data on obsolete pesticide stocks that was used during ASP-P1 preparation, the PDO, associated indicators, and components were unrealistically designed. A comprehensive inventories and Country Environmental and Social Assessments (CESAs) to determine the size and nature of stocks and associated disposal requirements were only envisaged to be conducted during implementation. • The sequential design of Component 1 was not realistically reflected in the planned implementation and disbursement schedule. The completion of an inventory was necessary to prepare a CESA, which at the same time had to be completed before disposal activities could be initiated. Potential delays and their subsequent impact on overall project implementation were not adequately taken into account in the forecasted disbursement schedule and overall project timeframe. 32 21. Government commitment: Adequate commitment at entry of the participating governments towards the issue of obsolete pesticides was an eligibility criterion for participation in ASP-P1. Governments had to ratify the relevant conventions and show a good level of readiness to implement planned activities. Please refer to each participating country’s Technical Annex for details. 22. Assessment of risks: Program-level risks were adequately identified and outlined in the PAD. They were presented in the form of a matrix and represented a summary of risk assessments undertaken during identification, pre-appraisal and appraisal missions to ASP-P1 participating countries. Even though the assessment of risks was comprehensive and included a broad range of mitigation measures in generic terms, an assessment of country-specific risks and mitigation measures was lacking. At the same time, it should be noted that it was difficult to assess country-specific risks in the absence of inventories of obsolete pesticides and CESAs, which were only planned to be undertaken during project implementation. 23. Quality at Entry Assessment (QEA): An assessment conducted by the Quality Assurance Group (QAG) on ASP-P1 in 2007, rated the QEA of the program moderately satisfactory. 33 The panel considered that despite the lengthy preparation time the project was approved prematurely, but concurred with the World Bank team that delaying Board presentation could have halted the hard- negotiated momentum among clients and partners and led to the partnership’s collapse. The panel further stated that more thought should have been given to the policy environment, that much of the 32 For example, procuring a consulting firm to conduct the CESA and a service provider for the disposal of obsolete pesticides, and overall preparation of the CESA were lengthy procedures that should have been allocated sufficient time. 33 AFRICA: 3A-W Africa Stockpiles 1 GEF (FY06) (P075776): Eighth Quality at Entry Assessment (QEA8) Fiscal Year 06-07, June 27, 2007. The assessment reviewed ASP-P1 on the basis of two of the participating countries – ASP-P1 Tunisia and ASP-P1 South Africa. 9 first year’s program should have been carried out prior to approval, and that the prospect for completing the approved project within the prescribed four-year-time frame was poor. 34 2.2 IMPLEMENTATION 24. ASP-P1 was implemented at the program-level, and in ASP-P1 participating countries. Implementation was characterized by a complex partnership, substantial delays, and varying performance across the participating countries. In 2010, activities at the program-level were terminated due to several partners having exhausted their resources, and ASP-P1 continued to be implemented solely at the country-level. 25. Key factors that affected implementation and outcomes at the program level, which simultaneously translated into implementation delays at the country-level included: • The envisaged governance structures as outlined in the PAD never became operational, and affected program implementation. The PCU, intended to coordinate, monitor, and evaluate the implementation of ASP-P1 among the partners, and the ASP-Steering Committee (SC), intended to serve as advisory structure, were never established. 35 Instead, an ASP Implementation Committee (ASPIC) was created as a forum for partners to oversee program implementation. The partners (i.e., FAO, CLI, PAN-UK, WWF, and the World Bank) met regularly through a combination of virtual and face-to-face meetings. However, ASPIC was never formalized, which resulted in an unclear overall governance structure, and unclear roles and responsibilities, cumbersome decision-making processes and coordination, and a lack of an independent M&E function. Eventually, following the GEF failure to approve ASP-P2 in 2009, future initiatives in the areas of pesticide management were prepared and managed individually by partners, outside the program. 36 • Lack of clearly defined roles, responsibilities, and partnership principles significantly affected collaboration among partners, and translated into implementation delays at the country-level. o Collaboration between FAO and the World Bank was challenging throughout implementation. The unclear roles of the two agencies, and their different organizational cultures, and internal procedures and requirements, translated into difficult discussions on standard document formats for project-related activities, delays and conflicting messages given to ASP-P1 participating countries, 37 exacerbating the already weak operational and technical capacity of the ASP-P1 participating countries. In 2008, FAO and the World Bank prepared and agreed on a memo, which defined exact roles, responsibilities, and procedures. While this memo improved collaboration between the two entities, disagreements remained. 38 34 Further ratings included: Environmental aspects satisfactory; strategic relevance and approach, technical, financial and economic aspects, poverty, gender and social development, fiduciary aspects, policy and institutional aspects, risk assessment moderately satisfactory; and World Bank inputs and processes moderately satisfactory. 35 NEPAD was intended to host the PCU, which did not materialize due to insufficient capacity. 36 ASP-P2 was not approved by GEF due to a lack of resources at the end of a GEF replenishment cycle at the time, and poor performance of ASP-P1. 37 Standard documents and generic Terms of Reference (TORs) that had been agreed on in the context of the ASP-P1 Operational Manual were reconsidered during implementation. 38 For example, FAO and the World Bank had different views on the roles, the national government staff could play in disposal operations: while FAO was in favor of including government staff to pursue cost-savings and increase national capacities, the World Bank opposed their participation as health and safety risks were high. 10 In 2010, difficulties subsided as FAO had exhausted its GEF funds and significantly minimized its support to the participating countries. To compensate for this, the World Bank increased its own support through technical and operational consultants funded by the MDTF. o Delayed financial support obtained from CLI. In the beginning of implementation, it was planned that CLI funds would be channeled through the World Bank and/or FAO; however, shortly after, this was judged to be too complex and time-consuming. Instead, it was agreed that CLI would channel its contributions through direct financing agreements with recipient governments to support Technical Advisors for Disposal (TADs) as well as parallel contracts with hazardous waste management companies. This processes proved to be equally lengthy, ultimately further delaying the technical support and the financial contributions from CLI. • World Bank institutional challenges contributed to program implementation delays. ASP-P1 was affected by a complex supervision arrangement within the World Bank. Since the program operated in countries from two different Regions (Africa and the Middle East and North Africa), different staff were involved. In addition, ASP-P1 was supervised by a program-level Task Team Leader (TTL) with full budget authority, a program-level co-TTL responsible for francophone ASP-P1 participating countries, and co-TTLs at the country-level with no budget authority. Particularly during the first three years, this arrangement translated into confusion among staff, partners, and Project Management Units (PMUs), cost overruns, and lengthy World Bank review and clearance processes of project-related documents. The program also experienced a high turnover in World Bank TTLs, 39 requiring transition periods for hand-over activities. Lastly, the World Bank’s internal systems were challenging for administration of an umbrella program with multiple sub-projects in more than one region, ultimately causing high transaction costs and inaccurate reporting. Efforts were undertaken to solve these system-related challenges, but without success. 40 • Insufficient fundraising efforts. Higher than estimated stocks of obsolete pesticides and contaminated soils together with the depreciation of the US dollar resulted in a funding gap for the ASP-P1 disposal component of at least US$10.6 million by 2010. 41 The lack of funds was subsequently mitigated by formally revising the PDO and associated indicators in four of the ASP-P1 participating countries where gaps existed. However, significant efforts to secure additional funds were not pursued by the partners and ultimately resulted in non-disposal of some of the inventoried obsolete pesticides and associated wastes. • Procurement of disposal services represented a challenge in all ASP-P1 participating countries: Procurement of the highly specialized disposal services under World Bank procurement guidelines was a challenging and time-consuming process, which was exacerbated by weak national procurement capacity in almost all ASP-P1 participating countries (see Section 2.4 in the Technical Annexes). 39 Approximately eight different TTLs led the program during preparation and supervision. 40 The second and third set of country-level projects (ASP-P1 Mali, Morocco, Tanzania (P103189); and ASP-P1 Ethiopia (P105711)) could not be processed under the project number which was used for ASP-P1 South Africa and Tunisia (P075776) since each Board presentation required a separate number. Resulting recording and reporting challenges caused high administrative transaction costs, and did not allow for a detailed expenditure overview at the program-level. 41 For more detail see Independent Evaluation of Design and Initial Implementation of Africa Stockpiles Program – Evaluation Report, May 2010, COWI – Document no. 69292-A-1 11 • The Level 1 restructuring in four ASP-P1 countries positively affected implementation and allowed eventual PDO achievement in three countries: At mid-term review, it was recognized that the PDO was unlikely to be achieved by the original closing date. It was agreed to restructure ASP-P1 to better reflect country-specific environments. With the discovery of higher than budgeted stocks of obsolete pesticides, including associated wastes, it was decided to adopt a blended approach of disposal overseas and lower cost safeguarding in-country based on risk profiling of the inventories of pesticides and associated wastes, and the availability of satisfactory storage facilities (i.e., obsolete pesticides and higher risk stocks of heavily contaminated soils being disposed of overseas while lower risk stocks of associated wastes being safeguarded in- country). In addition, the original PDO had assumed no new accumulations of obsolete pesticides during the life of the project, which had been proven difficult to achieve in the context of agricultural intensification and food security concerns; therefore it was agreed to focus on developing a strategy for management of future accumulations. The restructuring, which was approved in 2011 for ASP-P1 Ethiopia, ASP-P1South Africa, ASP-P1 Tanzania, and ASP-P1 Tunisia, eventually, allowed the countries (except ASP-P1 South Africa) to achieve targeted outcomes. 42 26. Learning Review of Regional Projects: A review conducted by the World Bank in 2009, rated the program moderately unsatisfactory. The review suggested to: (i) strengthen the program-level results framework and define country-specific ones; (ii) clarify roles and responsibilities among partners; and (iii) ensure low TTL turnover. 43 Furthermore, an independent evaluation of the program’s design and initial implementation, finalized in June 2009, stated that while the program’s partnership was valid, it also was a challenge. It stated that each of the specialized partners was characterized by differing organizational cultures, traditions, practices, procedures, and interests, and thus making a close collaboration challenging. The report emphasized the importance of finding common solutions on tasks and procedures. 44 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTILIZATION 27. Overall M&E – Rating Moderately Unsatisfactory. 28. M&E design – Rating Unsatisfactory: At the program-level, a complex M&E system, including a results framework, had been developed and was described in detail in the PAD and Operational Manual. However, the results framework and arrangements for results monitoring showed slight discrepancies, and PDO and associated indicators were vaguely formulated and did not cover all expected outcomes, having made it difficult to assess impacts. Moreover, M&E systems specific to the individual ASP-P1 participating countries were not prepared during preparation. Country-specific results frameworks, intended to be finalized by the TSU during project launch, were not prepared; 42 For ASP-P1 Mali, the Level 1 restructuring was prepared but not approved because of the March 2012 coup d’état and subsequent suspension of activities. For ASP-P1 Morocco, a Level 1 restructuring was not prepared because the project closed as originally scheduled in 2010. For ASP-P1 South Africa, a Level 1 restructuring was approved but the late request for an extension of the closing date, eventually prevented PDO achievement. 43 AFRICA: 3A-Africa Stockpiles 1 MMT GEF (FY07) (P0103189): Learning Review of Regional Projects, December 11, 2009. The assessment reviewed ASP-P1 on the basis of three of the participating countries –Mali, Morocco, and Tanzania 44 For more detail see Independent Evaluation of Design and Initial Implementation of Africa Stockpiles Program – Evaluation Report, May 2010, COWI – Document no. 69292-A-1 12 only country-specific indicators linked to the different project components were developed, but these did not allow for adequate monitoring of project implementation progress as they focused on measuring outputs as opposed to progress towards project outcomes. 29. M&E implementation and utilization – Rating Moderately Unsatisfactory: ASP-P1 lacked systematic monitoring and reporting at the program-level throughout implementation, largely due to failure to establish the PCU, which among other tasks, was expected to coordinate the independent M&E of the program. The ASPIC attempted to monitor implementation of activities during its regular meetings, and the World Bank produced comprehensive annual consolidated technical and financial reports, which were disseminated to all partners and donors. However, these reports did not include systematic monitoring of implementation progress against the PDO and Intermediate Outcome Indicators; thus, overall progress (including the technical assistance and cross-cutting activities intended to be implemented by FAO and the NGOs) was difficult to assess. 30. At the country-level, until mid-term review, ASP-P1 participating countries operated without an M&E system in place. Eventually in 2009, the TSU provided the countries with an M&E tool, however, it was considered too complex by some of the countries, and introduced too late in the process to be used efficiently. Only in 2011, a results framework was introduced in four ASP-P1 participating countries as part of their restructuring, which strengthened the M&E of the projects. 2.4 SAFEGUARDS AND FIDUCIARY COMPLIANCE 31. Safeguard compliance – Rating Satisfactory: ASP-P1 triggered two safeguard policies: OP 4.01 on Environmental Assessment and OP 4.09 on Pest Management. In accordance with the World Bank’s environmental and social policies and due to potentially significant adverse environmental impacts, ASP-P1 was classified as Category A, requiring full assessment. Environmental and social impacts were adequately assessed in a detailed Framework Environmental Assessment (FEA), as were their mitigation measures in an Environmental and Social Management Framework (ESMF). Formal country consultations were held in Addis Ababa (April 2002) to discuss details of the program’s design, which led to strong endorsement of the approach. 45 An Environmental and Social Assessment Synthesis Report was prepared based on these documents, adequately consulted in the ASP-P1 participating countries, and disclosed to the World Bank’s InfoShop in March 2004. 46 The document provided guidance to the ASP-P1 participating countries on how to prepare country-specific safeguard instruments, including CESAs. For safeguard compliance at the country-level, see Technical Annexes. 32. ASP-P1 did not finance the procurement of pesticides or any other agricultural chemicals, nor did it lead to their increased use. The prevention component was initially designed to reduce reliance on pesticide use by promoting IPM, improving pesticide procurement and management systems and building capacity to prevent future accumulation of pesticides. To meet the World Bank's safeguards requirements on Pest Management, the program needed to promote IPM, improve pesticide management, and build capacity to address Integrated Pest Management (IPM) and pesticide management. ASP-P1 provided adequate training to all participating countries on promoting IPM, on 45 The meeting brought together representatives from Botswana, Cameroon, Cote d’Ivoire, Ethiopia, Lesotho, Mali, Morocco, Mozambique, Namibia, Niger, Nigeria, South Africa, Swaziland, Tunisia and Tanzania. 46 The same document was posted on the World Bank’s Public Information Center (PIC) on June 29, 2005, and distributed to the Board on July 4, 2005. 13 the adherence to the International Code of Conduct on pesticide distribution and use, and on using the Pesticide Stock Management System (PSMS) (all countries have access to the system and many have been using it beyond ASP). The project did not include country specific IPM activities, since they were outside the scope of the project. The activities proposed by ASP-P1, which were executed in all participating countries largely met the requirements of the pest management policy. 33. Financial Management compliance – Rating Satisfactory: As the signatory of all ASP-P1 GEF trust fund grant agreements, as well as the CIDA and MDTF administration agreements, the World Bank was adequately supervising the use of these funds through regular missions, review of progress and financial reports, audits, follow-up meetings, and regular contact as needed. The World Bank produced monthly financial reports, which could be accessed by authorized users via the World Bank donor portal, or upon request. 34. Procurement – Rating Not Applicable: The vast majority of ASP-P1 procurement activities was carried out by the ASP-P1 projects at the country-level and details can be found in Section 2.4 in the Technical Annexes. The ASP partners each used their own procurement procedures to procure necessary services, consultancies and pay for operating costs without any reported problems. 2.5 POST-COMPLETION OPERATION/NEXT PHASE 35. Following the disapproval of ASP-P2 for funding by GEF (due to a lack of resources at the end of a GEF replenishment cycle at the time and poor implementation performance of ASP-P1), CLI, FAO, PAN-UK and the World Bank began to individually develop and implement pesticide-related projects. These projects have been implemented outside the ASP-P1 framework, but have been using ASP guidelines, tools, and experience, more specifically: • CLI has supported projects in Cameroon, Ghana, Kenya, Malawi, and Nigeria, which safeguarded a total of 811 tonnes of high-risk obsolete pesticides and containers. • FAO has implemented projects in other countries including Mozambique, and currently conducts a disposal and prevention project in several West African countries using GEF and other donor resources. 47 • The World Bank has been implementing a POPs project in Egypt, to help the government meet its obligations under the Stockholm Convention, and contribute to capacity. The World Bank is also currently discussing a possible follow-up project in Mali, which would focus on finalizing prevention and disposal activities. • PAN-UK has been strengthening civil society in ASP-P1 countries through capacity building and training in health and ecotoxicology monitoring, and supporting prevention activities. 3. ASSESSMENT OF OUTCOMES 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION 36. Relevance – Rating High: The elimination of obsolete pesticide stocks remains a development priority. Rural communities cannot hope to advance if the soil and water, upon which their 47 The project is conducted in Comité permanent Inter-Etats de Lutte contre la Sécheresse dans le Sahel (CILSS) member countries such as Burkina Faso, Cape Verde, the Gambia, Guinea Bissau, Mali, Mauritania, Niger, and Senegal. 14 livelihoods and health depend, are contaminated with pesticides, and urban populations cannot prosper if they are suffering from severe illnesses caused by pesticide poisoning. 37. National development strategies of ASP-P1 participating countries confirm the importance of eliminating hazardous waste (see Section 3.1 in Technical Annexes), as do World Bank priorities. The World Bank’s environment strategy from 2001, 48 which guided the institution’s work in this sector over the past decade highlighted pesticide residues as environmental and health risk factor, and today, the World Bank’s dual goal of ending extreme poverty and promoting shared prosperity is to be achieved in an environmentally sustainable manner, taking into account the impacts of climate change, environmental degradation, and ecosystem changes. 49 The World Bank’s current environment strategy places particular attention on helping countries to address environment-related health issues by supporting the creation of regulatory, economic and financial incentives to reduce pollution and increase clean production. 50 In Africa, the World Bank aims to ensure the availability of transparent environmental regulatory frameworks and competent institutions for monitoring environmental compliance. New analytical work at the regional level on hazardous waste, POPs, and the clean agenda is expected in the future. 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE 38. This assessment is based on indicators captured in the results framework of the PAD. 51 In 2011, project activities in four of the ASP-P1 participating countries were restructured, as it was recognized that their design required revision to permit PDO achievement; the restructuring subsequently permitted PDO achievement in three ASP-P1 countries. These revisions were, however, not captured at the program-level, as the PDO, associated indicators, and components at the program-level were not formally restructured. 39. PDO achievement – Rating Moderately Unsatisfactory: This assessment takes account of one PDO Outcome Indicator, which was partly (37%) achieved. 52 It also takes account of ten Results Indicators, of which four were fully (100%), four partly (50-83%), and two not (0%) achieved.53 The assessment is framed by the program’s envisaged outcomes (i.e., risk reduction and pesticide disposal, improved pesticide management, and service delivery to ASP-P1 countries). 40. Risk reduction and pesticide disposal. The level of risk emanating from publicly held obsolete pesticides and associated wastes to nearby communities and critical natural resources was successfully determined and quantified through the completion of national inventories in five of the ASP-P1 participating countries evaluated in this ICR (Results Indicator #1 – 83%). 54 The information obtained (location and state of protection of the storage sites, contaminant type and quantity, and 48 See Making Sustainable Commitments – An Environment Strategy for the World Bank, June 18, 2001, World Bank 49 See World Bank Group Strategy, August 16, 2013, World Bank 50 See How Environment Sustains Development – 2011 World Bank Group Environment Strategy: Toward a Green, Clean and Resilient World for All, January 4, 2012, World Bank 51 The indicators presented in the results framework differ from those presented in the arrangements for results monitoring. 52 The second PDO Outcome Indicator was not measurable; the third PDO Outcome Indicator was not taken into account as it corresponded with the second PDO Outcome Indicators. 53 Three additional Results Indicators were not taken into account as one was not applicable, one corresponded with a PDO Outcome Indicator, and one had already been achieved prior to program approval. 54 The originally planned countrywide inventory of publicly held obsolete pesticides and associated waste was not carried out in ASP-P1 South Africa, as it was considered cost-ineffective in an environment where most stocks were held by private farmers and not by the public sector (see Technical Annex IV for details). 15 general condition of the stockpiles) provided the governments with the baseline data and knowledge necessary to inform eventual disposal of stockpiles. 41. Despite the program’s complexity, challenges, and poor initial performance, mitigation efforts eventually resulted in complete disposal of inventoried publicly held obsolete pesticides in ASP-P1 Ethiopia, ASP-P1 Tanzania, and ASP-P1 Tunisia, having significantly reduced risks to the environment and public health. A total of 3,164 of the inventoried 8,949 tonnes of publicly held obsolete pesticides and associated waste were eliminated overseas in an environmentally and technically sound manner (PDO Outcome Indicator #1 – 35%) 55 by contracted specialized companies and in line with national and international regulations (Results Indicator #3 – 50%). 56 In South Africa, where obsolete pesticides were predominantly privately held, ASP-P1 succeeded, using CLI funding, in eliminating obsolete pesticides in one pilot province, totaling approximately 100 tonnes. 42. Improved pesticide management. The original target of not accumulating new obsolete pesticide stockpiles and associated waste was not measurable. At appraisal, it was estimated that approximately 4,750 tonnes of obsolete pesticides were scattered across the ASP-P1 countries; inventories conducted during implementation in the ASP-P1 participating countries revealed the existence of approximately 800 additional tonnes of obsolete pesticides. However, it is difficult to assess whether further stockpiles were accumulated during program implementation or whether the difference resulted from inaccurate estimates at the time of appraisal (PDO Outcome Indicator #2 – N/A). 43. The program was also successful in raising awareness on the health and environmental hazards of pesticides and encouraging safe pesticide handling through various outreach activities across ASP-P1 participating countries. An overarching communication strategy was developed by WWF, together with a communication strategy toolkit. On the basis of these tools and with support from WWF, national communication strategies were subsequently developed by all; however, they were only implemented by four ASP-P1 participating countries (Results Indicator #11 – 67%). Overall, WWF and PAN-UK actively participated in project activities related to awareness raising, monitoring, and communication (Results Indicator #10 – 100%). 44. Service delivery to ASP-P1 countries. The TSU, hosted by FAO, was fully funded and staffed shortly after the program became effective (Results Indicator #6 – 100%). However, it should be noted that the technical assistance the ASP-P1 participating countries were expected to receive, was not provided to the extent foreseen in the PAD, particularly in the areas of M&E, and preparation of contracts (e.g., preparation of technical specification and carrying out technical evaluations for project consultants and contractors). This lack of technical support was compensated with technical experts, temporarily funded by the World Bank, and the recruitment of TADs financed by CLI. A survey, specifically targeted at measuring ASP-P1 participating countries’ satisfaction with services received from partners, was not conducted (Results Indicator #7 – 0%). 45. The program did not succeed in launching ASP-P2. In April 2009, the World Bank and FAO jointly submitted a funding proposal to the GEF for US$20.7 million in support of ASP-P2. The second 55 Only eliminated obsolete pesticides and associated waste stockpiles are included. The stocks that were safeguarded (as alternative to elimination following the Level 1 restructuring in four ASP-P1 participating countries), and those that were held privately in the case of ASP-P1 South Africa, are not included in the assessment of PDO Outcome Indicator #1. 56 Disposal contracts were completed for ASP-P1 Ethiopia, ASP-P1 Mali, ASP-P1 Tanzania, and ASP-P1 Tunisia, but only implemented in ASP-P1 Ethiopia, ASP-P1 Tanzania, and ASP-P1 Tunisia. In South Africa, privately held obsolete pesticides were disposed of overseas in line with national and international regulations, however, with co-financing provided by CLI. 16 series of projects of the program was planned to be implemented in 22 additional African countries, and the proposal reflected recommendations obtained through the independent evaluation previously mentioned. However, ASP-P2 was not accepted by GEF due to a lack of resources at the end of a replenishment cycle at the time, and the poor performance of ASP-P1; instead, FAO and the World Bank were encouraged to submit funding requests for individual country or regional ASP projects once GEF funds became available (Results Indicator #8 – 50%). See Section 2.5 for details. 46. The PCU did not become operational due to a lack of capacity within NEPAD, the entity that had been selected to host the unit (Results Indicator #13 – 0%). Consequently, ASP-P1 was not successful in strengthening regional capacity for management of future program-related activities. 47. To inform sustainable decision-making in the ASP-P1 participating countries, a study on disposal technology options was prepared by WWF in June 2008. The study provided information on the treatment and disposal options of obsolete pesticides and associated wastes. Furthermore, a study on locust control issues was prepared by FAO as part of the Red Locust Control Program (Results Indicator #9 – 100%). 48. While implementation was not systematically monitored at the program-level, the World Bank produced technical and financial reports on a yearly basis. These reports provided annual overviews of implementation progress achieved by the program and ASP-P1 participating countries, and were shared with partners and donors (Results Indicator #12 – 100%). 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE 49. ASP-P1 aimed to directly contribute to the GEO of reducing the effects of POPs on the global environment through: (i) improving the quality of life in poor communities by reducing environmental risks; (ii) improving environmental protection; and (iii) enhancing the capacity of the agricultural sector to better manage crop pests. GEO achievement is measured by PDO achievement, and, thus is also rated Moderately Unsatisfactory. 3.4 EFFICIENCY 50. Efficiency – Rating Low: This rating represents a broad estimate, as financial and economic data to support this rating are scarce. At appraisal, neither an economic nor a financial analysis was undertaken for ASP-P1 at the program or at the country-level due to non-measurable benefits of reducing the risk of environmental contamination. A lack of reliable baseline data in combination with controversial aspects related to the valuation of human life and natural resources did not permit a cost-benefit analysis. For GEF purposes, the preparation of ASP-P1 at the program-level included an incremental cost analysis for achieving global benefits to address the issue of obsolete pesticides in the participating countries. 51. In general, ASP-P1 resource use could not be accurately evaluated because: (i) No accurate quantitative data were available on the financial and human resources invested in the program’s preparation by any of the partners; 57 (ii) with an envisaged PCU that never became operational, there was no M&E of the resources spent by FAO available to the World Bank, and thus no accurate data available; and (iii) the World Bank’s budgeting system made it difficult to adequately capture 57 See Independent Evaluation of Design and Initial Implementation of Africa Stockpiles Program – Evaluation Report, May 2010, COWI – Document no. 69292-A-1. 17 allocated funds and associated expenditures of a regional program with sub-projects, ultimately causing high transaction costs and inaccurate reporting. In addition, efficiency of capacity building and prevention activities is difficult to quantify due to time disconnect between project support and actual results. 52. When specifically considering efficiency of the disposal component, the unit cost for disposal (US$/tonne disposed) is used for measuring efficiency. Based on approximately US$15 million spent on the disposal of 3,240 tonnes of obsolete pesticides between 1990 and 2004, it was estimated that 300-900 tonnes could be disposed of per annum at the cost of US$3,400 per ton. At closing, 3,164 tonnes of obsolete pesticides and associated waste had been disposed of at the cost of approximately US$3,103 per tonne, which represents a slightly lower cost than originally anticipated. 58 Nevertheless, efficiency is rated low because, while ASP-P1 disbursed 75% of GEF resources, only 37% of inventoried publicly held obsolete pesticides and associated waste were disposed of at program closing. 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING 53. Overall outcome – Rating Moderately Unsatisfactory: On the basis of the program’s relevance (high), efficiency (low), and PDO achievement (moderately unsatisfactory), the overall outcome is rated moderately unsatisfactory. This rating takes into account that the program’s preparation and implementation were accompanied by substantial challenges, but also suffered from shortcomings that significantly affected the envisaged outcomes. However, it should be highlighted that despite the program’s complexity, challenges, and poor initial performance, mitigation efforts, most notably a Level 1 restructuring, eventually permitted PDO achievement in three of the ASP-P1 participating countries. Risks to the environment and public health were significantly reduced through complete disposal of inventoried obsolete pesticides in ASP-P1 Ethiopia, ASP-P1 Tanzania, and ASP-P1 Tunisia. Table 3.1 briefly provides an overview of PDO achievements at the individual country- level; for details, see Technical Annexes. Table 3.1: Overview of ASP-P1 PDO Achievement at the Country-Level ASP-P1 ASP-P1 ASP-P1 ASP-P1 ASP-P1 ASP-P1 Ethiopia Mali Morocco South Africa Tanzania Tunisia Outcome against Moderately Moderately Moderately Moderately Unsatisfactory Unsatisfactory Original Unsatisfactory Unsatisfactory Unsatisfactory Satisfactory PDO Not Outcome Closed as Moderately restructured Moderately Highly against scheduled and Satisfactory Satisfactory due to political Unsatisfactory Satisfactory Revised PDO not extended situation 58 The calculation takes disposed amounts of obsolete pesticides and associated GEF resources spent in three ASP-P1 countries into account: (i) ASP-P1 Ethiopia disposed of 450 tonnes at a cost of US$1.70 million; (ii) ASP-P1 Tanzania disposed of 967 tonnes at a cost of US$5.07 million; and ASP-P1 Tunisia disposed of 1,730 tonnes at a cost of US$3.05 million. Costs include those associated with repackaging, transport, and incineration of obsolete pesticides and other related activities such as for example the inventory, safeguarding efforts, associated studies, and advisory services. 18 Overall Moderately Moderately (Weighted - - Satisfactory Satisfactory Satisfactory Unsatisfactory Outcome) 59 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS 54. Poverty impacts, gender aspects, and social development: The disposal of obsolete pesticides and associated wastes had a direct positive impact on poor and farming communities, species and ecosystems, located in proximity to the identified sites. Baseline data on affected populations, species, and ecosystems was not conducted at appraisal; exact impacts are therefore difficult to measure. 55. Institutional change/strengthening: Since the PCU had not become operational, institutional change and strengthening occurred at the country level: (i) the regulatory and legal frameworks for pesticide management were improved in Ethiopia, Mali, Morocco, Tanzania, and Tunisia; (ii) the logistics of hazardous waste transportation have been internalized in Ethiopia, Tanzania, and Tunisia, and the knowledge gained can inform future disposal activities; (iii) the experience and skills gained in managing the project can be applied to other projects and subsequently contribute to success; (iv) the experience gained in Ethiopia, Mali, South Africa, Tanzania, and Tunisia, in preparing social and environmental impact assessments and management plans; (v) knowledge gained on sound pesticide management through the various training courses all participating countries participated in (see Section 3.6 in Technical Annexes). 56. Other unintended outcomes and impacts: The private sector through CLI provided valuable technical support to ASP-P1 Ethiopia, Mali, South Africa, and Tanzania through the funding of international TADs. CLI further funded the disposal of stocks in Ethiopia, South Africa, and Tanzania; and played a key role in mobilizing South Africa’s crop protection industry to forming a public-private partnership with the government to dispose of obsolete pesticide stocks. 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS 57. Beneficiary surveys were not carried out. ICR missions, including stakeholder workshops were carried out in Ethiopia and Tanzania (see Section 3.7 of corresponding Technical Annexes). All ASP- P1 participating countries provided completion reports, except ASP-P1 Morocco; information provided therein was incorporated in this ICR. 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME 58. The risk to development outcome varies across ASP-P1 participating countries and has been assessed at the country-level. In Ethiopia and Tanzania, the risk is rated moderate and confined to the re- accumulation of obsolete pesticides, and a potential weakening of the governments’ commitment to sustaining and building on the projects’ outcomes; however, if improved legislations will be enacted, the governments are well positioned to sustainably manage and reduce the re-accumulation of 59 Overall weighted outcomes have been calculated by weighting the separate outcome ratings in proportion to the share of actual grant disbursements made in the periods before and after approval of the revision. See Section 3.5 in Technical Annexes for details. 19 obsolete pesticides. In Mali, the development outcome risk is rated high due to Mali's post-conflict and transitional situation, its strong focus on agricultural intensification, and the relatively low priority obsolete stock elimination is receiving in the country's recovery plans. In Morocco, risk is also rated high due to the government’s lack of capacity and resources to safeguard and dispose of its stocks. In South Africa, the risk is rated moderate given the uncertainty about the government and the private industry to approve and implement the newly developed Integrated Waste Management Plan (IWMP) to collect and dispose of both private and public stocks. Finally, Tunisia’s development outcome risk is rated low based on the government’s demonstrated commitment and capacity, as well as the improved legislation for pesticide management. See Section 4 of the Technical Annexes for further detail. 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE 5.1 WORLD BANK PERFORMANCE 59. World Bank performance in ensuring quality at entry – Rating Moderately Unsatisfactory: The World Bank together with the other partners conducted numerous preparatory missions to all participating countries, which resulted in considerable background information, project plans and candid analysis of issues and challenges. FM and procurement assessments were also adequately conducted for all countries (see Technical Annexes for details), as well as for WWF, PAN-UK and PAN-Africa, which were to be recipients of DGF and MDTF funds during project implementation. However, although the PADs were comprehensive and described in detail the envisaged activities at the program-level, they were lacking country-specific details, most notably country-specific results frameworks. 60. Preparation at the program-level was, lengthy due to difficulties in finding common agreement among partners on the program’s design and implementation arrangements (see Section 2.1 for details). The World Bank’s authority during these negotiations was undermined by a high turnover in TTLs. The challenging partnership-building process ultimately resulted in the PADs giving insufficient attention to specifying and defining the collaboration including oversight, and roles and responsibilities of each partner involved, which significantly affected program- and country-level implementation (see Section 2.1 for details). 61. Quality of World Bank supervision – Rating Moderately Unsatisfactory: Issues caused by the lack of a PCU and unclear definition of the partnership arrangement at the program-level, dominated the first three years of program implementation, and equally affected project implementation at the country-level. Lengthy discussions between FAO and the World Bank on standard formats and guidelines, which ASP-P1 participating countries needed for project implementation, consequently delayed implementation progress. 60 In 2008, corrective action in form of a jointly written memo, which defined exact roles, responsibilities, and procedures, alleviated some of the difficulties; however, disagreements remained on a few issues (see Section 2.2 for details). 62. Since efforts to create an interim PCU secretariat had failed, the World Bank team proactively substituted for a number of the PCU tasks in order to ensure satisfactory program implementation. In 60 Standard documents, Terms of Reference (TORs), and procurement guidelines, required for project implementation and originally agreed on in the context of the ASP-P1 Operational Manual were reconsidered during implementation. 20 addition, the World Bank’s internal implementation arrangements were challenging, due to the need to bridge two World Bank administrative regions, a program level TTL and multiple co-TTLs, which finally often translated into lengthy review clearance processes of project-related documents. 61 Lastly, the World Bank’s internal systems provided challenges for administering an umbrella program with multiple sub-projects in more than one region, ultimately causing high transaction costs and inaccurate reporting. Efforts were undertaken to solve these system-related challenges, but without success. 63. Following mid-term reviews in 2008 and 2009, the World Bank team increased its efforts to effectively address the program’s implementation delays and maximize the likelihood of program success in ASP-P1 participating countries. Corrective actions included increased and more targeted operational and technical support through consultants, extension of closing dates, and reallocation of existing GEF funds to maximize outcomes. Project design weaknesses were addressed through the restructuring of four of the ASP-P1 participating countries, by both revising targets and better aligning project activities with country-specific environments (for details see Section 5.1 in the Technical Annexes). 64. Overall World Bank performance – Rating Moderately Unsatisfactory: The program suffered from shortcomings in preparation and design as well as from a PCU that did not become operational, which affected implementation at the program- and subsequently country-level, evident through substantial implementation delays. However, it must be acknowledged that the program was complex and particularly challenging in its implementation; and despite its complexity, challenges, and poor initial performance, the World Bank took important actions to improve implementation progress at the country-level, and eventually succeeded in supporting three participating ASP-P1 countries to achieve envisaged outcomes. 5.2 BORROWER PERFORMANCE 65. Overall Borrower performance – Rating Moderately Satisfactory: Borrower performance at the country-level varied across participating ASP-P1 countries; the rating represents an average. Table 5.1 provides an overview of borrowers’ individual performance. For details on performance at the government and implementing agency-level, refer to Section 5.2 of the individual Technical Annexes. Table 5.1: Overview of Borrower Performance at the Country-Level ASP-P1 ASP-P1 ASP-P1 ASP-P1 ASP-P1 ASP-P1 Ethiopia Mali Morocco South Africa Tanzania Tunisia Government Moderately Moderately Satisfactory Satisfactory Satisfactory Satisfactory Performance Satisfactory Unsatisfactory Implementing Moderately Moderately Moderately Agency Unsatisfactory Satisfactory Satisfactory Unsatisfactory Satisfactory Unsatisfactory Performance Overall Borrower Moderately Moderately Moderately Unsatisfactory Satisfactory Satisfactory Performance Satisfactory Satisfactory Unsatisfactory 61 The program was affiliated to two Regions (Africa and the Middle East and North Africa) with different procedures and regulations, and the supervision arrangement consisted of a program-level TTL with full and a co-TTL with no budget authority, which often caused confusion among staff. The program also experienced a high turnover in TTLs, requiring transition periods for hand-over activities. 21 6. LESSONS LEARNED 66. Lessons learned that were drawn at the program-level and could serve as guidance for similar programs in the future include the following: • The World Bank is not presently well equipped to handle multi-regional and multi-country programs. The World Bank's internal systems are not geared for administering an umbrella program with multiple sub-projects in more than one region. Possible World Bank support for similar approaches in the future should weigh transaction costs and efficiency constraints in designing regional programs. Unless strong justification is provided for designing the program with country-specific - or at least region-specific - projects should be considered. Furthermore, the World Bank's internal processing arrangements for regional programs should be made sufficiently flexible to accommodate their characteristics and needs rather than applying coding and budget systems that hinder successful implementation. • For multi-stakeholder projects, formalized and defined roles and responsibilities for all partners and their collaboration need to be agreed and operationalized prior to project effectiveness. ASP-P1 was a complex program, which depended on close collaboration and cooperation among several partners and drawing upon their comparative expertise in pesticide management, project design, implementation, communications, and monitoring and evaluation. To ensure clarity on, and commitment to, each entity’s roles and responsibilities, together with guidance on how the partnership should function, should be defined and agreed on during project preparation. Concise time and budget bound Terms of Reference (TORs) for all partners, and a clearly defined oversight and coordination entity would have greatly improved project design and implementation. • ASP-P1 would have benefited from a design that (i) defined envisaged outcomes and PDOs at the program-level, and (ii) individually-tailored PDOs at the country-level. The uniform approach ASP-P1 applied to all participating countries caused significant delays and transactions costs, and is not desirable. Future programs should place emphasis on tailoring individual country projects to the country-specific environments. • The dimension of the task should be carefully determined, and projects should be sufficiently well-prepared before implementation. Project activities such as a national obsolete pesticides inventory and a CESA are lengthy processes but crucial for determining the dimension of the disposal task ahead. Ideally, these activities should not be conducted during implementation; rather, they should either be conducted during preparation, or during a first implementation phase, followed by a second that comprises adequately timed and budgeted disposal activities. This could include an approach under which the inventory and the Country Environmental and Social Assessment (CESA) are prepared simultaneously. • While it is important to determine the dimension of the task before implementation, it is equally important to develop a flexible implementation approach. With the volume of obsolete pesticides to be disposed of being a shifting target, especially in larger and infrastructure-poor countries, it is important to develop a flexible implementation approach that can compensate for actual quantities that are larger or smaller than those estimated at the time of appraisal. 22 • The program's initial assumption that no further obsolete pesticides would be accumulated was unrealistic in view of food security concerns and ongoing agricultural intensification. Prevention activities are crucial for any future operations and need to be sufficiently reflected in the project’s PDO and associated indicators. Prevention activities were not pursued to the extent disposal activities were implemented, mainly as a result of not having been sufficiently reflected in the indicators. • Realistic and clear indicators and preparation of a reasonable results framework are of paramount importance. The PDO and associated indicators did not cover all expected outcomes that were expected to be achieved. In several instances they were vaguely formulated, making it difficult to assess progress and outcomes. • Project plans should recognize the complexity of project tasks. Project activities such as a national inventory and a CESA proved to be lengthy processes. Since their completion was a requirement for disposal activities to begin, and the majority of funds was expected to be disbursed during disposal activities, disbursement remained low for most of project implementation. To avoid low disbursement levels being considered as low performance, project implementation plans and schedules need to be realistic. • The participation of adequate technical experts is crucial during program preparation and implementation. Preparation of ASP-P1 would have benefitted from more specialized technical staff involvement, and, as confirmed by ASP-P1 sub-projects in Ethiopia, Tanzania, and Tunisia, technical advisors during disposal preparation activities proved to be very valuable and significantly contributed to successful disposal. • Knowledge exchange between participating countries of a regional project should be facilitated. Preparation of the CESA in the individual participating ASP-P1 countries was a lengthy and complex task; the program would have benefitted from a central Country Environmental and Social Assessment (CESA) advisor. Furthermore, the procurement process could have been streamlined to contract disposal and CESA consultants for all countries to gain efficiency. • It is important to comprehensively assess the capacity of African regional organizations before designating them as implementing agencies for donor-funded programs. The PCU did not become operational due to a lack of capacity within NEPAD, the entity that had been selected to host the unit. Consequently, ASP-P1 was not successful in strengthening regional capacity for management of future program-related activities. 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS 67. See Section 8.7 and 8.8. 23 8. ANNEXES 8.1 PROJECT COSTS AND FINANCING a. Project cost by component – All sources (in US$ million equivalent) – as of 06/30/2013 Component Appraisal Estimate Actual/Latest Actual as % of (in US$ million Estimate appraisal equivalent) (in US$ million estimate 62 equivalent) 1. Disposal of obsoletes pesticides 51.37 N/A N/A 2. Prevention of accumulation 4.31 N/A N/A 3. Capacity building 2.45 N/A N/A 4. Project management 1.88 N/A N/A Total 60.00 N/A N/A b. Financing – as of 06/30/2013 Sources of Funds Type of Appraisal Estimate Actual/Latest Actual as % Co- (in US$ million Estimate of appraisal financing equivalent) (in US$ estimate million equivalent) Global World Bank Grant 21.7 16.3 75% Environment FAO Grant 3.3 3.3 100% Facility (GEF) Development WB – FY05 Grant 1.5 1.5 100% Grant Facility (DGF) WB – FY06 Grant 1.2 1.2 100% Parallel Trust Fund Canada 2.3 2.3 100% Financing Multi-Donor Trust Fund (Denmark, EU, Parallel 7.1 4.2 59% Sweden, and Switzerland) Financing Belgium 4.1 N/A N/A Finland 1.0 N/A N/A Parallel Bi-later Financing France (FFEM) 1.8 1.7 94% Financing Japan 1.1 N/A N/A The Netherlands 1.0 1.0 100% African Development Parallel Trust Fund 10.0 0 0% Bank Financing Parallel FAO 0.1 N/A N/A Financing 62 Actual/latest estimates of cost by component and sources of funds were not available. 24 ASP-P1 Participating Countries’ In-Kind 3.8 2.7 71% Governments 63 Parallel CropLife International (CLI) 0 2.1 - Financing Total 60.0 N/A N/A 8.2 OUTPUTS BY COMPONENT 1. The following table provides a comprehensive account of qualitative and quantitative information of outputs realized against original components (achievements at the time of program closing in June 2013). Table 8.1: Outputs by Original Components as presented in the PAD Original Components Outputs achieved at the time of program closing in June 2013 as presented in the PAD Component 1 - Country operations: Achievements under this component are rated Moderately Unsatisfactory This was the principal component of Achievements under this component varied across the ASP-P1 participating the program, encompassing a full countries For a detailed account of outcomes achieved under this component in range of cleanup and prevention each ASP-P1 participating country, please see respective Technical Annexes. activities for ASP-P1 participating a. Country cleanup and disposal countries. • An inventory database is in place in and being used by all participating ASP- a. Country cleanup and disposal P1 countries, except for ASP-P1 South Africa where it was considered cost- b. Prevention activities ineffective by the government (Results Indicator #1 – 83%). c. Capacity building • CESA were completed, and publicly consulted and disclosed in five ASP-P1 d. Country project management and participating countries (Ethiopia, Mali, South Africa, Tanzania, and monitoring Tunisia). Activities recommended therein were implemented in four ASP-P1 participating countries (ASP-P1 Mali was closed before disposal activities began in 2012 due to the unstable political situation in the country) (Results Indicator #2 – 66%). • Disposal contracts, in line with national and international regulations, were completed in three ASP-P1 participating countries (Ethiopia, Tanzania, and Tunisia) (Results Indicator #4 – 50%). • Overall, 3,164 of the inventoried 8,543 tonnes of publicly held obsolete pesticides and associated waste were disposed of in four ASP-P1 countries (PDO Outcome Indicator #1 – 37%). b. Prevention activities • The original target of not accumulating new obsolete pesticide stockpiles and associated waste was not measurable. At appraisal, it was estimated that approximately 4,750 tonnes of obsolete pesticides were scattered across the ASP-P1 countries; inventories conducted during implementation in the ASP- P1 participating countries revealed approximately 800 additional tonnes of obsolete pesticides. However, it is difficult to assess whether further stockpiles were accumulated during program implementation or whether the difference resulted from inaccurate estimates used at appraisal (PDO Outcome Indicator #2 - N/A). • The International Code of Conduct on pesticide distribution and use was 63 Excludes ASP-P1 Morocco and South Africa, as data was not available. 25 adopted by all ASP-P1 participating countries during a FAO council session in 2002 (Results Indicator #5 – 100%). c. Capacity building • See Technical Annexes. d. Country project management and monitoring • See Technical Annexes. Component 2 – Technical support: Achievements under this component are rated Moderately Unsatisfactory FAO hosted a TSU, which was tasked The TSU was fully funded by two grants through FAO (Government of the with coordinating the technical input to Netherlands and GEF) and staffed; however, it exhausted its funds prior to the country projects, assuring the technical beginning of disposal activities (Results Indicator #6 – 100%). quality of country projects, and a. Preparation of Country Operational Manuals. Operational Manuals were consolidating M&E data at the prepared by the ASP-P1 participating countries in collaboration with the TSU, program level. with varying levels of quality. a. Preparation of Country Operational b. Preparation for disposal of obsolete pesticides. Manuals • In collaboration with the World Bank, the TSU prepared three guidance b. Preparation for disposal of obsolete notes to help ASP-P1 participating countries implement the project. pesticides Guidance notes were prepared on planning and M&E; procurement of • Support the design of strategies for disposal services; and preparation and implementation of the CESAs. the effective and safe disposal of • The TSU further supported, reviewed and technically approved the CESAs, obsolete pesticide stocks and EMPs, and disposal bidding documents, provided training on supervision of associated waste, health and safety safeguarding and disposal operations, soil decontamination techniques, and plans. operationalized FAO’s Pesticide Stock Management System (PSMS) in all • Support the development of the ASP-P1 participating countries, except ASP-P1 South Africa. Training was CESAs including EMPs. also provided in pesticide safeguarding. However, the nature and level of • Contract preparation – the TSU support provided to participating countries by the TSU, varied across ASP- will support countries in preparing P1 participating countries (see Technical Annexes for more detail). To technical specifications and compensate for the insufficient level of technical support in some of the carrying out technical evaluations participating countries, the World Bank financed technical experts that for project managers, consultants, provided temporary support in the areas of safeguarding, operations, and and contractors for obsolete pesticide management. pesticide prevention and disposal • Guidance on alternative disposal options was included in the Environmental activities. Management Tool Kit (EMTK - Volume 3). However, it was not clear, how • Advise on alternative disposal relevant this guidance was to the participating countries since it had been options, targeted to specific agreed that all countries would dispose of their stocks at incineration country or site conditions. facilities overseas. c. Prevention of accumulation • Noteworthy is also the provision of a TAD to almost all ASP-P1 • Support legislative and regulatory participating countries. The TADs were financed by CLI, and proved to be reforms to enhance pesticide very value adding and crucial in preparing for and overseeing the disposal of management. obsolete pesticides. • Support the identification of c. Prevention of accumulation weaknesses in the enforcement of • The TSU did not support prevention activities at the country-level to the legislation and enhance extent outlined in the PAD. enforcement measures. d. Monitoring and Evaluation • Support the design of enhanced • Project launch workshops were successfully supported and held in all ASP- pest management strategies to P1 participating countries (Tunisia – November 2005; South Africa - June improve efficiencies and to reduce 2006; Tanzania - October 2006; Morocco – November 2006; Mali – reliance on pesticides in various December 2006; and Ethiopia –September 2007). sectors. • Supervision support, however, was not provided as planned. Supervision 26 • Support the design and missions generally only took place once, instead of twice a year (missions implementation of appropriate conducted by the TSU were rarely announced and mission reports were not container management strategies. shared with ASP partners), and the collaboration between FAO and the d. Monitoring and Evaluation World Bank was cumbersome, ultimately causing delays at the country- • Support country project-launch level. The TSU participated in monthly coordination conference calls and in- workshops and supervision support person meetings with the other ASP partners, and managed its procurement with the World Bank at least twice and financial administration. However, the TSU did not report on its per annum. activities to the other ASP partners. Rather, it provided limited input into some World Bank reports. • Support countries to implement the detailed M&E system and enquiry • Support on implementing the detailed M&E system was not delivered as system. planned. Country-specific M&E systems, intended to be finalized by the TSU during project launch and included in the Operational Manuals, were • Support annual workshops for the not provided until 2009. Eventually in 2009, the TSU provided some of the PMUs on lessons learned, and ASP-P1 countries with an M&E system; however, it was considered • Support TSU management, complex and ultimately not used by most ASP-P1 countries. including overall coordination with • Yearly ASP Stakeholder Forums took place as platform for stakeholders and other ASP units, administration of ASP-P1 participating countries to share knowledge and experiences, and TSU activities, including discuss questions and concerns; they were, however, organized by other procurement, reporting and partners (WWF, PAN-UK, and the World Bank). financial administration. Component 3 – Crosscutting activities: Achievements under this component are rated Moderately Satisfactory The objective of this component was to Overall, NGO participation in project activities was achieved (Results Indicator support ASP-P1 knowledge #10 – 100%). management, build the capacity of a. Communications and knowledge management (WWF) local NGOs in pesticide-related • An overarching communication strategy was developed by WWF, together matters and identify alternative with a communication strategy toolkit. On the basis of these tools and with disposal technology and practices. This support from WWF, national communication strategies were subsequently component was intended to be developed by all, however, only implemented by four ASP-P1 participating implemented by WWF and PAN-UK. countries (Results Indicator #11 – 67%). WWF also coordinated the a. Communications and knowledge development of a program communication plan, which was partially management (WWF). implemented. • The communication strategy will • WWF developed and maintained an ASP-P1 website during most of the be further refined during project program’s lifetime (2006-2011). Focal points were trained on the implementation. management and maintenance of their respective country pages hosted on • A knowledge and information the ASP website and trained in the use of an online platform that facilitated system will be designed and communication between representatives of participating countries. implemented to store and retrieve • WWF published newsletters on the program’s activities, and produced a key documents and information. number of educational booklets for farmers and schoolchildren. It also • Treaty ratification and developed radio public service advertisements and short videos that were coordination with chemical aired on TV and radio in Ethiopia and Tanzania. Two workshops were held convention activities: countries in Nigeria and Ethiopia in August 2007, to raise awareness for obsolete that had not yet signed or ratified pesticides and the program, and to provide information on important safety the Stockholm Convention would issues. be supported to do so. • WWF participated in various conferences to promote ASP and pesticide • WWF would solicit private sector policy advocacy. Alliances were developed with crop production boards financial support for ASP in (e.g., in the coffee sector and with academic and research institutions such as cooperation with the World Bank Innovations in Civic Participation (ICIPE)). Private sector financial support and FAO. was not solicited by WWF; it was coordinated by CLI. b. NGO capacity building (PAN-UK). b. NGO capacity building (PAN-UK) • NGO capacity to address pesticide • PAN-UK organized an NGO exchange event in Rabat in April 2007, with 27 management issues would be built representatives from each ASP-P1 participating country; a regional agro- through sharing lessons learnt at ecology training event for NGOs from West Africa (Mali, Morocco, and workshops, networking, and Tunisia) in 2007; and a capacity building training on fundraising approaches coordination of NGO activities. and strategies in 2008. • Preparing and disseminating • PAN-UK finalized and disseminated a number of key publications: (i) An operational guidelines for NGO ASP NGO manual to present key issues associated with obsolete pesticides, involvement and training in ASP. and to provide information to help NGOs play a successful part in the • Designing and implementing implementation of national ASP-P1 projects; (ii) an ecological monitoring awareness raising activities and handbook and a guide to international conventions; and (iii) regular articles outreach program targeted at in PAN journals, various reports and publications on community stakeholders. monitoring, and translation of ASP posters and leaflets into local languages. c. Strategic studies (WWF). • CLI strengthened capacity building in both the public and private sector by • Identifying alternative disposal promoting sustainable management of agrochemical waste. Support was technologies for obsolete provided to National CropLife offices in ASP-P1 participating countries, to pesticides and remediation of become more closely engaged with the program. CLI further funded study contaminated soils and water tours to Kenya and Australia to provide ASP-P1 representatives from resources. Ethiopia, Tanzania, and South Africa with information and first hand • Identifying environmentally sound observations on private sector engagement in agrochemical waste locust-control strategies and the management. safe maintenance of emergency • PAN-UK supported, in collaboration with WWF, PAN-Africa and local pesticide stocks for the control of NGO networks, the design of awareness raising activities in ASP-P1 locust outbreaks, malaria, dengue participating countries to varying degrees. While support was provided to fever, and other vector-borne ASP-P1 Ethiopia, Mali, and Tanzania, PAN-UK was less active in Morocco, diseases. and Tunisia. Reasons were political environments in which local NGOs’ activities had been limited, and a lack of agreements determining the extent of responsibilities, roles and budgets between PMUs, and PAN-UK and WWF. c. Strategic Studies (WWF) • To inform sustainable decision-making in the ASP-P1 participating countries, a study on disposal technology options was prepared by WWF in June 2008. The study provided information on the treatment and disposal options of obsolete pesticides and associated waste (Results Indicator #9 – 100%). • PAN-Africa conducted research on locust control operations in West Africa; the results were used to develop a toolkit for national NGOs to advocate and lobby for strengthening national policies and raising awareness for alternatives to chemical pesticides for locust control. Component 4 – Project coordination: Achievements under this component are rated Moderately Unsatisfactory The objective of this component was to The PCU intended to coordinate, monitor, and evaluate the implementation of ensure overall coordination of ASP-P1, ASP-P1 between the partners, and a Steering Committee (SC) intended to serve including the other three components as advisory structure, were not established (see Section 2.2 for more details). and their activities. The PCU was Although NEPAD received funding to host the PCU, it did not assume this planned to be transferred to an African responsibility seemingly due to capacity issues (Results Indicator #13 – 0%). organization – e.g., NEPAD. a. Governance and coordination Components of the PCU included: • Instead, an unofficial governance body - the ASP-P1 Implementation a. Governance and coordination. Committee (ASPIC) - was created to oversee the program’s implementation. • Providing secretarial support for Key partners met regularly using a combination of virtual and face-to-face the ASP Steering Committee meetings, and managed to establish a workable division of work and • Coordinating preparation of ASP- responsibilities. However, despite having been prepared, TORs for the ASPIC were never approved, and thus the committee was acting without 28 P2 formalized principles. • Ensuring effective management of • A proposal for ASP-P2 was prepared by FAO and the World Bank (in ASP financial resources and collaboration with WWF, PAN-UK, and CLI), who jointly submitted a maintaining an audit regime in funding proposal to the GEF in 2009, for US$20.7 million. ASP-P2 was cooperation with the World Bank. planned to be implemented in 22 additional African countries, and the • Promoting ASP and liaising with proposal reflected the recommendations of the independent evaluation that donors and recipient countries. had been undertaken for ASP-P1 (Results Indicator #8 - 100%) in 2008. b. Project Monitoring and Evaluation ASP-P2 was not accepted by GEF due to a lack of resources at the end of a (M&E) GEF replenishment cycle at the time, and poor performance of ASP-P1.; • The PCU was intended to establish instead, FAO and the World Bank were encouraged to submit funding and implement the overall ASP requests for individual country or regional projects once GEF funds became M&E system to ensure that the available. program would be successfully • As the signatory of project and donor financial agreements, the World Bank implemented to achieve the PDO was responsible for managing the program’s financial resources. and resources effectively and Supervision was carried out through missions, audits, review of progress and efficiently used. It was planned to financial reports, review of expenditures, and regular contact with consolidate individual Financial governments and PMUs. Also, an independent program evaluation was Management Reports (FMRs) and commissioned by the World Bank at mid-term and was subsequently made monitoring program progress available to all stakeholders in 2008. against the program results • Yearly ASP Stakeholder Forums were organized by the ASP partners on a framework and each components rotational basis, as platform for stakeholders and ASP-P1 participating against its intended results, and to countries to share knowledge and experiences, and discuss questions and provide bi-annual consolidated concerns. The African Union, the Basel Convention Secretariat for ASP progress reports and work francophone countries, NGO networks, and representatives from the donor plans. community participated as well. c. ASP-PCU management b. Project Monitoring and Evaluation (M&E) • Funds from the Multi-Donor Trust • An overall M&E system to systematically monitor progress against the Fund (MDTF) were to financially results framework and each component, was not established. However, support the PCU for it to perform overall progress was monitored through supervision missions, discussed its central coordination functions. during regular meetings of the ASPIC, and captured in yearly technical and NEPAD was intended to be financial reports prepared by the World Bank (Results Indicator #12 – provided with a capacity 100%). Program-level Interim unaudited Financial Reports (IFRs) were not development grant from the consolidated due to the multiplicity of funding resources and various project Development Grant Facility (DGF) codes; however, the World Bank reported on and audited individual Trust to be prepared for taking on Funds as part of its ongoing fiduciary responsibility. increasing responsibility • A survey to determine client satisfaction was not conducted (Results throughout the life of ASP-P1. Indicator #7 – 0%). c. ASP-PCU management • NEPAD was intended to host the PCU, which did not materialize due to a lack in capacity. The PCU was subsequently initially taken on by the rotating chair of the ASPIC, then by a consultant hired by the World Bank and hosted by the TSU. In 2008 the PCU was to be placed in the World Bank office in Pretoria, South Africa. A recruitment effort however failed to identify a suitable candidate, and thus the PCU was never created. 29 8.3 ECONOMIC AND FINANCIAL ANALYSIS 2. As outlined in Section 3.4, an economic and financial analysis is not available. 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS 3. Please refer to Section 8.4 in the Technical Annexes for an overview of Task Team Leaders (TTLs). Staff time and costs associated with program preparation and supervision was not possible to determine as a result of multiple coding for the program in the system. 8.5 BENEFICIARY SURVEY RESULTS 4. Beneficiary survey results are not available, as no survey was conducted. 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS 5. A stakeholder workshop was not held at the program-level. For country-specific stakeholder workshop reports and results, please refer to Section 8.6 in the Technical Annexes. 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR 6. The Federal Democratic Republic of Ethiopia, the Republic of Mali, the Republic of South Africa, the United Republic of Tanzania, and the Republic of Tunisia submitted completion reports; messages contained therein were integrated into this ICR, and associated Technical Annexes (see Technical Annexes for details). ASP-P1 Morocco did not submit a completion report. 7. Comments on the draft ICR were received from the Federal Democratic Republic of Ethiopia, the Republic of Mali, the United Republic of Tanzania, and the Republic of Tunisia, and incorporated/outlined accordingly. See Section 8.7 of the Technical Annexes for details. 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS 8. The draft ICR was shared with all co-financiers and other partners/stakeholders. Comments on the program-level ICR were received from CLI, FAO, FFEM, WWF, and the European Commission. Comments and edits were incorporated accordingly and/or are outlined in the following. For comments received from FFEM, see Section 8.8 in the ASP-P1 Tunisia Technical Annex. 9. CLI: All suggested edits received from CLI were incorporated. 10. FAO: Comments received from FAO were incorporated. Additional comments included: a. On the program’s design having given insufficient attention to specifying and defining the partnership arrangement, and the roles and responsibilities of each partners involved (paragraph 20): There was a TOR for the PCU developed but never approved by the World Bank – all other partners agreed. b. On the collaboration between FAO and the World Bank (paragraph 25): There was a period when there was duplication of roles and advice to countries under one of the World Bank TTLs. This resulted in a breakdown of trust. c. On the delayed financial support obtained from CLI (paragraph 25): FAO had to provide this role which was unbudgeted until these people were hired again blurring the lines of responsibility as 30 FAO was not supposed to provide this support as defined by WB in the discussions leading up to project implementation. d. On the service delivery to ASP-P1 countries (paragraph 44): This statement is unacceptable and must be changed. The TSU was actively prevented from providing country-level support in the project design. We developed the guidance to countries for the CESA with the World Bank TTL and countries were trained in its developed although this was not foreseen in our project description. Training was also provided to countries on development of disposal tenders. The paragraph implies the TSU was supposed to do these activities when in fact we were told by the WB that this would be a conflict of interest and were actively prevented from providing this support. From the 2001 meeting in London we proposed the placement of chief technical advisors in each country as per FAO systems to manage the projects with the country teams. This was rejected. Crop Life offered to provide technical advisors and these took an age to materialize. The Bank sent in Safeguards consultants to provide this support and confused the countries as they were offering advice which was contradictory to FAO. This paragraph is not correct. The technical assistance provided by the TSU was as expected in our project document. If it did not reflect the support envisaged in the WB document then your document was incorrect. o World Bank response: Paragraph 44 was modified to partly reflect the above comment. However, in general, the description of Component 2 outlined in Annex 4 of the PAD (Report No: 32232-MNA) was taken as basis for this ICR’s evaluation. e. On no available data regarding FAO resources spent (paragraph 51): Again not accurate. Our reports were sent to GEF for our separate grant. f. Overall: The main issue was that this was never a true regional project with the associated economies of scale or synergies. It was a multi-country project in a single region (or two regions in the case of the Bank). The issue was compounded by a lack of mutual acceptance of roles and responsibilities and a lack of understanding as to the mode of operation of implementation of WB country activities by FAO which made the operation of the TSU as designed impossible. This was never really understood and no point was any adjustment made to compensate. 11. WWF: Suggested edits received from WWF were incorporated. Additional comments included: a. Component 3 on knowledge management was not implemented to the end as the CCAME effectively stopped being operational once funding for its activities stopped in December 2010. b. Inadequate resources were provided to WWF and PAN to execute the capacity building activities for NGOs. 12. Development and Cooperation - European Commission: Comments received from the European Commission included: a. The project closed in June 2013, addresses the issue of obsolete pesticides by developing and implementing disposal and prevention activities through the establishment of a new partnership between African countries and their collaborating partners (including the World Bank, the United Nations Food and Agriculture Organization (FAO), the pesticide industry represented by CropLife International (CLI), the Pesticides Action Network (PAN) in the UK and Africa (PAN- UK and PAN-Africa, respectively) and the World Wide Fund for Nature (WWF)). 31 b. The project was co-funded among others from the Multi-Donor Trust Fund (established by the EU together with Denmark, Sweden, and Switzerland) but it is not clear to us which component was supported by the Multi-Donor Trust Fund (supported by EU) through parallel financing. c. The report refers to a lot of results indicators (13) which are analyzed individually but not in correlation with each other’s. d. The ICR says: “The elimination of obsolete pesticide stocks remains a development priority. Rural communities cannot hope to advance if the soil and water, upon which their livelihoods and health depend, are contaminated with pesticides, and urban populations cannot prosper if they are suffering from severe illnesses caused by pesticide poisoning”. e. However, though technical support and strategic studies have been delivered and the NGOs participation was 100%, unfortunately the projects have shown unsatisfactory results concerning mainly the disposal implementation activities which are development priorities. Only 35% of the inventoried publicly held obsolete pesticides and associated waste were disposed in four (out of 6) ASP-P1 countries and the disposal contracts have been implemented only in three countries because country counter parts have not been found (the accomplishment of these targets is considered of 50%). f. Therefore, we are very much concerned on the unsatisfactory results in implementation of activities on the field which is affecting not only the objective of the project, but the development of the concerned countries. g. A deeper and correlated analysis of these indicators may reveal other aspects/conclusions that could be useful in future at least for the implementation of other projects, such as the effectiveness of technical support (including the quality of the studies), role and effectiveness of the participating NGOs, etc. 32 8.9 VERBATIM COMMENTS ON DRAFT ICR 33 34 35 TECHNICAL ANNEX I: ASP-P1 ETHIOPIA 36 ABBREVIATIONS AND ACRONYMS APL Adaptable Program Lending ASP-P1 Africa Stockpiles Program – Project 1 CESA Country Environmental and Social Assessment CLI CropLife International CPD Crop Protection Department EMP Environmental Management Plan EMTK Environmental Management Tool Kit EPA Environmental Protection Authority FAO United Nations Food and Agriculture Organization FM Financial Management GDP Gross Domestic Product GEF Global Environment Facility GEO Global Environment Objective ICR Implementation Completion and Results Report IP Implementation Performance IPM Integrated Pest Management ISR Implementation Status and Results Report IVM Integrated Vector Management M&E Monitoring & Evaluation MoARD Ministry of Agriculture and Rural Development NGO Non-Governmental Organization NIP National Implementation Plan NSC National Steering Committee PAD Project Appraisal Document PAN Pesticides Action Network PDO Project Development Objective PMT Project Management Team PMU Project Management Unit POP Persistent Organic Pollutants PRRP Pesticide Risk Reduction Program 37 PSMS Pesticide Stock Management System QAE Quality at Entry QAG Quality Assurance Group QSA Quality of Supervision TAD Technical Advisor for Disposal TOR Terms of Reference TSU Technical Support Unit TTL Task Team Leader USAID United States Agency for International Development WWF World Wildlife Fund 38 TABLE OF CONTENTS 1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN ........ 48 1.1 CONTEXT AT APPRAISAL............................................................................................................................................. 48 1.2 ORIGINAL PDO AND INDICATORS ............................................................................................................................. 49 1.3 REVISED PDO AND INDICATORS ................................................................................................................................ 50 1.4 MAIN BENEFICIARIES ................................................................................................................................................... 51 1.5 ORIGINAL AND REVISED COMPONENTS .................................................................................................................. 51 1.6 OTHER SIGNIFICANT CHANGES ................................................................................................................................. 53 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ....................................................................... 54 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY ............................................................................. 54 2.2 IMPLEMENTATION ........................................................................................................................................................ 55 2.3 MONITORING AND EVALUATION (M&E) ................................................................................................................. 56 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE ......................................................................................................... 57 2.5 POST-COMPLETION OPEARTION/NEXT PHASE ....................................................................................................... 58 3. ASSESSMENT OF OUTCOMES ........................................................................................................................................... 59 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION ....................................................................... 59 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE ................................................................................... 59 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE .................................................................................... 62 3.4 EFFICIENCY ..................................................................................................................................................................... 62 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING ................................................................................................. 63 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS ............................................................................. 63 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS ..................... 64 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME ............................................................................................ 64 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE .................................................................... 64 5.1 WORLD BANK PERFORMANCE ................................................................................................................................... 64 5.2 BORROWER PERFORMANCE ....................................................................................................................................... 65 6. LESSONS LEARNED .............................................................................................................................................................. 66 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS .................. 66 8. ANNEXES ................................................................................................................................................................................. 67 8.1 PROJECT COSTS AND FINANCING.............................................................................................................................. 67 8.2 OUTPUTS BY COMPONENT .......................................................................................................................................... 67 8.3 ECONOMIC AND FINANCIAL ANALYSIS .................................................................................................................. 71 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS.................................. 71 8.5 BENEFICIARY SURVEY RESULTS ............................................................................................................................... 73 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS ............................................................................................ 73 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR .................................................................. 74 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS ................................................... 75 8.9 VERBATIM COMMENTS ON THE DRAFT ICR ........................................................................................................... 76 39 A. Basic Information 64 Country: Ethiopia Project Name: Africa Stockpiles Program - Project 1 Project ID: P105711 L/C/TF Number: TF90543 ICR Date: 11/27/2013 ICR Type: Core ICR Lending Instrument: APL Borrower: Federal Democratic Republic of Ethiopia Original Total US$2.62 Disbursed Amount: US$2.23 million (06/30/2013) Commitment: million Revised Amount: - Environmental Category: A Global Focal Area: CHEM Implementing Agency: Crop Protection Department Co-financiers and Other External Partners: Government of the Federal Democratic Republic of Ethiopia, Food and Agriculture Organization (FAO), CropLife International (CLI), World Wildlife Fund (WWF), and Pesticide Action Network (PAN-UK and PAN-Africa). B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/17/2002 Effectiveness: 06/30/2007 07/13/2007 Appraisal: 04/19/2004 Restructuring: 06/30/2011 Approval: 06/19/2007 Mid-term Review: 06/30/2009 10/28/2009 Closing: 06/30/2011 12/31/2012; 05/31/2013; 06/28/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Global Environment Outcome: Moderate World Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of World Bank and Borrower Performance World Bank Ratings Borrower Ratings Quality at Entry: Moderately Unsatisfactory Government: Moderately Satisfactory Implementing Moderately Quality of Supervision: Moderately Unsatisfactory Agency/Agencies: Unsatisfactory Overall World Bank Moderately Unsatisfactory Overall Borrower Moderately Satisfactory 64 The information presented in the data sheet was inserted manually. Information provided in the system was incomplete resulting from coding issues experienced during implementation; these were not resolved prior project closing. 40 Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators - Performance (if any) Potential Problem Project at Quality at Entry Yes - any time (Yes/No): (QEA): Problem Project at any time Quality of Yes - (Yes/No): Supervision (QSA): PDO rating before Moderately Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total World Bank financing) Sanitation 33.3 33 Health 33.3 33 General Agriculture 33.3 34 Theme Code (as % of total World Bank financing) Environmental policies and institutions 50 50 Pollution management and environmental health 50 50 E. World Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Obiageli Katryn Ezekwesili Country Director: Guang Zhe Chen Mark D. Tomlinson Director (Regional Colin Bruce - Integration): Sector Manager: Magda Lovei Mary Barton-Dock Project Team Leader: Dinesh Aryal Peter Kristensen/Denis Jordy ICR Team Leader: Dinesh Aryal ICR Primary Author: Veruschka Schmidt F. Results Framework Analysis - At project approval, no country-specific result framework existed; only seven Project Development Objective (PDO) Indicators were defined. Following a Level 1 Restructuring in June 2011, a country-specific results framework was introduced. The original seven PDO Indicators were revised and reduced to two PDO Indicators, and in addition, seven Intermediate Outcome Indicators were introduced. 41 Project Development Objective (PDO) – as presented in the Global Environment Facility (GEF) Trust Fund Grant Agreement: 65 To assist the recipient in: a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and b) implementing measures to reduce and prevent future related risks. Revised PDO – as presented in the Level 1 Restructuring of June 2011 (Report No: 62794-ET): To assist the Federal Republic of Ethiopia to eliminate and/or safeguard inventoried publicly held obsolete pesticide stocks and associated waste, and develop a strategy for sustainable management of future accumulations. Global Environment Objectives (GEO) - as presented in the Project Appraisal Document (PAD (Report No: 39901-ET): Reduction of the effects of Persistent Organic Pollutants (POPs) on the global environment. Revised GEO – Not Applicable (a) PDO Indicators – Original PDO Indicators as presented in the GEF Trust Fund Grant Agreement are assessed against achievements realized at the time of the project’s Level 1 restructuring in June 2011; revised PDO Indicators are assessed against achievements realized at project closing in June 2013. Original Target Formally Actual Value Achieved at Values Indicator Baseline Value Revised Target Completion or Target (from approval Values Years documents) Original PDO Indicators Inventory database of publicly held obsolete pesticides stocks and associated Indicator 1: waste in place and in use by the PMT and the recipient's staff 66 Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 06/30/2011 - 05/31/2010 Target fully achieved (100%): An inventory was completed and validated in May Comments (incl. % 2010, including transfer of data into the FAO obsolete pesticide inventory database. achievement) The database was subsequently used for risk profiling of the stocks and preparation of the Country Environmental and Social Assessment (CESA). Completion of Country Environmental and Social Assessment (CESA), and Indicator 2: implementation of the measures contained therein Value (quantitative No Yes - No or qualitative) Date Achieved 04/19/2004 06/30/2011 - 06/30/2011 65 The PDO, indicators, and components that are stated in the Project Appraisal Document (PAD) slightly differ from those stated in the GEF Trust Fund Grant Agreement. This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities. 66 PMT refers to Project Management Team. Hereafter referred to as Project Management Unit (PMU) 42 Comments (incl. % Target not achieved (0%): A CESA was not completed by the time of restructuring. achievement) Inventoried publicly held obsolete pesticides stocks disposed of, as a result of the Indicator 3: completion of the disposal services contract(s) in accordance with national and international laws and regulations Value (quantitative 0% 100% - 0% or qualitative) Date Achieved 04/19/2004 06/30/2011 - 06/30/2011 Target not achieved (0%): The bidding document for the disposal tender was Comments (incl. % completed and cleared by the World Bank, but none of the publicly held obsolete achievement) pesticides was disposed of at restructuring, due to project implementation delays. Legal and regulatory framework for pesticide management improved, including Indicator 4: measures to strengthen compliance with the Basel Convention and the Rotterdam Convention Value (quantitative No Yes - Partly (50%) or qualitative) Date Achieved 04/19/2004 06/30/2011 - 06/30/2010 Target partly achieved (50%): A Pesticide Registration and Control Proclamation Comments (incl. % was approved in January 2010 by the Council of Ministers, and proclaimed by achievement) Parliament in June 2010. Measures to strengthen compliance with the Basel Convention and the Rotterdam Convention were not taken at restructuring. Indicator 5: Pesticide management strategy is developed Value (quantitative No Yes - No or qualitative) Date Achieved 04/19/2004 06/30/2011 - 06/30/2011 Comments (incl. % Target not achieved (0%): A pesticide management strategy was not developed by achievement) the time of restructuring due to project implementation delays. The training program under Part D 67 of the Project is implemented and the Indicator 6: knowledge so acquired is being used by the PMT and recipient's staff Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 06/30/2011 - 06/30/2011 Target fully achieved (100%): Project Management Unit (PMU) members and Comments (incl. % representatives from the ministry received training in the areas of pesticide achievement) management and utilization, pesticide store management, safeguarding, Financial Management (FM), and procurement. The Project Management Unit (PMU) is functional and its operation is Indicator 7: satisfactory to the World Bank Value (quantitative No Yes - Partly (50%) or qualitative) 67 See Section 1.5 for the project’s Component D (in the GEF Trust Fund Grant Agreement referred to as ‘Part D’) 43 Date Achieved 04/19/2004 06/30/2011 - 06/30/2011 Target partly achieved (50%): The PMU was functional, however, it was periodically Comments (incl. % missing key staff, and project implementation experienced significant delays achievement) particularly during the first half of the project. Revised PDO Indicators as per the Level 1 Restructuring in June 2011 Inventoried publicly held obsolete pesticides and associated waste disposed of Indicator 1: and/or safeguarded by the end of the project Value (quantitative 0 100 - Partly (35%) or qualitative) Date Achieved 06/30/2011 06/30/2013 - 06/30/2013 Target partly achieved (35%): 450 tonnes (35%) of the inventoried 1300 tonnes of Comments (incl. % publicly held obsolete pesticides and associated waste were disposed of or achievement) safeguarded. These include 450 tonnes of obsolete pesticides (against 450 tonnes inventoried), and 0 tonnes of associated waste (against 850 tonnes inventoried). A strategy for sustainable management of future accumulations adopted by the Indicator 2: Steering Committee by the end of the project Value (quantitative None Yes - Yes or qualitative) Date Achieved 06/30/2011 06/30/2013 - 06/25/2013 Comments (incl. % Target fully achieved (100%): A national pesticide management strategy for Ethiopia achievement) was developed and endorsed by the Steering Committee on June 25, 2013. (b) GEO Indicator(s) –The GEO was measured by the PDO and PDO indicators. (c) Intermediate Outcome Indicator(s) Original Target Actual Value Values (from Formally Revised Target Achieved at Indicator Baseline Value approval Values Completion or Target documents) Years Indicator 1: Percent of validated inventory data entered into PSMS Value (quantitative 100% - - 100% or qualitative) Date Achieved 06/30/2011 - - 05/31/2010 Comments (incl. % Indicator corresponds to original PDO Indicator #1 and was completed prior to achievement) restructuring. Therefore, the indicator is not taken into account for post-restructuring. Indicator 2: Approved CESA available for use by the disposal company Value (quantitative No Yes - Yes or qualitative) Date Achieved 06/30/2011 06/30/2013 - 12/26/2011 Comments (incl. % Target fully achieved (100%): Despite delays, a CESA was completed in February 44 achievement) 2011, found satisfactory by the World Bank on December 14, 2011, and publicly disclosed on February 29, 2012. Information contained therein was used by the disposal company. A final draft container management strategy submitted to the Steering Indicator 3: Committee for approval Value (quantitative No Yes - Yes or qualitative) Date Achieved 06/30/2011 06/30/2013 - 06/25/2013 Comments (incl. % Target fully achieved (100%): A container management strategy was endorsed by the achievement) Steering Committee and the Ministry of Agriculture on June 25, 2013. Indicator 4: Two planned communication tools implemented Value (quantitative 0 2 - 2 or qualitative) Date Achieved 06/30/2011 06/30/2013 - 06/30/2013 Target fully achieved (100%): Posters and other printed materials were prepared to Comments (incl. % raise awareness on the harmful nature of pesticides to human health and the achievement) environment; and a one-year TV and radio program was conducted in three local languages to raise awareness on pesticides use. A final draft sustainable management plan for obsolete pesticides and associated Indicator 5: wastes prepared and submitted to the Steering Committee for endorsement Value (quantitative No Yes - Yes or qualitative) Date Achieved 06/30/2011 06/30/2013 - 05/31/2013 Target fully achieved (100%): A draft national pesticide management strategy for Comments (incl. % Ethiopia was developed and submitted to the Steering Committee at the end of May achievement) 2013. It was subsequently endorsed on June 25, 2013. A Pesticide Registration and Control Proclamation drafted for submission to Indicator 6: Parliament Value (quantitative Yes - - Yes or qualitative) Date Achieved 06/30/2011 - - 06/30/2010 Comments (incl. % Indicator corresponds to original PDO Indicator #4 and was completed prior to achievement) restructuring. Therefore, the indicator is not taken into account for post-restructuring. Indicator 7: Availability of one new storage warehouse for pesticide management Value (quantitative 0 1 - 0 or qualitative) Date Achieved 06/30/2011 06/30/2013 - 06/30/2013 Comments (incl. % Target not achieved (0%): The construction of a new storage warehouse for pesticide achievement) management was aborted as a result of time constraints. 45 G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 05/30/2008 Moderately Unsatisfactory Moderately Unsatisfactory 0.25 2 12/01/2008 Unsatisfactory Highly Unsatisfactory 0.25 3 12/11/2008 Unsatisfactory Highly Unsatisfactory 0.25 4 05/29/2009 Unsatisfactory Unsatisfactory 0.28 5 12/23/2009 Unsatisfactory Unsatisfactory 0.29 6 06/22/2010 Unsatisfactory Moderately Satisfactory 0.38 7 03/27/2011 Unsatisfactory Moderately Satisfactory 0.51 8 07/13/2011 Moderately Satisfactory Moderately Satisfactory 0.63 9 01/08/2012 Moderately Satisfactory Moderately Satisfactory 0.67 10 10/08/2012 Moderately Satisfactory Moderately Satisfactory 0.74 11 05/29/2013 Moderately Satisfactory Moderately Satisfactory 1.25 06/30/2013 Final disbursement 2.23 H. Restructuring ISR Ratings at Amount Board Restructuring Disbursed at Restructuri Approved Reason for Restructuring & Key Changes Restructuring ng Date(s) PDO Made PDO IP in USD Change millions 08/01/2009 No Not Available Not Not • Reallocation of proceeds to make Available Available available funds for the construction of an improved pesticides storage facility 06/30/2011 Yes Unsatisfactory Moderately 0.5 • Revision of the PDO and indicators to Satisfactory reflect a shift from complete disposal to disposal and safeguarding of stocks • Introduction of a results framework for better M&E • Extension of closing date by 18 months to compensate for initial delays (06/30/2011 to 12/31/2012) • Reallocation of proceeds to accommodate disposal and/or safeguarding of all the additional stocks 12/03/2012 No Moderately Moderately Not available • Extension of closing date by five Satisfactory Satisfactory months to ensure disposal activity 46 ISR Ratings at Amount Board Restructuring Disbursed at Restructuri Approved Reason for Restructuring & Key Changes Restructuring ng Date(s) PDO Made PDO IP in USD Change millions completion (12/31/2012 to 05/31/2013) 05/24/2013 No Moderately Moderately 1.3 • Extension of closing date by one month Satisfactory Satisfactory to ensure reception of disposal certificates (05/31/2013 to 06/30/2013) If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body), enter ratings below: Outcome Rating Outcome Ratings Against Original PDO targets Moderately Unsatisfactory Against formally revised PDO targets Moderately Satisfactory Overall (weighted) rating Moderately Satisfactory I. Disbursement Profile – A disbursement profile is not available. 47 1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN 1.1 CONTEXT AT APPRAISAL 1. Country background: Ethiopia’s economy is primarily based on agriculture. At project approval in June 2007, the sector contributed around 46% to Ethiopia’s GDP and employed over 80% of the labor force. 68 Ethiopia’s highlands had been highly fertile, enabling the country to support a large population, but long-term output had been threatened by inefficient land tenure, soil degradation, and recurrent droughts. Pests and crop diseases further contributed to crop yield losses; at project appraisal, the average crop loss due to pests was estimated to be between 30% and 40%, annually. 69 2. Sector background: Although pesticide use in Ethiopia had been historically poor due to low economic performance and the presence of smallholder agricultural systems, efforts to increase and diversify food production since the late 1970s had resulted in an increased use of pesticides in the agricultural sector. Of the total pesticides imported, commercial farmers were estimated to have used about 80%, mainly on cotton; the remaining 20% were used for small-scale farming, household and industrial purposes, and disease vector control. Malaria had been prevented and controlled for several decades with the application of pesticides such as Dichlorodiphenyltrichloroethane (DDT). In addition, Ethiopia was one of the eastern African countries that suffered from migratory pests such as desert locust and armyworm outbreaks; the ensuing mobilization of international assistance to combat the pests often resulted in excessive emergency prevention stocks. 3. The accumulation of obsolete pesticides was aggravated by inappropriate storage conditions caused by poor storage facilities, poor management capacities of the stocks, unavailability of required application equipment, and a lack of trained staff on pesticide stock management. Product bans, unsuitable products or packaging, donation or purchase in excess of need, further contributed to this accumulation. Various obsolete, banned, and unwanted pesticides were accumulated throughout Ethiopia carrying significant environmental and public health risks. 4. Institutional background: At appraisal, the Environmental Protection Authority (EPA) was the designated national authority for the Stockholm, Rotterdam, Basel, and Bamako Conventions, which had all been ratified by the Government of Ethiopia. However, by the nature of the mandate, 70 EPA’s role and responsibilities had focused on policymaking, establishment of system-based standards and development of partnership with sector ministries. Consequently, the Ministry of Agriculture and Rural Development (MoARD), through its Crop Protection Department (CPD), had overall operational responsibility for cleanup and remediation operations. 5. Actions taken: Towards the end of the 1990s, a series of stockpile safeguarding and disposal operations was initiated, including an operation that was launched in 1999, and implemented with technical support of the Food and Agriculture Organization (FAO) and financial support of the United States Agency for International Development (USAID), the Netherlands and Sweden. 71 1,500 tonnes of obsolete pesticides were successfully disposed of under the project’s first phase (FAO-I); the 68 The Economist Intelligence Unit, Country Profile Ethiopia, 2007 69 Country Environmental and Social Assessment (CESA) and Management Plan of the African Stockpiles Program Project-1 Ethiopia, MTS Consulting Engineers PLC, February 2011. 70 Outlined in the Proclamation No. 295 71 FAO project titled Prevention and Disposal of Obsolete Pesticide Stocks in Ethiopia 48 second phase (FAO-II) started in 2004, and allowed disposal of an additional 1,100 tonnes with funds provided by Belgium, Japan, Finland and CropLife International (CLI). Project activities further included development and enforcement of pesticide policies, implementation of Integrated Pest Management (IPM) and Integrated Vector Management (IVM), and capacity building. ASP-P1- Ethiopia intended to complement these operations by financing the disposal of remaining 400 tonnes of obsolete pesticides that had not been disposed of under FAO-II as originally planned, but stored in two locations in Addis Ababa. 6. In preparation of ASP-P1, the Government of Ethiopia had initiated preparatory activities under its National Implementation Plan (NIP). 72 These activities included the completion of a preliminary inventory of obsolete pesticides across the country, which had identified over 900 sites with obsolete chemicals and contaminated soils. Very large quantities of empty containers and contaminated spray equipment were also found. 7. Rationale for World Bank assistance: Please refer to Section 1.1 in the program-level ICR. 8. Higher-level objectives: As noted in the Project Appraisal Document (PAD), the project intended to contribute to Ethiopia’s national development strategy 73 in the areas of agriculture and rural development, public health, poverty alleviation, and environmental protection. In particular, the project aimed to: (i) improve the quality of life in poor communities, by reducing environmental health risks; (ii) improve environmental protection; and (iii) enhance the capacity of the agricultural sector to better manage crop pests. Contributing to international efforts to eliminate Persistent Organic Pollutants (POPs), improving management of toxic chemicals, and promote alternatives to pesticide usage that include IPM and IVM represented further objectives. 1.2 ORIGINAL PDO AND INDICATORS 9. The Project Development Objective (PDO), indicators, and components that are described in the ASP-P1 Ethiopia PAD 74 slightly differ from those described in the associated GEF Trust Fund Grant Agreement. 75 This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities; in addition and as outlined in Section 1.3 and 1.5, changes under a Level 1 restructuring were introduced to the GEF Trust Fund Grant Agreement. 10. ASP-P1 Ethiopia’s original PDO as presented in the GEF Trust Fund Grant Agreement: 76 To assist the recipient in: a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and b) implementing measures to reduce and prevent future related risks. 11. At appraisal, only an overall ASP-P1 results framework had been prepared (see program-level ICR); results frameworks for the individual ASP-P1 countries, including Ethiopia, were intended to be 72 Federal Democratic Republic of Ethiopia - National Implementation Plan for the Stockholm Convention, September 2006. The Plan had been prepared with the support of the United Nations Environment Program (UNEP) and the United Nations Industrial Development Organization (UNIDO). 73 Ethiopia: Sustainable Development and Poverty Reduction Program, Ministry of Finance and Economic Development (MOFED), July 2002 74 Ethiopia – Africa Stockpiles Program, (Report No: 39901-ET), June 5, 2007 75 GEF Trust Fund Grant Agreement between the Federal Democratic Republic of Ethiopia and the International Bank for Reconstruction and Development, (Number TF090543 – ET), July 13, 2007 76 ASP-P1 Ethiopia’s PDO as presented in the PAD: Eliminate inventoried publicly held obsolete pesticide stockpiles and associated waste, and implement measures to reduce and prevent future related risks. 49 finalized during the individual ASP-P1 country project launches. However, country-specific PDO indicators were developed at appraisal. For ASP-P1 Ethiopia, these were as follows: Table 1.1: ASP-P1 Ethiopia Original PDO Indicators Original PDO Indicators Original PDO Indicators as presented in the PAD as presented in the GEF Trust Fund Grant Agreement a. Inventory database of publicly held obsolete pesticide a. Inventory database of publicly held obsolete pesticides stocks and associated wastes in place and being used stocks and associated waste in place and in use by the PMT by the Project Management Team (PMT) and the and the recipient's staff. stakeholders. b. Completion of CESA, and implementation of the measures b. Completion of a Country Environmental and Social contained therein. Assessment (CESA) and implementation of the c. Inventoried publicly held obsolete pesticides stocks disposed measures contained therein. of, as a result of the completion of the disposal services c. Inventoried publicly held obsolete pesticides stocks contract(s) in accordance with national and international disposed of, as a result of the completion of the laws and regulations. disposal services contract(s) in accordance with d. Legal and regulatory framework for pesticide management national and international laws and regulations. improved, including measures to strengthen compliance with d. Legal and regulatory framework for pesticide the Basel Convention and the Rotterdam Convention. management upgraded, including measures to e. Pesticide management strategy is developed. strengthen compliance with the Conventions. f. The training program under Part D of the project is e. Pesticide management strategy developed. implemented and the knowledge so acquired is being used by f. The training program under Component C of the the PMT and recipient's staff. 77 project is implemented and the knowledge so acquired g. The PMT is functional and its operation is satisfactory to the is being used by the PMT and stakeholders. World Bank. g. The PMT is functional and its operation is satisfactory to the World Bank. 1.3 REVISED PDO AND INDICATORS 12. The PDO and indicators for ASP-P1 Ethiopia were revised through a Level I restructuring on June 30, 2011. 78 The original PDO implied that all inventoried obsolete pesticides would be eliminated. With the discovery of higher than estimated stocks, including associated waste, it was decided to adopt a blended approach of disposal overseas and lower cost safeguarding in-country based on risk profiling of the inventoried pesticides and associated waste, and availability of satisfactory storage facilities (i.e. obsolete pesticides and higher risk stocks of heavily contaminated soils being disposed of overseas while lower risk stocks of associated waste being safeguarded in-country). In addition, the original PDO assumed no new accumulations of obsolete pesticides during the life of the project, which had been proven difficult to achieve in the context of agricultural intensification and food security concerns. Therefore, it was agreed that the project would instead focus on developing a strategy for management of future accumulations. 77 See Section 1.5 for the project’s Component D, here referred to as “Part D”. 78 The Africa Stockpiles Program – Project for Ethiopia, South Africa, Tanzania and Tunisia: Proposal to Restructure, (GEF/R2011-0022), June 22, 2011 50 13. ASP-P1 Ethiopia’s revised PDO: 79 To assist the Federal Republic of Ethiopia to eliminate and/or safeguard inventoried publicly held obsolete pesticide stocks and associated waste, and develop a strategy for sustainable management of future accumulations. 14. ASP-P1 Ethiopia PDO indicators were reduced in number and revised to focus on the revised project objective. In addition, a country-specific result framework was introduced for effective monitoring and evaluation of project progress (see Data Sheet). 15. ASP-P1 Ethiopia’s revised PDO Indicators: (i) Inventoried publicly held obsolete pesticides and associated waste disposed of and/or safeguarded by the end of the project; and (ii) a strategy for sustainable management of future accumulations adopted by the Steering Committee by the end of the project. 1.4 MAIN BENEFICIARIES 16. Please refer to Section 1.4 in the program-level ICR. 1.5 ORIGINAL AND REVISED COMPONENTS 17. The overall PDO of ASP-P1 Ethiopia was to be achieved through the implementation of four distinct components (outlined in Table 1.2). The table also shows revisions that were made through a Level I restructuring on June 30, 2011. 80 Table 1.2: ASP-P1 Ethiopia Original and Revised Components Original Components Original Components as Revised Components as presented in the PAD presented in the GEF Trust (Level I Restructuring June 30, Fund Grant Agreement 2011) 81 Component 1.A - Cleanup and disposal Part A - Obsolete pesticides Part A - Obsolete pesticides inventory: (Total: US$2.03 million; GEF US$1.03 inventory: No changes were introduced at million): a. Collection of data and restructuring. a. To verify and validate the existing compilation of a data base; obsolete pesticide data base, and conduct b. A risk-based prioritization of inventory on contaminated soils, buried the stocks identified in the data pesticides and contaminated equipment, base; and inventory and risk assessment including c. Preparation of the CESA. free standing pesticides, contaminated Part B – Disposal of obsolete Part B – Disposal and/or sites, soils, buried pesticides and pesticides: safeguarding of obsolete pesticides contaminated containers and equipment; a. Carrying out of a technological and associated waste: b. To implement the World Bank-approved assessment of available a. Carrying out a technological CESA for free standing obsolete treatment and disposal options; assessment of available safeguarding pesticides and to produce a second and disposal options; b. Implementation of the treatment CESA for the treatment of heavily and/or disposal technology b. Implementation of the safeguarding 79 In the GEF Trust Fund Grant Agreement, the term ‘Performance Monitoring Indicators’ was used; the restructuring paper (The Africa Stockpiles Program – Project for Ethiopia, South Africa, Tanzania and Tunisia: Proposal for Restructure, (GEF/R2011- 0022), June 22, 2011) used the term ‘project development objective level results indicators’. 80 The Africa Stockpiles Program – Project for Ethiopia, South Africa, Tanzania and Tunisia: Proposal to Restructure, (GEF/R2011-0022), June 22, 2011 81 Changes (outlined in more detail in the project’s restructuring paper, The Africa Stockpiles Program – Project for Ethiopia, South Africa, Tanzania and Tunisia: Proposal for Restructure, (GEF/R2011-0022), June 22, 2011) were introduced to the GEF Trust Fund Grant Agreement. 51 contaminated soils, buried pesticides and selected; and/or disposal technology selected; contaminated equipment; c. Improving the operations of c. Improving the operations of obsolete c. To safeguard obsolete pesticide stocks; obsolete pesticide stocks pesticide stocks collection centers, and collection centers, including including store stabilization and d. To dispose of the identified publicly held store stabilization and safety safety measures at those centers; and obsolete pesticides. measures at those centers; and d. Container management. d. Container management. The changes reflected the addition of safeguarding as an outcome and clearly defined associated waste as an integral part of the stocks to be treated. Component 1.B: Prevention of obsolete Part C – Prevention of obsolete Part C - Reduction of obsolete pesticide accumulation (Total: US$0.19 pesticide accumulation: pesticide and associated waste re- million; GEF: US$0.19 million): a. Awareness raising activities on accumulation: a. Strengthen the existing regulatory the program, health, and a. Awareness raising activities on the system for pesticide control and the environmental hazards of program, health and environmental homologation system of the pesticides; pesticides, and IPM; hazards of pesticides; b. Promote alternatives to chemical b. A review and update of the b. A review and update of the legal and pesticides through improvement of pest legal and regulatory framework regulatory framework to control management strategies, with a particular to control obsolete pesticide obsolete pesticide accumulation and attention to IPM for agriculture and IVM accumulation and strengthen strengthen enforcement mechanisms; for health care; enforcement mechanisms; c. Preparation of a sustainable pesticide c. Communication and awareness raising, c. Preparation of a pesticide management strategy; and; management strategy; and d. Strengthening the pesticide d. Strengthen the capacity of the d. Strengthening the pesticide management capacity of storage warehouses in carrying out pesticide management capacity of storage warehouses; management. warehouses. e. Provision of training to relevant staff of the recipient on health and safety issues, and environmental monitoring; and f. Strengthening awareness on the pesticide management legal framework and homologation systems. The changes reflected the integration of the capacity-building component in the revised prevention component as both aimed at improved pesticide management practices, and removal of the IPM activities where the potential impact was likely to be marginal. Component 1.C: Capacity building Part D – Capacity building: Part D – Capacity building: No (Total: US$0.30 million; GEF: US$0.20 a. Provision of training to relevant changes were introduced at million): staff of the recipient on restructuring. a. Laboratory capacity including chemical laboratory analysis, health and analysis; safety issues, and environmental b. Environmental assessment and monitoring; monitoring and evaluation training for b. Strengthening awareness on the the EPA and other stakeholders; and pesticide management legal c. Health and safety training. framework and homologation systems; and 52 c. Development and implementation of a pesticide storage and integrated pest management training program. Component 1.D: Project management Part E – Project management Part E – Project management and and M&E (Total: US$0.42 million; GEF: and monitoring: Supporting the monitoring: No changes were US$0.30 million): Activities included the PMT in carrying out project introduced at restructuring. establishment of a PMU in Ethiopia whose coordination, implementation, responsibility was to implement the monitoring and evaluation, country level activities, solicit external through the provision of technical technical support, monitor and evaluate advisory services, goods, and project activities and cooperate with the operational support. GEF-funded NIPs for the Stockholm Convention and similar activities. Country level management also included establishment of a National Steering Committee (NSC) representing key stakeholders and overseeing the work of the PMU. 1.6 OTHER SIGNIFICANT CHANGES Table 1.3: ASP-P1 Ethiopia Other Significant Changes Change Date Justification Approval Extension of closing date Extension of closing date by 18 months 06/30/2011 To compensate for initial delays, the disposal and Board from 06/30/2011 to 12/31/2012 safeguarding activities were expected to commence in 11/2011 and conclude in 12/2012. Extension of closing date by 5 months 12/03/2012 To ensure the completion of the disposal and Regional from 12/31/2012 to 05/31/2013 safeguarding contract, including reception of Vice certificates of disposal from disposal facilities. President Extension of closing date by 1 month 05/24/2013 To ensure reception of disposal certificates Country from 05/31/2013 to 06/30/2013 Director Reallocation of Proceeds Reallocation of proceeds from the 08/01/2009 To make funds available (US $265,000) for the Country categories Goods (-US$10,000), construction of an improved pesticides storage Director Training and Workshops (-US$20,000), facility as part of the prevention component. An Operating Costs (-US$80,000) and improved facility was expected to reduce the amount Unallocated (-US$155,000) to the of future obsolete pesticides through proper storage. category Works (+US$265,000). Reallocation of proceeds from the 06/30/2011 To accommodate the disposal and/or safeguarding of Board categories Goods (-US$414,000), all the additional stocks and associated waste Consulting Services (-US $77,000), inventoried and provide additional related training to Operating Costs (-US $47,000) and meet the first PDO indicator. Unallocated (-US $80,000) to the categories Disposal Services (+US $600,000) and Training and Workshops 53 (+US $18,000) Reallocation of US $310,000 from the 12/03/2012 To further increase the budget for disposal activities Regional Works category to the Disposal due to larger than budgeted stocks, which is critical Vice Services category. to meet the first PDO-level indicator. The request President was made to reallocate the amount which had originally been allocated for the construction of a pesticide storage warehouse (see above), which was had not been constructed. Redefinition of activities Redefinition of the term “Disposal 06/30/2011 Addition of in-country safeguarding as a possible Board Services” from services related to the cheaper end result for lower risk stocks; this was due treatment or disposal of obsolete to larger than budgeted stocks of obsolete pesticides pesticides and associated waste, to and associated waste. services related to the safeguarding and/or disposal of the same. 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY 18. ASP-P1 Ethiopia was prepared between 2001 and 2007, as part of preparatory activities at the program-level. It took more than five years from GEF eligibility in July 2001, and more than four years from concept note in July 2002 to project approval by the Board in June 2007. 82 19. Soundness of background analysis: Project preparation was conducted in close collaboration between the government and the World Bank, FAO, and CLI, through joint missions, and knowledge exchange. Considerable background analysis was undertaken, including a procurement capacity assessment in 2004, which appropriately recommended a procurement capacity-enhancement plan and a comprehensive procurement–training program. A Financial Management (FM) capacity assessment was also undertaken at appraisal concluding that an FM system would need to be in place prior to effectiveness. Overall, the government had acquired substantial experience through implementation of the above-mentioned project; 83 and was considered well prepared for pesticide disposal by the World Bank at appraisal in 2004, and by FAO in 2005. 84 20. Key preparatory inputs such as the PAD and to a lesser extent the Operational Manual, however only broadly laid out the project’s plans; detailed country-specific information regarding Monitoring and Evaluation (M&E), risks and risk mitigation measures particularly regarding procurement, and FM, were missing. As noted above, the government had acquired substantial experience through implementation of a previous project, 85 but no lessons learned seem to have been integrated and used during preparation. 82 The long preparation time resulted from a lengthy partnership building/negotiation process at the program-level resulting from the complex implementation arrangement and project design (see Section 2.1 in the program-level ICR). 83 FAO project titled Prevention and Disposal of Obsolete Pesticide Stocks in Ethiopia 84 See: Federal Democratic Republic of Ethiopia – Africa Stockpiles Program – Appraisal Mission: Aide-Memoire (March 28 – April 2, 2004; and African Development Bank – The Federal Democratic Republic of Ethiopia – Africa Stockpiles Program – Project 1: Appraisal Mission (July 20-26, 2005) Aide-Memoire 85 FAO project titled Prevention and Disposal of Obsolete Pesticide Stocks in Ethiopia 54 21. Assessment of project design: The project’s design (as for all ASP-P1 participating countries) was preconfigured at the program-level, and did not reflect any country-specific modifications or lessons learned from previous projects executed in Ethiopia. Design weaknesses at the program-level that translated into implementation delays at the country level, including ASP-P1 Ethiopia, are outlined in Section 2.1 of the program-level ICR. 22. Government commitment: The Government of Ethiopia had shown significant commitment towards the objectives of the ASP, particularly through the implementation of a previous similar project. In addition, the government had already established the institutional structures for coordinating the implementation of ASP-P1. MoARD, CPD, had already established a PMU in 2003. 23. Assessment of risks: Program-level risks were adequately identified and outlined in the PAD. They were presented in the form of a matrix and represented a summary of risk assessments undertaken during identification, pre-appraisal and appraisal missions to ASP-P1 participating countries. Even though the assessment of risks was comprehensive and included a broad range of mitigation measures in generic terms, an assessment of risks and mitigation measures specific to Ethiopia was lacking. At the same time, it should be noted that it was difficult to assess country-specific risks in the absence of inventories of obsolete pesticides and CESAs, which were only planned to be undertaken during project implementation. 2.2 IMPLEMENTATION 24. Throughout implementation, the project experienced long delays, eventually resulting in three extensions for a total of 24 months, and a Level 1 restructuring in 2011. 25. Key factors affecting implementation and outcomes included: • Poor project management and procurement capacity affected project implementation. The PMU staff that had managed the preceding pesticide project (FAO-II) was considered experienced and had therefore remained in place to take over the management of ASP-P1. However, FAO-II only closed in July 2008 and the transition period in which both projects were active was poorly managed; the PMU primarily focused on completing outstanding activities of FAO-II, and ASP- P1 start-up activities were delayed. Upon completion of FAO-II, key staff, including the project coordinator, unexpectedly left, resulting in significant additional delays as new staff had to be appointed and trained. The lack of adequate procurement staff coupled with cumbersome and lengthy procurement processes at the ministry-level, in particular, adversely affected the project’s progress throughout project implementation (see Section 2.4 for more details). • Delayed and insufficient technical support affected implementation performance. The PMU was supposed to receive highly technical and specialized expertise (required for implementation, supervision and monitoring of project activities) from FAO’s Technical Support Unit (TSU); however, this support was not delivered to the extent originally planned. 86 This was mainly due to poor collaboration between the World Bank and FAO, resulting from a complex and loosely defined stakeholder implementation arrangement at the program-level, and FAO having exhausted its GEF funds before the disposal activities began (see Section 2.2 in the program-level ICR). Urgently needed operational support from the World Bank was not sufficiently provided 86 In addition, equipment and vehicles intended to be provided by the TSU in the beginning of project implementation, were delayed by 5 months. 55 prior to mid-term review; however, increased supervision, operational and technical support thereafter, improved the level of support. For technical support during disposal activities, a Technical Advisor for Disposal (TAD) was eventually recruited and financed by CLI, which positively affected project implementation. • Higher than estimated obsolete pesticide stocks caused a funding gap. The completion of the national obsolete pesticides inventory in May 2010 revealed additional 200 tonnes of obsolete pesticides and 850 tonnes of associated waste, which had not been budgeted. At restructuring, the project was therefore facing an estimated funding gap of US$4.1 million for the disposal of all inventoried stocks and associated waste. The World Bank and the government had agreed to reallocate GEF proceeds toward disposal, dispose of the highest risk stocks and safeguard stocks with a lower risk profile. A round table with possible donors to raise additional funds was organized in early 2012; however, at project closure no additional funds had been raised. Insufficient project funds eventually resulted in non-disposal and/or safeguarding of the inventoried associated waste including contaminated soils. 26. Restructuring: At mid-term review, achievement of the PDO was considered unlikely within the original timeframe and budget. The World Bank recommended revising the PDO and associated indicators, introducing a results framework, and extending the project’s closing date. However, it was agreed that the project would only be restructured once project implementation showed positive progress; the PMU was subsequently provided with increased technical and operational support by the World Bank. 27. In June 2011, following implementation progress, the project was restructured. 87 Revisions reflected a shift from complete elimination of all inventoried publicly held obsolete stocks and associated waste to elimination overseas of the highest risk stocks and lower-cost in country safeguarding of the stocks. However, the revised PDO remained ambitious as project funds continued to be insufficient, even in view of revised targets. 2.3 MONITORING AND EVALUATION (M&E) 28. Overall M&E – Rating Moderately Unsatisfactory. 29. M&E design – Rating Unsatisfactory: Despite the lengthy preparation time, at appraisal, an M&E system specific for Ethiopia was not developed. A results framework, which was intended to be finalized by the TSU during project launch and included in the project’s Operational Manual, did not materialize. Only a set of PDO indicators linked to the different project components was developed in the PAD and the GEF Trust Fund Grant Agreement, however, these did not allow for adequate monitoring of project implementation progress. The indicators focused on measuring outputs as opposed to progress towards project outcomes. In particular, results expected under the prevention component were not adequately considered by these indicators. 30. M&E implementation – Rating Moderately Unsatisfactory: Until mid-term review, there was no M&E system in place. Eventually, in 2009, the TSU provided the PMU with and trained it in using a Disposal-Project Tracking Workbook to monitor and evaluate project-related activities. However, the tool was complex, and therefore not used by the PMU. Increased support provided by the World Bank 87 Following the restructuring, two additional extensions were processed for a total of six months in order to compensate for longer than anticipated disposal activities, and in particular to ensure reception of incineration certificates before project closure. 56 in 2010 included assisting the PMU with an 18-months component-based work plan and a simplified implementation progress-monitoring plan. In 2011, following the restructuring of the project, a results framework specific to Ethiopia was finally introduced and the PDO and associated indicators were revised to focus on the two main project achievements (disposal and/or safeguarding and reduction of future accumulations) which strengthened project M&E, albeit late in the process. 31. M&E utilization – Rating Moderately Satisfactory: The inventory of obsolete pesticide stocks was conducted by using inventory forms and a manual provided by the FAO Environmental Management Toolkit (EMTK). 88 The inventory data was entered into a database of the FAO web-based Pesticide Stock Management System (PSMS) to record and monitor the pesticides and manage their usage, and subsequently validated. The data was used by the PMU for risk profiling of the stocks,89 and to prepare the CESA and disposal bidding documents/contracts. FAO also provided training to the PMU in how to best use the database, which took place in 2010. The PSMS together with the EMTK proved to be very useful tools in managing obsolete pesticides stocks. 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE 32. Safeguard compliance – Rating Satisfactory: ASP-P1 triggered two safeguard policies: OP 4.01 on Environmental Assessment and OP 4.09 on Pest Management. 33. Compliance with OP 4.01 on Environmental Assessment: A CESA identifying risks and devising mitigation measures to address potential negative environmental and social impacts was completed to the World Bank’s satisfaction in December 2012, albeit with significant delays. The assessment concluded that MoARD had the capacity to safeguard medium and low risk pesticide stores, and included an Environmental Management Plan (EMP) together with a Health and Safety Plan for safeguarding obsolete pesticides before disposal. The CESA’s late completion resulted from: i) the late completion of the inventory of obsolete pesticide stocks (only finalized in May 2010) which was needed for the CESA; ii) a complicated and time-consuming consultant procurement process; and iii) a lengthy World Bank review and clearance process (which ultimately caused eight months of delay). Subsequently, project activities involving storage, transportation, disposal, and site remediation were implemented in compliance with the recommendations and requirements of the CESA, the associated EMP, and the Health and Safety Plan. 34. OP 4.09 on Pest Management: Since project activities themselves met requirements of a pest management plan (i.e., through pesticide management, IPM, and capacity building), individual country projects, including ASP-P1 Ethiopia, were not required to prepare self-standing documents. See Section 2.4 in the program-level for more details. 35. Financial Management compliance – Rating Moderately Satisfactory: During project implementation, quarterly Financial Monitoring Reports and annual audit reports were regularly prepared and submitted to the World Bank’s satisfaction, although the latter often delayed, particularly prior to 2010. The PMU encountered a shortage of staff, before in November 2009 a finance and administration officer and an accountant were recruited. Annual budgeting was being prepared, and its monitoring was satisfactory. Disbursement of total GEF funds was low throughout 88 The tool kit provides a practical set of methodologies to assist countries in the risk-based management of obsolete pesticides, including guidance on the management of collection/storage locations, based on international best practice. 89 Sites were evaluated by risk category (high, medium, and low). 57 implementation of the project, with only 26% in June 2011, when the project was restructured. Reasons were significant project implementation delays (often associated with procurement), and the project’s design having been back loaded in terms of expenditures with the more costly disposal activities programmed to take place only at the end of the project. 36. Procurement – Rating Unsatisfactory: Project implementation was significantly affected by continuous procurement-related delays, which were caused by low procurement capacity within the PMU and long authorization processes within the government. Following the departure of the initial procurement officer in June 2008, efforts were undertaken to recruit a replacement, however, with little success. 90 At project closure, the position still had not been filled. Procurement support from MoARD was equally cumbersome. The World Bank eventually mitigated the PMU’s weak procurement capacity through temporary technical assistance.91 2.5 POST-COMPLETION OPEARTION/NEXT PHASE 37. Post-completion: To further strengthen the legal and regulatory framework, a Pesticide Registration and Control Regulation was developed in addition to and based on the Pesticide and Control Proclamation that was proclaimed in 2010 (Proclamation No. 674/2010). The new regulation was validated by a national workshop in April 2013, which was organized by a similar project with funds from FAO and the Netherlands. 92 The regulation was subsequently approved by MoARD and is currently expected to be submitted to the Council of Ministers for further approval and final enactment. At the same time, the MoARD is determined to fully implementing Proclamation No. 674/2010. 38. Next phase: At project appraisal, the Ministry of Health was in possession of 450 tonnes of DDT that were stored near Addis Ababa. As Ethiopia was still relying on DDT for malaria control at appraisal, the stocks had not been declared obsolete and had therefore not been included in ASP-P1. 93 In 2012 however, Ethiopia eventually suspended the use of DDT. The 450 tonnes of DDT near Addis Ababa were subsequently declared obsolete, albeit too late to be included in ASP-P1. The Ministry of Health is currently in the process of conducting a nation-wide preliminary inventory exercise, before the stocks are planned to be adequately inventoried through PSMS in collaboration with Ministry of Agriculture. Additionally, following the development of a container management strategy under ASP- P1 Ethiopia, the government is currently exploring the possibility of conducting a pilot project in selected areas with support from Crop Life Ethiopia. 90 The general paucity of trained procurement specialists in Ethiopia and competition between projects caused a lengthy and unsuccessful recruitment process In November 2009, an experienced procurement officer had declined the final offer due to an unattractive remuneration scheme, and the following candidate resigned two months after his appointment in June 2010. Procurement support received from the MoARD was also characterized by a lack of skilled staff. 91 The World Bank provided temporary assistance in April 2010 by funding an international consultant team that conducted five project management support missions between July 2010 and January 2011 to help the PMU update and refine project management tools, and finalizing the bidding document for the disposal. 92 The Pesticide Risk Reduction Program (PRRP) aims to contribute to a sustainable pesticide management system by focusing on capacitating pesticide registration and post registration. 93 DDT is being phased out internationally under the Stockholm Convention on POPs. It is no longer permitted for use in agriculture. However, several countries, including Ethiopia, still rely on DDT for the control of mosquito vectors of malaria. 58 3. ASSESSMENT OF OUTCOMES 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION 39. Relevance – Rating Substantial: Despite having been too ambitious, the project’s original objectives, design and implementation were relevant when the project was prepared, and consistent with the World Bank’s Country Assistance Strategy for Ethiopia, which targeted to increase attention to environmental issues. 94 Even though they remained ambitious after they were revised, the project’s objective, design, and implementation remained and continue to be relevant to the national development priorities today. The Government of Ethiopia places strong emphasis on preserving the environment, reducing pollution, and supporting efforts to improve the quality of life in cities and rural areas, as evident from the country’s climate-resilient green economy strategy which focuses on achieving economic development goals in a sustainable way. 95 The project’s objective is also consistent with the World Bank’s Country Partnership Strategy for Ethiopia issued in 2012, 96 which supports a multifaceted approach addressing the challenges posed by climate change, including but not limited to interventions in the agriculture sector; better management of natural resources such as land, forestry, groundwater; and promotion of green technologies. 40. The project supported the Government of Ethiopia in complying with the Stockholm Convention on POPs, and the Basel Convention on the control of trans-boundary movements of hazardous wastes and their disposal. In addition, the government has shown increasing commitment to eliminating obsolete pesticides and introducing measures to reduce future related risks, as demonstrated by its engagement in a similar project, the Pesticide Risk Reduction Program (PRRP – Ethiopia), which is planned to be completed by the end of 2013 (see Section 2.5). 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE 41. Taking account of the project’s Level 1 restructuring, the project has been evaluated against its original and revised PDO by measuring achievements at the time of the project’s restructuring (June 2011) and project closing (June 2013), respectively. 42. Original PDO achievement – Rating Moderately Unsatisfactory: The assessment took the project’s seven PDO indicators into account, of which two were fully (100%), two partly (50%), and three not (0%) achieved. The three main project outcomes measured by indicators include: (i) sound disposal and management of obsolete pesticides; (ii) prevention of accumulation of new stockpiles of disposal and management of obsolete pesticides; and (iii) enhanced capacity and institutional strengthening of pesticide management. 97 43. Sound disposal and management of obsolete pesticides. The level of risks emanating from publicly held obsolete pesticides and associated wastes to adjacent and wider communities or critical natural resources was determined and quantified through the completion of a national inventory in May 2010 (PDO Indicator #1 – 100%). This information (geo-location of the storage sites, whether the storage 94 Country Assistance Strategy for the Federal Democratic Republic of Ethiopia, (Report No. 25591-ET), March 24, 2003 95 Ethiopia’s Climate-Resilient Green Economy Strategy – The path to sustainable development, Federal Democratic Republic of Ethiopia, 2011 96 Country Partnership Strategy for the Federal Democratic Republic of Ethiopia, (Report No. 71884-ET), August 29, 2012 97 Achievements were identified based on intended outcomes presented in Section B.1 and Annex I of the PAD (Report No: 39901-ET). 59 sites were protected or not, identity of the contaminant, its quantity, and general condition of the stockpiles) allowed the government to risk profile the stocks and to prepare the CESA. However, at restructuring, the CESA was still being finalized and not yet completed (PDO Indicator #2 – 0%). Due to substantial project implementation delays, the disposal had not begun at restructuring. The bidding document for the disposal tender was completed and cleared by the World Bank, but no publicly held obsolete pesticides had been disposed of yet (PDO Indicator #3 – 0%). 44. Prevention of accumulation of new stockpiles of obsolete pesticides. Better regulation and control of the registration, production, storage, and sale of pesticides was achieved by strengthening the legal and regulatory frameworks for pesticide management. A Pesticide Registration and Control Proclamation was approved in January 2010 by the Council of Ministers, proclaimed by Parliament in June 2010, and published in August 2010 in a federal newspaper. However, measures to strengthen compliance with the Basel Convention and the Rotterdam Convention had not been undertaken (PDO Indicatory #4 – 50%). Important steps had been made towards preparing a national pesticide management strategy, aiming to enhance the sustainability of MoARD’s organizational and financial provisions to deal with obsolete pesticide and associated waste; however, at restructuring the strategy had not been finalized yet (PDO Indicator #5 – 0%). In collaboration with WWF and in consultation with various ministries, NGOs and other stakeholders, a communication strategy was finalized in March 2009. 98 The strategy comprised an outreach program focusing on farmers and rural communities including a number of key events to support the prevention component. The project subsequently raised stakeholders’ awareness of the new proclamation, through a three-day workshop in May 2011. Awareness was not raised on the health and environmental hazards of pesticides and encouraging safe pesticide handling, as the envisaged radio, TV, and print media materials had not been distributed/aired by the time of restructuring. 45. Enhanced capacity and institutional strengthening of pesticide management. Between 2010 and 2011, PMU members and representatives from MoARD participated in several pesticide management trainings. Capacity was strengthened as became evident through successful completion of the inventory, and adequate preparation of safeguard instruments (PDO Indicator #6 – 100%). However, procurement capacity at the time of restructuring remained low (see Section 2.4), which continued to delay project implementation (PDO Indicator #7 – 50%). 46. Revised PDO achievement – Rating Moderately Satisfactory: The assessment took the project’s two revised PDO indicators into account, of which one was fully (100%) and the other partly (35%) achieved. In addition, seven Intermediate Outcome Indicators framed the evaluation. Two of these were not taken into account as they corresponded with Original PDO Indicators that had been achieved prior to restructuring; of those taken into account, four were fully and one was not achieved. Outcomes were slightly modified following the restructuring and included: (i) Sound disposal and management of obsolete pesticides and associated waste including safeguarding; (ii) reduction of re- accumulation of obsolete pesticides and associated waste through regulations, pest management, and stakeholder involvement; and (iii) enhanced capacity and institutional strengthening of pesticide management. 98 The communication strategy was developed during a workshop in May 2008 involving participants from various ministries, NGOs, and other stakeholders, and subsequently finalized during various consultative meetings. 60 47. Sound disposal and management of obsolete pesticides and associated waste, including safeguarding. The inventory had revealed a total of 449 tonnes of obsolete pesticides and 842 tonnes of associated waste, consisting of empty containers and contaminated material, equipment, and soil99 across 103 stores. Information obtained through the national inventory, was effectively used to inform the preparation of a CESA (Intermediate Outcome Indicator #2 - 100%). 100 48. The project reduced risks emanating from publicly held obsolete pesticides to communities, natural resources, and the global environment through the sound disposal of all inventoried obsolete pesticides. To maximize risk reduction and by recognizing that funds would be insufficient to dispose of and/or safeguard all inventoried obsolete pesticides and associated waste, the government and the World Bank agreed to give priority to disposing of obsolete pesticides over safeguarding associated waste as these were considered posing the larger risk. 49. As a result, at project closing, all inventoried 450 tonnes of Figure 3.1: Publicly Held Obsolete Pesticides and Associated Waste obsolete pesticides had been Inventoried and Eliminated/Safeguarded disposed of overseas; 101 900 however, none of the 850 800 Inventoried 2010 tonnes of associated waste had 700 been disposed of or 600 safeguarded (Revised PDO 500 Targets set at Indicator #1 – 35%). 102 An Tonnes 400 842 850 850 Restructuring June additional 395 tonnes of 2011 300 obsolete pesticides were 449 450 450 200 Eliminated June disposed of overseas in January 100 2013 2012 through parallel funding 0 by CLI. 103 These were legacy stocks from the FAO-II project. Partially Obsolete Associated Waste Safeguarded 50. Reduction of re-accumulation Pesticides (particularly (reported/not of obsolete pesticides and Contaminated verified) June 2013 Soils) associated waste through regulations, pest management, and stakeholder involvement. A national pesticide management strategy was developed and endorsed by the Steering Committee on June 25, 2013 (Revised PDO Indicator #2 – 100%; and Intermediate Outcome Indicator #5 – 100%). The strategy focused on sound pesticide management based on a holistic, integrated, and systematic approach. Equally endorsed by the Steering Committee on June 25, 2013 were action plans for a national container management strategy (Intermediate PDO Indicator #3 – 100%), and pesticide management. The project completed awareness raising activities as outlined in its communication strategy. In 2012 and 2013, posters and 99 Including 360 tonnes of contaminated building (contaminated concrete floor, contaminated walls), which required cleaning technologies not usable for contaminated soils. 100 The document was finalized on December 14, 2011, found satisfactory by the World Bank on February 21, 2012, and disclosed to the World Bank InfoShop on February 29, 2012. 101 Incineration certificates are dated to June 22, 2013. 102 450 tonnes (35%) of the inventoried 1,300 tonnes of publicly held obsolete pesticides and associated waste had been disposed of. 103 Incineration certificates are dated to January 20, 2012. 61 similar materials were prepared and distributed to raise awareness on the harmful nature of pesticides to human health and the ecosystem; and a one-year TV and radio program was conducted in three local languages to raise awareness on pesticides use (Intermediate PDO Indicator #4 – 100%). The construction of a new storage warehouse for temporary storage of imported pesticides and thus better pesticide management, was not initiated as a result of time constraints (Intermediate PDO Indicator #7 – 0%). 104 51. Enhanced capacity and institutional strengthening of pesticide management. In addition to the capacity building achieved prior to restructuring, the project did not achieve further institutional strengthening of pesticide management. 52. Weighted average PDO achievement – Rating Moderately Satisfactory: At restructuring in June 2011, the project had disbursed US$0.5 million, representing 21.5% of total disbursed funds of US$2.32 million. On this basis and as illustrated by the table below, project achievements against the original PDO are rated moderately unsatisfactory, whereas project achievements against the revised PDO are rated moderately satisfactory. The weighted average of the project’s PDO achievement is therefore rated moderately satisfactory. Table 3.1: Weighted Average PDO Achievement Against Original PDO Against Revised PDO Overall 1. Rating Moderately Unsatisfactory Moderately Satisfactory - 2. Rating Value 3 4 - Weight (% disbursed before/after 3. 21.5% 78.5% - PDO change) 4. Weighted value 0.7 3.1 3.8 Moderately 5. Final rating - - Satisfactory 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE 53. The project aimed to directly contribute to the GEO of reducing the effects of POPs on the global environment through: (i) improving the quality of life in poor communities, by reducing environmental risks; (ii) improving environmental protection; and (iii) enhancing the capacity of the agricultural sector to better manage crop pests. GEO achievement is measured by PDO achievement and, thus, is also rated moderately satisfactory based on the assessment above (see Section 3.2). 3.4 EFFICIENCY 54. Efficiency – Rating Substantial: This rating represents a broad estimate, as financial and economic data to support this rating are scarce. At appraisal, an economic or financial analysis was not 104 Its construction required the development of an Environmental and Social Impact Assessment (ESIA) which was only completed in April 2011. The assessment’s late completion was caused by a lengthy public consultation process and data collection and analysis, a lengthy procurement process, and the World Bank’s late advice on developing such assessment. In May 2012, it was agreed by the World Bank and PMU to cancel the store’s construction as the remaining project time was considered insufficient. 62 undertaken for ASP-P1 at either the program or at the country-level due to non-measurable benefits of reducing the risk of contaminating the environment. 55. Based on the fact that approximately 71% (US$1.70 million) of disbursed GEF funds (i.e., US$2.38 million) went towards disposal activities under ASP-P1 Ethiopia, the project’s efficiency was primarily assessed by its investment in the disposal of obsolete pesticides and associated waste. 56. During appraisal, the unit cost for disposal was estimated at US$3,400 per tonne (based on approximately US$15 million spent on the disposal of 3,240 tonnes of obsolete pesticides between 1990 and 2004 across Africa). (See Section 3.4 in the program-level ICR for details). 57. At project closing, 450 tonnes 105 of obsolete pesticides and associated waste had been disposed of at the cost of approximately US$3,778 per ton, which represents a slightly higher cost than originally anticipated. 106 Given the overall high cost involved, cleanup and safe disposal was prioritized on the basis of a detailed inventory of pesticide stockpiles and associated waste, determining the identity of the contaminant, its quantity, and its proximity to the people and environment. In comparison with the originally estimated US$3,400 per ton, the project achieved its goal with a slightly higher cost-benefit than originally estimated. However, the US$ depreciation since 2004 and an increase in the price of fuel, which has increased the cost of disposal in US$ terms, is not taken into account by this calculation; neither are varying distances from collection centers to harbors, varying distances of shipping routes, and responses to call of bid by different companies. 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING 58. Overall outcome - Rating Moderately Satisfactory: On the basis of the project’s efficiency (substantial), relevance (substantial), and PDO achievement (moderately satisfactory), the overall outcome is rated moderately satisfactory. The rating takes into account that despite substantial implementation challenges prior to mid-term review (see Section 2), and a shortage of funds to dispose of and/or safeguard all inventoried obsolete pesticides and associated waste, the project successfully reduced risks to the environment and public health by disposing all inventoried obsolete pesticides. In addition, the project succeeded in significantly improving measures to reduce future re- accumulations by developing a national pesticide management strategy, including action plans for a national container management strategy, and pest management. 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS 59. Poverty impacts, gender aspects, and social development: The disposal of obsolete pesticides and associated wastes had a direct positive impact on poor and farming communities, species and ecosystems, located in proximity to the sites carrying obsolete pesticide stocks and associated waste. Baseline data on affected populations, species, and ecosystems were not gathered at appraisal; exact impacts are therefore difficult to measure. 60. Institutional change/strengthening: The project strengthened the government’s institutional capacity on several levels: (i) The regulatory and legal framework for pest and pesticide management was improved; (ii) the logistics of hazardous waste transportation were internalized and the knowledge 105 Not including the additional 395 tonnes of obsolete pesticides that were disposed of through parallel funding by CLI. 106 Costs include costs associated with repackaging, transport, and incineration of obsolete pesticides and other related activities such as for example the inventory, safeguarding efforts, associated studies, and advisory services. 63 gained can be used to inform future disposal activities; (iii) the experience and skills gained in managing the project can be applied to other projects and subsequently contribute to success; and (iv) knowledge was also gained on sound pesticide management through the various training that were provided to governmental staff. 61. Other unintended outcomes and impacts: A positive outcome was the disposal of 395 tonnes stocks of obsolete pesticides under CLI funding in August 2011. The stocks represented legacy stocks from the earlier FAO-II project, and had been stored near Addis Ababa. An EMP and a Health and Safety Plan were developed, found satisfactory by the World Bank, and subsequently disclosed at the World Bank’s InfoShop and in country in April/May 2010. Disposal activities were subsequently implemented in compliance with World Bank safeguard policies, and in accordance with all necessary international protocols. 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS 62. An ICR mission, including a stakeholder workshop was conducted in the beginning of May 2013 in Addis Ababa, Ethiopia. For a detailed summary of the findings please see Section 8.6. A beneficiary survey was not conducted. 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME 63. Risk to the development outcome - Rating Moderate. The risk is confined to the re-accumulation of obsolete pesticides, and a potential weakening of government commitment to sustaining and building on the project’s outcomes, particularly with respect to prevention activities. 64. The project substantially strengthened pesticide legislation and national capacity, and together with the various activities agreed upon in the national strategy, the government is well positioned to sustainably manage and reduce future accumulations of obsolete pesticides. In addition, the government has shown continued commitment to reducing future related risks, as demonstrated by its engagement in a similar project, the PRRP, which is planned to be completed by the end of 2013 (see Section 2.5). 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE 5.1 WORLD BANK PERFORMANCE 65. World Bank performance in ensuring quality at entry – Rating Moderately Unsatisfactory: Project preparation of ASP-P1 Ethiopia was conducted in conjunction with preparatory activities at the program-level, which were lengthy (see Section 2.1 in the program-level ICR). While considerable time was invested in overall preparation, key project preparation documents for ASP-P1 Ethiopia only broadly laid out project implementation plans, and lacked country-specific information. The project’s design was characterized by several weaknesses, which translated into substantial project implementation delays (see Section 2.1 in the program-level ICR). FM and procurement assessments adequately outlined potential risks, however, corresponding resulting mitigation measures were weak. 64 66. Quality of World Bank supervision – Rating Moderately Unsatisfactory: Until 2011, the project was supervised by the Task Team Leader (TTL) responsible for ASP at the program-level, who was based at the World Bank’s headquarters in Washington DC. In 2011, a co-TTL was assigned in Ethiopia. Supervision missions took place regularly, albeit only once a year (except in 2010 when a number of technical and operational support missions were conducted). 67. During the first three years, project implementation had been experiencing substantial delays; support from the World Bank was limited, and the project was affected by collaboration issues among partners, especially the World Bank and FAO, at the program-level (see Section 2.2 in the program- level ICR). 107 Following the mid-term review, the World Bank team took corrective action by significantly increasing its support to more effectively address the project’s implementation delays. These included funding of an international consultant team that conducted five support missions between July 2010 and January 2011 to help the PMU update and refine project management tools, and finalize the disposal bidding document. In 2011, the World Bank prepared a Level 1 restructuring involving revision of the PDO to better respond to sector realities and country needs; however, the PDO remained ambitious. Final disposal activities were effectively supported through: (i) more frequent and ad-hoc project implementation support from the co-TTL; (ii) reallocation of GEF funds to accommodate the disposal of higher than estimated obsolete pesticides; and (iii) timely preparation of two additional extensions of the closing date to ensure the safe arrival of already shipped obsolete pesticide materials at their destination, PDO achievement, and project funding eligibility of all disposal-related activities. FM review missions were conducted regularly to ensure the project’s FM arrangements remained acceptable to the World Bank. 68. Overall World Bank performance – Rating Moderately Unsatisfactory: The project suffered from shortcomings in project preparation and design at the program-level. Following a resulting slow performance during the first half of the project, the World Bank team gradually improved its performance through corrective actions. 5.2 BORROWER PERFORMANCE 69. Government performance – Rating Moderately Satisfactory: Government commitment at entry towards the issue of obsolete pesticides was evident through the implementation of a similar previous project, described in Section 1.1. The government had therefore acquired substantial experience and was considered experienced and well prepared for pesticide disposal under ASP. In addition, the government had already established the institutional structures for coordinating the implementation of ASP-P1. MoARD through the CPD had established a PMU in 2003. However, during project implementation, MoARD’s cumbersome and long procurement processes significantly contributed to project implementation delays. 70. Implementing agency performance – Rating Moderately Unsatisfactory: Overall, the implementing agency’s performance was moderately unsatisfactory, with gradual improvement throughout project implementation. Significant implementation delays were caused by a poorly managed transition period between the preceding pesticide project (FAO-II) and ASP-P1, and key staff that had unexpectedly left upon completion of the FAO-II project, as new staff had to be appointed and trained 107 Significant delays were caused by lengthy World Bank review and clearance processes of project-related documents. It took eight months to review and clear the CESA TORs, as FAO and the World Bank struggled finding an agreement on a format that was in line with technical requirements; no objections were often provided with delays. 65 over time. Particular delays were caused by the lack of adequate procurement staff throughout project implementation (see Section 2.4 for more details). However, the agency’s performance improved gradually, and particularly during the last two years of project implementation, it demonstrated strong commitment towards achieving the PDO. 71. The agency complied with all World Bank loan covenants, and executed its fiduciary duties. Delays were noted in submitting Financial Monitoring Reports, however, no issues in view of FM or procurement were noted during project implementation. The agency equally ensured safeguard compliance. 6. LESSONS LEARNED 72. The most significant lessons learned are outlined in Section 6 of the program-level ICR, as the experience they are based on was often reflected across all country-specific projects. Lessons learned that were particularly relevant to ASP-P1 Ethiopia include: • Both, the implementing agency and the World Bank team need to be staffed with appropriate technical expertise, which is crucial for sound preparation and implementation of projects associated with pesticide disposal. As was evident during implementation, the disposal of obsolete pesticides and associated waste required highly specialized technical expertise, particularly for the preparation of disposal activities (i.e., procuring the specialized services). The technical support the project received, particularly from the TAD proved to be crucial for the project’s progress toward PDO achievement. • Knowledge exchange between participating countries of a regional project can be value adding. CLI had provided support to ASP-P1 Ethiopia and ASP-P1 Tanzania by financing a TAD. The TAD was represented by the same person, which automatically facilitated the knowledge exchange between the two countries with respect to disposal activities. Both countries confirmed the value added through this knowledge exchange. • Possible low procurement capacity should be adequately reflected when assessing risks. Project implementation was significantly affected by continuous procurement-related delays. The general paucity of trained procurement specialists in Ethiopia and competition between projects for procurement staff should be taken into account during preparation. 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS 73. See Section 8.7 and 8.8. 66 8. ANNEXES 8.1 PROJECT COSTS AND FINANCING a. Project cost by component – All sources (in US$ million equivalent) – as of 06/30/2013 Actual/Latest Appraisal Estimate Actual as % of Component Estimate (in US$ million appraisal (in US$ million equivalent) estimate equivalent) 1. Disposal of obsoletes pesticides 2.142 3.017 141% 2. Prevention of accumulation 0.194 0.3 155% 3. Capacity building 0.292 0.116 40% 4. Project management 0.421 0.675 160% Total 3.048 4.108 135% b. Financing – as of 06/30/2013 Actual/Latest Appraisal Estimate Actual as % of Type of Estimate (in Sources of Funds (in US$ million appraisal Co-financing US$ million equivalent) estimate equivalent) Borrower In-kind 0.428 0.428 100% Global Environment Facility Grant 2.620 2.380 91% (GEF) Parallel Crop Life International 0 1.300 - Financing Total 3.048 4.108 135% 8.2 OUTPUTS BY COMPONENT 1. Taking account of the project’s Level 1 restructuring in June 2011, the following table provides a comprehensive account of qualitative and quantitative information of outputs realized against both original (achievements at the time of restructuring in June 2011) and revised components (achievements at the time of project closing in June 2013). Table 8.1: Outputs by Original Components as presented in the GEF Trust Fund Grant Agreement Original Components as presented in the GEF Outputs achieved at the time of restructuring in June 2011 Trust Fund Grant Agreement Part A – Obsolete pesticides inventory: Achievements under this component are rated Moderately Unsatisfactory 67 Carrying out of a detailed inventoried of the a. An inventory of publicly held obsolete pesticides was completed and recipient’s publicly held obsolete pesticides validated in May 2010, including 100% data entry into an inventory stocks and associated waste through: database, which has been under the management and oversight of a. Collection of data and compilation of a FAO, as part of their PSMS (Original PDO Indicator #1 – 100%). database; b. The PMU and the government’s staff were trained by FAO in using b. A risk-based prioritization of the stocks the system (2010), and used the information on stocks identified in identified in the database; and the database for a risk-based prioritization of the different sites, and c. Preparation of the CESA. as basis for the preparation of the CESA. c. At restructuring, a CESA was still being finalized and not yet completed (Original PDO Indicator #2 – 0%). Part B – Disposal of obsolete pesticides: Achievements under this component are rated Moderately Unsatisfactory Treatment and/or disposal of publicly held a. A technological assessment of available treatment options was obsolete pesticides stocks and associated waste carried out by WWF in 2008. by: b. Disposal activities, started at the time of restructuring, thus a. Carrying out of a technological assessment of implementation of the treatment and/or disposal technology selected available treatment and disposal options; was not completed (Original PDO Indicator #3 – 0%). b. Implementation of the treatment and/or c. Preparations for the construction of a new storage warehouse for disposal technology selected; temporary storage of imported pesticides and thus better pesticide c. Improving the operations of obsolete pesticide management were initiated, including the development of an stocks collection centers, including improving Environmental and Social Impact Assessment (ESIA) which was safety measures at those centers; and completed in April 2011. d. Container management. d. Recycling of metal and/or plastic containers was addressed by a container management study, which was being conducted at the time of restructuring. Part C – Prevention of obsolete pesticide accumulation: Achievements under this component are rated Moderately Unsatisfactory Carrying out of activities aimed at preventing a. In collaboration with WWF, a communication strategy was finalized obsolete pesticide accumulation, including: in March 2009. The strategy comprised an outreach program a. Awareness raising activities on the program, focusing on farmers and rural communities including a number of health and environmental hazards of pesticides, key events to support the prevention component. The project and IPM; subsequently raised stakeholders' awareness of the new b. A review and update of the legal and proclamation, through a three-day workshop in May 2011. regulatory framework to control obsolete However, individual outreach activities envisaged under the pesticide accumulation and enforcement communication strategy were not implemented at restructuring. mechanisms; b. Better regulation and control of the registration, production, storage, c. Preparation of pesticide management strategy; and sale of pesticides was achieved by strengthening the legal and and regulatory framework for pesticide management. A Pesticide d. Strengthening the pesticide management Registration and Control Proclamation was approved in January capacity of storage warehouses. 2010 by the Council of Ministers, proclaimed by Parliament in June 2010, and published in August 2010 in a federal newspaper. However, measures to strengthen compliance with the Basel Convention and the Rotterdam Convention had not been undertaken (PDO Indicatory #4 - 50%). c. Important steps were made towards preparing a national pesticide management strategy, aiming to enhance the sustainability of MoARD's organizational and financial provisions to deal with obsolete pesticide and associated waste; however, at restructuring the strategy was not yet finalized (PDO Indicator #5 - 0%). 68 d. Preparations for the construction of a new storage warehouse for temporary storage of imported pesticides and thus better pesticide management were underway. Part D – Capacity building: Achievements under this component are rated Moderately Satisfactory. Carrying out of capacity building activities, a. Between 2010 and 2011, PMU members and representatives from including: MoARD participated in several trainings such as pesticide a. Provision of training to relevant staff of the management (December 2010), safeguards and preparing EMPs, and recipient on laboratory analysis, health and PSMS. Capacity was strengthened as became evident though safety issues, and environmental monitoring. application of the acquired knowledge during the inventory, and b. Strengthening awareness on the pesticide preparation of safeguard instruments (PDO Indicator #6 – 100%). management legal framework and However, despite the strengthened capacity, the PMU was still homologation systems; and facing low procurement capacity at the time of restructuring, which c. Development and implementation of a continued to delay project implementation (PDO Indicator #7 – pesticide storage and IPM training program. 50%). b. A Pesticide Registration and Control Proclamation was approved in January 2010 by the Council of Ministers, proclaimed by Parliament in June 2010, and published in August 2010 in a federal newspaper. Copies of the proclamation were also distributed to approximately 45 government representatives during an associated training. c. Training in store keeping, pesticide management, and maintaining the PSMS was provided to the PMU and representatives of the ministries. Part E – Project management: Achievements under this component are rated Moderately Unsatisfactory. Supporting the PMU in carrying out project An unclear partnership arrangement at the program-level hampered coordination, implementation, monitoring, and timely technical support, particularly during the first three years. evaluation, through the provision of technical • The highly technical and specialized expertise required for advisory services, equipment, and operational implementation, supervision and monitoring of project activities the support. PMU was supposed to receive from FAO’s TSU was not delivered to the extent planned. Valuable support was given in terms of equipment through the provision of the PSMS, and associated training. However, limited technical advisory services were given on the pre-qualification, bidding, and execution of the specialized services associated with procuring the selection processes of the CESA and disposal service providers, and M&E. • The urgently needed operational support from the World Bank was not sufficiently provided prior to mid-term review; however, increased supervision, operational and technical support in 2010, and the subsequent deployment of an in-country co-TTL, improved the level of support. 69 Table 8.2: Outputs by Revised Components as presented in the Level 1 Restructuring June 30, 2011 Revised Components (Level 1 Restructuring Outputs achieved at the time of project closing June 30, 2011) 108 Specific revisions are underlined Part A – Obsolete pesticidesinventory: Achievements under this component are rated Moderately Satisfactory Carrying out of a detailed inventoried of the a. See outputs achieved by Original Part B. recipient’s publicly held obsolete pesticides b. See outputs achieved by Original Part B. stocks and associated waste through: c. A CESA identifying mitigation measures to treat possible negative a. Collection of data and compilation of a environmental and social impacts that were considered likely to database; occur during implementation of the project, was completed in b. A risk-based prioritization of the stocks February 2012, found satisfactory by the World Bank in December, identified in the database; and 2011, and disclosed to the World Bank InfoShop on February 29, c. Preparation of the CESA. 2012. (Intermediate Outcome Indicator #2 - 100%). Part B – Disposal and/or safeguarding of obsolete pesticides and associated waste: Achievements under this component are rated Moderately Unsatisfactory. Safeguarding and/or disposal of publicly held a. See outputs achieved by Original Part B. obsolete pesticides stocks and associated waste b. 450 tonnes (35%) of the inventoried 1,300 tonnes of publicly held by: obsolete pesticides and associated waste had been disposed of a. Carrying out a technological assessment of overseas. These included 450 tonnes of obsolete pesticides (against available treatment and disposal options; 450 tonnes inventoried), but none of the 850 tonnes of associated b. Implementation of the safeguarding and/or waste due to exhausted project funds. Reportedly, the project disposal technology selected; safeguarded some of the associated waste in form of cleaning c. Improving the operations of obsolete pesticide contaminated buildings, but no exact data was available/could be stocks collection centers, including improving verified (Revised PDO Indicator #1 - 35%). Additional 395 tonnes safety measures at those centers; and of obsolete pesticides were disposed of overseas in January 2012 by d. Container management. a CLI contracted and funded UK company. The stocks represented legacy obsolete pesticides, which originally had not been included in ASP-P1 Ethiopia. c. Preparations for the construction of a new storage warehouse for temporary storage of imported pesticides and thus better pesticide management were underway. d. An action plan for a national container management strategy (Intermediate PDO Indicators #3 – 100%) was endorsed by the Steering Committee on June 25, 2013. Part C – Reduction of obsolete pesticides and associated waste re-accumulation: Achievements under this component are rated Moderately Satisfactory. Carrying out activities aimed at reducing a. The project completed awareness raising activities as outlined in its obsolete pesticide and associated waste re- communication strategy. In 2012 and 2013, posters and similar accumulation, including: materials were prepared and distributed to raise awareness on the a. Awareness raising activities on the program, harmful nature of pesticides to human health and the ecosystem; and health and environmental hazards of pesticides. a one-year TV and radio program was conducted in three local b. A review and update of the legal and languages to raise awareness on pesticides use (Intermediate PDO regulatory framework to control obsolete Indicator #4 - 100%). pesticide accumulation and strengthen b. See outputs achieved by Original Part C. enforcement mechanisms; c. A national pesticide management strategy was developed and 108 Restructuring Paper on a Proposed Project Restructuring of Africa Stockpiles Program – Project 1 to the Republic of Tunisia, (Report No: 62797-TN), June 20, 2011 70 c. Preparation of a sustainable pesticide endorsed by the Steering Committee on June 25, 2013 (Revised management strategy; PDO Indicator #2 - 100%; and Intermediate Outcome Indicator #5 - d. Strengthening the pesticide management 100%). The strategy focused on sound pesticide management based capacity of storage warehouses; on a holistic, integrated, and systematic approach. Equally endorsed e. Provision of training to relevant staff of the by the Steering Committee on June 25, 2013 were action plans for a recipient on health and safety issues, and national container management strategy (Intermediate PDO environmental monitoring; and Indicator #3 - 100%), and pesticide management. f. Strengthening awareness on the pesticide d. The construction of a new storage warehouse for temporary storage management legal framework and of imported pesticides and thus better pesticide management was homologation systems. aborted and not completed as a result of time constraints (Intermediate PDO Indicator #7 - 0%). e. See outputs achieved by Original Part D. f. See outputs achieved by Original Part D. Part D – Capacity building: Achievements under this component are rated Moderately Unsatisfactory. Carrying out of capacity building activities, a. See outcomes achieved under Original Part D. including: b. See outcomes achieved under Original Part D. a. Strengthening the recipient’s capacity to carry c. See outcomes achieved under Original Part D. out environmental assessments. b. Reviewing the recipient’s pesticide management practices and providing training on improved methodologies. c. Reviewing the implementation capacity of the PMU and providing requisite training. Part E – Project management: Achievements under this component are rated Moderately Satisfactory Supporting the PMU in carrying out project In addition to outputs achieved under Original Part D, the World Bank coordination, implementation, monitoring, and provided effective operational support during the final disposal evaluation, through the provision of technical activities. For technical support, a TAD was eventually recruited and advisory services, equipment, and operational financed by CLI, which positively affected project implementation. support. 8.3 ECONOMIC AND FINANCIAL ANALYSIS 2. As outlined in Section 3.4, an economic and financial analysis is not available. 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS a. Task Team Leaders Lending (Task Team, specialization, and unit as outlined in the PAD) Names Specialization Unit Responsibility/Specialty Abushakra, Hadi Chief Legal Counsel LEGMS Brackmann, Stefanie Environmental Specialist MNSRE Chihuguyu, Evelyn Assistant AFTS4 Di Leva, Charles Chief Counsel LEGEN Eshetu, Yimer Senior Financial Management Specialist AFTFM 71 Fissha, Azeb Extended Term Consultant AFTS2 Gorman, Steve Lead Environmental Specialist ENVGC Jordy, Denis Senior Agricultural Specialist AFTS4 Co-TTL Kaniaru, Muthoni Counsel LEGAF Konare, Amadou Senior Environmental Specialist AFTS1 Kranz, Frederick Senior Procurement Specialist MNACS Kristensen, Peter Senior Environmental Specialist AFTS4 TTL Maber, Steven Senior Operations Officer MNSRE TTL Ndiaye, Marie-Jeanne Language Program Assistant AFTS4 Permalpillai-Essex, Jeeva Lead Operations Officer AFTS3 Rajat, Narula Senior Finance Officer LOAG2 Temechew, Abiy Procurement Analyst AFTPC Tran, Lucie Operations Officer AFTS4 Tynan, Ellen Senior Environmental Specialist ENVMP Warner, Christopher Senior Environmental Specialist AFTS1 TTL Waters, Warren Regional Environmental and Safeguards AFTSD Advisor Supervision/ICR Names Specialization Unit Responsibility/Specialty Abate, Asferachew Environmental Specialist Co-TTL Aryal, Dinesh Senior Operations Officer TTL Badisso, Shimelis Procurement Specialist Woldehawariat Belaye, Abiye Demissie Financial Management Specialist Brackmann, Stefanie Environmental Specialist Doucouré, Djibril Safeguards Specialist Consultant Dwumfour, Edward Senior Natural Resource Management Co-TTL Specialist Fissha, Azeb Operations Analyst Freminatos Abraham, Financial Management Specialist Tafesse Fye, Serigne Omar Senior Environment Specialist Guibert, Yves Jordy, Denis Senior Environmental Specialist Co-TTL Kristensen, Peter Program Manager TTL Lagnaoui, Abdelaziz Senior Environmental Specialist Peled, Ayala Environmental Economist Prevoo, Dirk Senior Operations Officer TTL 72 Rechbauer, Gabriele Senior Environmental Operation Specialist Said, Sophia Team Assistant Schmidt, Veruschka Strategy Officer Tadesse, Meron Financial Management Analyst Temechew, Abiy Procurement Analyst Tsegaye, Lakech Team Assistant Tynan, Ellen Environmental Specialist TTL b. Staff time and cost –Staff time and costs associated with program preparation and supervision was not possible to determine as a result of poor record keeping in the system. 8.5 BENEFICIARY SURVEY RESULTS 3. Beneficiary survey results are not available, as no survey was conducted. 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS 4. An ICR mission including a stakeholder workshop was conducted in the beginning of May 2013 in Addis Ababa, Ethiopia. Stakeholders highlighted the following key points: • The most important achievements that resulted from prevention activities included the preparation of a strategic plan, awareness raising among different target groups, the development of a legal framework that resulted in better utilization of the registration/purchasing system, and capacity building among government officials and users. • The most important achievements that resulted from disposal activities included clearing up storage facilities, knowledge transfer from the international disposal company to national staff, identification of 850 tonnes of contaminated soils, and completion of a nation-wide inventory of obsolete pesticides. • Key challenges included a shortage and high turnover of PMU staff, improper project management, weak procurement capacity, project design problems, project overlaps between the preceding and ASP project, slow response from different countries regarding the shipment, a lack of M&E, a lack of laboratories to determine the level of contamination, lack of awareness on dangers among end-users of pesticides, and lack of heavy trucks for transportation. • The sustainability of project outcomes could be ensured through effective use of the gained opportunities, addressing the remaining associated waste, use of IPM and IVM, assessing alternative means for disposal, implementing improved legislation, increased and sustained government commitment, policies, strategies, and budget allocation, disposal and prevention strategies, continued awareness raising, and collaboration with the private sector. • The institutional and partnership arrangement was lacking a harmonized implementation system, resulting in weak awareness and coordination among and between stakeholders. • Key lessons learned were: (i) A pesticide management system needs to be in place; (ii) recruitment of local staff minimized costs; (iii) empty containers needed to be triple-rinsed 73 before they could be reused; (iii) knowledge sharing with other countries was value-adding; (iv) local stakeholders had to be involved from the beginning on; (v) emphasis on training was important; (vi) a pesticides poisoning/testing center should be in place; (vi) pesticides safety and management should be covered by the media; (vii) a system for container management needs to be in place; (viii) personal protective equipment needs to be available; and (ix) technical requirements per year need to be assessed in advance. 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR 5. The borrower submitted a completion report; messages contained therein were integrated into this document. 6. Comments received on the draft ICR were equally incorporated. Additional comments included the following: a. Inventory: We have satisfactorily complete inventory and therefore we do not accept the rating for inventory as Moderately Satisfactory. Please change this rating to Satisfactory. • World Bank response: The data sheet notes that Original PDO Indicator #1 (Inventory database of publicly held obsolete pesticides stocks and associated waste in place and in use by the PMU and the recipient’s staff) was fully achieved (100%). b. PDO Indicator 1: ASP-P1Ethiopia has disposed of all inventoried obsolete pesticides (450 tonnes) though the original target was 250 tonnes. Moreover, the project didn’t have any plan and resources to fully safeguard and or dispose all inventoried associated waste. Hence we strongly disagree on the downgraded rating of our performance of this PDO Indicator and recommend it to be updated as Satisfactory. • World Bank response: The Revised PDO Indicator #1 (as per the Level 1 Restructuring in June 2011) was: “Inventoried publicly held obsolete pesticides and associated waste disposed of and/or safeguarded by the end of the project”. The inventory noted 450 tonnes of obsolete pesticides and 850 tonnes of associated waste. At restructuring, it was agreed to dispose of 250 tonnes of obsolete pesticides and safeguard all other inventoried obsolete pesticides and associated waste. However, it was recognized that the Revised PDO Indicator #1 continued to be ambitious and funds insufficient. Therefore, the government and the World Bank agreed to give priority to disposing of obsolete pesticides over safeguarding associated waste as these were considered posing the larger risk. The Overall Outcome rating took these circumstances into account, because even though the Revised PDO Indicator #1 was only achieved 35% (i.e., 450 tonnes of obsolete pesticides disposed of against inventoried 1,300 tonnes of obsolete pesticides and associated waste), the Overall Outcome was rated Moderately Satisfactory. c. Efficiency: Our cost of disposal and safeguarding is about 3,011 US$/tonne but in the ICR it is stated that to be 3,778 US$/tonne. As our actual cost is less than the proxy (3,400 US$/tonne) we believe that ASP Ethiopia is cost effective and efficient. • World Bank response: The ICR’s calculation took disbursed GEF funds that were spent towards disposal activities as basis. This included costs associated with repackaging, transport, and incineration of obsolete pesticides and other related activities. 74 d. Performance of Implementing Agency: We would be grateful if the downgraded rating (Moderately Unsatisfactory) be upgraded to be at least Moderately Satisfactory as all efforts were made by MOA/APHRD for the successful completion of ASP-P1 Ethiopia. • World Bank response: The rating is based on the discussion outlined in Section 2.2 and 5.2, and found adequate. e. Overall performance rating: We deserve to be evaluated as Satisfactory as we have disposed 450 tonnes (almost double of the original plan), prepared pesticide mgt strategy, proclaimed pesticide registration and control proclamation, drafted pesticide registration and control regulation, raised awareness on pesticide issues, disposed additional 395 tonnes through CLI funding and implementing PRRP Ethiopia project, which mainly focuses on pesticide registration and post registration capacity building etc. 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS 7. No comments were received from co-financiers and other partners/stakeholders on the ASP-P1 Ethiopia draft ICR. 8. Comments of co-financiers and other partners/stakeholders on the program-level ICR are summarized in Section 8.8 in the program-level ICR. 75 8.9 VERBATIM COMMENTS ON THE DRAFT ICR 76 TECHNICAL ANNEX II: ASP-P1 MALI 77 ABBREVIATIONS AND ACRONYMS AELP Africa Emergency Locust Project APL Adaptable Program Lending ASP-P1 Africa Stockpiles Program – Project 1 CAS Country Assistance Strategy CESA Country Environmental and Social Assessment CLI CropLife International CMDT Compagnie Malienne pour le Développement des Textiles DNACPN Direction Nationale de l’Assainissement et du Contrôle des Pollutions et des Nuisances EC European Commission EMP Environmental Management Plan EMPRES Emergency Prevention System for Trans-boundary Animal and Plant Pests and Diseases FAO United Nations Food and Agriculture Organization FFEM French Global Environment Facility GDP Gross Domestic Product GEF Global Environment Facility GEO Global Environment Objective ICR Implementation Completion and Results Report IFR Interim Unaudited Financial Report IP Implementation Performance IPM Integrated Pest Management ISR Implementation Status and Results Report IVM Integrated Vector Management M&E Monitoring & Evaluation MDTF Multi-Donor Trust Fund NGO Non-Governmental Organization NIP National Implementation Plan PAD Project Appraisal Document PAN Pesticides Action Network PDO Project Development Objective 78 PIF Project Identification Form PMU Project Management Unit POP Persistent Organic Pollutants PSMS Pesticide Stock Management System QAE Quality at Entry QAG Quality Assurance Group QSA Quality of Supervision RAP Resettlement Action Plan SDC Swiss Agency for Development and Cooperation SIDA Swedish International Development Cooperation Agency TAD Technical Advisor for Disposal TOR Terms of Reference TSU Technical Support Unit TTL Task Team Leader WWF World Wildlife Fund 79 TABLE OF CONTENTS 1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN ........ 86 1.1 CONTEXT AT APPRAISAL .............................................................................................................................................. 86 1.2 ORIGINAL PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS .................................................................. 87 1.3 REVISED PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS .................................................................... 88 1.4 MAIN BENEFICIARIES..................................................................................................................................................... 88 1.5 ORIGINAL AND REVISED COMPONENTS ................................................................................................................... 88 1.6 OTHER SIGNIFICANT CHANGES ................................................................................................................................... 90 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ....................................................................... 91 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY .............................................................................. 91 2.2 IMPLEMENTATION .......................................................................................................................................................... 92 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTLIZATION ........................... 93 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE ........................................................................................................... 94 2.5 POST-COMPLETION OPERATION/NEXT PHASE ........................................................................................................ 96 3. ASSESSMENT OF OUTCOMES ........................................................................................................................................... 96 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION ........................................................................ 96 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE ..................................................................................... 96 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE ..................................................................................... 98 3.4 EFFICIENCY ...................................................................................................................................................................... 98 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING................................................................................................... 98 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS .............................................................................. 99 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS .................... 100 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME .......................................................................................... 100 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE .................................................................. 100 5.1 WORLD BANK PERFORMANCE .................................................................................................................................. 100 5.2 BORROWER PERFORMANCE....................................................................................................................................... 101 6. LESSONS LEARNED ............................................................................................................................................................ 101 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS ................ 102 8. ANNEXES ............................................................................................................................................................................... 102 8.1 PROJECT COSTS AND FINANCING ............................................................................................................................. 102 8.2 OUTPUTS BY COMPONENT ......................................................................................................................................... 103 8.3 ECONOMIC AND FINANCIAL ANALYSIS .................................................................................................................. 105 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS ................................. 105 8.5 BENEFICIARY SURVEY RESULTS .............................................................................................................................. 105 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS............................................................................................ 105 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR ................................................................. 105 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS ................................................... 107 8.9 VERBATIM COMMENTS ON DRAFT ICR ................................................................................................................... 107 80 A. Basic Information 109 Country: Mali Project Name: Africa Stockpiles Program - Project 1 Project ID: P103189 L/C/TF Number: TF57678 ICR Date: 11/27/2013 ICR Type: Core ICR Lending Instrument: APL Borrower: Republic of Mali Original Total US$2.55 Disbursed Amount: US$2.44 million (12/31/2012) Commitment: million Revised Amount: - Environmental Category: A Global Focal Area: CHEM Implementing Agency: Direction Nationale de l’Assainissement et du Contrôle des Pollutions et des Nuisances (DNACPN) Co-financiers and Other External Partners: Government of Republic of Mali, CropLife International (CLI), CropLife-Mali, European Commission (EC), Swedish International Development Cooperation Agency (SIDA), Swiss Agency for Development and Cooperation (SDC) and Denmark Ministry of Foreign Affairs, French Global Environment Facility (FFEM), The Netherlands, UN Food and Agriculture Organization (FAO), Pesticide Action Network (PAN-UK and PAN-Africa) and World Wildlife Fund (WWF). B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/17/2002 Effectiveness: 02/12/2007 12/26/2006 Appraisal: 04/19/2004 Restructuring: - - Approval: 12/05/2006 Mid-term Review: 02/11/2009 02/06/2009 Closing: 08/31/2010 12/31/2011, 06/30/2012, 12/31/2012 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Global Environment Outcome: High World Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of World Bank and Borrower Performance World Bank Ratings Borrower Ratings 109 The information presented in the data sheet was inserted manually. Information provided in the system was incomplete resulting from coding issues experienced during implementation; these were not resolved prior project closing. 81 Quality at Entry: Moderately Unsatisfactory Government: Satisfactory Implementing Quality of Supervision: Moderately Unsatisfactory Moderately Satisfactory Agency/Agencies: Overall World Bank Overall Borrower Moderately Unsatisfactory Moderately Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators - Performance (if any) Potential Problem Project at Quality at Entry Yes - any time (Yes/No): (QEA): Problem Project at any time Quality of Yes - (Yes/No): Supervision (QSA): PDO rating before Moderately Satisfactory Closing/Inactive status D. Sector and Theme Codes Original Actual Sector Code (as % of total World Bank financing) Sanitation 33.3 33 Health 33.3 33 General Agriculture 33.3 34 Theme Code (as % of total World Bank financing) Environmental policies and institutions 50 50 Pollution management and environmental health 50 50 E. World Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Gobind T. Nankani Country Director: Ousmane Diagana Mark D. Tomlinson Director (Regional Colin Bruce - Integration): Sector Director - Michel Wormser Sector Manager: Magda Lovei Mary Barton-Dock Project Team Leader: Dinesh Aryal Peter Kristensen/Denis Jordy ICR Team Leader: Dinesh Aryal ICR Primary Author: Ayala Peled Ben Ari 82 F. Results Framework Analysis - At project approval, a country-specific results framework was not presented; only six PDO Indicators were defined. Project Development Objective (PDO) – as presented in the GEF Trust Fund Grant Agreement: 110 To assist the recipient in: a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste and preparing measures for the future decontamination of soils in priority sites; and b) implementing measures to reduce and prevent future risks related to pesticides and waste. Revised PDO – Not Applicable Global Environment Objectives (GEO) – as presented in the PAD (Report No: 36273-AFR): Reduction of the effects of Persistent Organic Pollutants (POPs) on the global environment. Revised GEO – Not Applicable (a) PDO Indicators – Original PDO Indicators as presented in the GEF Trust Fund Grant Agreement are assessed against achievements realized at project closing in December 2012. Original Target Formally Actual Value Achieved at Values Indicator Baseline Value Revised Target Completion or Target (from approval Values Years documents) Original PDO Indicators The inventory database of publicly held obsolete pesticide stocks is in place and Indicator 1: being used by the Project Management Unit (PMU) and the recipient's staff Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 12/31/2012 - 12/1/2009 Target fully achieved (100%): An inventory was completed and validated in 2007, including transfer of data into the FAO obsolete pesticide inventory database. The Comments (incl. % database was subsequently used for preparation of a Country Environmental and achievement) Social Assessment (CESA), safeguarding and soil cleaning operations, and to prepare disposal bidding documents. Inventoried publicly held obsolete pesticide stocks have been disposed of, as a Indicator 2: result of the completion of the disposal services contract(s) in accordance with national and international laws and regulations Value (quantitative 0% 100% - 0% or qualitative) Date Achieved 04/19/2004 12/31/2012 - 12/31/2012 Comments (incl. % Target not achieved (0%): The inventoried stocks were not disposed of due to a long achievement) disposal procurement process and the closing of the project before disposal activities 110 The PDO, indicators, and components that are stated in the Project Appraisal Document (PAD) slightly differ from those stated in the GEF Trust Fund Grant Agreement. This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities. 83 commenced. A plan, in form and substance satisfactory to the World Bank, for the Indicator 3: decontamination of soils in priority sites has been prepared by the recipient Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 12/31/2012 - 12/31/2012 Target fully achieved (100%): The project carried out nation-wide soil contamination surveys in July 2007 and September 2009, and prepared an assessment of five Comments (incl. % contaminated sites in May 2009. The PMU subsequently prepared site-specific achievement) Environmental Management Plans (EMPs) for the decontamination of five priority sites. The legal and regulatory framework for pesticide management has been Indicator 4: improved and includes measures to strengthen compliance with the Basel Convention and the Rotterdam Convention Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 12/31/2012 - 12/31/2012 Target fully achieved (100%): The project conducted a legal review of Mali’s legislative and regulatory framework for pesticide management in 2007, and Comments (incl. % provided technical support for its improvement to various government entities. achievement) Consequently, project objectives and activities were integrated into national policies for hazardous waste management. The training program under Part B 111 of the project has been implemented, Indicator 5: and the knowledge so acquired is being used by the PMU and the recipient's staff Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 12/31/2012 - 12/31/2012 Target fully achieved (100%): An extensive training program was implemented Comments (incl. % throughout implementation on inventory taking, data maintenance, site achievement) decontamination and pesticides management, reaching approximately 400 governmental and non-governmental agents, as well as pesticide retailers. Indicator 6: The PMU is functional, and its operation is satisfactory to the World Bank Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 12/31/2012 - 12/31/2012 Comments (incl. % Target partly achieved (80%): PMU was functional and operated satisfactorily albeit achievement) procurement weaknesses. (b) GEO Indicator(s) – The GEO was measured by the PDO and PDO indicators. 111 See Section 1.5 for the project’s Component B (in the GEF Trust Fund Grant Agreement referred to as ‘Part B’). 84 (c) Intermediate Outcome Indicator(s) – Not Applicable G. Ratings of Project Performance in ISRs – Some DO and IP ratings and actual disbursement data are not available because ISRs reported on ASP-P1 as a whole, and only occasionally on country-specific project performance, including disbursements. Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 10/31/2005 2 06/30/2006 Not applicable as ASP-P1 Mali not yet launched 3 12/27/2006 4 06/28/2007 Satisfactory Moderately Satisfactory - 5 12/18/2007 Moderately Satisfactory Moderately Satisfactory - 6 05/30/2008 - - - 7 12/18/2008 Moderately Satisfactory Satisfactory - 8 03/25/2009 Moderately Satisfactory Satisfactory 1.06 9 12/29/2009 Moderately Satisfactory Moderately Satisfactory - 10 06/29/2010 Moderately Satisfactory Moderately Satisfactory - 11 03/27/2011 - - 1.14 12 12/25/2011 Moderately Satisfactory Satisfactory 2.04 13 06/29/2012 Moderately Satisfactory Satisfactory 2.32 12/31/2012 Final disbursement 2.44 H. Restructuring ISR Ratings at Amount Board Restructuring Disbursed at Restructuri Approved Reason for Restructuring ng Date(s) PDO Restructuring & Key Changes Made PDO IP in USD Change millions 04/01/2010 No Not Not Not • Extension of closing date by 16 months Available Available Available (08/31/2010 to 12/31/2011) • Reallocation of proceeds to accommodate delays in the signature of the ASP-Mali MDTF agreement due to internal processing 12/12/2011 No Not Not 2.04 • Extension of closing date by 6 months Available Available (12/31/2011 to 06/30/2012) 07/02/2012 No Moderately 2.32 • Extension of closing by 6 months Satisfactory Satisfactory (06/30/ 2012 to 12/31/2012) I. Disbursement Profile – A disbursement profile is not available (see explanation under G). 85 1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN 1.1 CONTEXT AT APPRAISAL 1. Country background: At project approval in December 2006, agriculture and livestock husbandry had long been the backbone of the economy, accounting (with related activities) for around 30-35% of GDP, as well as providing the bulk of export revenue until the advent of large-scale gold production in 2000. 112 Besides its importance in the overall economy, agriculture with main crops such as cereals and cotton had been the principal source of livelihood for a large segment of the Malian population. 2. Sector background: The agricultural sector had therefore accounted for the largest fraction of pesticides used in Mali, with pesticide imports having increased substantially during the 1990s. 113 The use of chemical pesticides had been a crucial factor in controlling pests and improving agricultural productivity. About 80% of the total agricultural pesticide use in Mali had been in the cotton sector; of which 95% were insecticides, distributed by the parastatal cotton development agencies and the Ministry of Agriculture. Within the health sector, the Ministry of Health had been responsible for the execution of public health pest control (i.e., mosquito and black fly control). From 1990 to 1992, about 630 tonnes of pesticides per year were imported annually for public health pest control, representing approximately 19% of total pesticide use. From 1993, data on pesticide use in the public health sector have become unavailable as a result of decentralization efforts and a shift in pest control responsibilities to the private sector. 3. Obsolete pesticides accumulation and soil contamination: Mali was considered a priority country to participate in ASP-P1 because of its large estimated volume of obsolete pesticides. 114 Accumulation of obsolete pesticides in Mali had been caused mainly by product bans due to their effects on public health and the environment, deterioration as a result of prolonged or poor storage, and poor procurement planning/uncoordinated donations of pesticides. Mali also had a number of sites with highly polluted soil as a result of spills and poor storage conditions. 115 4. Institutional context: Obsolete pesticides management is handled by the National Department for Sanitation and Pollution Control (Direction Nationale de l'Assainissement et du Contrôle des Pollutions et des Nuisances - DNACPN) within the Ministry of Environment and Sanitation. In 2002, a National Pesticides Management Committee (Comité National de Gestion des Pesticides - CNGP) was formed under government decision as the executive implementing body of existing pesticides management regulation. CNGP recently underwent review and resumed activities with support from the project. 5. Actions taken: From 2005 to 2006, and in the course of preparing the project, the Government of Mali carried out an inventory of obsolete pesticides, with support from the UN Food and Agriculture Organization (FAO). Close to 2,000 tonnes of the total inventory were considered having potential for 112 The Economist Intelligence Unit, Country Profile Mali, 2006 113 Volumes grew from 1,800 tonnes in 1991 to 4,100 tonnes in 1999 114 See ESMF Executive Summary (October 28, 2004). Pre-2005 estimates indicated that Mali possessed 500 tonnes of obsolete pesticides spread in 76 storage sites, as well as 40,000 tonnes of contaminated soil. 115 An African approach for Risk Reduction of Soil Contaminated by Obsolete Pesticides. Wageningen University and Research Centre, Wageningen (The Netherlands), 2009 86 dangerous environmental impacts, with the majority of the stocks having been inappropriately stored. 116 6. Rationale for World Bank assistance: Please refer to Section 1.1 in the program-level ICR. 7. Higher-level objectives: The project intended to contribute to the third pillar of Mali’s Country Assistance Strategy (CAS) 117, which focused on developing basic infrastructure and productive sectors. It aimed to support the development of the agriculture sector by reducing inputs of pesticides, to promote the use of Integrated Pest Management (IPM) in the cotton sector, to strengthen natural resource management by supporting the enhancement of pesticide management, and the conservation of scarce water supplies through supporting the cleanup of sites where underground water had been contaminated by pesticide spillage. The project further aimed to assist the government in developing strategies for cleaning up soils that had been contaminated with pesticides. Contributing to international efforts to eliminate Persistent Organic Pollutants (POPs), improving management of toxic chemicals, and promoting alternatives to pesticide usage that include IPM and Integrated Vector Management (IVM) represented further objectives. 1.2 ORIGINAL PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS 8. The Project Development Objective (PDO), and components that are stated in the Project Appraisal Document (PAD), 118 slightly differ from those stated in the GEF Trust Fund Grant Agreement. 119 This ICR bases its evaluation on the PDO, indicators, and components outlined in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities. Table 1.1: ASP-P1 Mali Original PDO Original PDO Original PDO as presented in the PAD as presented in the GEF Trust Fund Grant Agreement To assist the Government of Mali to dispose of publicly To assist the recipient in: a) eliminating inventoried publicly held obsolete pesticide stocks at priority-selected sites, to held obsolete pesticide stocks and associated waste and support the cleanup preparation of contaminated soils of preparing measures for the future decontamination of soils priority sites, and to support actions to help prevent the in priority sites; and b) implementing measures to reduce future accumulation of new stocks of obsolete pesticides and prevent future risks related to pesticides and waste. 9. At appraisal, only an overall ASP-P1 results framework had been prepared (see program-level ASP ICR); results frameworks for the individual ASP-P1 countries, including Mali, were intended to be finalized during the individual ASP-P1 country project launches. However, country-specific PDO indicators were developed at appraisal. For ASP-P1 Mali, these were as follows: 116 Particularly the contaminated site Nanguila (Kati) in the Koulikoro region was identified as posing high environmental risks. 117 Country Assistance Strategy for the Republic of Mali (Report No. 25663), July 7, 2003 118 Project Appraisal Document for the Republic of Mali, the Kingdom of Morocco, and the United Republic of Tanzania in support of the Africa Stockpile Program – Technical Annex 1, (Report No. 36273-AFR), November 7, 2006. 119 GEF Trust Fund Grant Agreement between the Republic of Mali and the International Bank for Reconstruction and Development, (Number TF057678 – MLI), December 26, 2006 87 Table 1.2: ASP-P1 Mali Original PDO Indicators Original PDO Indicators as presented in the PAD and the GEF Trust Fund Grant Agreement a. The inventory database of publicly held obsolete pesticide stocks is in place and being used by the Project Management Unit's (PMU) and the recipient's government staff. b. Inventoried publicly held obsolete pesticide stocks have been disposed of, as a result of the completion of the disposal services contract(s) in accordance with national and international laws and regulations. c. A plan, in form and substance satisfactory to the World Bank, for the decontamination of soils in priority sites has been prepared by the recipient. d. The legal and regulatory framework for pesticide management has been improved and includes measures to strengthen compliance with the Basel Convention and the Rotterdam Convention. e. The training program under Part C of the project has been implemented, and the knowledge so acquired is being used by the PMU and the recipient's government staff. f. The PMU is functional, and its operation is satisfactory to the World Bank. 1.3 REVISED PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS 10. Similar to ASP-P1 Ethiopia, ASP-P1South Africa, and Tunisia, ASP-P1 Mali was planned to undergo a Level 1 Restructuring to ensure the PDO and associated indicators were achievable and measurable. However, the restructuring of ASP-P1 Mali did not materialize due to delays in the processing of an additional GEF Grant for the project and the March 2012 coup d'état, which resulted in the temporary suspension of World Bank operations in the country in accordance with OP/BP 7.30 (Dealing with De Facto Governments). 1.4 MAIN BENEFICIARIES 11. Please refer to Section 1.4 in the program-level ICR. 1.5 ORIGINAL AND REVISED COMPONENTS 12. The PDO of ASP-P1 Mali was to be achieved through the implementation of three distinct components: Table 1.3: ASP-P1 Mali Original Components Original Components Original Components as presented in PAD as presented in the GEF Trust Fund Grant Agreement Component A – Country cleanup and disposal activities Part A – Cleaning and elimination of obsolete pesticides (Total: US$5.38 million; GEF: US$2.31 million): The and associated waste objective of this component was to provide an inventory of a. Carrying out a detailed inventory of the recipient’s obsolete pesticides, a risk assessment, a country publicly held obsolete pesticide stocks and associated environmental and social assessment (CESA), a minor waste through: a) provision of relevant training to the upgrade of collection sites and the disposal of the obsolete recipient; b) collection of data and compilation of a data pesticides. Main activities included: base; c) a risk-based prioritization of the stocks and a. Completing the detailed inventory and risk assessment of associated waste identified in the database; and d) freestanding obsolete pesticides; preparation of the CESA. b. Preparing CESA including an environmental management b. Disposal of obsolete pesticides and associated waste plan (EMP); through a) carrying out of a technological assessment of c. Preparing the disposal strategy, safeguarding obsolete available treatment and disposal options; b) pesticides and undertaking the actual disposal of the implementation of the treatment and/or disposal 88 freestanding obsolete pesticides; technology selected; c) improving the operations of d. Container assessment and disposal; obsolete pesticide stocks collection centers, including e. Buried pesticides assessment and disposal; and improving safety measures at those centers. f. Contaminated soil assessment and disposal. Component B - Prevention of obsolete pesticides Part B – Prevention of obsolete pesticide accumulation: accumulation (Total: US$1.77 million; GEF: US$0.24 a. Building capacity through the provision of training to million): farmers, civil society groups and relevant staff of the a. Strengthening the government’s capacity to manage recipient; pesticides stocks through the design and the b. A review and update of the legal and regulatory implementation of a program preventing the future framework to control obsolete pesticide accumulation accumulation of pesticides; and strengthen enforcement mechanisms; b. Promoting ongoing IPM and IVM efforts, particularly c. Promotion of improved pesticide management practices with small-scale farmers in sectors such as cotton. such as integrated pest management and integrated vector c. Strengthening small-scale farmers’ capacity to use and management, and improved management of empty manage pesticides; pesticide containers through the design and d. Raising awareness on general pesticide issues using an implementation of container management strategy; Non-Governmental Organizations (NGOs) to support the d. Awareness raising activities on the program, and the program; health and environmental hazards of pesticides, including e. Developing a container management strategy; and carrying out of workshops on reducing reliance on f. Strengthening laboratory services. pesticides in agricultural production, disease vector control and migratory pest control; e. Development of a long-term funding strategy for prevention activities and a plan to prevent pesticide accumulation; and f. Strengthening the recipient’s laboratory services. Component C - Project management (Total: US$1.18 Part C – Project management: Supporting the PMU in million; GEF US$0): carrying out project coordination, implementation, a. Project coordinator position primarily financed by the monitoring and evaluation, through the provision of Government; technical advisory services, goods and operational support. b. Pesticide management expert including for Monitoring and Evaluation (M&E). This position was planned to support the disposal operations. Therefore, the M&E function was included in its scope; c. Prevention and communication staff to support the prevention component and awareness raising; and d. Finance, procurement and administration staff to support the overall project. 89 1.6 OTHER SIGNIFICANT CHANGES Table 1.5: ASP-P1 Mali Other Significant Changes Change Date Justification Approval Extension of closing date Extension of the closing date of the 04/01/2010 To compensate for start-up delays, and allow Country GEF Trust Fund Agreement by 16 completion of the disposal contract, taking into Director months from 08/31/2010 to account the potentially long lag time between 12/31/2011. obsolete pesticides being exported from the country and the date of actual incineration (up to 12 months). Extension of the closing date of the 04/29/2011 Extension of the parent ASP MDTF as part of the Country Multi-Donor Trust Fund (MDTF) child overall restructuring of ASP-P1 projects. Director TF (TF096815) by 20 months from 04/30/2011 to 12/ 31/2012. Extension of the closing date of the 12/12/2011 To allow time to finalize all remaining project Country GEF Trust Fund Agreement by six activities by utilizing the project funds and the Director months from 12/31/2011 to proposed additional GEF financing. 06/30/2012. Extension of the closing date of the 07/02/2012 a) Continued implementation delays, especially due Regional GEF Trust Fund Agreement by six to the needed preparation of a Resettlement Action Vice months from 06/30/2012 to Plan (RAP) and the long disposal procurement President 12/31/2012. process; and b) the March 22, 2001 coup in Mali resulted in suspension of all operations in Mali in accordance with OP/BP 7.30. The extension was granted to allow for the possible resumption of operations after an assessment mission scheduled for June 2012. Reallocation of proceeds Reallocation of proceeds from the 04/01/2010 Delays in the signature of the ASP-P1 Mali MDTF Country categories Disposal Services (- agreement due to internal processing resulted in the Director $600,000), and unallocated (- additional use of the GEF Trust Fund, causing an $240,000) to categories Goods overdraw in almost all categories. The reallocation (+$180,000), Works (+$20,000), fixed the overdraw. The amount deducted from Consultant Services including Audits Disposal Services was later added to the Disposal (+$125,000), Training and Workshops category in the MDTF agreement. (+$150,000) and Operating Costs (+$365,000). 90 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY 13. ASP-P1 Mali was prepared between 2001 and 2006, as part of preparatory activities at the program- level. It took more than five years from GEF eligibility in July 2001, and more than four years from concept note in July 2002 to project approval by the Board in December 2006.120 14. Soundness of the background analysis: Project preparation was conducted in close collaboration between the government and the ASP partners, and it resulted in intputs of varying levels of quality. The government, the World Bank, FAO, CropLife International (CLI), French Global Environment Facility (FFEM), World Wildlife Fund (WWF) and Pesticides Action Network (PAN)-UK collaborated via joint missions and workshops, where each partner carried out a distinct role. 121 National stakeholders became involved around appraisal (2004), when a broad National Steering Committee (NSC) was created under a ministerial decree.122 The key preparatory outputs were the PAD and the Operational Manual: the PAD contained mostly ASP-P1 level information and plans with limited amount of country-specific information. Exception to this was the additional focus on contaminated soil, which had been recognized by all parties involved as a serious problem needing special attention. The Operational Manual, on the other hand, included considerable amount of background detail and plans specific to Mali, including a stakeholder analysis, an M&E framework, an analysis of the national legislative framework, environmental impact assessment procedures, an analysis of risks, and terms of reference for key tasks and budgets. There was a general agreement amongst the partners that the 2005-2006 inventory data might be inaccurate, and that the stocks of obsolete pesticides could range anywhere between 400 and 800 tonnes. Similarly, the lack of reliable data led to the estimate that soil treatment costs could range anywhere from US$2-6 million. Therefore, it was agreed that the inventory data would be validated early in implementation. 15. Assessment of project design: The project’s design (as for all ASP-P1 participating countries) was preconfigured at the program level, and did not reflect any country-specific modifications or lessons learned from pervious projects executed in Mali. Design weaknesses at the program-level that translated into implementation delays at the country level, including ASP-P1 Mali, are outlined in Section 2.1 of the program-level ICR. In the case of Mali, it was also agreed 123 that additional attention would be given to the issue of contaminated soil as a result of pesticide spills and leaks. Consequently, ASP-P1’s generic PDO was slightly modified to include the preparation of measures for future decontamination of soils, and treatment of contaminated soil was included under component A (Country Cleanup and Disposal Activities). Studies were to be undertaken to assess the scale of soil contamination, identify the adequate technologies, and estimate the costs of treatment of the soils. 120 The long preparation time resulted from a lengthy partnership building/negotiation process at the program-level resulting from the complex implementation arrangement and project design (see Section 2.1 in the program-level ICR). 121 During preparation, FAO provided technical support to the PMU and sponsored the temporary hiring of a national expert in obsolete pesticides; WWF and PAN-UK conducted workshops for journalists and francophone NGOs, respectively; and the World Bank Task Team focused its support on institutional arrangements, safeguard requirements, risks analysis, budgeting and fund raising. CLI and FFEM participated in technical and budgetary discussions. 122 Decree No. 04 /1516 of August 2, 2004, modified on September 8, 2006. 123 See for example, the PDF-B completion report which summarized key country-specific points of focus. 91 16. Government commitment: DNACPN’s strong commitment at entry was evident from its engagement in the inventory of free-standing obsolete pesticides with the support of FAO and the interest in continuing pesticide cleanup activities. In 2004, the government established a PMU within the Ministry of Environment and Sanitation, and the NSC, which included broad representation of all key stakeholders. Another evidence of the government’s strong commitment was the concern it expressed (as reflected in the various mission reports) about the lengthy preparation time, especially in view of risks posed by some contaminated sites. The government committed to co-financing corresponding to standard disbursement percentages (SDP) and to financing two technical project staff. 17. Assessment of risks: The PAD and the Operational Manual presented brief and partial risk assessments, which focused mainly on crosscutting and program-level risks. The PAD included one risk specific to Mali, which was related to the government’s lack of capacity or experience in treating contaminated sites or soils. As mitigation, the PAD proposed to outsource these activities to professional contractors. The Operational Manual identified risks emanating from the health impact of pesticide handing, accidents during transport and public opposition to the project. Corresponding mitigation measures were proposed, which correctly highlighted the need to conduct accurate inventories, hire external expertise and continuously seek additional funding in case additional stocks were revealed. Neither the PAD nor the Operational Manual included risks of delays due to the sequential design of country projects and political instability. Appropriate mitigation of the sequential design could have significantly increased the project’s success, while the country’s political situation was outside the scope of the project. 2.2 IMPLEMENTATION 18. The project experienced long delays, mainly in the procurement of a disposal company. Positively affecting project implementation were government's strong commitment to the project and the recruitment of an experienced Technical Advisor for Disposal (TAD) by CLI in mid-2009. 124 19. Key factors affecting implementation included: • Long disposal procurement process. The procurement of a disposal company took more than three years to complete due to: (i) low procurement capacity within DNACPN, which meant that some steps took longer to complete or had to be repeated; (ii) lack of clarity within the World Bank about the disposal procurement process (see Section 2.2 of the program-level ICR); and (iii) the long time it took to finalize the CESA, which had to be completed before the scale of interventions could be determined and shared with potential bidders. • A disposal-funding gap and delays in securing additional financing. Disposal activities were allocated a budget despite the uncertainty around the amount of stocks to be disposed of and the cost of soil treatment. An inventory validation exercise carried out in June 2007 found higher obsolete pesticide stocks (1,100 tonnes), indicating a disposal budget gap. While the exact scale of the gap was never fully identified by either the World Bank or the project, it was clear to all early in implementation that additional funds were needed in order to dispose of all stocks and 124 It should be noted, however, that the TAD was contracted with substantial delays (the ToRs were approved in September 2007 but the contract was signed in mid-2009) due to long discussions between CLI and the government about contracting procedures and mobilization of the funding within CLI. Given the TAD’s positive impact on the disposal contracting and PMU capacity, it is likely that an earlier recruitment could have shortened the disposal contracting period. 92 treat polluted soil. Additional financing was secured in March 2010 when the GEF Council approved a $3.19 million grant to ASP-P1 Mali, but this was only approved by the CEO in March 2012. Project/mission reports and discussions with World Bank task team members highlight the following reasons for this substantial delay: o Unsuccessful initial fund raising efforts. Until additional funds were sought from the GEF, the project was unsuccessful in securing any funds. In fact, project and mission reports do not confirm whether fund raising efforts were made. o Delayed submission of additional financing proposal to GEF. Following the February 2009 mid-term review and recognition that additional resources could not be secured by the government, the World Bank and the government decided to pursue additional financing from GEF. However, more than two and a half years had passed before the additional financing was approved (in March 2012).125 This was mainly the result of a World Bank TTL change, as well as the World Bank’s decision to merge the additional financing request with project restructuring and government’s reluctance to restructure the project. 126 • Temporary unavailability of the MDTF funds. The US$3.98 million MDTF closed in July 2009 (with an undisbursed balance of US$3.1 million) due to delayed donor agreement on an extension and amendment of the parent MDTF agreement. The process of establishing a new MDTF with the balance took 19 months to complete (on February 1, 2011). This delay was the result of internal processing delays, resulting, according to the PMU, in the delay of some project activities. It is unclear, however, to what extent this delay affected the project since the GEF funds were used during this period as backup, 127 and the disposal contract was only signed in October 2011, eight months after the new MDTF became available. • The March 2012 coup d'état and suspension of activities: The Additional Financing/Level I Restructuring Paper was submitted for management decision in March 2012, days before the military coup, which halted all operations in the country. 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTLIZATION 20. Overall M&E - Rating Moderately Unsatisfactory based on the following evaluation. 21. M&E design - Rating Unsatisfactory: The project’s M&E framework lacked the most important element – a project-level results framework. Although the appraisal mission developed a project-level results framework 128, it was not reflected in the PAD and the Operational Manual, both presenting only the overall ASP-P1 results framework (see program-level ASP ICR) and six country-specific 125 The additional financing was titled “Africa Stockpiles Programme - Project 1 – Supplemental Funds for Disposal and Prevention” (P118630). The Project Identification Form (PIF) included both Mali and Tunisia, while the CEO Memo included Mali alone after Tunisia had decided to withdraw from the request. 126 Discussion with the task team indicate the following two key reasons: (i) concerns about public criticism, and (ii) the Technical Support Unit’s (TSU’s) conflicting opinion that a restructuring was not necessary if some disposal tasks were carried by local staff, an option which was considered too risky and unacceptable to the World Bank. 127 In April 2010, the World Bank authorized a reallocation of GEF proceeds between categories in order to facilitate this. 128 See Africa Stockpiles Programme (ASP) Mali Country Project -Appraisal mission Aide Memoire (April 19 to 27, 2004), July 1, 2004 93 Performance Monitoring Indicators. These indicators did not permit adequate monitoring of project implementation progress because they focused on measuring outputs as opposed to progress towards project outcomes. Expected results under the prevention component, in particular, were not adequately considered by the indicators. 22. M&E implementation - Rating Moderately Satisfactory: The project was implemented without a country-specific results framework until the mid-term-review in February 2009, more than three years after effectiveness. Discussions with the PMU and the World Bank team indicate that the delays were caused by delayed recruitment of an M&E consultant and the dissatisfaction of the World Bank with the consultancy’s results. The PMU with assistance from a World Bank M&E specialist eventually developed an M&E manual and a project-specific results framework based on FAO guidelines and in agreement with World Bank instructions. The PMU was effective in collecting data and producing satisfactory progress reports. Performance evaluation was carried out internally by the disposal specialist who was also in charge of M&E, while external evaluation was carried by external consultants and an FFEM-assigned committee according to specific terms of reference. 129 23. M&E utilization - Rating Satisfactory: The inventory validation exercise and the national database that was created using this information were the key mechanisms for risk evaluation, site cleanup prioritization and resource allocation. The inventory data were also used to assess the disposal funding gap and dictated which safeguard instruments should be prepared and followed by the project. In order to enhance the usefulness of the national database, the project provided training on data management to key holders of pesticide stocks. Since the database was, and still is, hosted by FAO’s online Pesticide Stock Management System (PSMS), the government received training on data management offline to be periodically up-loaded back to FAO’s server. 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE 24. Safeguard compliance - Rating Satisfactory: ASP-P1 triggered two safeguard policies: OP 4.01 on Environmental Assessment and OP 4.09 on Pest Management. 25. ASP-P1 Mali complied with OP 4.01 on Environmental Assessment: The CESA was completed in December 2009 and disclosed in the World Bank InfoShop on April 8, 2010, more than three years after effectiveness. The long process was due to weak procurement capacity, lack of clarity of the PMU and the consultant on the scope of work, poor quality of the first drafts and a long authorization process within the World Bank. Stakeholders and concerned communities were consulted twice during the preparation of the CESA. The report outlined mitigation measures for the safe handling of hazardous substances during preparatory cleanup works, repackaging, intermediary storage, transport, and site rehabilitation. 130 The CESA highlighted the need to prepare separate EMPs for a number of highly contaminated sites and a RAP for one site where residents had to be temporarily relocated. While the CESA was never implemented (the disposal work had not commenced by project closing), the site-specific EMPs were implemented satisfactorily during site cleanup operations, as confirmed by World Bank reports prepared by a World Bank-hired safeguard specialist. 129 The committee comprised representatives from MEDD France, French Ministry of Foreign Affairs, French Development Agency, French Ministry of Agriculture and Forestry, PAN-Europe and Union of Plant Protection Industries. It oversaw FFEM- funded activities. Agricultural Research for Development and two disposal companies (SYPRED and ADIVALOR) provided technical expertise to the committee. 130 The CESA did not include mitigation measures for the final disposal since incineration was to take place in an overseas licensed facility, and therefore no impact was foreseen in Mali. 94 26. OP 4.09 on Pest Management: Since project activities themselves met requirements of a pest management plan (i.e., through pesticide management, Integrated Pest Management (IPM), and capacity building), individual country projects, including ASP-P1 Mali, were not required to prepare self-standing documents. See Section 2.4 in the program-level for more details. 27. ASP-P1 Mali complied with OP 4.12 on Involuntary Resettlement to a satisfactory extent: A RAP was prepared for planned safeguarding and soil remediation works in Nanguila (Koulikoro region), which involved the temporary relocation of inhabitants during the cleanup operation. The RAP was approved, disclosed in November 2010, and subsequently implemented in full compliance with safeguard requirements, as confirmed by various World Bank implementation support missions. 28. Financial Management compliance - Rating Satisfactory: The project had in place appropriate financial management staffing 131 and software, and accounts were maintained properly. The project submitted Interim unaudited Financial Report (IFRs) and independent audits in a timely and overall satisfactory manner. Any ineligible expenditure was reconciled. Highlighted weaknesses were unclear distribution of financial management tasks between the Administrative and Finance Officer and the Accountant, and some deficiencies in record keeping, internal control system management and fixed asset management. The project undertook the necessary corrective measures. GEF disbursement was low during the first half of the project (14% at mid-term compared to a 66% disbursement estimate), but increased substantially during the second half of the project (95% disbursement at closing). The low disbursement rate at mid-term is mainly attributable to the project's design having been back loaded in terms of expenditures with the more costly disposal activities (over 66% of the GEF grant budget) programmed to take place only at the end of the project. 29. Procurement - Rating Unsatisfactory: Procurement undertaken by DNACPN suffered from substantial delays, particularly in the purchasing of personal protective equipment for the disposal and cleanup operations, selecting a consulting firm to conduct the CESA and selecting a disposal company. The delays were attributable to a number of factors, the main one being DNACPN’s weak procurement capacity and the lack of a dedicated procurement staff132, combined with unclear guidance from the World Bank. The PMU also used a procurement plan, which had not been approved by the World Bank, and finalized contracts without the World Bank’s prior to approval. World Bank suggestions for increasing the procurement capacity of the PMU, by either hiring a procurement officer or providing training to other PMU staff, were not accepted by the PMU, highlighting the lack of qualified specialists in the country, and the already overloaded work program of the PMU members. Procurement received some reinforcement in July 2011 when the Africa Emergency Locus Project’s (AELP’s) procurement specialist began providing cross-support to the project under a coordination agreement between the two projects. 131 With the exception of a temporary absence of the administrative and financial manager (between March and June 2010) due to personnel exchange 132 Early in implementation the dual functioning of financial management and procurement was judged by the World Bank to be problematic because it both slowed down procurement and constituted a conflict of interest within the PMU where purchasing and payments were carried out by the same person. The World Bank repeatedly recommended hiring a procurement specialist but none was hired due to lack of qualified candidates in Mali. 95 2.5 POST-COMPLETION OPERATION/NEXT PHASE 30. Post-completion: During implementation, inventoried low risk stocks at the various sites were repacked in metal containers and centralized in a store 35 km outside Bamako by trained local staff. Higher risk stocks have remained untreated since the 2005-2006 inventory in more than ten stores across the country, including two stores in Gao and one store in Timbuktu. The condition of the stocks is overall unknown; however, discussions with the PMU indicate that it is likely that some stores are in poor condition, leaking and accessible by the public. Despite this situation, no post completion activities have taken place since project closing. 31. Next phase: Discussions between the World Bank and the GEF Secretariat confirmed that the approved US$3.19 million additional financing could be used in a follow-up disposal and prevention project in Mali, and a letter from government dated March 20, 2013 confirmed the government’s interest in pursuing this option. In April 2013, the World Bank conducted an identification mission, where it also sought additional sources of financing. The World Bank task team confirmed that if a follow-up project would be launched, most of the proceeds would be used to pay for the disposal and/or safeguarding of the 1,100 tonnes of obsolete pesticide stocks and treatment of sites with highly contaminated soil. 32. DNACPN had offered to create a permanent ASP unit, which would be responsible for updating the national obsolete pesticide and waste inventory, prepare for the disposal operation, oversee decontamination of polluted sites, and conduct various prevention activities. It is unclear at this point whether this unit will be formed by the government. 3. ASSESSMENT OF OUTCOMES 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION 33. Relevance – Rating Substantial: Pesticide management, and obsolete pesticide and site cleanup were and continue to be relevant to the country's current transitional and development priorities, but are not key priorities. The Plan for the Sustainable Recovery of Mali for 2013-2014 (May 2013) only includes a pesticide cleanup operation in Gao.133 The Plan is part of the country’s Transitional Roadmap of January 29th 2013 and the Strategic Framework for Growth and Poverty Development 2012-2017 of December 2011. Obsolete pesticide management is relevant to the Interim Strategy Note (ISN) for Mali for the period FY14-15, 134 and its objective of increasing agriculture performance and improving resilience to weather-related shocks as a measure for resuming economic activity. The project supports the Government of Mali in complying with the Stockholm Convention on POPs, the Basel Convention on Control of Trans-boundary Movements of Hazardous Waste and their Disposal, and the Rotterdam Convention on the Prior Informed Consent Procedure for Certain Hazardous Chemicals and Pesticides in International Trade. 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE 34. The project has been evaluated against its PDO by measuring achievements at the time of the project closing (December 31, 2012). 133 The document is presented in the IMF Poverty Reduction Strategy Paper (PRSP), (Country Report No. 13/111). 134 Report No. 76233-ML, May 20, 2013 96 35. PDO achievement – Rating Moderately Unsatisfactory: The assessment took the project’s six PDO indicators into account, of which four were fully achieved (100%), one was partly achieved (80%) and one was not achieved (0%). Indicator #2 (disposal of the stocks), which signifies the most important project achievement, received a higher weight in the rating calculation. The three main project outcomes measured by indicators include: 135 (i) Sound disposal and management of obsolete pesticides and contaminated sites; (ii) prevention of accumulation of new stockpiles of obsolete pesticides; and (iii) enhanced capacity and institutional strengthening of pesticide management. 36. Sound disposal and management of obsolete pesticides and contaminated sites. The level of risks emanating from publicly held obsolete pesticides and associated waste to adjacent and wider communities or critical natural resources was determined and quantified through the completion of a national inventory of publicly held obsolete pesticides in 2005-2006 and its validation in June 2007 (PDO Indicator #1 – 100%). This information was used to prepare a CESA, conduct an assessment of contaminated sites (PDO Indicator #3 – 100%), safeguard low risk stocks, clean up a number of priority sites and contract a disposal company. It is estimated that over four million people (half of them are women) living in vicinity to the contaminated sites benefitted from soil cleanup operations and obsolete pesticide safeguarding activities.136 Inventoried stocks were not disposed of under this contract due to a very long disposal procurement process and the closing of the project before the disposal work commenced (PDO Indicator #2 - 0%). 37. Prevention of accumulation of new stockpiles of obsolete pesticides. The project helped improve the legal and regulatory framework for pesticide management (PDO Indicator #4 - 100%) by conducting a legal review of Mali's legislative and regulatory framework for pesticide management, in 2007, followed by technical support for its improvement based on the results. The project supported the National Directorate of Agriculture in the development of procedures of obtaining approval for the importation/export and the sale of pesticides. It also supported DNACPN in the development of laws for empty container management and management of seized pesticides. Project objectives and activities were integrated into national policies for hazardous waste management (such as the Politique Nationale d'Assainissement, endorsed by the Council of Ministers in 2009), where an entire chapter was devoted to obsolete pesticides management at both the public and private sectors. Awareness raising and sensitization efforts were made to educate relevant governmental agents about the introduced changes. New accumulations of associated waste were addressed by the project by developing and implementing an empty container strategy in 29 rural communities from 2009 to 2012 in the cotton zones of the Malian Cotton Company (Compagnie Malienne pour le Développement des Textiles - CMDT). The project also increased government and public awareness with respect to the harmful effects of pesticide mishandling and contact with obsolete pesticides. It developed, validated and implemented a communication plan, which included various communication tools, such as films, radio, newspapers, as well as through workshops for NGOs. It is estimated that over 1.4 million people were affected by project communication, either via face-to-face workshops or through radio programs. 135 Achievements were identified based on intended outcomes presented in Section B.1 and Annex I of the PAD (Report No: 36273-AFR) 136 Source : Nombre de bénéficiaires du PASP-Mali (2007 à 2012), ASP-Mali Project Management Unit, 4/29/2013 97 38. Enhanced capacity and institutional strengthening of pesticide management. A total of 4,451 governmental and non-governmental staff received training on various topics related to pesticide and waste management, communication and M&E (Original PDO Indicator #5 - 100%). These skills were used to validate the pesticide inventory, enter the data into the PSMS data base, and participate in successful safeguarding and site cleanup operations. The project also invested resources in reactivating the CNGP, the executive pesticides regulating body; however, it is unclear whether this strengthened the institutional settings for pesticide management. 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE 39. The project aimed to directly contribute to the GEO of reducing the effects of POPs on the global environment through: (i) improving the quality of life in poor communities, by reducing environmental risks; (ii) improving environmental protection; and (iii) enhancing the capacity of the agricultural sector to better manage crop pests. GEO achievement is measured by PDO achievement and, thus, is also rated moderately unsatisfactory (see Section 3.2). 3.4 EFFICIENCY 40. Efficiency – Rating Low: This rating represents a broad estimate, as financial and economic data to support this rating are scarce. At appraisal, an economic or financial analysis was neither undertaken for ASP-P1 at the program level nor at the country level due to non-measurable benefits of reducing the risk of contaminating the environment. A lack of reliable baseline data in combination with controversial aspects related to the valuation of human life and natural resources did not permit a cost-benefit analysis. In addition, efficiency of capacity building and prevention activities is difficult to quantify due to the temporal disconnect between project support and actual results. For GEF purposes, the ASP-P1 underwent an incremental cost analysis to demonstrate the incremental benefit associated with the participation of GEF in the program. The analysis demonstrated that GEF support would lead to the channeling of considerable additional resources to the program ($35 million in addition to the $25 million from GEF), as well as important domestic (safeguarding local populations and environments and having in place meaningful prevention mechanisms) and global benefits (removal of trans-boundary pollution threats). 41. Although 77% of the entire project budget was disbursed at the end of the project, it was not used to dispose of any obsolete pesticide stocks, demonstrating low project disposal efficiency. In view of safeguarding and site cleanup activities, the project was successful in decontaminated five polluted sites using low cost techniques (e.g., land farming) and employing trained local staff, as well as collecting low risk stocks from numerous sites, also by local staff at relatively low cost. . These cost savings, coupled with the fact that the project achieved substantial prevention and capacity building results, alleviate the overall project efficiency rating. 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING 42. Overall Outcome - Rating Moderately Unsatisfactory: On the basis of the project's efficiency (low), relevance (substantial), and PDO achievements (moderately unsatisfactory), the overall outcome is rated moderately unsatisfactory. The overall outcome rating takes into account that even though the key result – disposal - did not take place, important environmental and health impacts expected of the project in view of POPs-related illnesses which are likely to have been prevented in some of the most 98 polluted sites (through safeguarding and soil clean up). Prevention (improved legislation, awareness raising and improved container management practices) and capacity building activities conducted by the project have the potential of sustaining these achievements. 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS 43. Poverty impacts, gender aspects, and social development: The safeguarding and site cleanup operations had potentially a direct impact on nearby poor and farming communities, species and ecosystems. Since baseline data on affected populations, species and ecosystems was not conducted at appraisal, exact health and environmental impacts associated with the removal of POPs from the environment, are difficult to measure. However, a project beneficiaries report calculated that close to 1.5 million people were reached by the project’s communication and awareness campaigns, and that over 4 million people live in vicinity to the contaminated sites. These people’s health is likely to be positively affected by the removal of POPs from their environments (water, soil, crops, fish, livestock, etc.) through the site cleanup and safeguarding activities. 137 44. Institutional change/strengthening: The project strengthened the government’s institutional capacity on several levels: (i) the regulatory and legal framework for pesticide management was strengthened; (ii) the government gained experience in the preparation of social and environmental impact assessments and management plans; (iii) the experience and skills gained in carrying out inventory, creating and utilizing a database, participating in safeguarding and soil remediation activities, and managing a multi-stakeholder project, can be applied to other projects and subsequently contribute to success; and (iv) knowledge gained on sound pesticide management through the various training and courses, which were provided to governmental and non-governmental staff (the beneficiary report calculated a total of 4,451 governmental and non-governmental staff receiving training on various topics related to pesticide and waste management, communication and M&E). 45. Other unintended outcomes and impacts: ASP-P1 Mali had an additional focus on soil treatment due to a number of sites known for having been polluted by pesticide spills and leaks. This provided an opportunity for the program to explore the issue of soil remediation and offer solutions by conducting assessments and carrying out pilot cleanup operations. In July 2008, a site in the town of Molodo, which contained large quantities of highly toxic obsolete pesticides contaminating local ground water supplies, was successfully cleaned up using a 'landfarming' technique for the first time in the country. 138 The operation was carried out with strong support from the TSU and Wageningen University. The results of the Molodo operation and similar operations in Dialakoroba, Niogoméra, Sévaré, Gao and Nara were analyzed and disseminated to other ASP countries in a workshop organized by FAO in Mali in 2010 and in 2009 at an international soil decontamination conference in Baltimore, Maryland. 137 Source : Nombre de bénéficiaires du PASP-Mali (2007 à 2012), ASP-Mali Project Management Unit, April 29, 2013 138 Land farming is a soil decontamination technique which involves taking polluted soil and encouraging the biodegradation activity of soil microbes 99 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS 46. A specific completion, and/or ICR mission was not carried out, and thus no stakeholder workshop was held. The project submitted a draft completion report in March 2013, 139 which according to the PMU and the World Bank team, had not been shared with national stakeholders. The report identified the following key factors as having a negative impact on project implementation: (i) delays in mobilizing the MDTF, FFEM and additional GEF funds, resulting in implementation delays and a disposal funding gap; (ii) inability of the project to disburse funds due to a three months absence (March to June 2010) of a financial manager to sign transactions; and (iii) complex World Bank safeguards and procurement procedures. 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME 47. Risk to the development outcome - Rating High: Given Mali's post-conflict and transitional situation and the relatively low priority obsolete stocks cleanup is receiving in the country's recovery plans, it is unlikely that the obsolete pesticide stocks will be disposed of by the country without a donor- funded disposal project. Furthermore, Mali has a number of identified polluted sites which need cleaning up, but the government does not have the budget or skills to carry out such operations independently. In terms of prevention of new accumulations of obsolete pesticide stocks, the improved legislation and regulation is unlikely to have significant on-the-ground effect unless the government dedicates adequate resources to enforce them, which is also questionable. Mali's strong focus on agricultural intensification as a measure for economic development is likely to overpower any such efforts in the long run. 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE 5.1 WORLD BANK PERFORMANCE 48. World Bank performance in ensuring quality at entry - Rating Moderately Unsatisfactory: While World Bank performance during preparation was adequate, program-related factors negatively affected quality at entry. Considerable efforts were invested by the World Bank in preparing the project, which involved agreements on project scope, budget, implementation arrangements, results, M&E arrangements, project work plans, FM and procurement. The World Bank also invested considerable efforts in securing project co-financing by meeting with potential donors and seeking collaboration with ongoing projects. Nonetheless, the preparation of the project was conducted in conjunction with preparatory activities at the program-level, which were lengthy (see Section 2.1 in the program-level ICR). In addition, while some of the fund raising efforts were successful, the delays during preparation resulted in some co-financing becoming unavailable. 140 49. Quality of World Bank supervision - Rating Moderately Unsatisfactory: The project was supervised by the program co-TTL responsible for francophone countries (Mali, Morocco and Tunisia) who was 139 MEA/DNACPN’s draft progress report for the period June 2006-December 2012 140 Funds were secured from the Netherlands, FFEM and MDTF. Co-financing from PASAOP was no longer available since the project had closed at the time ASP-P1 Mali was approved (on December 31, 2006). 100 based in Washington DC and later in Dakar, and a Mali-based co-TTL. The two co-TTLs ensured knowledge exchange between Mali and other countries on one hand, and contact with stakeholders on the other. Supervision missions by both TTLs took place regularly, albeit only once a year, with additional technical support and fiduciary missions in between. Aide Memoires were detailed, providing candid description of issues and risks. However, significant delays by the World Bank in the processing of the restructuring/additional GEF financing and MDTF funds as explained above and the unclear guidance on the disposal procurement process negatively affected project performance. 50. Overall World Bank performance – Rating Moderately Unsatisfactory: Based on the above- presented evaluation, the overall borrower performance is rated moderately unsatisfactory. 5.2 BORROWER PERFORMANCE 51. Government performance - Rating Satisfactory: The government of Mali continued to show commitment to the project and supported it throughout implementation. Commitment was evident from a) the integration of project objectives and activities in national policies; b) the timely and adequate provision of counterpart financing; and c) the close collaboration between the PMU and government officials during implementation, which facilitated overall project management. A wide variety of government representatives participated in the NSC, the CNGP, and thematic groups created for the project 141, training sessions and safeguarding operations (e.g., a soil contamination forum). The government initial reluctance to restructure the project contributed to the delays in obtaining the additional financing. However, this was not the result of lack of commitment to the project but rather from an aspiration to accomplish the original, more ambitions PDO. 52. Implementing Agency performance - Rating Moderately Satisfactory: DNACPN demonstrated high level of performance in the areas of stakeholder mobilization, coordination of activities, awareness raising, safeguard implementation and FM. M&E was properly conducted and progress reporting was done in a satisfactory manner. DNACPN's performance was hampered by weak procurement capacity and the refusal to receive offered World Bank support, as well as the low efficiency in the use of resources. DNACPN complied with all World Bank legal covenants, and executed its fiduciary duties. 53. Overall Borrower performance – Rating Moderately Satisfactory: Based on the above-presented evaluation, the overall borrower performance is rated moderately satisfactory. 6. LESSONS LEARNED 54. The most significant lessons learned are drawn at the program-level, as the experience they are based on was often reflected across all country-specific projects; they are therefore discussed in the program-level ICR. Lessons learned that were particularly relevant to ASP-P1 Mali include: 142 141 Such as the soil decontamination group, which comprised eleven focal points of the main relevant institutions including DNACPN, Crop Protection Office, National Center for Locust Control, Health Directorate, Food Safety Agency, the Air Force, Laboratory of Toxicology and Environmental Quality control and the Central Veterinary Laboratory; this group provided information, technical support and advice to the PMU on issues related to soil decontamination. In addition, a prevention working group comprised 16 focal points of the main institutions involved in pesticides management in Mali. The group provided information related to pesticides management to the PMU and met on a semi-annual basis to review and evaluate progress made under the prevention component, including the National Prevention Plan developed by the project. 142 The lessons learned are summarized from the ASP-P1 draft progress report for June 2006-December 2012. 101 • Soil decontamination activities should be tailor-made for each site. ASP-P1 Mali's unique experience in soil decontamination demonstrated that each site has its own characteristics and requires different implementation considerations. Land farming or sealing contaminated soil in concrete are affordable solutions if conditions permit. • National strategies for container management should be developed through a community- based approach. Such an approach could guarantee the full involvement of associations and producer groups, administrative and municipal authorities, decentralized technical services and NGOs. Agreements should be signed on container centralization and collection, which clearly define roles and responsibilities. • Developing and implementing a pesticide management communication plan is a prerequisite for the implementation of sound pesticide management. The audience of the communication plan should be defined by stakeholders and target groups. • The importance of joint development of a national prevention plan. A participatory process could ensure the involvement of all stakeholders in the prevention of new accumulations of obsolete pesticide stocks. 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS 55. See Section 8.7 and 8.8. 8. ANNEXES 8.1 PROJECT COSTS AND FINANCING a. Project Cost by Component – All sources (in US$ million equivalent) – as of 12/31/2012 Actual/Latest Appraisal Estimate Actual as % of Component Estimate (in US$ million appraisal (in US$ million equivalent) estimate equivalent) 1. Disposal of obsoletes pesticides 5.38 2.37 44% 2. Prevention of accumulation 1.77 1.87 105% 3. Project management 1.18 2.23 188% Total 8.33 6.47 78% b. Financing – as of 12/31/2012 Actual/Latest Appraisal Estimate Actual as % of Type of Estimate (in Sources of Funds (in US$ million appraisal Co-financing US$ million equivalent) estimate equivalent) Counterpart Borrower 0.75 0.98 130% Funding/In-kind 102 Global Environment Facility Grant 2.55 2.44 95% (GEF) Multi-Donor Trust Fund (MDTF) Grant 3.98 1.89 18% French Global Environment Parallel Co- 0.75 0.69 92% Facility (FFEM) financing/Grant Parallel Co- Netherlands 0.22 0 - financing/Grant Parallel Co- UN Food and Agriculture financing/Grant 0.08 0.18 225% Organization (FAO) and In-kind Parallel Co- CropLife International (CLI) financing/ In- - 0.29 - kind Total 8.33 6.47 78% 8.2 OUTPUTS BY COMPONENT 1. The following table provides a comprehensive account of qualitative and quantitative information of outputs realized against original components (achievements at the time of project closing on December 31, 2012). Table 8.1: Outputs by Original Components as presented in the GEF Trust Fund Grant Agreement Original Components as presented in the GEF Outputs achieved at the time of closing in December 2012 Trust Fund Grant Agreement Part A – Cleaning and elimination of obsolete pesticides and associated waste: Achievements under this component are rated Moderately Unsatisfactory Carrying out of a detailed inventory of the a. 40 government and non-governmental agents received training in recipient’s publicly held obsolete pesticide conducting an inventory of obsolete pesticides and associated waste stocks and associated waste through: in 2007. a. provision of relevant training to the recipient; b. Data were collected, validated (on June 12, 2007) and compiled into b. collection of data and compilation of a data the inventory database which had been under the management and base; oversight of FAO, as part of their Pesticide Stock Management c. a risk-based prioritization of the stocks and System (PSMS). associated waste identified in the database; and c. Risk-based prioritization of the different sites was carried out to d. preparation of the CESA. determine which sites require emergency clean up and safeguarding. The data were also used as basis for the preparation of the CESA Treatment and/or disposal of obsolete pesticides (Original PDO Indicator #1 – 100%). stocks and associated waste through: d. A CESA, identifying mitigation measures to treat possible negative a. carrying out of a technological assessment of environmental and social impacts that were considered likely to available treatment and disposal options; occur during implementation of the project, was completed in December 2009, and found satisfactory by the World Bank. b. implementation of the treatment and/or disposal technology selected; and e. National soil contamination surveys were carried out in July 2007 and September 2009, and an assessment of available soil treatment c. improving the operations of obsolete pesticide options was carried out in May 2009 with the support of the TSU stocks collection centers, including improving and University of Wageningen (Original PDO Indicator #3 – 100%). safety measures at those centers. Treatment and disposal options for obsolete pesticide stocks were assessed as part of the CESA and site-specific EMPs. 103 f. The obsolete stocks (1,100 tonnes), were not collected or disposed of at the time of project closing (Original PDO Indicator #2 – 0%). g. The following cleanup/safeguarding operations took place: inventoried low risk stocks at the various sites were repacked in metal containers and centralized in a store 35km outside Bamako; Elimination of 65 tonnes of obsolete pesticides and waste and soil decontamination in Gao; Safeguarding of obsolete pesticides, veterinary products and empty containers in Koutiala, Dialakoroba, Ménaka, Molodo, Niogoméra, Sikasso, Tombouctou, Sévaré, Kerbaye, Baguineda, Kéniéba, Kayes and Yélimané and in 7 rural communities in Koutiala and Kita; Safeguarding of obsolete pesticides and decontamination in the industrial park of Bamako and a site in Bla; Decontamination of highly polluted sites in Molodo, Dialakoroba and Niogoméra (through landfarming) and in Sévaré and Nara. Part B – Prevention of obsolete pesticide accumulation: Achievements under this component are rated Satisfactory a. Building capacity through the provision of a. A total of 4,451 governmental and non-governmental staff received training to farmers, civil society groups and training on various topics related to pesticide and waste relevant staff of the recipient; management, communication and M&E (Original PDO Indicator #5 b. Review and update of the legal and regulatory – 100%) framework to control obsolete pesticide b. Reviewing and updating legal and regulatory framework to control accumulation and strengthen enforcement re-accumulation and strengthen enforcement (Original PDO mechanisms; Indicator #4 – 100%): c. Promotion of improved pesticide management • Technical support was provided to the National Directorate of practices such as integrated pest management Agriculture in the review of the Decree on pesticides management; and integrated vector management, and in the development of procedures for obtaining import/export and improved management of empty pesticide sale authorizations; and in the development of permits and containers through the design and certificates for shops, storage facilities and transport of pesticides implementation of container management of control; strategy; • Project objectives and activities were integrated into national d. Awareness raising activities on the program, policies for hazardous waste management (such as the Politique and the health and environmental hazards of Nationale d’Assainissement, endorsed by the Council of Ministers pesticides, including carrying out of workshops in 2009), where an entire chapter was devoted to obsolete on reducing reliance on pesticides in pesticides management at both the public and private sectors; agricultural production, disease vector control • Technical support was provided to DNACPN in the development and migratory pest control; of laws for empty container management and management of e. Development of a long-term funding strategy seized pesticides; in the development of laws for notification of for prevention activities and a plan to prevent seizure and detention of obsolete pesticides; and in the pesticide accumulation; and development of a Decree on Principle Pollutant Payer; f. Strengthening the recipient’s laboratory • Sensitization of Plant Protection agents and representatives from services. the Police Force and Customs on laws related to pesticides control. c. Development and implementation of an empty container strategy for 29 rural communities in the CMDT cotton zones. d. Development, validation, and implementation of a project communication plan via various communication tools to raise awareness of project activities (project logo, documentary films, sketches, reports, microprograms, folders, calendars, etc.); broadcasting of project activities via the radio and newspapers; workshop for 40 NGOs on pesticide management through PAN- Mali as a means to encourage NGO participation in project 104 activities. e. Development of a national prevention plan: supporting the National Health Directorate in the development of a strategic prevention plan for chemical poisoning; establishment of a Prevention Working Group, comprising 24 members from national entities, associations and NGOs; dynamization of the National Pesticides Management Committee; f. The project supported the Central Veterinary Laboratory in the development of a plan for the Laboratory of Toxicology and Environmental Quality Control. Part C – Project management: Achievements under this component are rated Moderately Unsatisfactory Supporting the PMU in carrying out project The PMU received technical advisory services from the TSU (on coordination, implementation, monitoring and inventory taking, data management, M&R, soil assessment and evaluation, through the provision of technical remediation techniques in partnership with the University of advisory services, goods and operational support. Wageningen, and safeguarding of low risk stocks), CLI (on disposal through the CLI-funded TAD), WWF (on communication), PAN- UK/Africa (on civil society engagement in project activities) and the World Bank (on procurement, financial management, safeguards and M&E). Project funds were used to purchase needed goods and other consultant services. The support to the PMU was hampered by: (i) technical disagreements between the TSU and the World Bank on the content of the M&E system, and the need for restructuring; (ii) significant delays in recruiting the TAD due to long negotiations between CLI and the government and delays within CLI in mobilizing the funds, and (iii) inconsistent procurement advice from the World Bank about the disposal contract. 8.3 ECONOMIC AND FINANCIAL ANALYSIS 2. As outlined in Section 3.4, an economic and financial analysis is not available. 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS 8.5 BENEFICIARY SURVEY RESULTS 3. Beneficiary survey results are not available, as no survey was conducted. 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS 4. A specific completion, and/or ICR mission was not carried out, and thus no stakeholder workshop was held. 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR 5. The borrower submitted a completion report; messages contained therein were integrated into this document. 105 6. Comments and edits received on the draft ICR were equally incorporated. Additional comments included suggested lessons learned: • A national inventory should be a preliminary step during project preparation, particularly in the technical and financial arrangements for the component associated with the removal and cleaning. This could avoid the financing gaps and the necessity to seek additional funds for the elimination of additional stocks of obsolete pesticides and related waste. • If the PMU consists of specialists who are directly recruited by the implementing agency, as was the case for ASP-Mali, it is useful and necessary to add counterparts to capitalize on experiences and to strengthen the capacity of these structures. The selection criteria of these counterparts should be based on the profile and long-term availability of applicants. • National teams composed of technical pesticide management officers, who are well trained and equipped with adequate facilities and equipment, are able to secure a certain category of obsolete pesticides and related waste. This category includes stocks of small amounts (less than 500 kg) and low risk (not belonging to Classes Ia and Ib WHO). Using national teams can strengthen national capacities in securing stockpiles of obsolete pesticides and related waste. Once these capabilities are integrated into the missions and activities of the departments concerned, they could perform similar operations without input and external funding after the closure of the project. Using national teams can also shorten lead times and reduce the costs of security operations and disposal. • With regard to decontaminated soils, each site has its own characteristics. The history of the sites and nature of contaminants determine the technical options available for cleanup operations. Land-farming and confinement techniques in the contaminated concrete layer allow for solving soil decontamination problems with local human resources and at lower costs. • A national empty container management strategy using community-based approaches guarantees the involvement of associations and producer groups, administrative and municipal authorities, structured and decentralized technical services and NGOs. It also enhances sustainability. The signing of agreements on local collection/grouping of empty packaging, which define the roles and responsibilities of each other, would demonstrate the commitment of those concerned. It is also a prerequisite for a community management strategy targeting empty containers. • Development and implementation of a specific communication plan on pesticides management is a prerequisite for the implementation of sound management of pesticides. This plan must take internal and external communication into account. The external communication plan should be defined by stakeholders and target groups. • The national prevention plan represents a road map for implementing activities targeted at preventing the accumulation of obsolete pesticide stocks. Its development should be based on a coordinated and responsible approach involving all stakeholders. • The working group that focuses on implementing the national prevention plan should include all stakeholders. Group members should be nominated focal points by their respective stakeholder group. As such, they would be responsible for collecting information and data relating to their respective stakeholder group and contribute the same to the group work in order to contribute to a comprehensive overview of national pesticide management. 106 • Involving NGOs in project implementation allows for training and raising the awareness of local producers. Priority activities should relate to: (i) the development of, translation into local languages, and dissemination of informational guides on the hazards and risks and good management of pesticides; (ii) the design and dissemination of micro programs on good management practices, pesticides and empty containers; (iii) community monitoring of hazards related to the use of pesticides; (iv) promoting Integrated Pest Management (IPM); and (v) independent monitoring of operations for disposal of obsolete pesticides and related waste. 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS 7. No comments were received from co-financiers and other partners/stakeholders on the ASP-P1 Mali draft ICR. 8. Comments of co-financiers and other partners/stakeholders on the program-level ICR are summarized in Section 8.8 in the program-level ICR. 8.9 VERBATIM COMMENTS ON DRAFT ICR 107 108 109 110 111 112 113 114 TECHNICAL ANNEX III: ASP-P1 MOROCCO 115 ABBREVIATIONS AND ACRONYMS APL Adaptable Program Lending ASP-P1 Africa Stockpiles Program – Project 1 CAS Country Assistance Strategy CESA Country Environmental and Social Assessment CLI CropLife International CPS Country Partnership Strategy DELM Direction de l’Epidémiologie et de la lutte contre les Maladies DPVCTRF Direction de la Protection des Végétaux des Contrôles Techniques et de la Répression des Fraudes DSPR Direction de la Surveillance et de la Prévention des Risques DSSPA Direction de la Sécurité Sanitaire des Produits Alimentaires FAO United Nations Food and Agriculture Organization GDP Gross Domestic Product GEF Global Environment Facility GEO Global Environment Objective ICR Implementation Completion and Results Report IFR Interim Unaudited Financial Report IP Implementation Performance IPM Integrated Pest Management ISR Implementation Status and Results Report IVM Integrated Vector Management MADRPM Ministère de l’Agriculture, du Développement Rural et des Pêches Maritimes MATEE Ministère de l’Aménagement du Territoire, de l’Eau et de l’Environnement M&E Monitoring & Evaluation NGO Non-Governmental Organization NIP National Implementation Plan ONSSA Office National de la Santé Sanitaire des Aliments PAD Project Appraisal Document PAN Pesticides Action Network PDO Project Development Objective 116 PMU Project Management Unit POP Persistent Organic Pollutants PSMS Pesticide Stock Management System QAE Quality at Entry QAG Quality Assurance Group QSA Quality of Supervision SAICM Strategic Approach to International Chemicals Management TAD Technical Advisor for Disposal TOR Terms of Reference TSU Technical Support Unit TTL Task Team Leader UNDP United Nations Development Program UNEP United Nations Environment Program WHO World Health Organization WWF World Wildlife Fund 117 TABLE OF CONTENTS 1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN ...... 124 1.1 CONTEXT AT APPRAISAL ............................................................................................................................................ 124 1.2 ORIGINAL PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS ................................................................ 125 1.3 REVISED PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS .................................................................. 125 1.4 MAIN BENEFICIARIES................................................................................................................................................... 125 1.5 ORIGINAL AND REVISED COMPONENTS ................................................................................................................. 126 1.6 OTHER SIGNIFICANT CHANGES ................................................................................................................................. 127 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ..................................................................... 127 2.2 IMPLEMENTATION ........................................................................................................................................................ 128 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTLIZATION ......................... 130 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE ......................................................................................................... 130 2.5 POST-COMPLETION OPERATION/NEXT PHASE ...................................................................................................... 131 3. ASSESSMENT OF OUTCOMES ......................................................................................................................................... 132 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION ...................................................................... 132 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE ................................................................................... 132 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE ................................................................................... 133 3.4 EFFICIENCY .................................................................................................................................................................... 133 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING................................................................................................. 134 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS ............................................................................ 134 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS .................... 134 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME .......................................................................................... 134 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE .................................................................. 134 5.1 WORLD BANK PERFORMANCE .................................................................................................................................. 134 5.2 BORROWER PERFORMANCE....................................................................................................................................... 135 6. LESSONS LEARNED ............................................................................................................................................................ 135 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS ................ 136 8. ANNEXES ............................................................................................................................................................................... 136 8.1 PROJECT COSTS AND FINANCING ............................................................................................................................. 136 8.2 OUTPUTS BY COMPONENT ......................................................................................................................................... 137 8.3 ECONOMIC AND FINANCIAL ANALYSIS .................................................................................................................. 138 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS ................................. 138 8.5 BENEFICIARY SURVEY RESULTS .............................................................................................................................. 138 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS............................................................................................ 138 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR ................................................................. 138 8.8 COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS .................................................... 139 118 A. Basic Information 143 Country: Morocco Project Name: Africa Stockpiles Program - Project 1 Project ID: P103189 L/C/TF Number: TF57778 ICR Date: 11/27/2013 ICR Type: Core ICR Lending Instrument: APL Borrower: Kingdom of Morocco Original Total US$4.0 million Disbursed Amount: US$0.29 million (06/30/2010) Commitment: Revised Amount: - Environmental Category: A Global Focal Area: CHEM Implementing Agency: Direction de la Protection des Végétaux des Contrôles Techniques et de la Répression des Fraudes (DPVCTRF) Co-financiers and Other External Partners: Government of the Kingdom of Morocco, CropLife International (CLI), UN Food and Agriculture Organization (FAO), CropLife Morocco, Pesticide Action Network in the United Kingdom (PAN-UK), PAN-Africa, and World Wildlife Fund (WWF). B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/17/2002 Effectiveness: 02/12/2007 02/05/2007 Appraisal: 04/19/2004 Restructuring: - - Approval: 12/05/2006 Mid-term Review: 11/30/2008 12/11/2009 Closing: 06/30/2010 06/30/2010 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Unsatisfactory Risk to Global Environment Outcome: High World Bank Performance: Moderately Unsatisfactory Borrower Performance: Unsatisfactory C.2 Detailed Ratings of World Bank and Borrower Performance World Bank Ratings Borrower Ratings Quality at Entry: Moderately Unsatisfactory Government: Moderately Unsatisfactory Implementing Quality of Supervision: Moderately Satisfactory Unsatisfactory Agency/Agencies: Overall World Bank Overall Borrower Moderately Unsatisfactory Unsatisfactory Performance: Performance: 143 The information presented in the data sheet was inserted manually. Information provided in the system was incomplete resulting from coding issues experienced during implementation; these were not resolved prior project closing. 119 C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators - Performance (if any) Potential Problem Project at Quality at Entry Yes - any time (Yes/No): (QEA): Problem Project at any time Quality of Yes - (Yes/No): Supervision (QSA): PDO rating before Unsatisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total World Bank financing) Sanitation 33.3 33 Health 33.3 33 General Agriculture 33.3 34 Theme Code (as % of total World Bank financing) Environmental policies and institutions 50 50 Pollution management and environmental health 50 50 E. World Bank Staff Positions At ICR At Approval Vice President: Inger Andersen Daniela Gressani/Gobind T. Nankani Country Director: Neil Simon M. Gray Theodore O. Ahlers Director (Regional Colin Bruce - Integration): Sector Director: - Inger Andersen/Michel Wormser Sector Manager: Charles Cormier Vijay Jagannathan Project Team Leader: Dinesh Aryal Peter Kristensen/Denis Jordy ICR Team Leader: Dinesh Aryal ICR Primary Author: Ayala Peled Ben Ari F. Results Framework Analysis - At project approval, a country-specific results framework was not presented; only six PDO Indicators were defined. 120 Project Development Objective (PDO) – as presented in the GEF Trust Fund Grant Agreement: 144 To assist the recipient, in the project area, in: (a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and (b) implementing measures to reduce and prevent future related risks. Revised PDO – Not Applicable Global Environment Objectives (GEO) - as presented in the PAD (Report No: 36273-AFR): Reduction of the effects of Persistent Organic Pollutants (POPs) on the global environment. Revised GEO – Not Applicable (a) PDO Indicators – Original PDO Indicators as presented in the GEF Trust Fund Grant Agreement are assessed against achievements realized at project closing on June 30, 2010. Original Target Formally Actual Value Achieved at Values Indicator Baseline Value Revised Target Completion or Target (from approval Values Years documents) Original PDO Indicators Inventory database of publicly held obsolete pesticide stocks in place and being Indicator 1: used by the Project Management Unit (PMU) and stakeholders Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 06/30/2010 - 01/01/2010 Comments (incl. % Target fully achieved (100%): An inventory was completed and validated in 2010. achievement) The database was subsequently used to prepare a draft disposal tender. Completion of the County Environmental and Social Assessment (CESA) and Indicator 2: implementation of the measures contained therein Value (quantitative No Yes - No or qualitative) Date Achieved 04/19/2004 06/30/2010 - 06/30/2010 Target not achieved (0%): The procurement process for a consulting firm to conduct Comments (incl. % the CESA began in February 2008 with the preparation of TORs. By project closing, achievement) a consulting firm was selected but not contracted. Inventoried publicly held obsolete pesticide stocks disposed of, as a result of the Indicator 3: completion of the disposal services contract(s) in accordance with national and international laws and regulations Value (quantitative 0% 100% - 0% or qualitative) Date Achieved 04/19/2004 06/30/2010 - 06/30/2010 144 The Project Development Objective (PDO) and components that are stated in the Project Appraisal Document (PAD) slightly differ from those stated in the GEF Trust Fund Grant Agreement. This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities. 121 Target not achieved (0%): Morocco adopted Tunisia’s short list of pre-qualified Comments (incl. % disposal firms, and a first draft of the tender document was prepared by the project. achievement) However, a disposal firm was not selected and no disposal activity took place by project closing. Legal and regulatory framework for pesticide management upgraded, including Indicator 4: measures to strengthen compliance with the Conventions Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 06/30/2010 - 06/30/2010 Comments (incl. % Target fully achieved (100%): Several pesticide/waste management-related legal achievement) texts and regulations were improved. The training program under Part D 145 of the project is implemented and the Indicator 5: knowledge so acquired is being used by PMU and stakeholders Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 06/30/2010 - 05/01/2009 Target partly achieved (50%): In 2007 and 2008 stakeholders participated in Comments (incl. % inventory and date management training. PMU staff participated in two World Bank achievement) fiduciary training workshops in 2008 and in 2009. The training was not entirely effective in increasing the PMU and stakeholders’ capacities in the subject matters. Indicator 6: PMU is functional and its operation is satisfactory to the World Bank Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 06/30/2010 - 06/30/2010 Target partly achieved (50%): The PMU was staffed as needed throughout project Comments (incl. % implementation; however, it lacked capacity in procurement, financial management, achievement) M&E, planning and keeping up with timelines. Reporting was also delayed. (b) GEO Indicator(s) –The GEO was measured by the PDO and PDO indicators. (c) Intermediate Outcome Indicator(s) – Not Applicable G. Ratings of Project Performance in ISRs – Some DO and IP ratings and actual disbursement data are not available because ISRs reported on ASP-P1 as a whole, and only occasionally on country-specific project performance, including disbursement. Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 10/31/2005 Not applicable as ASP-P1 Morocco not yet launched 2 06/30/2006 145 See Section 1.5 for the project’s Component D (in the GEF Trust Fund Grant Agreement referred to as ‘Part D’) 122 3 12/27/2006 4 06/28/2007 Moderately Satisfactory Moderately Satisfactory - 5 12/18/2007 Moderately Satisfactory Moderately Unsatisfactory - 6 05/30/2008 - - - 7 12/18/2008 Moderately Unsatisfactory Unsatisfactory 0.23 8 03/25/2009 Unsatisfactory Highly Unsatisfactory 0.28 9 12/29/2009 Unsatisfactory Highly Unsatisfactory 0.28 10 06/29/2010 Unsatisfactory Highly Unsatisfactory 0.29 06/30/2010 Final disbursement 0.29 H. Restructuring – Not Applicable I. Disbursement Profile – A disbursement profile is not available (see explanation under G). 123 1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN 1.1 CONTEXT AT APPRAISAL 1. Country background: After expanding significantly in the 1980s, agricultural production stagnated in the 1990s because of the increased incidence of drought, which damaged output of the drought- sensitive cereals that dominated Moroccan agriculture. The agricultural sector had been considered inefficient; modern, large-scale farms characterized certain parts of the country, elsewhere, however, farming suffered from the fragmentation of the land, low rates of mechanization and fertilizer use, and limited use of selected seeds. Locusts had been occasionally a problem in the far south, as in 2003-04, although crop damage had been small. 2. Sector background: At project approval, Morocco had imported approximately 9000 tonnes of pesticides per year, with the majority having been insecticides and fungicides. Large-scale farmers primarily used chemicals in controlling pests and diseases, whereas small farmers often used alternatives such as applying soil management or crop rotation. 3. Actions taken: Recognizing the potential hazard obsolete stocks of pesticides were posing to people, animals and to the environment, the Government of Morocco had developed legislative measures as well as qualified structures (laboratories and services) to handle pesticide issues. Since 2002, Morocco had taken a series of measures allowing the country to become eligible for international donations to reduce and eliminate pesticide stockpile and toxic waste. The Government of Morocco had initiated activities under its National Implementation Plan (NIP). 146 In preparation of ASP-P1, a preliminary inventory had been conducted and identified 700 tons of obsolete stocks at approximately 225 sites. These stockpiles had been largely stocked for combating locust infestations and were held largely by the Ministries of Agriculture and Health. Only a small portion of the stocks was held by private companies. 4. Rationale for World Bank assistance: Please refer to Section 1.1 in the program-level ICR. 5. Higher-level objectives: The project intended to contribute to Morocco’s national development strategy, in the areas of public health, poverty alleviation, environmental protection, and strengthening the agricultural sector. In particular, the ASP-P1 Morocco aimed to (i) improve the quality of life in poor communities by reducing environmental health risks, (ii) improve environmental protection, and (iii) enhance the capacity of the agricultural sector to better manage crop pests. In addition, the project aimed to consolidate various approaches to obsolete pesticide cleanup, generate practical lessons and produce practical tools that would facilitate its replication in other countries and regions. Contributing to international efforts to eliminate Persistent Organic Pollutants (POPs), improving management of toxic chemicals, and promoting alternatives to pesticide usage that include Integrated Pest Management (IPM) and Integrated Vector Management (IVM), represented further objectives. 146 National Implementation Plan for Stockholm Convention on Persistent Organic Pollutants (POPs), May 2006, Kingdom of Morocco – Ministry of Territory Management, Water and Environment. 124 1.2 ORIGINAL PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS 6. The Project Development Objective (PDO), indicators, and components that are stated in the Project Appraisal Document (PAD) 147 slightly differ from those stated in the GEF Trust Fund Grant Agreement. 148 This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities. 7. ASP-P1 Morocco’s original PDO as presented in the GEF Trust Fund Grant Agreement: 149 To assist the recipient in: a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and b) implementing measures to reduce and prevent future related risks. 8. At appraisal, only an overall ASP-P1 results framework had been prepared (see program-level ASP ICR); results frameworks for the individual ASP-P1 countries, including Morocco, were intended to be finalized during the individual ASP-P1 country project launches. However, country-specific PDO indicators were developed at appraisal. For ASP-P1 Morocco, these were as follows: Table 1.1: ASP-P1 Morocco Original PDO Indicators Original PDO Indicators as presented in the PAD and GEF Trust Fund Grant Agreement a. Inventory database of publicly held obsolete pesticide stocks in place and being used by the Project Management Unit (PMU) and stakeholders. b. Completion of Country Environmental and Social Assessment (CESA) and implementation of the measures contained therein. c. Inventoried publicly held obsolete pesticides stocks disposed of, as a result of the completion of the disposal services contract(s) in accordance with national and international laws and regulations. d. Legal and regulatory framework for pesticide management upgraded, including measures to strengthen compliance with the Conventions. e. The training program under Part D of the project is implemented and the knowledge acquired is being used by the PMU and stakeholders. 150 f. The PMU is functional and its operation is satisfactory to the Bank. 1.3 REVISED PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS 9. Not Applicable. 1.4 MAIN BENEFICIARIES 10. Please refer to Section 1.4 in the program-level ICR. 147 Project Appraisal Document for the Republic of Mali, the Kingdom of Morocco, and the United Republic of Tanzania in support of the Africa Stockpile Program – Technical Annex 2, (Report No. 36273-AFR), November 7, 2006. 148 GEF Trust Fund Grant Agreement between the Kingdom of Morocco and the International Bank for Reconstruction and Development, (Number TF057778-MOR), February 5, 2007. 149 ASP-P1 Morocco PDO as presented in the PAD was: To eliminate inventoried publicly held obsolete pesticide stockpiles and associated waste, and implement measures to reduce and prevent future related risks. 150 See Section 1.5 for the project’s Component D, here referred to as “Part D”. 125 1.5 ORIGINAL AND REVISED COMPONENTS 11. The PDO of ASP-P1 Morocco was to be achieved through the implementation of four distinct components (outlined in Table 1.2). Table 1.2: ASP-P1 Morocco Original Components Original Components Original Components as presented in the PAD as presented in the GEF Trust Fund Grant Agreement Component 1.A – Disposal of obsolete pesticides (Total: Part A – Obsolete pesticide inventory: Carrying out a US$2.51 million; GEF: US$2.26 million): Country cleanup detailed inventory of the recipient's publicly held obsolete and disposal activities were intended to be based on a pesticides stocks including: comprehensive and detailed inventory of pesticide stockpiles a. Collection of data and compilation of a database; and related contamination, determining the level of risk to b. A risk-based prioritization of the stocks identified in the adjacent and wider communities or critical natural resources database; and (such as water supplies). The cleanup was intended to include c. Preparation of the CESA. the repackaging of obsolete pesticides, transport and Part B – Disposal of obsolete pesticides: Disposal of publicly destruction of pesticides based on chemical analysis, needs held obsolete pesticides stocks and associated waste on the assessment and technical capacity. basis of an adequate disposal technology, and establishment, as needed, of obsolete pesticide stocks collection centers. Component 1.B - Prevention of obsolete pesticides Part C – Prevention of obsolete pesticide accumulation: accumulation (Total: US$0.78 million; GEF: US$0.55 Carrying out activities aimed at preventing obsolete pesticide million): Activities in this component aimed at strengthening accumulation, including: the pesticide management through improving pesticide a. Awareness raising activities on the health and registration, licensing, enforcement of import controls, stock environmental hazards of pesticides; management, waste management, and formulation of effective b. A review of the legal and regulatory framework to control procurement strategies, as well as promotion of alternatives to obsolete pesticide accumulation and enforcement chemical pesticides through improvement of pest control mechanisms with a view to upgrading said framework and strategies with particular attention to IPM for agriculture and mechanisms; IVM for health care. Activities further included the c. Preparation and implementation of an IPM strategy; and development of awareness raising programs. d. A review of pesticide management practices and preparation and implementation of a training plan. Component 1.C - Capacity building (Total: US$0.27 million; Part D – Capacity building: Provision of technical GEF: US$0.27 million): Capacity building activities included assistance, training and equipment to strengthen the capacity the implementation of a training program and awareness of PMU and the staff of relevant ministries and other raising campaigns. Environmental monitoring was to be stakeholders, in environmental monitoring, pesticide and supported through the provision of equipment and staff other chemical management practices, and in carrying out training for effective pesticide and other chemical management awareness raising campaigns. practices. Component 1.D - Project management and Monitoring and Part E – Project management and evaluation: Supporting Evaluation (Total: US$1.84 million; GEF: US$0.35 million): the PMU in carrying out project coordination, Activities included the establishment of a PMU: its implementation, monitoring and evaluation, through the responsibility was to implement the country level activities, provision of technical advisory services, equipment and solicit external technical support, monitor and evaluate project operational support. activities and coordinate their implementation with the GEF- funded NIPs for the Stockholm Convention and similar initiatives. Furthermore, the establishment of a steering committee representing key stakeholders, such as Non- Governmental Organizations (NGOs) and the private sector, and overseeing the work of the respective PMU. 126 1.6 OTHER SIGNIFICANT CHANGES 12. No changes were made to ASP-P1 Morocco. The project closed as originally scheduled on June 30, 2010. 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES 13. Project preparation took place between 2001-2006, as part of preparatory activities at the program- level. It took more than five years from GEF eligibility in July 2001, and more than four years from concept note in July 2002 to have the project approved by the World Bank Board in December 2006. 151 14. Soundness of the background analysis: Preparation was conducted in close collaboration between the World Bank and the Food and Agriculture Organization (FAO), with CropLife International (CLI) joining around appraisal (April 2004); mission reports do not indicate the participation of the World Wildlife Fund (WWF) and the Pesticides Action Network (PAN-UK/PAN-Africa) during preparation. The key preparatory inputs were the PAD and the Operational Manual with varying levels of quality: the PAD contained mostly ASP-P1 level information and plans with very little country-specific information, while the Operational Manual presented a considerable amount of country information. 152 It was based on country background analyses, which had been carried out by national working groups. The World Bank carried out comprehensive procurement and Financial Management (FM) assessments in September 2004. While it was recognized by the World Bank, FAO, CLI and the government that the inventory carried out in 2002 had to be updated in order to determine the disposal budget, 153 the new inventory was postponed to the implementation phase, most likely due to budgetary constraints. Consequently, a tentative disposal budget was agreed on. 154 15. Assessment of project design: The project’s design (as for all ASP-P1 participating countries) was preconfigured at the program level, and did not reflect any country-specific modifications or lessons learned from pervious projects executed in Morocco. Design weaknesses at the program level that translated into implementation delays at the country level, including ASP-P1 Morocco, are outlined in Section 2.1 of the program-level ICR. At the project level, design weaknesses were inherent in the complex implementation arrangements, where overall project management was with the Ministry of Agriculture, Rural Development and Fisheries (Ministère de l’Agriculture, du Développement Rural et des Pêches Maritimes - MADRPM), but it was one of at least two identified executing entities, with unclear definition of responsibilities and budget allocations. 155 Another key weakness was the 151 The long preparation time resulted from a lengthy partnership building/negotiation process at the program-level resulting from the complex implementation arrangement and project design (see Section 2.1 in the program-level ICR). 152 Information included considerable detail onthe legislative framework for pesticide management, stakeholder analysis, national disposal options, processes for establishing national prevention and empty containers plans, procurement and financial management procedures, health and safety procedures and an M&E framework. 153 See Royaume du Maroc - Programme Africain Relatif aux stocks de pesticides obsolètes - Première phase, Mission de préparation, Aide-mémoire, December 2003 154 US$2.265 million for component A, of which US$165,000 for inventory and US$2.1 million for disposal. 155 Within MADRPM, the Direction de la Protection des Végétaux, du Contrôle technique et de la Répression des Fraudes (DPVCTRF) was the main executing agency and host of the PMU; the Direction de la Surveillance et de la Prévention des Risques (DSPR) within MATEE was the second executing agency. The Operational Manual also includes the Direction de l’Epidémiologie et de la lutte contre les Maladies (DELM) within the Ministry of Health (Ministère de Sante) as a third executing agency; however this is not mentioned in the PAD. 127 agreed disbursement arrangements, which relied on government’s central budgeting and disbursement system (see more under Section 2.2). 16. Government commitment: During preparation, the World Bank reported on strong government commitment as was evident from its proactivity in starting to analyze the baseline situation before it was formally agreed on. 156 In addition, in 2003, the government established multi-stakeholder thematic working groups and a steering committee. Finally, the government took the necessary actions to process the ratification of the Stockholm Convention on POPs, which was a condition for joining the program. The process was finalized in June 2004. 17. Assessment of risks: The PAD and the Operational Manual presented brief and partial risk assessments, which focused mainly on crosscutting and program-level risks; corresponding mitigation measures were proposed. The World Bank procurement and FM/disbursement assessments carried out during preparation identified respective risks, which were specific to ASP-P1 Morocco, and offered mitigation measures. The risks, which were also reflected in the PAD and the Operational Manual, correctly highlighted possible future delays in commitments and payment as a result of Morocco’s pre-financing system; 157 weak FM capacity of the implementing agency; and the joint responsibility of the executing agencies to produce Interim Unaudited Financial Reports (IFRs). The appraisal mission in April 2004 identified weak inter-ministerial coordination of prevention activities as a project risk; however, this risk was not included in the PAD or the Operational Manual. 2.2 IMPLEMENTATION 18. The implementation of ASP-P1 Morocco experienced significant delays throughout implementation, which ended with the closing of the project on June 30, 2010 as scheduled (with no extensions). During the December 2009 mid-term review, and in view of unsatisfactory project performance, the World Bank and the government agreed on an action plan of corrective measures; its timely implementation served as a minimum condition for a closing date extension. However, by June 30, 2010 the action plan had only been partially implemented, leading to a World Bank decision to close the project. 19. Key factors that affected implementation included: • Institutional challenges most notably an internal government restructuring, halted project implementation. Project management was substantially affected by an internal government restructuring process, which resulted in a lack of institutional leadership, a need to redefine a number of project aspects (such as disbursement arrangements and counterpart financing), and a need for repeated training. 158 In addition, project management was affected by (i) an unclear 156 See Royaume du Maroc - Programme Africain Relatif aux stocks de pesticides obsolètes - Première phase. Mission de préparation, Aide-mémoire, December 2003 157 Whereby payments are made directly to suppliers, regardless of the source of funding, by the Moroccan Treasury (Trésorerie Générale du Royaume) from the Budget funds upon submission of supporting documentation and payment orders by the project unit. Reimbursement is made directly from the grant account. 158 MADRPM underwent a significant restructuring process. In May 2009, the PMU was temporarily transferred from DPVCTRF to the Department of Health Safety of Food Products (Direction de la Sécurité Sanitaire des Produits Alimentaires - DSSPA) under MADRPM. In January 2010 the PMU was again transferred to the newly established ONSSA, which became the agency responsible for pesticide management and for project implementation. In July 2009, the PMU was mainly composed of former DPVCTRF staff who were assigned to DSSPA until ONSAA was established. These changes impacted the PMU because it had 128 definition of roles and responsibilities of PMU members from both MADRPM and the Ministry of Spatial Planning, Water and Environment (Ministère de l’Aménagement du Territoire, de l’Eau et de l’Environnement – MATEE); (ii) a lack of full time staff 159 and the staff’s weak fiduciary and project management capacity; (iii) ineffective collaboration between stakeholders (most likely due to unclear definition of roles, responsibilities and budgets); and (iv) low-level representation in the Steering Committee. Challenging disbursement arrangements and a lack of flexibility in budget planning further affected project implementation. • Lengthy procedures to mobilize GEF funds in absence of a project special account: The project was dependent on the government’s budget because designated/special accounts were largely not allowed in Morocco; in order to access the GEF funds, each executing agency had to develop and submit annual budgets to the government, to be approved as part of the national budget approval process, and once approved, changes to the budget were difficult to make. The lengthy process also caused project activities to only begin nine months after effectiveness. In addition, in order to effect payments, each contract had to go through the Ministry of Finance’s control in addition to the World Bank approval process, which took considerable time. 160 • Higher than estimated amounts of obsolete pesticides caused delays: It took seven months for the project to complete an inventory (December 2007-July 2008), and 22 months to enter all data into FAO’s Pesticide Stock Management System (PSMS) and have it validated (March 2008-January 2010; originally 15 months were allocated for data validation). A data validation workshop took place in February 2010. The underlying reasons for these delays were: (i) increase in the number of sites (340 instead of 225 originally planned); (ii) difficulties in carrying out an inventory during the summer and Ramadan; (iii) repeated visits to some sites due to removal of stocks and poor quality of the collected data/photos as a result of insufficient training; and (iv) the large number of pesticides found at the sites, resulting in multiple data sheets to be entered into the PSMS. 161 • Insufficient and delayed technical support affected implementation performance: FAO’s Technical Support Unit (TSU) support was insufficient (e.g., inventory, PSMS management and M&E). In addition, while the recruitment of the Technical Advisor for Disposal (TAD) was initiated in January 2007, she was only recruited by CLI in January 2010. The valuable support, however, came too late and was not used by the PMU before project closing. to relocate to a new facility with unreliable internet connection, which was necessary to maintain the PSMS and overall project management. 159 The coordinator, because of his other responsibilities as the Head of Division within the DPVCTRF, could not commit all of his time to the project; the four technical officers were similarly involved in other activities in DPVCTRF 160 Any budget allocation request, which had not been specifically included in the GEF Grant Agreement, such as MATEE’s and NGOs’ budgets, was not approved, and activities were consequently not implemented. Sources: Programme Africain Relatif aux Stocks de Pesticides Périmés, Mission de Supervision au Maroc 14-17 Mai, 2007, Aide-Mémoire and Back-to-Office Report : Morocco - Africa Stockpiles Programme – Implementation Support Mission (P0103189 / TF057778). 161 Sources: Programme Africain relatif aux Stocks des Pesticides Périmés, PASP – Maroc : État d’avancement, July 2009 (PowerPoint Presentation); and Programme Africain Relatif aux Stocks de Pesticides Périmés (Phase 1) - Mission d’achèvement du 7 au 9 juin 2010, Aide Mémoire (Draft), July 25, 2010. 129 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTLIZATION 20. Overall M&E - Rating Unsatisfactory. 21. M&E design – Rating Unsatisfactory: The design of the M&E framework, as presented in the Operational Manual, was generally appropriate but lacked the most important element – a project- level results framework. Although the appraisal mission developed a project-level results framework, 162 it was not reflected in the PAD and the Operational Manual, both presenting only the overall ASP-P1 results framework (see program-level ASP ICR) and six country-specific Performance Monitoring Indicators.163 These indicators did not allow for adequate monitoring of project implementation progress because they focused on measuring outputs as opposed to progress towards project outcomes. Expected results under the prevention component, in particular, were not adequately considered by the indicators. The PAD further indicated that a country-specific framework would be finalized during project launch. Other key M&E elements, such as strategies and responsibilities, were detailed in the Operations Manual. The inventory was expected to provide baseline and target values for disposal, and overall M&E was expected to be carried out using standardized methodologies for data collection and quality assurance. The responsibility for data collection rested with the PMU with data provided by the various project stakeholders, assisted by the TSU and external consultants. 22. M&E implementation and utilization – Rating Unsatisfactory: A project-specific results framework was developed with the government during the February 2008 World Bank mission; however, it was not officially introduced to the project (through a restructuring or addition to the Operations Manual), and it is unclear whether the PMU used it to produce progress reports. The late completion of the national database (in January 2010) left little time to use these data for a CESA and disposal activities. 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE 23. Safeguard compliance – Rating Moderately Satisfactory: ASP-P1 triggered two safeguard policies: OP 4.01 on Environmental Assessment and OP 4.09 on Pest Management. 24. ASP-P1 Morocco complied with OP 4.01 on Environmental Assessment: The project did not begin to prepare the CESA by the project closing due to considerable delays in the completion of data validation of the inventory as mentioned above, coupled with weak PMU procurement capacity. In terms of on-the-ground activities’, the World Bank observed at mid-term that some health and safety procedures had not been properly applied during the inventory process by the trained teams. No other field activities took place, which necessitated consideration of OP 4.01. Note that the safeguard rating applies to activities that took place before the project closed in June 2010. Activities not conducted by the project closing (i.e., preparation of a CESA and EMP) were not factored into the rating. 162 See Africa Stockpiles Program (ASP) - Morocco - Appraisal mission Aide Memoire (April 22 to 27, 2004). 163 Lengthy discussions between the partners during project preparation on whether to have an M&E system at the program-level alone, or also at the country-level, may have contributed to the absence of an M&E system at the country-level at project launch. However, no information was available on whether it was a deliberate decision not to have an M&E system in place at the country-level. 130 25. OP 4.09 on Pest Management: Since project activities themselves met requirements of a pest management plan (i.e., through pesticide management, IPM and capacity building), individual country projects, including ASP-P1 Morocco, were not required to prepare self-standing documents. See Section 2.4 in the program-level for more details. 26. Financial Management Compliance – Rating Unsatisfactory: Despite training sessions provided by the World Bank, the PMU’s financial management capacity was insufficient to ensure smooth management of funds and timely preparation of reports. Financial management was further weakened by the above-mentioned disbursement arrangements and institutional changes. In December 2007, following considerable procurement delays, the PMU recruited an administrative, financial and procurement consultant, who was tasked to strengthen the PMU’s fiduciary capacity and to develop a financial management manual for the project. Poor performance of the consultant, partly caused by delayed payments by the government’s treasury department, resulted in the termination of the contract in June 2009. In December 2009, another fiduciary consultant was identified; however, the recruitment was not finalized due to the institutional changes and proximity to the closing date. 27. GEF disbursement was low throughout the project’s lifetime. The first disbursement was recorded in December 2008 (5.9%), two years after project approval, mainly on inventory equipment, inventory training, a vehicle and operating costs. In May 2009, disbursement increased to 7.4% and remained at this level until project closing. The low level of disbursement is linked to the above-mentioned implementation delays (the CESA and disposal contract, which together were allocated over 56% of the GEF grant amount, were not implemented by project closing), and cumbersome disbursement procedures. 28. Procurement – Rating Unsatisfactory: Project procurement was considerably delayed due to the PMU’s weak capacity and the centralized contract awarding system. A number of training sessions and the above-mentioned recruitment of a consultant did not strengthen procurement capacity. Despite the World Bank authorization to use ASP-Tunisia’s list of pre-qualified disposal firms, the disposal procurement process did not advance as expected, and at project closing, only a first draft of the tender document was available. 2.5 POST-COMPLETION OPERATION/NEXT PHASE 29. Post completion: After the project closed, two related activities were carried out by the government with donor support: (i) a national team composed of ten technical staff from the Ministry of Public Health received training by the Strategic Approach to International Chemicals Management (SAICM) project, led by the Ministry of Public Health. Consequently, 50 tonnes of Dichlorodiphenyltrichloroethane (DDT) were packed and stored in Oued Zem. This DDT stock is expected to be disposed of under a GEF project titled 'Alternatives to DDT in the Middle East and North Africa', led by the World Health Organization (WHO) and the United Nations Environment Program (UNEP); (ii) in 2011, at the request of the government, FAO implemented a small project, which identified and qualified priorities for a national program on sound POPs and pesticides management. The following priorities were identified: (i) revising pesticide legislation; (ii) establishing a registration system for pesticides used in agriculture, public and animal health; (iii) 131 technical and analytical support for pesticide quality control; and (iv) extension services and farmer field schools. 164 30. Next phase: In January 2012, the GEF approved a concept for a follow-up disposal and prevention project titled ‘Disposal of Obsolete Pesticides including POPs and Implementation of Pesticides Management Programme’, to be managed by FAO. 165 The project is designed to address the above- mentioned priorities, in addition to (i) disposing of the stocks and remediation of contaminated soil identified by ASP-P1 Morocco, as well as stocks of a recently banned pesticide (endosulfan); (ii) elimination of legacy empty containers and development of a container management system; and (iii) development of a network for information exchange within and among pesticide using sectors. The full project proposal is currently being developed by FAO. 3. ASSESSMENT OF OUTCOMES 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION 31. Relevance – Rating High: Even though the project’s objectives were ambitious, they were and they continue to be highly relevant to Morocco, especially in view of the implementation of the Green Morocco Plan, and its focus on increasing agricultural productivity through intensification of production systems. The project is similarly relevant to the most recent Country Partnership Strategy (CPS) 166 Program Area 2.6 (Agricultural Sector Reform), which looks to increase competitiveness and diversification of the agricultural sector; and Program Area 3.3 (Solid Waste Management) to support disposal practices complying with international social and environmental standards. Project objectives and components also support the Government of Morocco in complying with the Stockholm Convention on POPs, the Basel Convention on Control of Trans boundary Movements of Hazardous Waste and their Disposal, and the Rotterdam Convention on the Prior Informed Consent Procedure for Certain Hazardous Chemicals and Pesticides in International Trade. 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE 32. The project has been evaluated against its PDO by measuring achievements at the time of the project closing (June 30, 2010). 33. PDO achievement – Rating Unsatisfactory: The assessment took the project’s six PDO indicators into account, of which two were fully achieved (100%), two were partly achieved (50%) and two were not achieved (0%). The three main project outcomes measured by the indicators include: 167 (i) sound disposal and management of obsolete pesticides; (ii) prevention of accumulation of new stockpiles of obsolete pesticides; and (iii) enhanced capacity and institutional strengthening of pesticide management. 164 Source: Disposal of Obsolete Pesticides including POPs and Implementation of Pesticides Management Programme, GEF Project Identification Form (FAO), January 5, 2012. 165 See http://www.thegef.org/gef/project_detail?projID=4738 for the project concept. 166 Country Partnership Strategy for the Kingdom of Morocco for the Period FY10-13, (Report No. 50316-MA), December 30, 2009. 167 Achievements were identified based on intended outcomes presented in Section B.1 and Annex I of the PAD (Report No: 36273-AFR). 132 34. Sound disposal and management of obsolete pesticides and contaminated sites. The level of risks emanating from publicly held obsolete pesticides and associated waste to adjacent and wider communities or critical natural resources was determined and quantified through the completion of a national inventory of publicly held obsolete pesticides in 2007-2008 and its validation in 2008-2010 (PDO Indicator #1 - 100%). However, this information was not used to prepare a CESA (PDO Indicator #2 - 0%) or contract a disposal company by project closing. Inventoried stocks were not disposed of under a disposal contract due to considerable delays in the inventory and validation activities, and the lack of a CESA as a guiding document (PDO Indicator #3 - 0%). 35. Prevention of accumulation of new stockpiles of obsolete pesticides. Project and parallel government- led activities taking place during the project lifetime helped improve the legal and regulatory framework for pesticide management (PDO Indicator #4 - 100%) by improving several pesticide/waste management-related legal texts and regulations. In 2008, the ASP NGO network conducted three workshops on pesticide and empty container management. In June 2009, both PAN- Morocco (established in February 2009) and the ASP NGO network collaborated to develop and disseminate a brochure on the proper management of pesticides. Other awareness raising tools were to be developed by the project but did not materialize due to procurement delays. 36. Enhanced capacity and institutional strengthening of pesticide management. In October 2007, 21 plant protection agents received inventory training from the TSU; and in March 2008, ten plant protection agents received data management training using the PSMS from the TSU. PMU staff participated in two World Bank fiduciary training workshops in 2008 and in 2009. The training sessions were not entirely effective in increasing the PMU and stakeholders’ capacities in the subject matters (Original PDO Indicator #5 – 50%). 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE 37. The project aimed to directly contribute to the GEO of reducing the effects of POPs on the global environment through: (i) improving the quality of life in poor communities by reducing environmental risks; (ii) improving environmental protection; and (iii) enhancing the capacity of the agricultural sector to better manage crop pests. GEO achievement is measured by PDO achievement (see Section 3.2). 3.4 EFFICIENCY 38. Efficiency – Rating Low: This rating represents a broad estimate, as financial and economic data to support this rating are scarce. At appraisal, an economic or financial analysis was neither undertaken for ASP-P1 at the program- nor at the country-level due to non-measurable benefits of reducing the risk of contaminating the environment. A lack of reliable baseline data in combination with controversial aspects related to the valuation of human life and natural resources neither allowed for a cost-benefit analysis. In addition, efficiency of capacity building and prevention activities is difficult to quantify due to time disconnect between project support and actual results. For GEF purposes, the ASP-P1 underwent incremental cost analysis of achieving global benefits to address the issue of obsolete pesticides in the participating countries. 39. The project was effective for over three years (February 2007 to June 2010) but in fact, implementation took place for about two and a half years (November 2007-June 2010). Within this time period little on the ground achievements were recorded although a fully staffed PMU was in 133 place. Of the 7.4% total disbursement at closing, 56% were spent on operating costs while the rest was spent mainly on inventory equipment and a project vehicle, demonstrating low project efficiency. 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING 40. Overall Outcome – Rating Unsatisfactory: On the basis of the project’s efficiency (low), relevance (high), and PDO achievements (unsatisfactory), the overall outcome is rated unsatisfactory. The overall outcome rating takes into account the few project achievements and lack of any tangible environmental and health impacts. 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS 41. Not Applicable 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS 42. A three days mission took place shortly before project closing (June 7-9, 2010); however, it did not include a stakeholder workshop. The government did not prepare a completion report. 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME 43. Risk to the development outcome - Rating High: The government of Morocco is aware of the risks posed by obsolete pesticides, but lacks the capacity and resources to safeguard and dispose of the stocks. According to FAO, 168 without a donor-supported project, inventory and disposal would not be completed, and heavily contaminates sites could not be cleaned up. Pesticides management will continue to be addressed periodically without coordination and connection to a national strategic framework. FAO’s planned follow up project could fill these gaps. 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE 5.1 WORLD BANK PERFORMANCE 44. World Bank performance in ensuring quality at entry – Rating Moderately Unsatisfactory: The preparation of the project was conducted in conjunction with preparatory activities at the program- level, which were lengthy (see Section 2.1 in the program-level ICR). The World Bank made considerable preparation efforts, which involved agreements on project scope, implementation arrangements, work plans, FM and procurement. However, it was not effective in clearly defining project implementation arrangements and embedding clear responsibilities and budgets in either the PAD or the GEF Trust Fund Grant Agreement, causing significant implementation problems later on. In addition, the World Bank did not agree with the government on simpler disbursement arrangements. This worsened quality at entry, which was already adversely affected by problematic program-related design factors. 168 See Disposal of Obsolete Pesticides including POPs and Implementation of Pesticides Management Programme, GEF Project Identification Form (FAO), January 5, 2012. 134 45. Quality of World Bank supervision – Rating Moderately Satisfactory: The project was supervised by the program co-TTL responsible for francophone countries (Mali, Morocco and Tunisia) and a country co-TTL, both based in Washington DC. Five supervision and/or technical support missions (including the mid-term review mission) and a closing mission were conducted between February 2007 and June 2010, averaging twice a year. Aide Memoires were detailed, providing candid description of issues and risks. Expertise brought in these missions (operations, pesticide management, procurement, FM and safeguards) were adjusted to meet the perceived project needs. In addition, World Bank fiduciary staff based in the Rabat office provided additional support to the PMU. However, the negative effect of repeated TTLs changes (four in total) is noted. In addition, the World Bank was not proactive in offering adjustments to the project, such as restructuring, in a timely manner. 46. Overall World Bank performance – Rating Moderately Unsatisfactory: Based on the above- presented evaluation, the overall borrower performance is rated moderately unsatisfactory. 5.2 BORROWER PERFORMANCE 47. Government performance – Rating Moderately Unsatisfactory: The government of Morocco showed commitment to the issue of pesticide management by working to improve a number of legal and regulatory texts during the life of the project. However, the government did not provide the necessary high-level oversight to the project via the steering committee and dictated cumbersome disbursement procedures. While the national authorities were unanimous in confirming their high- level commitment to the World Bank, 169 this was clearly disconnected from project implementation performance. 48. Implementing Agency performance – Rating Unsatisfactory: The PMU’s management weaknesses were exacerbated by the restructuring of MADRPM and the shifting of the unit between directorates/agencies. Although the PMU demonstrated adequate technical capacity, its procurement, FM and planning capacities did not improve despite training and other support it received. 49. Overall Borrower performance – Rating Unsatisfactory: Based on the above-presented evaluation and the project’s unsatisfactory outcome rating, the overall borrower performance is rated unsatisfactory. 6. LESSONS LEARNED 50. The most significant lessons learned are drawn at the program level, as the experience they are based on was often reflected across all country-specific projects; they are therefore discussed in the program-level ICR. Lessons learned that were particularly relevant to ASP-P1 Morocco include: • Activities and disbursement schedules should be realistic. Projects involving a number of executing agencies are inherently complex and should dictate realistic operational and disbursement timetables accordingly. • Steering committees should ensure the sustained participation of decision-making staff. The participation of lower level staff in meetings greatly limits the effectiveness of such a committee 169 See Back-to-Office Report: Africa Stockpiles Program-Morocco (P103189/GEF TF057778) – Mid-Term Review, Dec 7-11, 2009 135 to fulfill its role as a conduit between government priorities, operational progress and results achieved; further, it greatly limits its ability to dictate corrective measures. 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS 51. See Section 8.7 and 8.8. 8. ANNEXES 8.1 PROJECT COSTS AND FINANCING a. Project Cost by Component – All sources (in US$ million equivalent) – as of 06/30/2010 Actual/Latest Appraisal Estimate Actual as % of Component Estimate (in US$ million appraisal (in US$ million equivalent) estimate equivalent) 4. Disposal of obsoletes pesticides 2.51 N/A 170 N/A 5. Prevention of accumulation 0.78 N/A N/A 6. Capacity building 0.27 N/A N/A 7. Project management 1.85 N/A N/A 8. Contingencies 0.57 N/A N/A Total 5.98 N/A N/A b. Financing – as of 06/30/2010 Actual/Latest Appraisal Estimate Actual as % of Type of Estimate (in Sources of Funds (in US$ million appraisal Co-financing US$ million equivalent) estimate equivalent) Counterpart Borrower 1.98 N/A N/A Funding Global Environment Facility Grant 4.0 0.29 7.3% (GEF) Total 5.98 0.29 4.8% 170 Actual/latest estimates of cost by component and sources of funds were not available. 136 8.2 OUTPUTS BY COMPONENT 1. The following table provides a comprehensive account of qualitative and quantitative information of outputs realized against original components (achievements at the time of project closing in June 2010). Table 8.1: Outputs by Original Components as presented in the GEF Trust Fund Grant Agreement Original Components as presented in the GEF Outputs achieved at the time of closing in June 2010 Trust Fund Grant Agreement Part A – Obsolete pesticide inventory: Achievements under this component are rated Moderately Unsatisfactory Carrying out a detailed inventory of the a. Inventory began in December 2007 and was completed in July 2008. recipient's publicly held obsolete pesticides The inventory covered 340 sites, revealing 480 tonnes of obsolete stocks including: pesticides plus 230m3 (February 2009 estimate: 756 tonnes); 600 m3 a. Collection of data and compilation of a of soil; 800 m3 of equipment; 670 m3 of material; 730 m3 plus database; 22,240 containers/drums; and 904m3 of building material. Data b. A risk-based prioritization of the stocks entry began in March 2008, and was fully entered and validated in identified in the database; and January 2010. The project conducted a validation workshop in c. Preparation of the CESA. February 2010. b. N/A c. The CESA’s procurement process began in February 2008 with the preparation of TORs. By project closing a consulting firm was selected but not contracted. Part B – Disposal of obsolete pesticides: Achievements under this component are rated Unsatisfactory Disposal of publicly held obsolete pesticides No concrete results at project closing: Tunisia’s short list of pre- stocks and associated waste on the basis of an qualified disposal firms was adopted ad a first draft of the tender adequate disposal technology, and establishment, document was prepared. as needed, of obsolete pesticide stocks collection No collection centers were established. centers. Part C – Prevention of obsolete pesticide accumulation: Achievements under this component are rated Moderately Unsatisfactory Carrying out activities aimed at preventing a. In 2008, the PMU organized communication material for posters obsolete pesticide accumulation, including: and pamphlets about the project and good plant protection practices a. Awareness raising activities on the health and (not printed and distributed). In November 2008, TORs for the environmental hazards of pesticides; production of two television spots on pesticide storage/empty b. Review of the legal and regulatory framework containers management and good plant protection practices were to control obsolete pesticide accumulation and prepared and received no objection from the World Bank. This did enforcement mechanisms with a view to not materialize due to procurement difficulties. In 2008, the ASP upgrading said framework and mechanisms; NGO network with the financial support of PAN and other donors c. Preparation and implementation of an IPM conducted thee workshops on pesticide and empty container strategy; and management. In June 2009, both PAN-Morocco (established in d. Review of pesticide management practices and February 2009) and the ASP NGO network collaborated to develop preparation and implementation of a training and disseminate a brochure on the proper management of pesticides plan. in external funding to conduct training sessions for farmers. b. Improvement of several pesticide/waste management-related legal texts and regulations as a result of project and a number of government entities’ actions. c. In February 2009, TORs were prepared for a diagnostic study of the 137 management of pesticides, which was to inform the preparation of a national prevention plan leading to a national strategy. The procurement process was not successful since only one firm showed interest despite three successive publications. d. For IPM and training activities, see Part D below. Part D – Capacity building: Achievements under this component are rated Moderately Unsatisfactory Provision of technical assistance, training and In October 2007, 21 plant protection agents received inventory equipment to strengthen the capacity of the PMU training from the TSU; and in March 2008, ten plant protection agents and the staff of relevant ministries and other received data management training using the PSMS from the TSU. stakeholders, in environmental monitoring, A communication strategy was developed and validated by WWF in pesticide and other chemical management June 2007 and by the World Bank in February 2008. It included a practices, and in carrying out awareness raising number of outreach activities and training sessions on the impact of campaigns. pesticides on the environment, human health, and also initiatives to promote IPM with small farmers. An international TAD was hired by CLI in January 2010. In the short period of her work, the TAD supported the preparation of the disposal tender. Part E - Project Management and Evaluation: Achievements under this component are rated Moderately Unsatisfactory Supporting the PMU in carrying out project Participation of PMU staff in two World Bank fiduciary training coordination, implementation, monitoring, and workshops: Bamako in May 2008 and Rabat in May 2009. evaluation, through the provision of technical Limited M&E support from the TSU at project onset. advisory services, equipment, and operational Operational and pesticide management support from the World Bank support. in July 2009. 8.3 ECONOMIC AND FINANCIAL ANALYSIS 2. As outlined in Section 3.4, an economic and financial analysis is not available. 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS 3. As outlined in Section 3.4, an economic and financial analysis is not available. 8.5 BENEFICIARY SURVEY RESULTS 4. Beneficiary survey results are not available, as no survey was conducted. 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS 5. A specific completion, and/or ICR mission was not carried out, and thus no stakeholder workshop was held. 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR 6. The borrower did not submit a completion report. 7. The draft ICR was shared with the borrower; however, no comments were received. 138 8.8 COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS 8. No comments were received from co-financiers and other partners/stakeholders on the ASP-P1 Morocco draft ICR. 9. Comments of co-financiers and other partners/stakeholders on the program-level ICR are summarized in Section 8.8 in the program-level ICR. 139 TECHNICAL ANNEX IV: ASP-P1 SOUTH AFRICA 140 ABBREVIATIONS AND ACRONYMS ASP-P1 Africa Stockpiles Program – Project 1 AVCASA Association of Veterinary and Crops Associations of South Africa CAS Country Assistance Strategy CESA Country Environmental and Social Assessment CLI CropLife International DAFF Department of Agriculture, Forestry, and Fisheries DEA Department of Water and Environmental Affairs DDT Dichlorodiphenyltrichloroethane EA Environmental Assessment EMP Environmental Management Plan EMTK Environmental Management Toolkit FAO United Nations Food and Agriculture Organization FM Financial Management GDP Gross Domestic Product GEF Global Environment Facility GEO Global Environment Objective IFR Interim Financial Report IPM Integrated Pest Management IVM Integrated Vector Management IWMP Integrated Waste Management Plan MDTF Multi-Donor Trust Fund M&E Monitoring & Evaluation NGO Non-Governmental Organization NIP National Implementation Plan NRP National Retrieval Project PAD Project Appraisal Document PAN Pesticides Action Network PDO Project Development Objective PMU Project Management Unit POPs Persistent Organic Pollutants 141 PSMS Pesticide Stock Management System QEA Quality at Entry Assessment TAD Technical Advisor for Disposal TOR Terms of Reference TSU Technical Support Unit TTL Task Team Leader UNEP United Nations Environment Program WWF World Wildlife Fund 142 TABLE OF CONTENTS 1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN ...... 152 1.1 CONTEXT AT APPRAISAL ............................................................................................................................................ 152 1.2 ORIGINAL PDO AND INDICATORS ............................................................................................................................. 153 1.3 REVISED PDO AND INDICATORS ............................................................................................................................... 154 1.4 MAIN BENEFICIARIES................................................................................................................................................... 154 1.5 ORIGINAL AND REVISED COMPONENTS ................................................................................................................. 155 1.6. OTHER SIGINIFICANT CHANGES .............................................................................................................................. 156 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ..................................................................... 157 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY ............................................................................ 157 2.2 IMPLEMENTATION ........................................................................................................................................................ 159 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTILIZATION ........................ 161 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE ......................................................................................................... 162 2.5 POST-COMPLETION OPERATION/NEXT PHASE ...................................................................................................... 163 3. ASSESSMENT OF OUTCOMES ......................................................................................................................................... 163 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION ...................................................................... 163 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE ................................................................................... 164 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE ................................................................................... 166 3.4 EFFICIENCY .................................................................................................................................................................... 167 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING................................................................................................. 167 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS ............................................................................ 167 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS .................... 168 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME .......................................................................................... 168 5. ASSESSMENT OF BANK AND BORROWER PERFORMANCE .................................................................................. 169 5.1 WORLD BANK PERFORMANCE .................................................................................................................................. 169 5.2 BORROWER PERFORMANCE....................................................................................................................................... 169 6. LESSONS LEARNED ............................................................................................................................................................ 170 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS ................ 170 8. ANNEXES ............................................................................................................................................................................... 171 8.1 PROJECT COSTS AND FINANCING ............................................................................................................................. 171 8.2 OUTPUTS BY COMPONENT ......................................................................................................................................... 172 8.3 ECONOMIC AND FINANCIAL ANALYSIS .................................................................................................................. 174 8.4 WORLD BANK LENDINGA AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS .............................. 175 8.5 BENEFICIARY SURVEY RESULTS .............................................................................................................................. 176 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS............................................................................................ 176 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR ................................................................. 176 8.8 COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS .................................................... 176 143 A. Basic Information 171 Country: South Africa Project Name: Africa Stockpiles Program - Project 1 Project ID: P075776 L/C/TF Number: TF55763 ICR Date: TBC ICR Type: Core ICR Lending Instrument: APL Borrower: Republic of South Africa Original Total US$1.7 million Disbursed Amount: US$0.71 million (06/30/2012) Commitment: Revised Amount: - Environmental Category: A GEF Focal Area: CHEM Implementing Agency: Ministry of Water and Environmental Affairs - Department of Environmental Affairs (DEA) Co-financiers and Other External Partners: Government of Republic of South Africa, CropLife International (CLI), Pesticide Action Network (PAN-UK and PAN-Africa), and World Wildlife Fund (WWF). B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/17/2002 Effectiveness: 09/30/2005 04/04/2006 11/25/2009, 05/03/2011, Appraisal: 03/28/2004 Restructurings: 06/30/2011 Approval: 09/08/2005 Mid-term Review: 03/2008 01/23/2009 Closing: 09/30/2009 06/30/2012 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Global Environment Outcome: Moderate World Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of World Bank and Borrower Performance World Bank Ratings Borrower Ratings Quality at Entry: Moderately Government: Satisfactory 171 Information presented in the datasheet was inserted manually. Information provided in the system was incomplete resulting from coding issues experienced during implementation; these were not resolved prior project closing. 144 Unsatisfactory Moderately Moderately Quality of Supervision: Implementing Agency: Unsatisfactory Unsatisfactory Overall World Bank Moderately Overall Borrower Moderately Performance: Unsatisfactory Performance: Unsatisfactory C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments - Performance (if any) Potential Problem Project at Quality at Entry Yes - any time (Yes/No): (QEA): Problem Project at any time Quality of Supervision Yes - (Yes/No): (QSA): PDO rating before Moderately Closing/Inactive status: Unsatisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total World Bank financing) Sanitation 33.3 33 Health 33.3 33 General Agriculture 33.3 34 Theme Code (as % of total World Bank financing) Environmental policies and institutions 50 50 Pollution management and environmental health 50 50 E. World Bank Staff Positions At ICR At Approval Christiaan J. Poortman/Gobind T. Vice President: Makhtar Diop Nankani Country Director: Asad Alam Theodore O. Ahlers Director (Regional Integration): Colin Bruce Mark Tomlinson Sector Director: - Inger Andersen Sector Manager: Jonathan S. Kamkwalala Vijay Jagannathan Project Team Leader: Dinesh Aryal Steven P. Maber ICR Team Leader: Dinesh Aryal ICR Primary Author: Veruschka Schmidt 145 F. Results Framework Analysis - At project approval, no country-specific result framework existed; only six PDO Indicators were defined. Following a Level 1 Restructuring in June 2011, a country-specific results framework was introduced. The original six PDO Indicators were revised and reduced to two PDO Indicators, and in addition, four Intermediate Outcome Indicators were introduced. Project Development Objective (PDO) - as presented in the GEF Trust Fund Grant Agreement: 172 To assist the recipient in: (a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and (b) implementing measures to reduce and prevent future related risks. Revised PDO - as presented in the Level 1 restructuring of June 2011 (Report No: 62795-ZA): To assist the Recipient in the development and piloting of a sustainable system to identify, collect and dispose of obsolete pesticides and associated waste. Global Environment Objectives (GEO) - as presented in the PAD (Report No: 32232-MNA): Reduction of the effects of Persistent Pollutants (POPs) on the global environment. Revised GEO – Not Applicable (a) PDO Indicator(s) - Original PDO Indicators as presented in the GEF Trust Fund Grant Agreement are assessed against achievements realized at the time of the project’s Level 1 restructuring in June 2011; revised PDO Indicators are assessed against achievements realized at project closing in June 2012. Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised Target approval Completion or Target Values documents) Years Original PDO Indicators Inventory database of publicly held obsolete pesticides stocks in place and Indicator 1: being used by the PMU and the recipient’s government staff Value (quantitative or No Yes - No qualitative) Date Achieved 04/19/2004 09/30/2009 - 06/30/2011 Target not achieved (0%): In 2007, it was recognized that the ASP-P1 country implementation approach was not applicable in South Africa. Carrying out an Comments (incl. % inventory was considered cost-ineffective in an environment where most stocks achievement) were held privately; in addition, the government was already using a national register to maintain pesticide data. Completion of CESA, and implementation of the measures contained Indicator 2: therein Value (quantitative or No Yes - No qualitative) 172 The PDO, indicators, and components that are stated in the Project Appraisal Document (PAD) slightly differ from those stated in the GEF Trust Fund Grant Agreement. This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities. 146 Date Achieved 04/19/2004 09/30/2009 - 06/30/2011 Target not achieved (0%): In 2007, it was recognized that the project Comments (incl. % implementation approach used in other ASP-P1 countries was not applicable in achievement) South Africa. A CESA was subsequently not prepared. Inventoried publicly held obsolete pesticides stocks disposed of, as a result of Indicator 3: the completion of the disposal services contract(s) in accordance with national and international laws and regulations Value (quantitative or 0 250 - Approx. 17 qualitative) Date Achieved 04/19/2004 09/30/2009 - 06/30/2011 Target partly achieved (6.8%): 102.5 tonnes of obsolete pesticides were collected Comments (incl. % and stored in Limpopo province in 2006. Of those, approximately 17 tonnes were achievement) publicly held non-hazardous stocks and disposed of locally in 2009 using GEF funds. Legal and regulatory framework for pesticide management improved, Indicator 4: including measures to strengthen compliance with the Basel Convention and the Rotterdam Convention Value (quantitative or No Yes - Yes qualitative) Date Achieved 04/19/2004 09/30/2009 - 06/30/2011 Target fully achieved (100%): A national Pesticides Management Policy, which Comments (incl. % supported the objectives of the conventions, was adopted and published by achievement) DAFF in 2010. The training program under Part D 173 of the project is implemented and the Indicator 5: knowledge so acquired is being used by the PMU and recipient’s government staff Value (quantitative or No Yes - Partly (80%) qualitative) Date Achieved 04/19/2004 09/30/2009 - 06/30/2011 Target partly achieved (80%): A training program was not developed. However, training in financial management, procurement, project management, CESA Comments (incl. % preparation, and Pesticide Stock Management System (PSMS) application was achievement) received, and a study tour to Australia was conducted; additional envisaged procurement training was not received. Indicator 6: The PMU is functional and its operation is satisfactory to the World Bank Value (quantitative or No Yes - Partly (80%) qualitative) Date Achieved 04/19/2004 09/30/2009 - 06/30/2011 Comments (incl. % Target partly achieved (80%): PMU was functional and performing moderately achievement) satisfactorily. However, procurement was conducted without following World 173 See Section 1.5 for the project’s Component D (in the GEF Trust Fund Grant Agreement referred to as ‘Part D’). 147 Bank guidelines in a couple of instances, subsequently resulting in a failure to procure a disposal contractor, and financial management staff did not allocate sufficient time to the project. Revised PDO Indicators as per the Level 1 Restructuring in June 2011 A system for identifying, collecting and disposing of private and public Indicator 1: obsolete pesticides stocks and associated waste developed through collaboration with the private sector association Private and public Private and public obsolete pesticides obsolete pesticides stocks and associated stocks and associated waste identified in Value (quantitative or No system in place waste identified, - three provinces qualitative) collected and (100%), collected disposed of (using the (33%), and disposed booking line) of in one province (33%) Date Achieved 06/30/2011 06/30/2012 - 06/30/2012 Target partly achieved (55%): A booking line to facilitate identification, collection, and disposal of privately and publicly held stocks was developed in Comments (incl. % collaboration with the private sector, tested in two provinces, and subsequently achievement) introduced to all other provinces in South Africa. Stocks were identified in three provinces (100%), collected (33%), and disposed of in one province (33%). Indicator 2: Three pilot safeguarding and/or disposal operations implemented Value (quantitative or 0 3 - 1 qualitative) Date Achieved 06/30/2011 06/30/2012 - 06/30/2012 Target partly achieved (33%): At Limpopo province, 100 % of obsolete Comments (incl. % pesticides were disposed of using CLI. At Western Cape and Free State achievement) provinces, obsolete pesticides were identified, but not collected, safeguarded and/or disposed of within the implementation period. (b) GEO Indicator(s) –The GEO was measured by the PDO and PDO indicators. (c) Intermediate Outcome Indicator(s) Actual Value Original Target Values Formally Achieved at Indicator Baseline Value (from approval Revised Completion or Target documents) Target Values Years Indicator 1: Booking line developed, tested and maintained Value (quantitative or No booking line in Booking line Booking line - qualitative) place developed, tested and developed, tested, 148 maintained and maintained Date Achieved 06/30/2011 06/30/2012 - 06/30/2012 Target fully achieved (100%): A booking line was developed in collaboration Comments (incl. % with the private sector, tested in two provinces, subsequently introduced to all achievement) other provinces in South Africa, and maintained by the private sector industry. Indicator 2: Number of site-specific Environmental Management Plans Value (quantitative or 1 3 - 1 qualitative) Date Achieved 06/30/2011 06/30/2012 - 06/30/2012 Target partly achieved (33%): For Limpopo province, an EMP was prepared by Comments (incl. % the disposal contractor, and approved by the World Bank in 2010. For Western achievement) Cape and Free State provinces, EMPs were not prepared as the disposal contractor was not recruited prior project closing. Indicator 3: Number of sites in which identified stocks are collected Value (quantitative or 1 3 - 1 qualitative) Date Achieved 06/30/2011 06/30/2012 - 6/30/2012 Target partly achieved (33%): In Limpopo province, all stocks were collected in Comments (incl. % 2007. In Western Cape and Free State provinces, stocks were not collected, as achievement) the disposal contractor was not recruited prior project closing. Indicator 4: Training/study tours implemented 2 (Study tour to Value (quantitative or 1 Australia; auditing and - 1 qualitative) monitoring courses) Date Achieved 06/30/2011 06/30/2012 - 06/30/2012 Target partly achieved (50%): Government staff participated in a study tour to Comments (incl. % Australia in 2008 to examine the country’s collection and disposal system. No achievement) information was available on whether auditing and monitoring courses were conducted. G. Ratings of Project Performance in ISRs - Some DO and IP ratings and actual disbursement data are not available because ISRs reported on ASP-P1 as a whole, and only occasionally on country-specific project performance, including disbursement. Date ISR Actual Disbursements No. DO IP Archived (US$ millions) 1 10/31/2005 - - 0 2 06/30/2006 Satisfactory Satisfactory 0 3 12/27/2006 Satisfactory Satisfactory 0 4 06/28/2007 Moderately Satisfactory Moderately Satisfactory 0 5 12/18/2007 Moderately Satisfactory Satisfactory 0 149 6 05/30/2008 - - 0.08 7 12/18/2008 Moderately Satisfactory Satisfactory - 8 03/25/2009 Moderately Satisfactory Satisfactory 0.29 9 12/29/2009 Unsatisfactory Unsatisfactory 0.37 10 06/29/2010 Unsatisfactory Unsatisfactory 0.37 11 03/27/2011 - Unsatisfactory 0.55 12 12/25/2011 Unsatisfactory Unsatisfactory 0.71 13 06/29/2012 Unsatisfactory Unsatisfactory 0.71 06/30/2012 Final Disbursement 0.71 H. Restructuring ISR Ratings Amount Board at Restructuring Disbursed at Restructuring Approved Reason for Restructuring & Restructuring Date(s) GEO Key Changes Made GEO IP in US$ Change millions • Extension of closing date by 17 months (11/30/2009 to 04/30/2011) • Reallocation of proceeds to accommodate costs associated with a storage facility for the stocks at Limpopo, disposal of stocks at two Not Not 11/25/2009 No 0.37 additional provinces, and project Available Available management • Increase of the percentage of expenditure to be financed to 100% for all but Consultant Services categories due to budgetary constraints within the Government Not Not • Extension of closing date by 14 months 05/03/2011 No 0.55 Available Available (04/30/2011 to 06/30/2012) • Revision of the PDO and indicators to reflect a shift from complete disposal to the development and piloting of a sustainable system to identify, collect, Not Not and dispose of stocks in the private and 06/30/2011 Yes 0.55 Available Available public sectors • Reallocation of proceeds due to changes associated with a) costs incurred by the storage of stocks collected at Limpopo; b) fewer 150 ISR Ratings Amount Board at Restructuring Disbursed at Restructuring Approved Reason for Restructuring & Restructuring Date(s) GEO Key Changes Made GEO IP in US$ Change millions workshops needed and training carried out through parallel financing (CLI); c) disposal of and/or safeguard additional stocks; and d) increased operating costs because of previous extension. • Change in the financing plan: addition of US$0.51 million from CLI • Introduction of a revised safeguards instrument • Introduction of a results framework for better M&E If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body), enter ratings below: Outcome Rating Outcome Ratings Against Original PDO targets Moderately Unsatisfactory Against formally revised PDO targets Moderately Unsatisfactory Overall (weighted) rating Moderately Unsatisfactory I. Disbursement Profile – A disbursement profile is not available (see explanation under G). 151 1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN 1.1 CONTEXT AT APPRAISAL 1. Country background: South Africa’s agricultural share of the economy has been outstripped by those of the mining and secondary industries. In the 1930s, agriculture contributed about 20% of the GDP; at project approval in 2005, the sector’s share of the GDP had dropped to 2.6%. 174 Despite the farming industry's declining share, it has remained vital to the economy, development, and stability of South Africa, with the industry employing approximately 30% of the workforce. 2. Sector background: To boost production, chemical pesticides had been widely used in the agricultural sector. These included large quantities of Dichlorodiphenyltrichloroethane (DDT) between the 1950s and early 1980, before a 1983 ban on all uses of DDT as an active ingredient for agricultural purposes was introduced. The environment in which pesticides were applied, however, had been largely uncontrolled, as pesticides had primarily been used by unskilled farm workers with a low level of literacy and by households in domestic applications. The same was relevant for chemicals sold to the public or used in products. Chemicals sold to the industry, on the other hand, had been sold within a controlled environment where workers were skilled or semi-skilled and the use could be managed. Another use of pesticides concerned the health sector to prevent and control malaria. Since 1946, DDT had been used for malaria vector control in South Africa for indoor residual spraying. 175 In 1996, the government stopped the usage of DDT and relied on pyrethroid insecticides. However, DDT was re-introduced for indoor residual spraying in 2000, at the height of a malaria epidemic. 3. Obsolete pesticide accumulation: The accumulation of obsolete pesticides in South Africa had resulted from pesticide bans, insufficient control, and a lack of awareness of their efficient use. In general, government involvement in pesticide purchasing had been limited and the import and distribution was conducted mainly by the private sector, and large amounts of obsolete pesticides had been accumulated by private farmers. In addition, the industry had accumulated large amounts of pesticide stocks due to the unavailability of local destruction technologies, and the high scale of commercial farming throughout the country also resulted in significant amounts of obsolete stocks. 4. Institutional background: Between 1947 and 2010, management of pesticides was governed by the National Department of Agriculture (NDA, currently titled Department of Agriculture, Forestry and Fisheries - DAFF). 176 In 2010, DAFF approved and published the national Pesticides Management Policy, 177 which stated that DAFF would receive a statutory mandate to regulate pesticides in collaboration with other relevant departments through Memoranda of Understanding; and that 174 The Economist Intelligence Unit, Country Profile South Africa, 2005 175 In 1996, DDT was replaced with pyrethroids before a subsequent increase in infection rates had forced the reintroduction of DDT in 2000. 176 Under the Fertilizers, Farm Feeds, Agricultural Remedies and Stock Remedies Act of 1947. 177 Fertilizers, Farm Feeds, Agricultural Remedies and Stock Remedies Act, Act No 36 of 1947. Pesticide Management Policy for South Africa. GG No 33899, GN 1120 of 2010. DAFF 2010. 152 collection programs for obsolete pesticides would be instituted by DAFF. However, since 2008, pesticide waste has been managed by the Department of Water and Environmental Affairs (DEA). 178 5. Actions taken: In 1997, DAFF launched a National Retrieval Project (NRP) aimed at disposing unwanted pesticides nation-wide. With national funds, pesticides were centralized in 45 collection depots, and approximately 1,040 tonnes of obsolete pesticides, including 70 tonnes of DDT and other Persistent Organic Pollutants (POPs), were disposed of overseas and locally. 179 Different surveys undertaken in the Stellenbosch and neighboring areas in 1995, 2000, and 2003, however, concluded that since the NRP, new stocks had accumulated. In addition to government owned stocks that had been used to combat migratory pests, there were significant quantities scattered amidst the estimated 55,000-60,000 commercial farms throughout the country. In 2004, the Association of Veterinary and Crops Associations of South Africa (AVCASA) and the Endangered Wildlife Trust (a local Non- Governmental Organization (NGO)) initiated a project, which collected pesticides directly from households. Obsolete pesticides were dropped off at participating nurseries countrywide, to then be collected and repackaged for disposal. The initiative collected approximately 12 tonnes of pesticides, including 2% of POPs, which were stored in a landfill facility. These stocks were subsequently integrated into ASP-P1 South Africa. 6. Rationale for World Bank assistance: Please refer to Section 1.1 in the program-level ICR. 7. Higher-level objectives: The project intended to contribute to South Africa’s national development strategies in the areas of public health, poverty alleviation, environmental protection, and to strengthen the agricultural and rural development sector. In particular, the project aimed to (i) improve the quality of life in poor communities by reducing environmental health risks, (ii) improve environmental protection, and (iii) enhance the capacity of the agricultural sector to better manage crop pests. Contributing to international efforts to eliminate POPs, improving management of toxic chemicals and promoting alternatives to pesticide usage that include Integrated Pest Management (IPM) and Integrated Vector Management (IVM) represented further objectives. 1.2 ORIGINAL PDO AND INDICATORS 8. The Project Development Objective (PDO), indicators, and components that are stated in the Project Appraisal Document (PAD) 180 slightly differ from those stated in the GEF Trust Fund Grant Agreement. This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities. Table 1.1: ASP-P1 South Africa Original PDO Original PDO Original PDO as presented in the PAD as presented in the GEF Trust Fund Grant Agreement 178 Under the National Environmental Management: Waste Act of 2008 DEA, which is responsible for regulation of hazardous and non-hazardous waste. 179 The most toxic portion of the stocks was incinerated in the UK while the less toxic stocks were disposed of at a commercial hazardous waste site in South Africa. 180 South Africa and Tunisia: Africa Stockpile Program – Project 1: PAD (Report No: 32232-MNA), August 19, 2005 153 To eliminate 250 tonnes of inventoried publicly held obsolete To assist the recipient in: (a) eliminating inventoried pesticides stockpiles and associated wastes, 181 and implement publicly held obsolete pesticide stocks and associated measures to reduce and prevent future related risks. waste; and (b) implementing measures to reduce and prevent future related risks. 9. At appraisal, only an overall ASP-P1 results framework had been prepared (see program-level ICR); results frameworks for the individual ASP-P1 countries, including South Africa, were intended to be finalized during the individual ASP-P1 country project launches. However, country-specific PDO indicators were developed at appraisal. For ASP-P1 South Africa, these were as follows: Table 1.2: ASP-P1 South Africa Original PDO Indicators Original PDO Indicators as presented in the PAD and the GEF Trust Fund Grant Agreement a. Inventory database of publicly held obsolete pesticide stocks in place and being used by the Project Management Unit (PMU) and the government's staff. b. Completion of Country Environmental and Social Assessment (CESA) and implementation of the measures contained therein. c. Inventoried publicly held obsolete pesticides stocks disposed of, as a result of the completion of the disposal services contract(s) in accordance with national and international laws and regulations. d. Legal and regulatory framework for pesticide management improved, including measures to strengthen compliance with the Basel Convention and the Rotterdam Convention. e. The training program under Part D of the project is implemented and the knowledge so acquired is being used by the PMU and the recipient's government staff. 182 f. The PMU is functional and its operation is satisfactory to the World Bank. 1.3 REVISED PDO AND INDICATORS 10. The PDO and associated indicators for ASP-P1 South Africa were revised through a Level 1 restructuring on June 30, 2011. 183 11. ASP-P1 South Africa’s revised PDO: To assist the recipient in the development and piloting of a sustainable system to identify, collect and dispose of obsolete pesticides and associated waste. 12. ASP-P1 South Africa’s revised PDO Indicators: (i) A system for identifying, collecting and disposing of private and public obsolete pesticide stocks and associated waste developed through collaboration with the private sector association; and (ii) three pilot safeguarding and/or disposal operations implemented. 1.4 MAIN BENEFICIARIES 13. The original project design as described in the PAD targeted: (i) rural communities and individuals living in proximity to publicly held stocks of obsolete pesticides across the country, and (ii) government agencies, principally DEA and DAFF, which were expected to benefit from sustainable 181 The GEF Trust Fund Grant Agreement defined associated waste as materials that are contaminated by obsolete pesticides, including inter alia containers, equipment, and heavily contaminated soils where the World Bank agrees that those soils are within the project scope. 182 See Section 1.5 for the project’s Component D, here referred to as “Part D”. 183 Restructuring Paper on a Proposed Restructuring of the Africa Stockpiles Program to the Republic of South Africa, (Report No: 62795-ZA), June 20, 2011. 154 mechanisms aimed at managing accumulations of publicly held obsolete pesticides, and targeted training. Following the expansion of the project's scope to include private stocks and reduction of activities to focus on the Limpopo, Western Cape and Free State provinces, the main beneficiaries of the project became: (i) small scale farmers in the three provinces, and (ii) the pesticide industry, which became an active player in both the cleanup of the three provinces and the development of long-term mechanisms for collecting and disposing stocks. 1.5 ORIGINAL AND REVISED COMPONENTS 14. The PDO of ASP-P1 South Africa was to be achieved through the implementation of the following components. The table also shows revisions that were made by the June 2011 Level 1 restructuring. 184 Table 1.3: ASP-P1 South Africa Original and Revised PDO Indicators Original Components as presented Original Components as presented in Revised Components (Level I in the PAD 185 the GEF Trust Fund Grant Restructuring June 30, 2011) Agreement Component 1.A – Disposal of Part A – Obsolete pesticide inventory: Part A – Identification and collection of obsolete pesticides (Total: US$0.73 a. Collection of data and compilation of obsolete pesticides and associated waste: million; GEF: US$0.97 million): a data base; a. Establishing a booking line where farmers a. To undertake a detailed pesticide b. A risk-based prioritization of the in selected provinces voluntarily declare inventory; stocks identified in the data base; and obsolete pesticides and associated waste b. To produce a CESA including an c. Preparation of the CESA. in their possession; Environmental Management Plan b. Preparing EMPs; and (EMP); c. Collecting and transporting identified c. To safeguard obsolete stockpiles obsolete pesticides and associated waste and associate waste; to collection centers. d. To undertake a technological Part B – Disposal of obsolete Part B - Safeguarding and/or disposal of assessment of disposal options for pesticides: obsolete pesticides and associated waste: the inventory and disposal a. Carrying out a technological a. Carrying out of a technological activities; and assessment of available treatment assessment of available safeguarding and e. Disposal. and disposal options; disposal options; and b. Implementation of the treatment b. Implementing the safeguarding and/or and/or disposal technology selected; disposal technology selected in three pilot and provinces. c. Improving the operations of obsolete pesticide stocks collection centers, including improving safety measures at those centers. Component 1.B - Prevention of Part C – Prevention of obsolete accumulation (Total: US$0.45 pesticide accumulation: million; GEF: US$0.30 million): a. Awareness raising activities on the a. Raise awareness about activities health and environmental hazards of to be undertaken; pesticides; b. Review regulations, laws and b. A review and update of the legal and enforcement and prepare regulatory framework to control 184 Restructuring Paper on a Proposed Restructuring of the Africa Stockpiles Program to the Republic of South Africa, (Report No: 62795-ZA), June 20, 2011 185 Indicated GEF $amounts are not in line with GEF $amounts presented in Annex 8.1 as a result of inconsistencies in the PAD. 155 revisions; Obsolete Pesticide accumulation and c. Prepare a draft IPM strategy and strengthen enforcement mechanisms; initiate four pilot projects; and c. Preparation and implementation of d. Review pesticide management an IPM strategy; and practices and prepare a revised d. A review of pesticide management training plan. practices and preparation and implementation of a training plan. Component 1.C - Capacity Part D – Capacity building: Part C – Awareness raising and capacity building (Total: US$0.34 million; Strengthening the capacity of the PMU building: Carrying out of activities aimed at GEF: US$0.23 million): to implement the project through the raising awareness on the health and a. Strengthen the project provision of training. environmental hazards of pesticides and management skills of the PMU; strengthening the capacity of the PMU to and implement the project through the provision b. Training program and workshops. of training. Component 1.D - Project Part E – Project management: Part D – Project management: Supporting management and Monitoring and Supporting the PMU in carrying out the PMU in carrying out project Evaluation(M&E) (Total: US Project coordination, implementation, coordination, implementation, monitoring $0.28 million; GEF: US$0.30 monitoring and evaluation, through the and evaluation through the provision of million): provision of technical advisory technical advisory services, equipment and a. Establish a PMU at the National services, equipment and operational operational support. Department of Environmental support. Affairs and Tourism (DEAT); and b. Support the further design and implementation of a project Monitoring and Evaluation system. 1.6. OTHER SIGINIFICANT CHANGES Table 1.4: ASP-P1 South Africa Other Significant Changes Change Date Justification Approval Extension of closing date Extension of closing date by 17 months from 11/25/2009 To compensate for delays, and to ensure Regional 11/30/2009 to 04/30/2011. implementation of the sustainability option. VP Extension of closing date by 14 months from 05/03/2011 To allow for the completion of the Level 1 Regional 04/30/2011 to 06/30/2012. restructuring and remaining disposal and VP prevention activities. Reallocation of proceeds Reallocation of proceeds from the Unallocated 11/25/2009 To accommodate rent costs of the storage Regional category (-US$165,000) to the Goods facility and continuation of project management VP (+US$80,000), Works (+US$35,000) and during the extended period, and budgetary Operating Costs (+US$50,000) categories. constraints within the Government of South Increase of the percentage of refunding of Africa, which had kept certain activities from expenditures to 100% for all categories but moving forward. 156 Consultant Services. Reallocation of proceeds from the categories 06/30/2011 To accommodate a) costs incurred by the Board Consultant Services (US$215,000), storage of Limpopo stocks; b) fewer workshops Workshops and Training (US$75,000) and needed and necessary training carried out using Unallocated (US$245,000) to the categories parallel financing; c) funding needs for Goods (+US$50,000), Works (+US$250,000), additional stocks; and d) increased operating Disposal Services (+US$165,000) and costs because of the previous extension. Operating Costs (+US$70,000). Redefinition of activities Adjustment of safeguards instruments for the 06/30/2011 Following implementation of the EMP and Board safeguarding and disposal work independent review during the Limpopo pilot. Redefinition of the term “Disposal Services” 06/30/2011 Addition of in country safeguarding as a Board from services related to the treatment or possible cheaper end result for lower risk disposal of obsolete pesticides and associated stocks; this was due to larger than budgeted waste, to services related to the safeguarding stocks of obsolete pesticides and associated and/or disposal of the same. waste. Change of financing plan Change in the financing plan: addition of 06/30/2011 To reflect CLI’s parallel financing of the Board US$0.51 million from CLI. privately held hazardous waste disposal from the Limpopo Province, an activity that was not funded by the GEF Grant. CLI’s contribution was provided through a direct agreement between CLI and DEA. Addition of risks Addition of a Moderate-L risk: Inventory is 06/30/2011 To compensate for potential delays in the Board inaccurate. implementation of the collection and disposal contract given past performance. 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY 15. ASP-P1 South Africa was prepared between 2001 and 2005, as part of preparatory activities at the program-level. It took more than four years from GEF eligibility in July 2001, and more than three years from concept note in July 2002 to project approval by the Board in September 2005. 186 16. Soundness of background analysis: While a considerable amount of background analysis was conducted during project preparation, key data were missing, which could have qualified and quantified needed interventions. The government's laboratory, financial and procurement capacity was assessed, existing IPM practices and relevant national legislation were analyzed, and previous clean up experience was taken into account. Although the World Bank team recommended carrying out a rapid inventory and an assessment of disposal technological options during preparation, 187 these and a number of other preparatory activities were postponed to the implementation phase. Key 186 The long preparation time resulted from a lengthy partnership building/negotiation process at the program-level resulting from the complex implementation arrangement and project design (see Section 2.1 in the program-level ICR). 187 The appraisal Aide-Memoire also recommended carrying out a pesticides management baseline and a brief audit of the project management capability of PMU staff. 157 preparatory outputs such as the PAD and the Operational Manual were therefore lacking: (i) a precise estimate of the nature and amount of stocks to determine the focus (public vs. private stocks) and budget of the project; (ii) an assessment of disposal technological options (also to determine costs); and (iii) an analysis of pesticide use in the country, identifying the sources of stocks. They were further lacking a country-specific results framework and a disbursement estimate, and contained only preliminary work plans and cost tables. World Bank preparation mission reports demonstrate good collaboration between DEA, DAFF, the World Bank, FAO and CLI. Missions were carried out jointly and there was general consensus on the project design by the key stakeholders. 188 Representatives from relevant central and provincial departments, NGOs, the pesticide industry and research institutions provided recommendations on the design of the inventory. There is no indication, however, that communities were consulted during preparation. These consultations were planned to take place during implementation. DEA and DAFF agreed on their respective responsibilities for implementation in line with their departmental focus: DEA intended to focus on the inventory, the CESA, and disposal activities (Component A), while DAFF intended to lead prevention activities (Component B). 17. Assessment of project design: The project’s design (as for all ASP-P1 participating countries) was preconfigured at the program-level, and did not reflect country-specific needs and modifications. Using a broad generic program-level design affected project implementation performance of ASP-P1 South Africa: (i) the project’s focus on publicly held obsolete pesticide stocks was not relevant to South Africa’s environment, as the majority of stocks was privately held, and a substantial amount of publicly held obsolete pesticides had already been disposed of in prior years; and (ii) the envisaged inventory and subsequent data maintenance through application of FAO’s Pesticide Stock Management System (PSMS) were not considered cost-effective in light of large quantities of obsolete pesticides being held privately and an already existing national register that was used by the government to maintain pesticide data. 18. No country-specific lessons learned were spelled out in the PAD, while some were highlighted in the Operational Manual. The key lesson learned for South Africa emerged from previous disposal initiatives that had been undertaken in the country, which demonstrated that prevention strategies were as important as the actual disposal of stocks. It should be noted that previous cleanup operations showed that in South Africa, unlike other ASP-P1 projects, government involvement in pesticide purchasing had been extremely limited and the import and distribution was conducted mainly by the private sector. 189 19. Government commitment: Government showed high level of commitment towards the issue of pesticide management throughout preparation. During project pre-appraisal, high-level government officials expressed their interest in launching the project; 190 at appraisal, the project was considered a priority for the government.191 It should be noted that at the time South Africa requested World Bank/GEF assistance through ASP-P1, the country was in the early process of preparing a new legislation to strengthen the regulation of chemical and hazardous waste; the project was intended to be a key instrument to implement this policy. 188 Appraisal Aide-Memoire, June 2004. 189 See Operations Manual, page vi 190 See pre-appraisal Aide Memoire, March 2003 191 See appraisal Aide Memoire, June 2004 158 20. Assessment of risks: While country-specific risks were not discussed in the PAD, the Operational Manual included a comprehensive risk assessment. Risks within the control of the project were correctly associated with poor project planning, a possible lack of funds and public opposition against to project activities resulting from insufficient awareness. Risks outside the control of the project were also adequately identified and included changes in national priorities or government implementation arrangements, delays related to local bureaucracy/issuance of licenses and permits, seasonal weather patterns, and a lack of support from farmers and the pesticide industry. Correlating risk mitigation measures for controllable risks were identified in the Operational Manual. 2.2 IMPLEMENTATION 21. The project was subject to two extensions for a total of 31 months, and a Level 1 restructuring in 2011. 22. Key factors affecting implementation and outcomes included: • Recognition of project approach not being appropriate for South Africa affected implementation of originally envisaged disposal activities. During the first year of project implementation, the envisaged project design was judged being inadequate by all partners involved, as it was not applicable to the South African context. The existing design that was applied to all ASP-P1 participating countries, including South Africa, was based on the assumption that pesticide stocks were held by public agencies and that the private sector would be relatively underdeveloped; however, in South Africa the majority of stocks was held by private farmers and not by the government - public stocks were limited to controlling migratory pests and a substantial amount of publicly held obsolete pesticides had already been disposed of in prior years.192 To allow for a better assessment of the issue of obsolete pesticide stocks in the South African context that could inform the development of an improved project implementation approach, the World Bank and partners quickly reacted by agreeing to pilot the identification and collection of both public and private stocks in one province, Limpopo. 193 • The first phase of the pilot was completed in 2007, with CLI having played a key role in the planning and implementation. The pilot revealed higher than estimated stocks of obsolete pesticides, and the already presumed high proportion of privately held stocks. The pilot confirmed the need to adopt a different collection and disposal model for the project, limiting the project’s geographic coverage and encompassing both private and public stocks because: (i) a country-wide obsolete pesticide cleanup was not considered suitable as it was estimated to cost about US$3 million more than budgeted at appraisal; 194 and (ii) the cleanup of predominantly privately-held stocks using public funds would not have been supported by DEA, as South 192 During the late 1990s, pesticides were centralized in 45 collection depots, and approximately 1,040 tonnes of obsolete pesticides, including 70 tonnes of DDT and other POPs, were disposed of overseas and locally with national funds. See Section 1.1 for more information. 193 Limpopo, a province in the north of South Africa, was considered well prepared for participation in the pilot, and was characterized by a strong agricultural sector. 194 Following the collection of approximately 102.5 tonnes of obsolete pesticides at Limpopo alone, 700-900 tonnes were estimated to be scattered across South Africa, versus 250 tonnes estimated at appraisal. 159 African law required the producer or generator of pollution to pay for the costs of avoiding pollution, cleaning up, or remedying its effects. 195 • Development of a different collection and disposal model, including the creation of a public- private partnership was a lengthy process. The government and the World Bank agreed to halt originally envisaged project activities until a better adapted project model for the disposal of obsolete pesticides in South Africa was identified. Instead, the focus was placed on facilitating a dialogue between the government and the private sector, to establish a partnership for collection and disposal of obsolete stocks, which the new approach could be based on; however, given the stakes involved, the process was lengthy and slow. The mid-term review, conducted in January 2009, consequently resulted in a sharp downgrade of ratings and confirmed the need to restructure the project for better alignment with the country's setting. Eventually in March 2010, and informed by the Limpopo pilot and an Australian pesticide collection system that had been examined during a study tour, DEA and the private industry association, AVCASA were able to reach an agreement. It was decided to implement two additional pilot cleanup operations (with utilization of remaining GEF funds), intended to help inform the development of an industry-run scheme for dealing with existing and future accumulations of both public and private stocks in the country. • Challenges associated with institutional arrangements delayed and eventually excluded implementation of prevention activities from the project. The implementation of prevention activities was planned to be implemented by DAFF. 196 However, the necessary co-operation between DEA and DAFF suffered from difficulties in advancing budgetary resources between the departments and a missing special account, and subsequently resulted in prevention activities still not having started in 2010. Upon review of activities included under this component, it finally became apparent that most prevention activities would be covered by DAFF under the newly gazetted National Pesticide Policy (2010), and by AVCASA under its stewardship program. The two entities subsequently informed the World Bank that GEF funding would no longer be needed as prevention activities would be integrated in DAFF's work program, and disconnected from the project. • Restructuring to reflect the project’s modified disposal model was implemented late in the process. Only in June 2011, ASP-P1 South Africa underwent a Level 1 restructuring to correct the identified design flaws. In addition to the above-mentioned reasons, the delay also resulted from the World Bank’s preference to restructure ASP-P1 South Africa together with three other ASP-P1 participant countries. The restructuring included: (i) a revised PDO and associated indicators to reflect the changed focus and geographic coverage; (ii) a revised project description to reflect the modified PDO and the fact that prevention activities were carried out with DAFF’s own funds and no longer part of the project; (iii) endorsement of a country-specific results framework which met current World Bank/GEF requirements; and (iv) revised safeguard requirements including a shift from a CESA to independently reviewed Environmental Management Plans (EMPs). 195 Including 12.5 tonnes of pesticides, which were collected in 2002 by the NGO Endangered Wildlife and safeguarded by CLI. It was agreed to include these stocks for disposal. 196 DEA and DAFF signed a Memorandum of Understanding in 2006, to formalize the departments’ work relationship and ensure joint responsibility for project outputs. 160 • Procurement issues associated with the disposal contract prevented disposal initiation at remaining two pilot areas. Following a failed bidding process due to inadequate procurement according to World Bank procedures for the disposal services in April 2011, disposal bidding documents were rejected again by the World Bank in April 2012 due to an improper technical assessment and non-adherence of procurement requirements as spelled out in the bidding document. Based on the experience gained in other ASP countries and previous procurement failures within the framework of the project in South Africa, the World Bank considered the remaining time frame too short to successfully contract a service provider, and asked DEA to consider requesting an extension of the project’s closing date. However, such a request was sent to the World Bank only three days before actual project closing, preventing the extension process to be completed in time. The project was subsequently closed without the service provider having been contracted. 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTILIZATION 23. Overall M&E - Rating Moderately Unsatisfactory. 24. M&E design – Rating Moderately Unsatisfactory: Despite the lengthy preparation time, an M&E system specific for South Africa was not developed at appraisal. 197 A results framework, which was intended to be finalized by the TSU during project launch and included in the project’s Operational Manual, also did not materialize. Only a set of PDO indicators linked to the different project components was developed in the PAD and GEF Trust Fund Grant Agreement, however, these did not allow for adequate monitoring of project implementation progress. Their design focused on measuring outputs as opposed to progress towards project outcomes. Expected results under the prevention component, in particular, were not adequately considered by the indicators. 25. M&E implementation – Rating Moderately Unsatisfactory: M&E staffing was adequate, and progress reports were submitted timely throughout project implementation. However, they only focused on describing implementation activities and lacked an evaluation or assessment of performance. Following revision of the disposal approach that had informally been initiated in 2007, several of the indicators agreed on at appraisal had become irrelevant; they were not modified accordingly. Only in 2011, the PDO and associated indicators were officially revised as part of the Level 1 restructuring; a results framework including intermediate outcome indicators was finally introduced. However, the amendments were introduced too late as the project closed a year later. 26. M&E utilization – Rating Moderately Satisfactory: The identification and collection of obsolete pesticides during the Limpopo pilot in 2007, was conducted under application of the FAO Environmental Management Toolkit (EMTK). 198 The data was, however, only captured in an Excel spreadsheet, which did not represent an adequate M&E instrument. In 2011, a booking line, where farmers voluntarily declare stocks in their possession, was tested and established in the two pilot 197 Lengthy discussions between the partners during project preparation on whether to have an M&E system at the program-level alone, or also at the country-level, may have contributed to the absence of an M&E system at the country-level at project launch. However, no information was available on whether it was a deliberate decision not to have an M&E system in place at the country-level. 198 The tool kit provides a practical set of methodologies to assist countries in the risk-based management of obsolete pesticides, including guidance on the management of collection/storage locations, based on international best practice. 161 provinces, Free State and Western Cape. The generated data was successfully used to determine waste collection points in these two provinces, which were subsequently assessed for suitability using the environmental assessment criteria outlined in the EMTK. However, a collection schedule was never developed as the disposal contractor was not recruited prior to project closing. The booking line and the government’s existing national register replaced the PSMS database, which was considered an unnecessary tool. 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE 27. Safeguard compliance – Rating Moderately Satisfactory: ASP-P1 triggered two safeguard policies: OP 4.01 on Environmental Assessment and OP 4.09 on Pest Management. 28. During project implementation, the World Bank continuously assessed the requirement of a CESA in view of the planned modifications to the disposal approach. In 2010, it was agreed that in the absence of a nation-wide inventory, the assessment of environmental and social risks would be based on the obsolete pesticides data from the Limpopo pilot. An EMP for the disposal of the Limpopo stocks (conducted in 2011), was prepared and reviewed by two independent reviewers. The approach was cleared by the World Bank, and recommended for disposal activities in the two remaining provinces. 29. OP 4.09 on Pest Management: Since project activities themselves met requirements of a pest management plan (i.e., through pesticide management, Integrated Pest Management (IPM), and capacity building), individual country projects, including ASP-P1 South Africa, were not required to prepare self-standing documents. See Section 2.4 in the program-level for more details. 30. Compliance with OP 4.01 on Environmental Assessment is summarized in the following: • The collection and inventory of obsolete pesticides in Limpopo province was not in full compliance with OP 4.01 on Environmental Assessment. The pilot operation in Limpopo did not make use of the inventory taking method proposed by FAO and the Environmental and Social Management Framework because it was not considered suitable for this specific case by the partners (i.e., obsolete pesticide stocks mainly held by private farmers). The pilot operation required a different approach consisting of close cooperation between pesticide distributors and farmer cooperatives to identify, package and deliver obsolete pesticide stocks to 24 existing pesticide distribution centers. Environmental assessments were carried out for each of the distribution centers. Farmers were instructed on safely transporting the stocks and a comprehensive communications and awareness campaign was implemented. 199 When the collected stocks were inventoried, it became apparent that some of the stocks were posing higher than estimated risks and that the proposed inventory procedure would not be adequate. By applying due diligence, an experienced national waste management company was subsequently recruited by DEA, to conduct the inventory and repackage the stocks for safe interim storage. Two no-objection letters were submitted to the World Bank requesting approval for the service provider; however, in an attempt to minimize exposure risks, DEA began conducting the inventory and repackaging activities before the World Bank responded to the requests. • Disposal of obsolete pesticides from Limpopo province was conducted in compliance with OP 4.01 on Environmental Assessment. In 2010, DEA initiated the disposal of the Limpopo stocks with CLI funding. A risk assessment report and an EMP were developed, peer reviewed by DEA, 199 Awareness was raised through radio programs, press releases, publications and SMS alerts to all commercial farmers. 162 an independent consultant and the ASP-South Africa NGO, and cleared by the World Bank on November 15, 2010. Disposal activities were subsequently implemented in compliance with safeguard policies, and in accordance with all necessary protocols, including the Basel Convention guidelines. Final incineration was achieved in August 2011. 200 • Disposal of obsolete pesticides in Western Cape and Free State was not conducted – therefore OP 4.01 did not apply. Following the successful independent review and implementation of the EMP during disposal activities at Limpopo, the World Bank and DEA agreed that the same approach would be used for disposal activities at the other two provinces. It was agreed that the selected disposal contractor would prepare the EMPs for the operations. However, eventually the EMPs were not prepared because the contractor had not been hired prior to project closure. 31. Financial Management compliance – Rating Moderately Satisfactory: The project submitted acceptable Interim unaudited Financial Reports (IFRs) and audit reports. However, several delays were noted, resulting from: (i) a small value of transactions at the beginning of the project, leading the World Bank to waive the first two audits (for FY07 and FY08); (ii) a long authorization process within DEA for audits; and (iii) the inability of DEA’s financial system to produce the agreed IFRs. The project did not fully comply with Financial Management (FM) arrangements; although it was supported by adequately qualified departmental staff, their availability to the project was lower than agreed. 32. Procurement – Rating Moderately Unsatisfactory: The key challenge for procurement above the shopping threshold was the reconciliation of South African procurement practices with the World Bank’s procurement guidelines. Changes in procurement staff at the beginning of the project, the lack of clearly identified procedures in the Operational Manual and the decision to start the Limpopo pilot before receiving procurement training, led the PMU to initiate procurement of services and consultants using government, rather than World Bank procurement procedures. The procurement of a service contractor for the disposal at Western Cape and Free State provinces failed twice.201 Although the World Bank had recognized the weak procurement capacity (for World Bank procedures) at appraisal, additional procurement training was not provided during project implementation. 2.5 POST-COMPLETION OPERATION/NEXT PHASE 33. Post-completion: No information could be obtained on post-completion activities. 34. Next phase: No information could be obtained on future activities. 3. ASSESSMENT OF OUTCOMES 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION 35. Relevance – Rating Substantial: The revised PDO is relevant to South Africa's current development and global priorities. Objectives are in line with the 2010 National Pesticide Policy and the National 200 As reported by CLI, disposal certificates indicate destruction of stocks on August 22, 2011. 201 In April 2011, the bidding process failed due to inadequate procurement according to World Bank procedures for the disposal services; in April 2012, disposal bidding documents were rejected again by the World Bank due to an improper technical assessment and non-adherence with bidding document requirements. 163 Environmental Management: Waste Act of 2008; they also support the Government of South Africa in complying with the Stockholm Convention on POPs, and the Basel Convention on the control of trans-boundary movements of hazardous wastes and their disposal. The objectives are also fully in line with the World Bank's Country Partnership Strategy, 202 which emphasizes the need to continue and support pollution management as part of a continent-wide program to dispose of obsolete pesticides. The engagement of the pesticide industry in discussions with government about long-term management of stocks is in line with the World Bank’s priority of supporting private-public partnerships in South Africa, and helping to remove constraints to the industry's engagement. 203 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE 36. Taking account of the project's Level 1 restructuring in June 2011, the project is evaluated against its original and revised PDO by measuring project achievements at the time of restructuring (June 2011) and project closing (June 2012), respectively. Associated project achievements, which were not adequately reflected in the results framework are also factored into the evaluation of PDO achievement. 37. Original PDO achievement – Rating Moderately Unsatisfactory. The assessment took the project’s six PDO indicators into account, of which one was fully (100%), three partly (7-80%), and two not (0%) achieved. 38. The three main project outcomes measured by indicators include: (i) sound disposal and management of obsolete pesticides; (ii) prevention of accumulation of new stockpiles of obsolete pesticides; and (iii) enhanced capacity and institutional strengthening of pesticide management. 39. Sound disposal and management of obsolete pesticides. The level of risks emanating from publicly held obsolete pesticides and associated waste to adjacent and wider communities or critical natural resources was not determined and reduced to the extent originally planned. The originally planned countrywide inventory of publicly held obsolete stocks (PDO Indicator #1 - 0%) was not carried out. 204 In the absence of the originally planned inventory, preparation of a CESA was neither completed (PDO Indicator #2 – 0%). Efforts were instead focused on developing an improved approach for sound disposal and pesticide management, encompassing both, privately and publicly held obsolete pesticides. To inform the development of this approach, a pilot operation was conducted in Limpopo province in 2007, where privately held obsolete pesticides were packaged and delivered to existing pesticide distribution centers in the province. All stocks were centralized to a main storage location in the province, and repackaged for interim storage. Approximately 17 tonnes of publicly 202 Country Partnership Strategy for the Republic of South Africa for the Period 2008-2012, (Report No. 38156-ZA), December 12, 2007. 203 As explained, the original project design was not entirely relevant to the obsolete pesticides situation in South Africa, where most stocks are held by private farmers rather than the public sector; nor was it relevant to the storage infrastructure and government's existing capacity to manage obsolete pesticides. On the other hand, the revised design responded adequately to the main origin of pesticide stocks, existing infrastructure and capacity of government, as well as to the industry's capability to manage the stocks. Implementation closely followed the government's policies regarding the 'polluter pays' principle, and the 2010 Pesticides Management Policy's requirement from the industry to prepare an Integrated Waste Management Plan (IWMP). 204 It was considered cost-ineffective in an environment where most stocks were held by private farmers, and not by the public sector; in addition, application of the FAO PSMS for maintaining the inventory was not considered value adding as the government was using an existing national register to maintain pesticide data. 164 held obsolete pesticides were classified as non-hazardous waste and disposed of locally in 2009 (PDO Indicator #3 – 6.8 %), thus having reduced risks emanating from these stocks.205 40. Prevention of accumulation of new stockpiles of obsolete pesticides. Prevention activities were not implemented as envisaged at appraisal. Instead, beginning in 2010, efforts to prevent future accumulation of obsolete pesticides were implemented under the respective work programs of DAAF and AVCASA. Activities were funded directly by these two entities and were therefore implemented independently from the project. The project strengthened pesticide management by improving pesticide registration, licensing, and enforcement of import controls. The legal and regulatory framework for pesticide management was strengthened in 2010, when DAFF adopted and published a national Pesticides Management Policy, which supported the objectives of the Basel and Rotterdam conventions. Direct project support was not needed due to the government's strong engagement (PDO Indicator #4 - 100%). 41. Enhanced capacity and institutional strengthening of pesticide management. The PMU had received fiduciary training as well as training in project management. Training was also provided in preparing a CESA and using the PSMS. In addition, relevant government staff participated in a study tour to Australia, to examine the country’s pesticide collection system (PDO Indicator #5 – 80%). However, the procurement training did not strengthen the capacity of the PMU because procurement staff changed following the training, and no additional training was provided. The PSMS and CESA trainings were not useful, as these tools were not used during project implementation. 42. Revised PDO achievement – Rating Moderately Unsatisfactory. The assessment took the project’s two revised PDO indicators into account, which were both only partly achieved (55% and 33%). In addition, four Intermediate Outcome Indicators framed the evaluation, of which one was fully (100%), and three partly (33-50%) achieved. Outcomes were modified following the restructuring and included: (i) improved ability to identify, collect and dispose of obsolete pesticides and associated waste, and (ii) gained experience in removing obsolete pesticides (public and private) and associated waste. 43. Improved ability to identify, and collect obsolete pesticides and associated waste. The project improved the government’s ability to identify, collect and dispose of obsolete pesticides and associated waste through the testing and establishment of a booking line, where farmers voluntarily declare stocks in their possession (Intermediate Outcome Indicator #1 - 100%). The booking line was operated by the private sector, supervised by the government, and was used to determine the scale of the collection and treatment operations. Following its successful piloting, the system was introduced to the remaining provinces in South Africa. The system was adopted from a model used in Australia, which the government was able to examine during a study tour that was financed by CLI (Intermediate Outcome Indicator #4 - 50%). As outlined further below, private and public obsolete pesticides and associated waste were identified in three provinces, however, only collected, and disposed of in one province (Revised PDO Indicator #1 – 55%). Moreover, the booking line and its application in the two provinces informed the development of an Integrated Waste Management Plan (IWMP), which intends to dictate a private sector-run scheme for collection and disposal as stipulated by national law. The IWMP is in line with the country’s National Environmental Management Waste 205 For more details see Aide-Memoire: South Africa – Africa Stockpiles Program (ASP) – Mid-Term review, January 23 – February 6, 2009; African Stockpiles Program (ASP) South Africa – Progress Report Period March 2008 – December 2008. 165 Act of 2008, and was submitted to the DEA for adoption in June 2011. If approved, the plan is considered representing a sustainable prevention mechanism against re-accumulation of obsolete pesticide stocks; however, at ICR drafting, the plan had not yet been approved as reported by CLI. A privately run recycling scheme already existed for empty plastic pesticide containers. 206 44. Through the Limpopo pilot disposal operation, experience was gained in removing obsolete pesticides (public and private) and associated waste. However, the project failed to remove obsolete pesticides in the remaining provinces, due to procurement issues and a subsequent missed opportunity to extend the project. The project reduced the risks emanating from obsolete pesticides and associated waste to communities, natural resources, and the global environment in one of the three pilot provinces, Limpopo Province. In addition to the 17 tonnes of publicly held stocks that had already been disposed of prior to restructuring, 92.5 tonnes 207 of privately held obsolete pesticides were disposed of with funds provided by CLI. A risk assessment report and an EMP had been developed for this province, peer reviewed by DEA, an independent consultant and the ASP-South Africa NGO representative, and cleared by the World Bank on November 15, 2010 (Intermediate Outcome Indicator #2 – 33%). Stocks were shipped to the UK for disposal by following all necessary protocols, including the Basel Convention guidelines, and eliminated in August 2011. 208 Approximately 120 tonnes of obsolete pesticides and associated waste were identified in the two additional provinces, consisting of about 70% privately held, and 30% publicly held obsolete pesticides. However, the project did not complete the disposal and/or safeguarding of these stocks as the disposal contractor was not hired prior to project closing due to the above-mentioned procurement issues and a missed opportunity to extend the project (Revised PDO Indicator #2 – 33%; Intermediate Outcome Indicator #3 – 33%, respectively). 45. Weighted average PDO achievement – Rating Moderately Unsatisfactory: Table 3.1: Weighted Average PDO Achievement Against Original PDO Against Revised PDO Overall 1. Moderately Moderately Rating - Unsatisfactory Unsatisfactory 2. Rating Value 3 3 - 3. Weight (% disbursed before/after PDO 77.0% 23.0% - change) 4. Weighted value 2.3 0.7 3.0 5. Moderately Final rating - - Unsatisfactory 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE 46. The project aimed to directly contribute to the GEO of reducing the effects of POPs on the global environment through: (i) improving the quality of life in poor communities, by reducing 206 For more information on the empty plastic pesticide container recycling scheme, please visit http://www.avcasa.co.za/. 207 These included 84 tonnes of obsolete pesticides collected at Limpopo province, 2.5 tonnes collected at Gauteng Municipality, and associated packaging materials. 208 As reported by CLI, disposal certificates indicate destruction of stocks on August 22, 2011. 166 environmental risks; (ii) improving environmental protection; and (iii) enhancing the capacity of the agricultural sector to better manage crop pests. GEO achievement is measured by PDO achievement, and, thus, also rated moderately unsatisfactory (see Section 3.2). 3.4 EFFICIENCY 47. Efficiency – Rating Low. This rating represents a broad estimate, as financial and economic data to support this rating are scarce. 48. At appraisal, an economic or financial analysis was neither undertaken for ASP-P1 at the program nor at the country-level due to non-measurable benefits of reducing the risk of contaminating the environment. Based on approximately US$15 million spent on the disposal of 3,240 tonnes of obsolete pesticides between 1990 and 2004, it was estimated that 300-900 tonnes could be disposed of per annum at the cost of US$3,400 per ton. (see Section 3.4 in the program-level ICR for details). 49. Despite the modification of the project’s design following the recognition that most obsolete pesticide stocks in South Africa were being held privately, only a limited amount of pesticides was disposed of under GEF funds by the end of the project. Actual unit cost per ton disposed of does therefore not allow for an accurate cost efficiency assessment. The project’s low efficiency rating is therefore based on the limited use of resources for disposal of obsolete pesticides in relation to lengthy project implementation. Delays were caused by a lack of implementation progress resulting from the lengthy process that was associated with adapting the project’s design to the South Africa’s country-context. About 35% of the funds were spent on the storage of the Limpopo stocks (5 years at a monthly rate of $7,660) following the government's decision not to pay for the disposal of private stocks, and the time it took to reach a financing agreement with the private sector to dispose of the collected stocks. Prevention activities eventually showed significant results with development of the 2010 Pesticide Management Policy, and capacity building efforts demonstrated an important impact in view of the dialogue with the private sector; however, these were financed by the government directly, or mostly by the private sector, respectively. 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING 50. Overall Outcome – Rating Moderately Unsatisfactory: On the basis of the project’s efficiency (low), relevance (high), and PDO achievement (moderately unsatisfactory), the overall outcome is rated moderately unsatisfactory. However, it should be noted that the effective collaboration between the government and industry resulted in the development and submission of an IWMP. If the plan is approved by DEA as envisaged, the cleanup of obsolete pesticide stocks could be systematically guided in future; this would be a sustainable and strong outcome, which, however, goes beyond the revised objectives of the project. 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS 51. Poverty impacts, gender aspects, and social development: The project had a direct positive impact on farming communities in Limpopo province, where obsolete pesticide stocks were collected and disposed of, thereby eliminating a prominent local development risk. Indirectly, the country’s population as a whole benefitted from a revised legal framework for pesticide management, which is expected to decrease the social burden pesticides pose on communities, females and males alike. Industry’s engagement in the dialogue ensured a more integrated approach toward future cleanup 167 operations. While the impact of improved pesticide management legislation was foreseeable, the impact of the project on the private sector (farmers and industry) was not initially foreseen given that the original project design focused on the public sector. 52. Institutional change/strengthening: While the project did not have a major impact on government’s already adequate capacity to implement clean up and prevention activities as planned, it did have a catalyzing effect on government’s regulatory role concerning pesticide management through improved policy formulation and entering into constructive dialogue with industry. 53. Other unintended outcomes and impacts: Despite its design inadequacies and implementation shortcomings, the project had an unintended impact on the way obsolete pesticides are expected to be managed in South Africa in the future, and the roles government and industry will play. The development and utilization of a booking line, a system that was adapted from Australia, demonstrated the feasibility to identify private stocks countrywide with limited government involvement and funding. 209 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS 54. No beneficiary survey was conducted and no stakeholder workshop was held. An ICR mission was planned but could not be carried out within the implementation period. The government’s completion report contained limited information and World Bank’s request for additional information did not reveal any particular issues. 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME 55. Risk to the development outcome - Rating Moderate: DAFF and DEA, the governmental departments mandated with managing pesticides and pesticide waste, respectively, demonstrated existing good capacity to implement their mandates and continuous strong commitment to the issue. The project further increased the government's capacity by sensitizing key stakeholders to the Australian pesticide collection system, and by supporting the dialogue with the private sector industry. This experience led the pesticide industry association to develop the IWMP and submit it for approval to DEA. If approved, the IWMP will regulate responsibilities and lay out plans for future collection operations with the private sector industry playing a key implementing role. 56. However, despite the commitment and capacity of both industry and government, and the appropriate regulatory framework in place, the development outcome could be at risk if: (i) the IWMP is not approved by the minister or approved with considerable delay; (ii) industry and/or other responsible entities do not implement the IWMP or lack capacity to perform satisfactorily; and (iii) a financial mechanism is not set up or does not raise sufficient funds for a meaningful impact. 209 Government provides indemnity to stock holders who register. 168 5. ASSESSMENT OF BANK AND BORROWER PERFORMANCE 5.1 WORLD BANK PERFORMANCE 57. World Bank performance in ensuring quality at entry – Rating Moderately Unsatisfactory: Project preparation of ASP-P1 South Africa was conducted in conjunction with preparatory activities at the program-level, which were lengthy (see Section 2.1 in the program-level ICR). While considerable time was invested in overall preparation, project documents for ASP-P1 South Africa only broadly laid out implementation plans, and lacked country-specific information (see Section 2.1). Most importantly, despite the lengthy preparation process, ASP-P1 South Africa was launched with a project design that turned out to be not applicable to the South African context, as recognized in the beginning of implementation. This resulted in considerable delays, and the need to redesign the project during implementation. 58. Quality of World Bank supervision – Rating Moderately Unsatisfactory: Project supervision was carried out via missions, email and telephone exchanges. One launching mission and five supervision missions took place, averaging one mission per year. Until mid-2008 the project was intermittently supervised by the ASP Task Team Leaders (TTLs) in Washington DC and co-TTLs based in the Pretoria World Bank Office. Beginning mid-2008 until project closing, the project was not assigned a co-TTL (unlike all other ASP projects) and was supervised by the ASP TTL from Washington DC. This may explain the infrequency of World Bank missions to the project. Procurement and FM were supervised by World Bank staff based in Pretoria: FM supervision was carried out in all missions, while procurement supervision was conducted only in the first two missions (in June 2006 for training and March 2007). One official post procurement review was carried out in May 2011. Safeguards were supervised by staff from the Pretoria and Washington DC offices during all missions except the mid-term review in February 2009 (due to unavailability of a safeguard specialist to join the mission). During the lifetime of the project, the project was supervised by six different TTLs/co- TTLs. 59. Despite early recognition of the need to modify the design, the project was only restructured in 2011. During the first five years of the project, project activities were not implemented according to the provisions of the GEF Trust Fund Grant Agreement; amended activities such as the piloted collection, storage and eventual disposal of obsolete stocks in Limpopo province were conducted without the necessary M&E framework in place to monitor the activities’ performance or progress. 60. Overall World Bank performance – Rating Moderately Unsatisfactory: Based on the above presented evaluation, the overall borrower performance is rated moderately unsatisfactory. 5.2 BORROWER PERFORMANCE 61. Government performance – Rating Satisfactory: Government performance was overall satisfactory throughout implementation as reflected by its commitment to the issue. Government commitment at entry continued to be strong as was evident from its strong engagement during the Limpopo pilot. Both DEA and DAFF engaged in a close dialogue with AVCASA and CropLife South Africa, and an effective public-private partnership was formed. This relationship was later maintained and the government continued to engage South Africa’s private sector industry in agreed follow up activities. Government approval of the National Pesticide Policy in 2010 demonstrated continued commitment to the issue of pesticide management. DAFF’s decision to implement prevention activities 169 independently from the project affected project performance; however, this seems to have been the result of administrative difficulties between the project and DAFF and not due to a lack of commitment. 62. Implementing Agency performance – Rating Moderately Unsatisfactory: During project preparation DEA collaborated well with the World Bank and other ASP partners. DEA prepared a comprehensive Operations Manual, which provided considerable information considering the lack of key baseline data. However, a staff turnover at the beginning and at the end of the project delayed project activities; in addition, the misplacement of legal documents in one occasion, lengthy authorization procedures within and outside the department affected timely submission of audits, and countersigning of amendments. DEA executed its fiduciary duties, however, not in full compliance with World Bank legal covenants. As previously described, the bidding process for disposal failed twice due to improper procurement. In addition, audits and IFRs were often submitted delayed. However, DEA provided co-financing and in-kind support to the project as planned. 63. Overall borrower performance – Rating Moderately Unsatisfactory: Taking into account that the project’s overall outcome is rated moderately unsatisfactory, and the implementation agency’s performance often delays implementation progress, overall borrower performance is rated moderately unsatisfactory. 6. LESSONS LEARNED 64. The most significant lessons learned are outlined in Section 6 of the program-level ICR, as the experience they are based on was often reflected across all country-specific projects. Lessons learned that were particularly relevant to ASP-P1 South Africa include: • Future projects should place particular emphasis on tailoring individual country projects to the country-specific environments. The uniform approach ASP-P1 was applying to all participating countries was not appropriate for the South African context, and caused significant delays and transaction costs. • Private-public partnerships are important for the collection and disposal of privately-held obsolete pesticides. In countries which hold significant amounts of privately-held stocks should seek to form private-public partnerships between the relevant governmental agency and the private sector, and include private stocks in the disposal and prevention components. 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS 65. See Section 8.7 and 8.8. 170 8. ANNEXES 8.1 PROJECT COSTS AND FINANCING a. Project Cost by Component – All sources (in US$ million equivalent) – as of 05/15/2013 Component Appraisal Estimate Actual/Latest Actual as % of (in US$ million equivalent) Estimate appraisal (in US$ million estimate equivalent) 5. Disposal of obsoletes pesticides 0.97 N/A 210 N/A 6. Prevention of accumulation 0.30 N/A N/A 7. Capacity building 0.23 N/A N/A 8. Project management 0.30 N/A N/A Total 1.80 0.71 39% b. Financing – as of 05/15/2013 Sources of Funds Type of Appraisal Restructuring Actual/Latest Actual as % of Co- Estimate Estimate Estimate (in appraisal financing (in US$ June 2011 US$ million estimate million (in US$ million equivalent) equivalent) equivalent) Counterpart Borrower 0.07 0.70 N/A N/A Funding Global Environment Facility Grant 1.73 1.70 N/A N/A (GEF) Parallel CropLife International (CLI) 0 0.51 N/A N/A Financing Total 1.80 2.91 0.71 39% 210 Actual/latest estimates of cost by component and sources of funds were not available. 171 8.2 OUTPUTS BY COMPONENT 1. Taking account of the project’s Level 1 restructuring in June 2011, the following table provides a comprehensive account of qualitative and quantitative information of outputs realized against both original (achievements at the time of restructuring in June 2011) and revised components (achievements at the time of project closing in June 2012). Table II.1: Outputs by Original Components as presented in the GEF Trust Fund Grant Agreement Original Components as presented in the GEF Outputs achieved at the time of restructuring in June 2011 Trust Fund Grant Agreement Part A – Obsolete pesticides inventory: Achievements under this component are rated Unsatisfactory Carrying out of a detailed inventory of the Most of the originally envisaged project activities were not applicable recipient’s publicly held obsolete pesticides to the South African context, and, therefore, could not be implemented stocks through: as planned. a. Collection of data and compilation of a a. Carrying out an inventory was considered cost-ineffective in an database; environment where most stocks were held privately; in addition, the b. A risk-based prioritization of the stocks government was using an existing national register to maintain identified in the database; and pesticide data (Original PDO Indicator #1 – 0%). c. Preparation of the CESA. b. A risk-based prioritization was subsequently not possible. c. In 2010, it was agreed that in absence of a nation-wide inventory, a CESA would not be the appropriate safeguard tool, as it required inventory data for detailed analysis and prioritization (Original PDO Indicator #2 – 0%). Part B – Disposal of obsolete pesticides: Achievements under this component are rated Moderately Unsatisfactory Treatment and/or disposal of publicly held a. The technological assessment study was launched. It was, however, obsolete pesticides stocks and associated waste only finalized after the restructuring; therefore, this activity was not by: completed. a. Carrying out of a technological assessment of b. Obsolete pesticides classified as non-hazardous waste under national available treatment and disposal options; and WHO regulations (17 tonnes) were disposed of locally in 2009 b. Implementation of the treatment and/or (Original PDO Indicator #3 – 6.8%). The hazardous portion of the disposal technology selected; and stocks was not disposed of by the time of the restructuring. c. Improving the operations of obsolete pesticide c. The need for improving the operations of obsolete pesticide stocks stocks collection centers, including improving collection centers, including improving safety measures at those safety measures at those centers. centers, was not reviewed at appraisal, as large pesticide stores in South Africa already had to meet minimum safety standards which met the guidelines established by FAO. Part C – Prevention of obsolete pesticide accumulation: Achievements under this component are rated Moderately Satisfactory Carrying out of activities at preventing obsolete a. The South African NGO Network prepared and distributed literature pesticide accumulation, including: on the appropriate handling of pesticides. Furthermore, NGOs a. Awareness raising activities on the health and conducted community monitoring and awareness-raising activities environmental hazards of pesticides; in Kwa-Zulu province where pesticides had been implicated b. A review and update of the legal and wildlife-poisoning incidents. Awareness raising activities were also regulatory framework to control obsolete conducted in preparation of collection activities as part of the pesticide accumulation and enforcement Limpopo pilot. mechanisms; b. Regulatory framework gaps analysis was carried out by DAFF's c. Preparation and implementation of an IPM legal unit. DAFF used the gap analysis to draft the required 172 strategy; and amendment and revisions to an existing policy. In 2010 a National d. A review of pesticide management practices Pesticide Policy was approved by the government and published and preparation and implementation of a (Original PDO Indicator #4 - 100%). training plan. c. Since IPM did not represent an official government policy, it was not integrated into DAFF's prevention work program. Moreover, the IPM budget allocation was deemed too small to have a meaningful impact. d. No training plan for the improvement of pesticide management practices was prepared; however, during the course of implementation the PMU received PSMS and CESA training and participated in a study tour to Australia to learn about a local system for stocks collection via a public-private partnership. (Original PDO Indicator #5 - 80%) Part D – Capacity building: Achievements under this component are rated Moderately Satisfactory. Strengthening the capacity of the PMU to The PMU was trained on procurement, financial management, CESA implement the project through the provision of preparation, project management and PSMS (Original PDO Indicator training. #6 – 80%). The procurement training did not strengthen the capacity of the PMU because procurement staff changed after the training and no additional procurement training was provided. The PSMS and CESA training were not useful as these tools were not used during project implementation in South Africa. They were provided as part of the ASP-wide training by FAO and the World Bank, respectively. Part E – Project Management: Achievements under this component are rated Moderately Satisfactory. Supporting the PMU in carrying out project DEA received equipment and funding for operational support; a TAD coordination, implementation, monitoring and was hired by CLI and was housed in DEA for 2 years (2006-2008); evaluation, through the provision of technical and the above-mentioned training was provided to the PMU. advisory services, equipment and operational support. Table II.2: Outputs by Revised Components as presented in the Level 1 Restructuring June 30, 2011 Revised Components (Level 1 Restructuring Outputs achieved at the time of project closing June 30, 2011) 211 Part A – Identification and collection of obsolete pesticides and associated waste: Achievements under this component are rated Moderately Unsatisfactory. 211 Restructuring Paper on a Proposed Restructuring of the Africa Stockpiles Program to the Republic of South Africa, (Report No: 62795-ZA), June 20, 2011 173 Identification and collection of obsolete a. A booking line was established and piloted for Western Cape and pesticides and associated waste by: Free State provinces, before it was expanded to additional provinces a. Establishing a booking line where farmers in in 2012. The booking line was successful in identifying 120 tonnes selected provinces voluntarily declare obsolete of obsolete pesticides and associated waste from the public sector pesticides and associated waste in their (30%) and private sector (70%) (Intermediate Indicator #1 – 100%). possession; b. A risk assessment report and an EMP had been developed for b. Preparing EMPs; and Limpopo province, peer reviewed by DEA, an independent c. Collecting and transporting identified obsolete consultant and the ASP-South Africa NGO representative, and pesticides and associated waste to collection cleared by the World Bank on November 15, 2010. No EMPs were centers. prepared for the two additional provinces because the disposal contractor was not recruited prior to project closing (Intermediate Outcome Indicator #2 – 33%). c. Obsolete pesticides and associated waste were collected and transported to collection centers in Limpopo province, during a pilot operation in 2007. Collection of identified obsolete pesticides in Free State and Western Cape provinces was not conducted as the disposal contractor was not recruited prior to project closing (Intermediate Outcome indicator #3– 33%). Part B – Safeguarding and/or disposal of obsolete pesticides and associated waste: Achievements under this component are rated Moderately Unsatisfactory. Safeguarding and/or disposal of obsolete a. A study on disposal technology options was finalized in early 2012. pesticides stocks and associated waste by: The study involved identifying South Africa’s waste profile, a. Carrying out of a technological assessment of identification of environmentally sound waste management available safeguarding and disposal options; technologies available and benchmarking them with the best and international technologies. b. Implementing the safeguarding and/or disposal b. The hazardous portion of the Limpopo stocks (92.5 tonnes) was technology selected in selected provinces. shipped for incineration to the UK, and disposed of on August 22, 2011. A certificate of incineration was received as reported by CLI. The disposal of obsolete pesticides at the two remaining pilot provinces, Western Cape and Free State, was not completed as the disposal contractor had not been recruited prior to project closing (Revised PDO Indicator #2 – 33%). Part C – Capacity building: Achievements under this component are rated Moderately Satisfactory Carrying out of activities aimed at raising See Outputs by Original Components C, D and E. awareness on the health and environmental hazards of pesticides and strengthening the capacity of the PMU to implement the project through the provision of training. Part D – Project management: Achievements under this component are rated Moderately satisfactory Supporting the PMU in carrying out project See Outputs by Original Component E. A new TAD was not recruited coordination, implementation, monitoring and to support the PMU; however, the TSU supported the preparation of evaluation through the provision of technical the disposal bidding documents. advisory services, equipment and operational support. 8.3 ECONOMIC AND FINANCIAL ANALYSIS 2. As outlined in Section 3.4, an economic and financial analysis is not available. 174 8.4 WORLD BANK LENDINGA AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS a. Task Team Leaders Lending (Task Team, specialization, and unit as outlined in the PAD) Names Specialization Unit Responsibility/Specialty Abushakra, Hadi Chief Legal Counsel LEGMS Adebowale, Modupe Senior Finance Officer LOAG2 Allan, Christine Operations Analyst MNSRE Bakayoko, Siaka Senior Financial Management MNACS Specialist Brackmann, Stefanie Environmental Specialist MNSRE Chummun, Zakia Language Program Assistant MNSRE Di Leva, Charles Chief Counsel LEGEN Ebro, Kristyn Communications Officer ESDVP Gorman, Steve Lead Environmental Specialist ENVGC Kaniaru, Muthoni Counsel LEGAF Kranz, Frederick Senior Procurement Specialist MNACS Krishnakumar, V.S. Manager ASPTC Lagnaoui, Abdelaziz Senior Environmental Specialist ESDQC Lotayef, Dahlia Senior GEF Operations MNSRE Coordinator Maber, Steven Senior Operations Officer MNSRE TTL Mani, Angeline Language Program Assistant MNSRE Melkonian, Hovsep Senior Finance Officer LOAG2 Newton, Murray Consultant ENVMP Olowo-Okere, Edward Lead Financial Management AFTFM Specialist Opsal, Knut Senior Social Development MNSRE Specialist Parish, Matthew T. Counsel LEGMS Tynan, Ellen Environmental Specialist ENVMP Warner, Christopher Senior Environmental Specialist AFTS1 Supervision/ICR Names Specialization Unit Responsibility/Specialty Aka, Henri Procurement Specialist Aryal, Dinesh Senior Operations Officer TTL Baimu, Evarist Senior Counsel Bouzaher, Aziz Lead Environmental Specialist Co-TTL Brackmann, Stefanie Environmental Specialist Butler, Yesmeana Program Assistant 175 Chitalu, Fenwick Financial Management Specialist da Silva, George Disbursement Specialist Gxaba, Thandiwe Senior Environmental Specialist Harlley, Jemima Program Assistant Jordy, Denis Senior Environmental Specialist Co-TTL Lagnaoui, Abdelaziz Senior Environmental Specialist Maber, Steve Senior Operations Officer TTL McCue, Theresa Disbursement Analyst Morita, Sachiko Counsel Mphande, Donald Financial Management Specialist Mpundu, Marjorie Senior Counsel Msiwa, Tandile Gugu Financial Management Specialist Nieuwoudt, Christiaan Finance Analyst Johannes Odendaal, Erika Program Assistant Peled, Ayala Environmental Economist Consultant Prevoo, Dirk Senior Operations Officer TTL Rechbauer, Gabriele Senior Environmental Operation Consultant Specialist Schmidt, Veruschka Strategy Officer CFRPA Tete, Patrick Umah Senior Financial Management Specialist Tynan, Ellen Environmental Specialist TTL b. Staff time and cost – Staff time and costs associated with project preparation and supervision was not possible to determine as a result of poor recording in the system. 8.5 BENEFICIARY SURVEY RESULTS 3. Beneficiary survey results are not available, as no survey was conducted. 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS 4. A specific completion, and/or ICR mission was not carried out, and thus no stakeholder workshop was held. 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR 5. The recipient submitted a completion report; messages contained therein were integrated into this document. 6. The draft ICR was shared with the borrower; however, no comments were received. 8.8 COMMENTS OF COFINANCIERS AND OTHER PARTNERS/STAKEHOLDERS 7. No comments were received from co-financiers and other partners/stakeholders on the ASP-P1 South Africa draft ICR. 176 8. Comments of co-financiers and other partners/stakeholders on the program-level ICR are summarized in Section 8.8 in the program-level ICR. 177 TECHNICAL ANNEX V: ASP-P1 TANZANIA 178 ABBREVIATIONS AND ACRONYMS AfDB African Development Bank APL Adaptable Program Lending ASP-P1 Africa Stockpiles Program – Project 1 CAS Country Assistance Strategy CESA Country Environmental and Social Assessment CLI CropLife International DDT Dichlorodiphenyltrichloroethane DNOC Dinitro-o-Cresol DoE Division of Environment EMP Environmental Management Plan EMTK Environmental Management Tool Kit FAO United Nations Food and Agriculture Organization FM Financial Management GEF Global Environment Facility GEO Global Environment Objective GIZ Deutsche Gesellschaft für Internationale Zusammenarbeit IMDG International Maritime Dangerous Goods IPCS International Program on Chemical Safety IPM Integrated Pest Management IVM Integrated Vector Management M&E Monitoring & Evaluation MAFSC Ministry of Agriculture, Food Security, and Cooperative MoA Ministry of Agriculture NEMC National Environment Management Council NGO Non-Governmental Organization NSAF National Stakeholders Advisory Forum PAD Project Appraisal Document PAN Pesticides Action Network PCT Project Coordination Team PDO Project Development Objective 179 POP Persistent Organic Pollutants PSMS Pesticide Stock Management System SIDA Swedish International Development Cooperation Agency TAD Technical Advisor for Disposal TORs Terms of Reference TSU Technical Support Unit TTL Task Team Leader UNEP United Nations Environment Program UNITAR United Nations Institute of Training and Research WWF World Wildlife Fund 180 TABLE OF CONTENTS 1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN ...... 190 1.1 CONTEXT AT APPRAISAL ............................................................................................................................................ 190 1.2 ORIGINAL PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS ................................................................ 191 1.3 REVISED PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS .................................................................. 192 1.4 MAIN BENEFICIARIES................................................................................................................................................... 193 1.5 ORIGINAL AND REVISED COMPONENTS ................................................................................................................. 193 1.6 OTHER SIGNIFICANT CHANGES ................................................................................................................................. 194 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ..................................................................... 195 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY ............................................................................ 195 2.2 IMPLEMENTATION ........................................................................................................................................................ 196 2.3 MONITORING AND EVALUATION (M&E) ................................................................................................................. 198 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE ......................................................................................................... 198 2.5 POST-COMPLETION OPERATION/NEXT PHASE ...................................................................................................... 200 3. ASSESSMENT OF OUTCOMES ......................................................................................................................................... 200 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION ...................................................................... 200 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE ................................................................................... 201 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE ................................................................................... 204 3.4 EFFICIENCY .................................................................................................................................................................... 204 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING................................................................................................. 205 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS ............................................................................ 205 3.7 SUMMARY OF FINDING OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS ...................... 206 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME .......................................................................................... 206 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE .................................................................. 206 5.1 WORLD BANK PERFORMANCE .................................................................................................................................. 206 5.2 BORROWER PERFORMANCE....................................................................................................................................... 207 6. LESSONS LEARNED ............................................................................................................................................................ 207 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS ................ 208 8. ANNEXES ............................................................................................................................................................................... 208 8.1 PROJECT COSTS AND FINANCING ............................................................................................................................. 208 8.2 OUTPUTS BY COMPONENT ......................................................................................................................................... 209 8.3 ECONOMIC AND FINANCIAL ANALYSIS .................................................................................................................. 213 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS ................................. 213 8.5 BENEFICIARY SURVEY RESULTS .............................................................................................................................. 214 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS............................................................................................ 214 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR ................................................................. 215 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS ................................................... 216 8.9 VERBATIM COMMENTS ON DRAFT ICR ................................................................................................................... 217 181 A. Basic Information 212 Country: Tanzania Project Name: Africa Stockpiles Program - Project 1 Project ID: P103189 L/C/TF Number: TF56693 ICR Date: 06/28/2013 ICR Type: Core ICR Lending Instrument: APL Borrower: United Republic of Tanzania Original Total US$6.87 Disbursed Amount: US$6.87 million (05/31/2013) Commitment: million Revised Amount: - Environmental Category: A Global Focal Area: CHEM Implementing Agency: National Environment Management Council (NEMC) Co-financiers and Other External Partners: Government of the United Republic of Tanzania, Food and Agriculture Organization (FAO), CropLife International (CLI), World Wildlife Fund (WWF), and Pesticide Action Network (PAN-UK and PAN-Africa). B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/17/2002 Effectiveness: 02/12/2007 12/19/2006 Appraisal: 04/19/2004 Restructuring: 06/30/2011 Approval: 12/05/2006 Mid-term Review: 08/31/2008 10/16/2009 Closing: 06/30/2010 12/31/2011; 12/31/2012; 05/31/2013. C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Global Environment Outcome Moderate World Bank Performance: Moderately Unsatisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of World Bank and Borrower Performance World Bank Ratings Borrower Ratings Quality at Entry: Moderately Unsatisfactory Government: Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Satisfactory Agency/Agencies: Overall World Bank Moderately Unsatisfactory Overall Borrower Satisfactory 212 The information presented in the data sheet was inserted manually. Information provided in the system was incomplete resulting from coding issues experienced during implementation; these were not resolved prior project closing. 182 Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators - Performance (if any) Potential Problem Project at Quality at Entry Yes - any time (Yes/No): (QEA): Problem Project at any time Quality of Yes - (Yes/No): Supervision (QSA): PDO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total World Bank financing) Sanitation 33.3 33 Health 33.3 33 General Agriculture 33.3 34 Theme Code (as % of total World Bank financing) Environmental policies and institutions 50 50 Pollution management and environmental health 50 50 E. World Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Gobind T. Nankani Country Director: Philippe Dongier Mark D. Tomlinson Regional Integration Colin Bruce - Director Sector Director - Michel Wormser Sector Manager: Magda Lovei Mary Barton-Dock Project Team Leader: Dinesh Aryal Peter Kristensen/Denis Jordy ICR Team Leader: Dinesh Aryal ICR Primary Author: Veruschka Schmidt F. Results Framework Analysis - At project approval, no country-specific results framework existed; only six PDO Indicators were defined. Following a Level 1 Restructuring in June 2011, a country-specific results framework was introduced. The original six PDO Indicators were revised and reduced to three PDO Indicators, and in addition, six Intermediate Outcome Indicators were introduced. 183 Project Development Objective (PDO) – as presented in the GEF Trust Fund Grant Agreement: 213 To assist the recipient in: (a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and (b) implementing measures to reduce and prevent future related risks. Revised PDO – as presented in the Level 1 Restructuring of June 2011 (Report No: 62796-TZ): Assist the recipient in: (i) disposing and/or safeguarding inventoried publicly held obsolete pesticide stocks and associated waste; and (ii) developing a strategy for sustainable management of future accumulations. Global Environment Objectives (GEO) - as presented in the PAD (Report No: 36273-AFR): Reduction of the effects of Persistent Organic Pollutants (POPs) on the global environment. Revised GEO – Not Applicable (a) PDO Indicators – Original PDO Indicators as presented in the GEF Trust Fund Grant Agreement are assessed against achievements realized at the time of the project’s Level 1 restructuring in June 2011; revised PDO Indicators are assessed against achievements realized at project closing in May 2013. Original Target Formally Actual Value Achieved at Values Indicator Baseline Value Revised Target Completion or Target (from approval Values Years documents) Original PDO Indicators Inventory database of publicly held obsolete pesticides stocks in place and being Indicator 1: used by the Project Coordination Team (PCT) and the recipient’s government staff. Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 06/30/2010 - 11/20/2009 Target fully achieved (100%): An inventory was completed and validated in 2009, Comments (incl. % and data was entered into the FAO web-based Pesticide Stock Management System achievement) (PSMS). The database was subsequently used for site prioritization, preparation of the CESA, and disposal activities. Completion of Country Environmental and Social Assessment (CESA), and Indicator 2: implementation of the measures contained therein. Value (quantitative No Yes - No or qualitative) Date Achieved 04/19/2004 06/30/2010 - 06/30/2011 Comments (incl. % Target not achieved (0%): A CESA was still being finalized and not completed at achievement) restructuring. Indicator 3: Inventoried publicly held obsolete pesticides stocks disposed of, as a result of the 213 The PDO, indicators, and components that are stated in the Project Appraisal Document (PAD) slightly differ from those stated in the GEF Trust Fund Grant Agreement. This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities. 184 completion of the disposal services contract(s) in accordance with national and international laws and regulations. Value (quantitative 0% 100% - 0% or qualitative) Date Achieved 04/19/2004 06/30/2010 - 06/30/2011 Target not achieved (0%): A draft disposal tender document had been prepared and Comments (incl. % submitted to the World Bank for review in May 2011. None of the inventoried achievement) publicly held obsolete pesticide stocks had been disposed of at restructuring. Legal and regulatory framework for pesticide management improved, including Indicator 4: measures to strengthen compliance with the Conventions. Value (quantitative No Yes - No or qualitative) Date Achieved 04/19/2004 06/30/2010 - 06/30/2011 Comments (incl. % Target not achieved (0%): The legal and regulatory framework for pesticide achievement) management had not been improved by the time of restructuring. Indicator 5: Integrated Pest Management (IPM) strategy is developed and implemented. Value (quantitative No Yes - No or qualitative) Date Achieved 04/19/2004 06/30/2010 - 06/30/2011 Target not achieved (0%): An IPM strategy was neither developed nor implemented, Comments (incl. % because in 2010, it was agreed by the PMU and the World Bank that instead of achievement) developing an IPM strategy, a national workshop on IPM would be held. However, the workshop had not been held by the time of restructuring. The training program under Part D 214 of the project is implemented and the Indicator 6: knowledge so acquired is being used by the recipient. Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 06/30/2010 - 06/30/2011 Target fully achieved (100%): PMU members received training on project Comments (incl. % management, conducting an inventory and PSMS, preparing a CESA, safeguards, achievement) storekeeping, and procurement. The acquired knowledge was continuously applied. Revised PDO Indicators as per the Level 1 Restructuring in June 2011 Quantity of inventoried publicly held obsolete pesticides and associated waste Indicator 1: disposed of and/or safeguarded. Value (quantitative 700 tonnes (gross) 0 - 575 tonnes (net) or qualitative) 665 tonnes (net) Date Achieved 06/30/2011 05/31/2013 - 05/31/2013 Target fully achieved (100%): 575 tonnes (net) of the inventoried 665 tonnes (net) of Comments (incl. % publicly held obsolete pesticides and associated waste were disposed of. The original achievement) quantity of 665 tonnes of obsolete pesticides turned out to be an overestimate, and 214 See Section 1.5 for the project’s Component D (in the GEF Trust Fund Grant Agreement referred to as ‘Part D’). 185 the actual value was 575 tonnes. Thus, with disposal of 575 tonnes, the target was fully achieved. Quantity of inventoried heavily contaminated soil disposed of and/or Indicator 2: safeguarded. Value (quantitative 0 300 tonnes (net) - 392 tonnes (net) or qualitative) Date Achieved 06/30/2011 05/31/2013 - 05/31/2013 Comments (incl. % Target exceeded (131%): 392 tonnes (net) of the inventoried 300 tonnes (net) of achievement) heavily contaminated soil were disposed of. Comprehensive strategy for dealing with future stocks of obsolete pesticides and Indicator 3: associated wastes adopted by the steering committee. Strategy finalized Strategy finalized Value (quantitative No strategy in place and adopted by the - and adopted by the or qualitative) Steering Committee Steering Committee Date Achieved 06/30/2011 05/31/2013 - 05/31/2013 Target fully achieved (100%): The comprehensive strategy included action plans on pesticide management, IPM, and container management, finalized and adopted by the Comments (incl. % Steering Committee on April 22, 2013; and draft legislation for a Plant Protection achievement) Bill and a Pesticide Management Bill, submitted to the Ministry of Agriculture for review. (b) GEO Indicator(s) –The GEO was measured by the PDO and PDO indicators. (c) Intermediate Outcome Indicator(s) Original Target Actual Value Values (from Formally Revised Target Achieved at Indicator Baseline Value approval Values Completion or Target documents) Years Validated inventory data entered on Pesticide Stock Management System Indicator 1: (PSMS) Value (quantitative Yes - - Yes or qualitative) Date Achieved 06/30/2011 - - 11/20/2009 Indicator corresponds to original PDO Indicator #1 and was completed prior to Comments (incl. % restructuring. Therefore, the indicator is not taken into account for post-restructuring achievement) assessment. Indicator 2: Country Environmental and Social Assessment (CESA) completed Value (quantitative No Yes - Yes or qualitative) Date Achieved 06/30/2011 05/31/2013 - 06/27/2012 Comments (incl. % Indicator corresponds to original PDO indicator #2. Target fully achieved (100%): A 186 achievement) CESA was completed on June 15, 2012, and disclosed to the World Bank InfoShop on June 27, 2012. Indicator 3: Four communications actions implemented Value (quantitative 0 4 - 4 or qualitative) Date Achieved 06/30/2011 05/31/2013 - 05/31/2013 Target fully achieved (100%): Awareness was raised through: (i) two TV spots aired Comments (incl. % in 2013; (ii) two radio spots aired in 2013; (iii) participation in five exhibitions; (iv) achievement) outreach visits to three zones in Tanzania. A final draft container management plan submitted for endorsement by the Indicator 4: Steering Committee No. A study Value (quantitative conducted and Yes - Yes or qualitative) reviewed. Date Achieved 06/30/2011 05/31/2013 - 04/22/2013 Comments (incl. % Target fully achieved (100%): A container management plan was drafted, and achievement) endorsed by the Steering Committee on April 22, 2013. A final draft sustainability roadmap developed and submitted for endorsement Indicator 5: by the Steering Committee Value (quantitative No Yes - Yes or qualitative) Date Achieved 06/30/2011 05/31/2013 - 04/22/2013 Target fully achieved (100%): A draft sustainability roadmap was finalized and Comments (incl. % endorsed by the Steering Committee in June 2010. The roadmap identified key steps achievement) towards sustainable pesticide management in Tanzania. Three training and/or field visits implemented on improved pesticide Indicator 6: management practices Value (quantitative 0 3 - 3 or qualitative) Date Achieved 06/30/2011 05/31/2013 - 05/31/2013 Comments (incl. % Target achieved (100%): Field visits were conducted during disposal activities to achievement) three different zones to raise awareness on improved pesticide management practices. G. Ratings of Project Performance in ISRs – Some DO and IP ratings and actual disbursement data are not available because ISRs reported on ASP-P1 as a whole, and only occasionally on country-specific project performance, including disbursement. Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 10/31/2005 Not applicable as ASP-P1 Tanzania not yet launched 2 06/30/2006 187 3 12/27/2006 4 06/28/2007 Moderately Satisfactory Moderately Satisfactory - 5 12/18/2007 - - - 6 05/30/2008 - - - 7 12/18/2008 Moderately Unsatisfactory Unsatisfactory - 8 03/25/2009 Moderately Satisfactory Moderately Satisfactory - 9 12/29/2009 Moderately Satisfactory Moderately Satisfactory 0.82 10 06/29/2010 Moderately Satisfactory Satisfactory - 11 03/27/2011 - Satisfactory - 12 12/25/2011 Moderately Satisfactory Satisfactory - 13 06/29/2012 Satisfactory Satisfactory 2.73 13-1 12/09/2012 Satisfactory Satisfactory - 13-2 05/29/2013 Satisfactory Satisfactory - 05/31/2013 Final disbursement 6.87 H. Restructuring ISR Ratings at Amount Board Restructuring Disbursed at Restructuri Approved Reason for Restructuring ng Date(s) PDO Restructuring & Key Changes Made PDO IP in USD Change millions 07/01/2010 No Not Not Not • Extension of closing date by 18 months Available Available Available to compensate for delays (06/30/2010 to 12/31/2011). 06/30/2011 Yes Moderately Satisfactory 2.1 • Revision of the PDO and indicators to Satisfactory reflect a shift from complete disposal to disposal and safeguarding of stocks • Introduction of a results framework for better M&E • Extension of closing date by 12 months to compensate for delays (12/31/2011 to 12/31/2012). • Reallocation of proceeds to accommodate field activities, training and workshops related to the prevention component. 12/13/2012 No Not Not Not • Extension of closing date by five months Available Available Available to allow time for the shipment of stocks to the incineration facilities thus avoiding contamination on transit; and 188 ISR Ratings at Amount Board Restructuring Disbursed at Restructuri Approved Reason for Restructuring ng Date(s) PDO Restructuring & Key Changes Made PDO IP in USD Change millions reception of certificates of disposal from the incineration facilities to enable payment and avoid reputational risks. (12/31/2012 to 05/31/2013). • Reallocation of proceeds to maximize the amount of hazardous materials to be disposed of. If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below: Outcome Rating Outcome Ratings Against Original PDO targets Unsatisfactory Against formally revised PDO targets Highly Satisfactory Overall (weighted) rating Satisfactory I. Disbursement Profile – A disbursement profile is not available (see explanation under G). 189 1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN 1.1 CONTEXT AT APPRAISAL 1. Country background: Tanzania’s agriculture sector had been highly labor-intensive, employing an estimated two-thirds of the country’s workforce. At project approval in December 2006, the sector accounted for just under 50% of GDP, with considerable under-utilized land, and the land that was in use, had been dominated by small-scale subsistence farmers. 215 2. Sector background: Following their introduction in the 1940s and 1950s, pesticides in Tanzania had been primarily used in agriculture for crop protection, as pests accounted for more than 40% of crop losses annually. The launching of the economic recovery programs and liberalization of trade in Tanzania had resulted in a rise of the amount of pesticides imported into the country from 500 tonnes in 2000 to 2,500 tonnes in 2003. To a lesser extent, pesticides had been used in the public health sector and other areas including for protecting buildings from insect pests. 3. Obsolete pesticide accumulation: Tanzania had accumulated large stockpiles of obsolete pesticides as a result of incorrect needs assessment and procurement; uncoordinated donations of pesticides; poor storage conditions; and poor stock management. 4. Institutional background: Pesticide legislation was updated in 1997 to reflect international agreements. 216 The Ministry of Agriculture, Food Security, and Cooperative (MAFSC) was responsible for approval and registration of pesticides, while the National Environmental Management Council (NEMC) represented the principal agency in responsible for regulating the environment, including chemical waste and disposal. The Division of Environment (DoE) – Vice Presidents’ Office (under which NEMC was situated) had the overall function of formulating policies on environment, coordination and monitoring environmental issues, environmental planning and carrying out policy oriented environmental research. At approval, the importation and distribution of pesticides was handled by the private sector, while the Government concentrated on regulating and controlling the chain. Although some of the pesticides used in agriculture were similar to those used in public health, pesticides for animal husbandry and public health were regulated under the Ministry of Livestock Development and Ministry of Health, respectively. 217 The Office of the Vice-President, and the ministries had adopted legal and regulatory provisions to improve the management of pesticides and to prevent impacts on human health and the environment.218 5. Actions taken: Considerable efforts had been undertaken by the government to safeguard and dispose of obsolete pesticides,219 before in 1997-1998, the Government of the Netherlands supported 215 The Economist Intelligence Unit, Country Profile Tanzania, 2006 216 The Plant Protection Act was enacted in 1997, dealing specifically with plant protection and pesticide control. Regulations were put in place in 1999 but enforcement of the Act was weak. 217 Pesticide and Poverty – A Case Study on Trade and Utilization of Pesticides in Tanzania: Implication to Stockpiling, Agenda for Environment and Responsible Development in collaboration with Pesticide Action Network UK, August 2006. 218 These included the Environmental Management Act (2004), which included a provision for the development of a regulation for hazardous waste management, the Occupational Safety and Health Act (2003), which ensured prevention to workers exposure to chemical and pesticide substance, and the Industrial and Consumer Chemical Act (2003). 219 In example, in 1986, the Swedish International Development Cooperation Agency (SIDA) provided assistance to repack large quantities of pesticides at the coastal region; in 1995, the Government of the Netherlands funded the removal of hazardous waste 190 Tanzania in carrying out a nationwide inventory of obsolete stockpiles, which identified about 1000 tonnes of obsolete pesticides. 70% of the pesticides were estimated to be publicly held and included large volumes of organ-chlorinated products, notably Dichlorodiphenyltrichloroethane (DDT). 220 The majority of the stores did not meet the basic requirements for environmentally sound storage of hazardous chemicals as set by the Food and Agriculture Organization (FAO) and the United Nations Environment Program (UNEP). Several other activities were subsequently undertaken. These included an inventory that was organized in collaboration with UNEP to assess status of specifically POPs in Tanzania, a capacity-building project on risk management decision-making undertaken by the Ministry of Health 221, and Integrated Pest Management (IPM) and Integrated Vector Management (IVM) programs organized by the Ministry of Agriculture. 222 In recognition of the problem identified by this inventory, Tanzania made an official appeal for assistance to the international donor community during the FAO Third Consultation on Prevention and Disposal of Obsolete Pesticide Stocks, held in Rome in March 1998. Tanzania was subsequently considered for ASP participation, and a country workshop was held in August 2002 in Dar es Salaam to discuss and evaluate the scope of the pesticide problem. 6. Rationale for World Bank assistance: Please refer to Section 1.1 in the program-level ICR. 7. Higher-level objectives: The project intended to support Tanzania’s Country Assistance Strategy (CAS) 223 to help realizing an environmentally sustainable rural development strategy, and support commercial agriculture to become more dynamic and productive. The project further aimed to assist the government in implementing policies focusing on the conservation of natural resources and management of the environment, and raising awareness for the sustainable use of natural resources. Contributing to international efforts to eliminate POPs, improving management of toxic chemicals, and promoting alternatives to pesticide usage that include IPM and IVM represented further objectives. 1.2 ORIGINAL PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS 8. The Project Development Objective (PDO), indicators, and components that are stated in the ASP-P1 Tanzania Project Appraisal Document (PAD) 224 differ from those stated in the GEF Trust Fund Grant Agreement. 225 This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the guiding document for the project’s implementation activities. Table 1.1: ASP-P1 Tanzania Original PDO Original PDO as presented in the PAD Original PDO from the islands of Zanzibar and Pemba; also in 1995, the Deutsche Gesellschaft für Internationale Zusammenarbeit (GIZ) provided technical assistance to incinerate 57 tonnes of Dinitro-o-Cresol (DNOC) in a cement kiln in Dar es Salaam. 220 Dichlorodiphenyltrichloroethane (DDT) 221 With support of the United Nations Institute of Training and Research (UNITAR) and the International Program on Chemical Safety (IPCS) 222 With support of the GIZ. 223 Country Assistance Strategy for the United Republic of Tanzania, (Report No. 16554-TA), May 6, 1997 224 Project Appraisal Document for the Republic of Mali, the Kingdom of Morocco, and the United Republic of Tanzania in support of the Africa Stockpile Program – Technical Annex 3, (Report No. 36273-AFR), November 7, 2006 225 GEF Trust Fund Grant Agreement between the United Republic of Tanzania and International Bank for Reconstruction and Development, (Number TF056693-TA), 2006 191 as presented in the GEF Trust Fund Grant Agreement To assist the Government of Tanzania to dispose of To assist the recipient in: (a) eliminating inventoried publicly publicly held obsolete pesticide stocks at priority- held obsolete pesticide stocks and associated waste; and (b) selected sites and to support priority actions to help implementing measures to reduce and prevent future related prevent the future accumulation of new stocks of risks. obsolete pesticides. 9. At appraisal, only an overall ASP-P1 results framework was prepared (see program-level ICR); results frameworks for the individual ASP-P1 countries, including Tanzania, were intended to be finalized during the individual ASP-P1 country project launches. However, country-specific PDO indicators were developed at appraisal. For ASP-P1 Tanzania, these were as follows: Table 1.2: ASP-P1 Tanzania Original PDO Indicators Original PDO Indicators as presented in the PAD and the GEF Trust Fund Agreement a. Inventory database of publicly held obsolete pesticides stocks in place and being used by the Project Coordination Team’s (PCT) and the recipient’s government staff. b. Completion of Country Environmental and Social Assessment (CESA) and implementation of measures contained therein. c. Inventoried publicly held obsolete pesticides stocks disposed of, as a result of the completion of the disposal services contract(s) in accordance with national and international laws and regulations. d. Legal and regulatory framework for pesticide management improved, including measures to strengthen compliance with the Conventions. e. Integrated Pest Management (IPM) strategy is developed and implemented. f. The training program under Part D of the project is implemented and the knowledge so acquired is being used by the recipient. 226 1.3 REVISED PROJECT DEVELOPMENT OBJECTIVE AND INDICATORS 10. The PDO and associated indicators for ASP-P1 Tanzania were revised through a Level I restructuring on June 30, 2011. 227 The original objective was based on the incorrect assumption that project funds would be sufficient to dispose of all inventoried stocks, and that demand for pesticides could be substantially reduced through project-funded prevention activities despite evidence of increasing demand for pesticides as a result of agricultural intensification. After it became evident that there were higher than estimated volumes of associated waste, particularly contaminated soils, and that future stocks could not be fully prevented, the original PDO, indicators and related components were revised to reflect a shift from complete elimination of all inventoried publicly-held obsolete stocks and associated waste to elimination overseas and lower-cost in-country safeguarding of the stocks. This also reflected a transition from prevention of accumulation of stocks to the development of a strategy to deal with future obsolete pesticides and associated waste re-accumulation, endorsed by the government. 226 See Section 1.5 for the project’s Component D, here referred to as “Part D”. 227 Revisions were approved on June 30, 2011, on the basis of: Restructuring Paper on a Proposed Project Restructuring of Africa Stockpiles Program – Project 1 to the United Republic of Tanzania, (Report No: 62796-TZ), June 20, 2011 192 11. ASP-P1 Tanzania’s revised PDO: Assist the recipient in: (i) disposing and/or safeguarding inventoried publicly held obsolete pesticide stocks and associated waste; and (ii) developing a strategy for sustainable management of future accumulations. 12. ASP-P1 Tanzania PDO indicators were reduced in number and revised to focus on the updated project objective. In addition, a country-specific results framework was introduced for effective monitoring and evaluation of project progress. 13. ASP-P1 Tanzania’s revised PDO Indicators: (i) Quantity of inventoried publicly held obsolete pesticides and associated waste disposed of and/or safeguarded; (ii) quantity of inventoried heavily contaminated soil disposed of and/or safeguarded; and (iii) comprehensive strategy for dealing with future stocks of obsolete pesticides and associated wastes adopted by the steering committee. 1.4 MAIN BENEFICIARIES 14. Please refer to Section1.4 in the program-level ICR. 1.5 ORIGINAL AND REVISED COMPONENTS 15. The PDO of ASP-P1 Tanzania was to be achieved through the implementation of four distinct components (outlined in Table 1.3). The table also shows revisions that were made through a Level I restructuring on June 30, 2011. Table 1.3: ASP-P1 Tanzania Original and Revised Components Original Components as presented Original Components as presented in the Revised Components (Level I in the PAD GEF Trust Fund Grant Agreement Restructuring June 30, 2011) Component 1.A – Cleanup and Part A – Obsolete pesticide inventory: Part A – Obsolete pesticide disposal (Total: US$5.83 million; a. Collection of data and compilation of a inventory: No changes were GEF: US$5.48 million): data base; introduced at restructuring. a. Designing an obsolete pesticide b. A risk-based prioritization of the stocks data base, inventory and risk identified in the data base; and assessment for free standing c. Preparation of the CESA. pesticides, contaminated sites, soils, buried pesticides and contaminated equipment; b. Producing a CESA; Part B – Disposal of obsolete pesticides: Part B – Disposal and/or c. Safeguarding obsolete pesticide a. Carrying out of a technological safeguarding of obsolete pesticides stocks; and assessment of available treatment and and associated waste: d. Disposing of the identified, disposal options; a. Carrying out a technological publicly held, obsolete b. Implementation of the treatment and/or assessment of available treatment pesticides. disposal technology selected; and disposal options; c. Improving the operations of obsolete b. Implementing the safeguarding pesticide stocks collection centers and/or disposal technology including store stabilization and safety selected; and measures at those centers; and c. Improving the operations of d. Container management. obsolete pesticide stocks collection centers, including store stabilization and safety measures at those centers. The changes reflected the addition of 193 safeguarding as an outcome and clearly defined associated waste as an integral part of the stocks to be treated. Component 1.B - Prevention of Part C – Prevention of obsolete pesticide Part C - Reduction of obsolete obsolete pesticide accumulation accumulation: pesticide re-accumulation: (Total: US$0.28 million; GEF: a. Awareness raising activities on the a. Awareness raising activities on the US$0.27 million): Program, and health and environmental program, and health and a. Raising awareness of ASP and hazards of pesticides; environmental hazards of obsolete pesticide disposal b. A review and update of the legal and pesticides; options; regulatory framework to control b. A review and update of the legal b. Reviewing and revising pesticide Obsolete Pesticide accumulation and and regulatory framework to laws and regulations; and strengthen enforcement mechanisms; and control obsolete pesticide c. Producing and piloting c. Preparation and implementation of an accumulation and strengthen implementation of an IPM IPM strategy. enforcement mechanisms; strategy. c. Preparation and implementation of Component 1.C - Capacity Part D – Capacity building: an IPM strategy; and building (Total: US$0.15 million; a. Strengthening the recipient’s capacity to d. Preparation of a pesticide and GEF: US$0.14 million): Approx. carry out environmental assessments; container management strategies. 14 training courses were planned to b. Reviewing the recipient’ s pesticide The change reflected the transition be provided to the government, management practices and providing from prevention (zero new farmers and the NGO sector. The training on improved methodologies; and accumulation) to reduction of new training and capacity building c. Reviewing the implementation capacity accumulations and the addition of a intended to strengthen: of the PCT and providing requisite pesticide and container management a. Disposal activities including training. strategies as key outputs. inventory, risk assessment, CESA and disposal; b. Prevention activities including pesticide legislation, management and IPM; and c. Project management and monitoring and evaluation. Component 1.D - Project Part E – Project management: Supporting management and monitoring the PCT in carrying out Project (Total: US$1.22 million; GEF: coordination, implementation, monitoring US$0.99 million): and evaluation, through the provision of Provide overall project technical advisory services, equipment and management, monitoring and operational support. reporting; and Finance the project coordination team and the project for effective implementation. 1.6 OTHER SIGNIFICANT CHANGES Table 1.4: ASP-P1 Tanzania Other Significant Changes Change Date Justification Approval Extension of closing date 194 Extension of closing date by 18 07/01/2010 To compensate for start-up delays, and allow completion Country months from 06/30/2010 to of the disposal contract, taking into account a potential Director 12/31/2011. lag time between obsolete pesticides being exported from the country and the date of actual incineration (up to 12 months), which was inadequately assessed during appraisal. Extension of closing date by 12 06/30/2011 To compensate for delays experienced during the Board months from 12/31/2011 to disposal tender process and allow completion of all core 12/31/2012. activities, including the treatment of large quantities of highly contaminated soils. Extension of closing date by 5 12/13/2012 To allow time for the shipment of stocks to the Regional months from 12/31/2012 to incineration facilities thus avoiding contamination on Vice 05/31/2013. transit; and reception of certificates of disposal from the President incineration facilities to enable payment and avoid reputational risks. Reallocation of proceeds Reallocation of proceeds from the 06/30/2011 To accommodate field activities, training and workshops Board categories Goods (-US$60,000), related to the prevention component to take place during Works (-US$40,000), Consultant the extended period. Services (-US$50,000) and Unallocated (-US$620,000) to the categories Training and Workshops (+US$370,000) and Operating Costs (+US$400,000). Reallocation of proceeds from the 12/13/2012 To maximize the amount of hazardous materials to be Regional categories Consultant Services (- disposed of. Vice US$170,000), Training and President Workshops (-US$285,000) and Operating Costs (-US$90,000) to the categories Goods (+US$85,000) and Disposal Services (+US$460,000). Redefinition of activities Redefinition of the term “Disposal 06/30/2011 Addition of in country safeguarding as a possible Board Services” from services related to cheaper end result for lower risk stocks as a result of the treatment or disposal of larger than budgeted stocks of obsolete pesticides and obsolete pesticides and associated associated waste. waste, to services related to the safeguarding and/or disposal of the same. 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY 16. ASP-P1 Tanzania was prepared between 2002 and 2006, as part of preparatory activities at the program-level. It took more than four years from GEF eligibility in July 2001, and more than three 195 years from concept note in July 2002 to project approval by the World Bank Board in December 2006. 228 17. Soundness of background analysis: Project preparation was conducted jointly by FAO and the World Bank. Preparation activities were further characterized by close collaboration with CropLife International (CLI) and the African Development Bank (AfDB), as mission reports show. 229 18. Background analysis was comprehensive. A preliminary procurement capacity assessment was conducted by the World Bank in July 2003. It was noted that no procurement specialist was envisaged to be part of PMU staffing; the World Bank correctly emphasized that procurement staff would be vital to avoid delays in project implementation. In May 2004, NEMC’s procurement capacity was assessed again, and considered inadequate. The World Bank recommended that NEMC hire a procurement specialist on a short-term basis for twelve months to support the day-to-day procurement. Regular procurement training to improve procurement skills was further recommended. Assessments of the project’s management arrangements and Financial Management (FM) capacity were equally undertaken. Key preparatory outputs such as the PAD and Operational Manual, however, were lacking detailed country-specific information regarding Monitoring and Evaluation (M&E), and risks and risk mitigation measures. 19. Assessment of project design: The project’s design (as for all ASP-P1 participating countries) was preconfigured at the program-level, and did not reflect any country-specific modifications or lessons learned from pervious projects executed in Tanzania. Design weaknesses at the program level that translated into implementation delays at the country level, including ASP-P1 Tanzania, are outlined in Section 2.1 of the program-level ICR. 20. Government commitment: The Government of Tanzania had demonstrated strong commitment to increasing pesticide management efforts, which was translated into the above-mentioned projects and operations (see Section 1.1). In addition, before project approval, the government had ratified the Basel (April 1993), Rotterdam (April 2002), and Stockholm (April 2004) Conventions. 21. Assessment of risks: Program-level risks were adequately identified and outlined in the PAD. They were presented in the form of a matrix and represented a summary of risk assessments undertaken during identification, pre-appraisal and appraisal missions to ASP-P1 participating countries. Even though the assessment of risks was comprehensive and included a broad range of mitigation measures in generic terms, an assessment of risks and mitigation measures specific to Tanzania was lacking. At the same time, it should be noted that it was difficult to assess country-specific risks in the absence of inventories of obsolete pesticides and CESAs, which were only planned to be undertaken during project implementation. 2.2 IMPLEMENTATION 22. Throughout implementation, the project experienced long delays, eventually resulting in three extensions for a total of 35 months, and a Level 1 restructuring in 2011. 23. Key factors affecting implementation and outcomes included: 228 The long preparation time resulted from a lengthy partnership building/negotiation process at the program-level resulting from the complex implementation arrangement and project design (see Section 2.1 in the program-level ICR). 229 Pre-Appraisal mission March 9-14, 2003 (NEMC, World Bank, and FAO); World Bank Appraisal mission April 25-30, 2004 (NEMC, World Bank, FAO, and CLI); and AfDB Appraisal mission July 15-20, 2005 (NEMC, AfDB, and World Bank). 196 • Low project management capacity affected project implementation in the beginning. The project was approved by the Board in December 2006 and became effective two weeks later. However, implementation started slowly due to a shortage of key personnel in the PMU, 230 substantial procurement delays due to the absence of a procurement officer during the first ten months of the project and a resulting lengthy procurement of personal protective equipment that was required for the inventory (see Section 2.4 for more detail). Ineffective communication within the PMU further contributed to initial project implementation delays. • Complex and loosely defined implementation arrangement at the program-level affected project implementation. The loosely defined implementation arrangement at the program level had caused collaboration issues between FAO and the World Bank, which subsequently affected the effective and timely technical and operational support the agencies were foreseen to provide (see Section 2.2 in the program-level ICR). Equally affected was the involvement of the local Non- Governmental Organization (NGO) network, whose Terms of Reference (TORs) and methods of procurement were unclear. Efforts were subsequently undertaken by the PMU, the Pesticide Action Network (PAN-UK) and World Bank to better integrate these activities into the project’s plans, and to provide additional guidance on procurement. For technical support during disposal activities, a Technical Advisor for Disposal (TAD) was eventually recruited and financed by CLI, which (although considerably delayed) positively affected project implementation. • Strong collaboration with the local NGO network positively supported project activities. A network of local NGOs with experience in health and environmental issues, formed in December 2002. Once integrated into the project plans, it substantially supported the project’s efforts in raising awareness at the local level through sensitizing selected communities, and training community members in independently identifying and monitoring chemical effects at the local level. • Discovery of large amounts of contaminated soils resulted in a funding gap. While the amount of obsolete pesticides inventoried was much lower than estimated at appraisal, large amounts of heavily contaminated soils had been discovered, which had not been budgeted for. In 2010, the project therefore faced an estimated funding gap of $2.4 million, for all inventoried stocks were disposed of adequately. The World Bank and the government subsequently agreed to dispose of obsolete pesticides and high-risk contaminated soils and safeguard low risk soil in country (as captured at restructuring). • Restructuring. At mid-term review in October 2009, the project was considered delayed and its PDO unlikely to be achieved on the basis of the original indicators (see Sections 1.2 and 1.3). The World Bank recommended revising the PDO, introducing a country-level results framework, and extending the project’s closing date. The project was eventually restructured in June 2011, and subsequently facilitated PDO achievement. 231 Further, in response to issues discussed during the mid-term review concerning disbursement, the ceiling for the designated account was increased from US$100,000 to US$200,000 based on projected expenditures. 230 The PMU coordinator position had only been appointed in January 2008. Until then, NEMC’s director had been coordinating the project in addition to his role as director, which led to a heavy workload and subsequent project implementation delays. 231 It took over two years to restructure the project, as ASP-P1 countries were planned to be restructured together and some of the other countries were delays due to political reluctance to restructure the project. 197 2.3 MONITORING AND EVALUATION (M&E) 24. Overall M&E – Rating Moderately Unsatisfactory based on the following evaluation. 25. M&E design – Rating Unsatisfactory: Despite the lengthy preparation time, at appraisal, an M&E system specific for Tanzania was not developed.232 A results framework, which was intended to be finalized during project launch and included in the project’s Operational Manual, did not materialize. Only a set of PDO indicators linked to the different project components was developed and outlined in the PAD and Operational Manual, however, these did not allow for adequate monitoring of project implementation progress. The indicators focused on measuring outputs as opposed to progress towards project outcomes. 233 Expected results under the prevention component, in particular, were not adequately considered by the indicators. Other key M&E elements, such as strategies, responsibilities, timelines, and budget were properly detailed in the Operational Manual. 26. M&E implementation – Rating Moderately Unsatisfactory: Until the mid-term review, the PMU compensated the lack of an M&E system with using Excel sheets to monitor project implementation. Eventually in 2009, the TSU provided the PMU with a simplified M&E system, which was subsequently used by the PMU. Monthly and bi-annual progress reports were timely prepared and consisted of quantitative and qualitative information on progress. In 2011, following the restructuring of the project, a results framework specific to Tanzania was finally introduced, and the PDO and associated indicators were revised to focus on the two main project achievements (disposal and/or safeguarding and reduction of future accumulations), which strengthened the M&E of the project, albeit late in the process. 27. M&E utilization – Rating Moderately Satisfactory: All identified sites in the country together with their respective stocks of obsolete pesticides, were subjected to risk assessments based on the Environmental Management Tool Kits (EMTKs) 234 developed by FAO. Data from each site was entered into the Pesticide Stock Management System (PSMS), a FAO web-based database, to calculate risk factors related to the product and environmental conditions, to prioritize each site, and to record pesticides. For the preparation of the CESA, the PSMS prioritization criteria were used to rank all sites into high, medium, and low risk. The PSMS together the EMTK proved to be very useful tools in managing obsolete pesticides stocks. 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE 28. Safeguard compliance – Rating Moderately Satisfactory: ASP-P1 triggered two safeguard policies: OP 4.01 on Environmental Assessment and OP 4.09 on Pest Management. 29. ASP-P1 Tanzania complied with OP 4.01 on Environmental Assessment: An environmental safeguard review that was conducted by the World Bank at mid-term review noted that during inventory activities, staff had not undergone health checks as required, and that contaminated soils had not been 232 Lengthy discussions between the partners during project preparation on whether to have an M&E system at the program-level alone, or also at the country-level, may have contributed to the absence of an M&E system at the country-level at project launch. However, no information was available on whether it was a deliberate decision not to have an M&E system in place at the country-level. 233 See Tanzania Africa Stockpiles Program – Project 1 (ASP-P1) – Joint Mid-Term Review Mission (October 5-16, 2009) Aide Memoire. 234 The tool kit provides a practical set of methodologies to assist countries in the risk-based management of obsolete pesticides, including guidance on the management of collection/storage locations, based on international best practice. 198 adequately safeguarded at two high-risk sites (Vikuge and Morogoro). 235 However, shortly after, corrective measures were undertaken; awareness-raising activities were conducted, and contaminated soils were adequately demarcated at these two sites. 236 Local NGOs significantly supported consultation activities and independent monitoring of health and safety compliance during inventory activities. 30. A CESA identifying mitigation measures to treat possible negative environmental and social impacts that were considered likely to occur during implementation of the project was completed in October 2011, with guidance of a World Bank safeguards specialist. The assessment had been found satisfactory by the World Bank, and associated Environmental Management Plans (EMPs) that were linked to a Health and Safety Management System were equally completed. The assessment’s delayed completion 237 was caused by: a) the sequential design of the project (the inventory was only finalized in November 2009); b) procurement delays; and c) a lengthy World Bank review and clearance process, particularly of the TORs. Project activities involving storage, transportation, disposal, and site remediation were implemented in compliance with the recommendations and requirements of the CESA, the associated EMPs, and the Health and Safety Management System which ensured that all workers were adequately protected against chemical hazards and using safe working practices. A World Bank safeguard specialist monitored and ensured compliance with safeguard requirements through participation in supervision missions and specific safeguard missions. 31. OP 4.09 on Pest Management: Since project activities themselves met requirements of a pest management plan (i.e., through pesticide management, IPM, and capacity building), individual country projects, including ASP-P1 Tanzania, were not required to prepare self-standing documents. See Section 2.4 in the program-level for more details. 32. Financial Management compliance – Rating Moderately Satisfactory: Financial Monitoring Reports (FMRs) were prepared and submitted on a quarterly basis to the World Bank’s satisfaction, as were audit reports. Shortcomings included occasional variances that were found during financial management reviews but which were subsequently corrected; and a missing audit committee during the first year of project implementation, which was eventually established in March 2008 following World Bank recommendations. 238 33. Disbursement: Disbursement levels were low during the first three years of project implementation with only 13% disbursement at mid-term review. A six months delay in opening a special account and a low account ceiling (US$100,000) contributed to low disbursement rates as did general delays in effecting disbursements, and procurement shortcomings (see below). Disbursement levels only increased substantially towards the end of the project, attributable to the increase in the ceiling (US$200,000) and the beginning of disposal activities (which were allocated the majority of project funds). 235 See Annex 3 on Safeguards in Tanzania Africa Stockpiles Program – Project 1 (ASP-P1) – Joint Mid-Term Review Mission (October 5-16, 2009) Aide Memoire. 236 See Annex 5 Safeguard Compliance Assessment in Tanzania Africa Stockpiles Program – Project 1 (ASP-P1) – Implementation Support Mission (November 8-12, 2010) Aide Memoire. 237 In relation to the project having been effective in December 2006. 238 A World Bank mission in 2007 had noted that an audit committee is required to be in place according to the 2001 Public Finance Act. 199 34. Procurement – Rating Moderately Satisfactory: All services were procured to the World Bank’s satisfaction; however, occasional delays and discrepancies in procurement plans delayed overall project activities, particularly during the first half of project implementation. Initial delays were caused by the absence of a procurement specialist during the first ten months of implementation. 239 The subsequent arrangement, which included a procurement officer, supplies officer, and a part-time procurement consultant, did not prove to be effective; the consultant’s support was not sought by the PMU as intended, therefore poor procurement documentation and procurement delays continued. In particular, the lengthy procurement of the personal protective equipment (February - November 2008), hindered project implementation to progress as the pesticide inventory and subsequent activities could not be started without the equipment. In 2009, the PMU received procurement training, 240 which substantially improved its procurement capacity. 2.5 POST-COMPLETION OPERATION/NEXT PHASE 35. Post completion: At project completion, two draft legislations had been formulated and were awaiting cabinet clearance; they included a Plant Protection Bill and a Pesticide Management Bill. 241 36. Next phase: A follow-on project is currently being prepared to focus on the remaining contaminated soils that continue to be in the country; ASP-P1 Tanzania only managed to address a portion of these contaminated soils. The envisaged project intends to focus on adequate hazardous waste disposal facilities including the establishment of an engineered bio-reactor landfill, capacity building, and enhancing institutional collaboration. A concept note is being prepared by the government for GEF funding consideration. 3. ASSESSMENT OF OUTCOMES 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION 37. Relevance – Rating High: Despite having been too ambitious, the project’s original objectives were relevant when the project was prepared. The project’s revised objective, design, and implementation are considered highly relevant to the current national development priorities. The Government of Tanzania places strong emphasis on preserving the environment and mainstreaming environmental concerns into all future policy measures, as evident from the country’s five year development plan for the period 2011/12-2015/16. 242 The project’s objective is also consistent with the World Bank’s FY12-15 Country Assistance Strategy for Tanzania, 243 which supports environmental capacity 239 The project’s procurement officer left the project in January 2007, shortly after project approval. The procurement unit was subsequently staffed with only one supplies officer, who had inadequate experience with World Bank procurement procedures. 240 Two staff attended procurement trainings that were conducted by the Eastern and Southern African Management Institute in cooperation with the World Bank. 241 The Plant Protection Act intends to prevent the introduction or spread of plant pests, provide for phytosanitary control measures, and facilitate trade in plants and plant products. The Pesticide Management Act intends to regulate the manufacture, formulation, importation into and exportation from the country, transport, storage, distribution, sale, use, and disposal of pesticides. 242 The Tanzania Five Year Development Plan 2011/12-2015/16 – Unleashing Tanzania’s Latent Growth Potentials, June 2011, United Republic of Tanzania – President’s Office, Planning Commission 243 Country Assistance Strategy for the United Republic of Tanzania for the Period FY12-15, (Report No. 60269-TZ), May 9, 2011 200 building in most sectors, and enhanced sustainability and improved management of natural resources. The project has also been supporting the Government of Tanzania in complying with the Stockholm Convention on POPs, and the Basel Convention on the control of trans-boundary movements of hazardous wastes and their disposal. 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE 38. Taking account of the project’s Level 1 restructuring in June 2011, the project has been evaluated against its original and revised PDO by measuring achievements at the time of the project’s restructuring (June 2011) and project closing (May 2013), respectively. 39. Original PDO achievement - Rating Unsatisfactory: The assessment took the project’s six PDO indicators into account, of which two were fully (100%), and four not (0%) achieved. The three main project outcomes measured by the indicators include: (i) sound disposal and management of obsolete pesticides; (ii) prevention of accumulation of new stockpiles of obsolete pesticides; and (iii) enhanced capacity and institutional strengthening of pesticide management. 244 40. Sound disposal and management of obsolete pesticides. The level of risks emanating from publicly held obsolete pesticides and associated waste to nearby communities or critical natural resources was determined and quantified through the completion of a national inventory of publicly held obsolete pesticides in November 2009 (PDO Indicator #1 – 100%). All identified sites in the country together with their respective stocks of obsolete pesticides were subject to risk assessments based on the EMTKs. Data from each site was entered into the PSMS, which calculates risk factors related to the product and environmental conditions, and assists in prioritizing the different sites. These efforts were substantially supported by the local ASP NGO network through independent monitoring of inventory activities to ensure safety and health standards. The data informed the preparation of the CESA and the disposal and safeguarding of stockpiles. However, at restructuring, the CESA was not yet completed (PDO Indicator #2 – 0%). Consequently, disposal activities had not begun at restructuring (PDO Indicator #3 – 0%). 245 However, it should be noted that the project conducted an emergency safeguarding operation at a high-risk site in Morogoro 246 in October 2010. 247 The operation had been advised by the World Bank and was conducted under application of an approved Environmental Management Plan (EMP). The operation reduced the risk emerging from obsolete pesticides and particularly contaminated soil, and provided the project with experience in preparing a satisfactory EMP and carrying out safeguarding. 41. Prevention of accumulation of new stockpiles of obsolete pesticides. Tanzania’s legal and regulatory framework for pesticide management had not been strengthened at restructuring (PDO Indicator #4 – 0%); the envisaged IPM strategy was not developed (PDO Indicator #5 – 0%). It should be noted that in 2010, it was agreed by the World Bank and the PMU that such strategy would not be developed, but instead a national workshop on IPM would be held, however, the workshop had not taken place 244 Achievements were identified based on intended outcomes presented in Annex 1 in Project Appraisal Document for the Republic of Mali, the Kingdom of Morocco, and the United Republic of Tanzania in support of the Africa Stockpile Program – Technical Annex 3, (Report No. 36273-AFR), November 7, 2006 245 Only a draft disposal tender document had been prepared and submitted to the World Bank for review on May 26, 2011 246 Morogoro is a municipality in the central part of Tanzania, 190 km west of Dar es Salaam. 247 The emergency safeguarding operation included fencing off the contaminated site with a barbed wire fence, putting in place danger signs and hazard tape to prevent people from entering. During disposal activities, 392 tonnes of contaminated soil were excavated from this site. 201 by the time of restructuring. The project raised awareness on the health and environmental hazards of pesticides and encouraging safe pesticide handling. A communication strategy was developed in collaboration with the World Wildlife Fund (WWF), and its implementation was launched in 2009. Activities included awareness raising visits to all agricultural zones (particularly to accompany the inventory), development of briefing materials, and press conferences. A network of local NGOs with experience in health and environmental issues and formed in December 2002, had substantially supported these efforts through sensitizing selected communities, and training community members in independently identifying and monitoring chemical effects at the local level. A National Stakeholders Advisory Forum (NSAF) 248 was established and held on July 31, 2009, to provide a forum of knowledge exchange and coordination of activities implemented by local NGOs. 42. Enhanced capacity and institutional strengthening of pesticide management. Between 2005 and 2010, PMU members and representatives from the Ministry of Agriculture (MoA) received training in project management, conducting a national inventory, entering data into the PSMS, preparing a CESA, safeguarding of hazardous waste, and procurement.. Capacity was strengthened as became evident through application of the acquired knowledge while conducting the inventory, and preparing safeguard instruments (PDO Indicator #6 – 100%). 43. Revised PDO achievement – Rating Highly Satisfactory: The assessment took the project’s three revised PDO indicators into account, of which two were fully (100%) achieved and one exceeded its target (131%). In addition, six Intermediate Outcome Indicators framed the evaluation. One of those was not taken into account as it corresponded with an Original PDO Indicator that had been achieved prior to restructuring; all others were fully (100%) achieved. Outcomes were slightly modified following the restructuring and included: (i) Sound disposal and management of obsolete pesticides and associated waste including safeguarding; (ii) reduction of re-accumulation of obsolete pesticides and associated waste through regulations, pest management, and stakeholder involvement; and (iii) enhanced capacity and institutional strengthening of pesticide management. 44. Sound disposal and management of obsolete pesticides and associated waste including safeguarding: The inventory was effectively used to prepare the CESA (Intermediate Outcome Indicator #2 – 100%). 249 The CESA led to important improvements in project design, such as the proposal to consolidate low-risk stores into main collection centers using country capacity. Such a strategy led to important cost savings, as it significantly reduced the disposal contractors’ fieldwork. A total of 665 tonnes 250 of obsolete pesticides and 2,515 tonnes of associated waste including large amounts of contaminated soils were inventoried. Since the amount of associated waste (particularly contaminated soils) had turned out to be much higher than estimated and the required funding for disposal subsequently beyond the allocated budget, it was agreed at restructuring that the disposal contract would include all inventoried obsolete pesticides (665 tonnes), and of the inventoried heavily contaminated soils only those that were posing a significant risk to human and animal health (300 tonnes). At project closure, all inventoried obsolete pesticides (575 tonnes) were disposed of overseas in an environmentally and technically sound manner (Revised PDO Indicator #1 – 100%). During disposal activities, it became clear that the total amount of obsolete pesticides was lower than 248 The NSAF consisted of representatives from local NGOs. PMU members, the private sector, media, the national registrar for pesticides. 249 The document was finalized on June 15, 2012, and disclosed to the World Bank InfoShop on June 27, 2012. 250 The 665 tonnes of obsolete pesticides represents net weight, and equals 700 tonnes gross weight. 202 anticipated; only representing 575 instead of 665 tonnes. As a result, even though the project disposed of fewer tonnes than originally targeted, the Revised Indicator #1 was fully achieved as all inventoried obsolete pesticides were eliminated. The generated cost savings were subsequently applied to the disposal of high risk contaminated soils to maximize the outcome. Eventually, the project disposed of 392 tonnes of the inventoried heavily contaminated soils (Revised PDO Indictor #2 – 131%), exceeding its target level of 300 tonnes. 45. Reduction of re-accumulation of obsolete pesticides and associated waste through regulations, pest management, and stakeholder involvement. The project implemented measures targeted at sustainably managing and reducing the re-accumulation of obsolete pesticides and associated wastes through the following activities: A comprehensive strategy to better regulate and control the registration, production, storage, and sale of pesticides in Tanzania was adopted by the Steering Committee on April 22, 2013 (Revised PDO Indicator #3 – 100%). The strategy intends to serve as basis and guidance for dealing with future stocks, and is based on a Figure 3.1: Publicly Held Obsolete Pesticides and Associated Waste sustainability roadmap Inventoried and Eliminated developed in 2010, which 3000 identified key steps towards sustainable pesticide 2500 Inventoried 2009 management in Tanzania (Intermediate Outcome 2000 Targets set at Indicator #5 – 100%). Overall, Restructuring June Tonnes 1500 the strategy was composed of 2011 2515 action plans for pesticide 1000 Eliminated May 2013 management and IPM including container 500 665 665 575 management (Intermediate 300 392 0 Outcome Indicator #4 – 100%), which were informed Obsolete Pesticides Associated Waste by various studies and (particularly reviews. 251 At project Contaminated Soils) completion, two draft legislations had been formulated and were awaiting cabinet clearance; they included a Plant Protection Bill and a Pesticide Management Bill.252 In addition to the awareness raising activities highlighted before, the project raised awareness on the health and environmental hazards of pesticides and encouraged safe pesticide handling by having implemented four different communication actions (Intermediate Outcome Indicator #3 – 100%). These included the development of two TV and two 251 These included an assessment of contaminated soil and buried pesticides, a study on pesticide management, and a review of pesticide legislation. 252 The Plant Protection Act intends to prevent the introduction or spread of plant pests, provide for phytosanitary control measures, and facilitate trade in plants and plant products. The Pesticide Management Act intends to regulate the manufacture, formulation, importation into and exportation from the country, transport, storage, distribution, sale, use, and disposal of pesticides. 203 radio spots, broadcasted in early 2013, raising awareness through participation in five exhibitions,253 and outreach activities during disposal activities in three different zones of Tanzania (Intermediate Outcome Indicator #6 – 100%). 46. Enhanced capacity and institutional strengthening of pesticide management. In addition to the capacity building achieved prior to restructuring, the project further strengthened the government’s capacity through additional training. Training was provided in International Maritime Dangerous Goods (IMDG) Code, safety and health, and preparation and implementation of national prevention actions plans. 47. Weighted average PDO achievement – Rating Satisfactory: At restructuring in June 2011, the project had disbursed US$1.96 million, representing 28.5% of total disbursed funds of US$6.87 million. On this basis and as illustrated by the table below, the weighted average of the project’s PDO achievement is rated moderately satisfactory. Table 3.1: Weighted Average PDO Achievement Against Original PDO Against Revised PDO Overall 1. Rating Unsatisfactory Highly Satisfactory - 2. Rating Value 2 6 - 3. Weight (% disbursed before/after PDO 28.5% 71.5% - change) 4. Weighted value 0.6 4.3 4.9 5. Final rating - - Satisfactory 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE 48. The project aimed to directly contribute to the GEO of reducing the effect of POPs on the global environment through: (i) improving the quality of life in poor communities, by reducing environmental risks; (ii) improving environmental protection; and (iii) enhancing the capacity of the agricultural sector to better manage crop pests. GEO achievement is measured by PDO achievement, and, thus, is also rated satisfactory (see Section 3.2). 3.4 EFFICIENCY 49. Efficiency – Rating Substantial: This rating represents a broad estimate, as financial and economic data to support this rating are scarce. At appraisal, an economic or financial analysis was neither undertaken for ASP-P1 at the program nor at the country-level due to non-measurable benefits of reducing the risk of contaminating the environment. 50. Based on the fact that approximately 74% (US$5.07 million) of disbursed GEF funds (US$6.87 million) went towards disposal activities under ASP-P1 Tanzania, the project’s efficiency was primarily assessed by its investment in the disposal of obsolete pesticides and associated waste. The project spend all available funds, and succeeded in achieving its revised PDO by disposing all inventoried obsolete pesticides, and a larger amount of contaminated soils than targeted. 253 Exhibitions included Farmers Day (August 2012), Earth/World Environment Day (June 2012), Public Service Week (July 2012), 50 Years Independence Anniversary event (December 2012), and Maji (Water) Day (March 2012). 204 51. During appraisal, the unit cost for disposal was estimated at US$3,400 per tonne (based on approximately US$15 million spent on the disposal of 3,240 tonnes of obsolete pesticides between 1990 and 2004 across Africa). (See Section 3.4 in the program-level ICR for details). At project closing, 967 tonnes of obsolete pesticides and associated waste had been disposed of at the cost of approximately US$5,243 per tonne, which represents a slightly higher cost than originally anticipated. 254 However, the US$ depreciation since 2004 and an increase in the price of fuel, which has increased the cost of disposal in US$ terms, is not taken into account by this calculation; neither are varying distances from collection centers to harbors, varying distances of shipping routes, and responses to call of bid by different companies. 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING 52. Overall Outcome - Rating Satisfactory: On the basis of the project’s efficiency (substantial), relevance (high), and PDO achievement (satisfactory), the overall outcome is rated satisfactory. The overall outcome rating takes into account that despite substantial implementation challenges prior to mid-term review (see Section 2), the project successfully reduced risks to the environment and public health by disposing of all inventoried obsolete pesticides and a substantial amount of highly contaminated soils. In addition, the project succeeded in significantly improving measures to reduce future re-accumulations by establishing the prerequisites for a sustainable system of pest and pesticide management. 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS 53. Poverty impacts, gender aspects, and social development: The disposal of obsolete pesticides and associated wastes had a direct positive impact on poor and farming communities, species and ecosystems, located in proximity to sites carrying obsolete pesticide stocks and contaminated soils. Baseline data on affected populations, species, and ecosystems was not conducted at appraisal; exact impacts are therefore difficult to measure. 54. Institutional change/strengthening: The project strengthened the government’s institutional capacity on several levels: (i) an improved regulatory and legal framework for pest and pesticide management was drafted and is awaiting final enactment; (ii) the logistics of hazardous waste transportation have been internalized and the knowledge gained can inform future disposal activities; (iii) the experience and skills gained in managing the project, particularly in the areas of procurement and FM, could be applied to other projects and subsequently contribute to success; and (iv) knowledge gained on sound pesticide management through the various training that were provided to governmental staff. 55. Other unintended outcomes and impacts: The mid-term review drew attention to 660 tonnes of unwanted sulfur stocks. The stocks could not be included in the project, but the World Bank offered specialist advice to the Government in finding a solution to the problem. 255 The CESA TORs were subsequently modified to include the sulfur as an annex to the main report, and eventually the stocks were repacked and used as input by a fertilizer manufacturer. 254 Costs include costs associated with repackaging, transport, and incineration of obsolete pesticides and other related activities such as for example the inventory, safeguarding efforts, associated studies, and advisory services. 255 The sulfur stocks could not be included in the project as (i) the sulfur stocks were privately owned and not part of the publicly held stocks the ASP was looking at, and (ii) it presented a much lower risk to the environment and human populations than the soil contamination or obsolete pesticide stocks. 205 3.7 SUMMARY OF FINDING OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS 56. An ICR mission including a stakeholder workshop was conducted on April 29-30, 2013 in Morogoro, Tanzania. For a detailed summary of the findings, please see Section 8.6. A beneficiary survey was not conducted. 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME 57. The risk to the development outcome is rated moderate: The risk is confined to the re-accumulation of obsolete pesticides, and a potential weakening of government commitment to sustaining the project’s outcomes, particularly with respect to prevention activities. However, the project set the basis for strengthened pesticide legislation and national capacity; if enacted and together with the various activities agreed upon in the comprehensive strategy for dealing with future stocks of obsolete pesticides and associated waste, the government is well positioned to sustainably manage and reduce the re-accumulation of obsolete pesticides. The government’s commitment that has been visible particularly following the project’s restructuring is likely to continue as visible by a follow-on project that is currently being prepared to focus on the remaining contaminated soils that continue to be in the country. 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE 5.1 WORLD BANK PERFORMANCE 58. World Bank performance in ensuring quality at entry – Rating Moderately Unsatisfactory: Project preparation of ASP-P1 Tanzania was conducted in conjunction with preparatory activities at the program-level, which were lengthy (see Section 2.1 in the program-level ICR). While considerable time was invested in overall preparation, key project preparation documents for ASP-P1 Tanzania only broadly laid out project implementation plans, and lacked country-specific information. The project’s design was characterized by several weaknesses, which translated into substantial project implementation delays (see Section 2.1). FM and procurement assessments and subsequent recommendations were adequate. 59. Quality of World Bank supervision – Rating Moderately Satisfactory: The project was supervised by the Task Team Leader (TTL) responsible for ASP at the program-level, and a co-TTL specifically assigned to ASP-P1 Tanzania. Supervision missions took place regularly, albeit only once a year prior to 2009. 60. During the first three years, World Bank performance was affected by issues at the program-level that resulted in delays. 256 Following the mid-term review, the World Bank team improved its support and addressed more proactively the project’s implementation delays. Efforts included: (i) effective discussions with high-level representatives from the government on how to improve project 256 World Bank performance had contributed to initial project implementation delays through: (i) The late opening of a special account for the project, which delayed disbursements of first installments by six months; (ii) collaboration issues with FAO, which had resulted in the PMU raising concern about differing guidance received from the two agencies; and (iii) lengthy review and clearance processes for non-objections. 206 implementation; (ii) more efficient project implementation support through local World Bank staff; (iii) an extension of the project’s closing date to compensate for start-up delays; and (iv) the preparation (with subsequent approval by the Board) of a Level 1 restructuring involving revision of the PDO to respond to sector realities and country needs, and to ensure PDO achievement. Successful disposal activities were ensured through: (i) the reallocation of existing GEF funds to maximize the amount of hazardous materials to be disposed of; and (ii) the timely preparation of an additional extension of the closing date to ensure the safe arrival of already shipped obsolete pesticide materials at their destination, PDO achievement, and project funding eligibility of all disposal-related activities. 61. FM reviews were regularly conducted by World Bank FM specialists. This included reviewing project FM arrangements (i.e., budgeting, accounting, reporting, funds flow, internal controls and external audit arrangements), and the implementation of internal and external audit queries. 62. Overall World Bank performance – Rating Moderately Unsatisfactory: The project suffered from shortcomings in project preparation and design at the program-level, which caused substantial delays during project implementation. However, following slow performance during the first half of the project, the World Bank team improved its performance through corrective actions, which ultimately contributed to successful PDO achievement. 5.2 BORROWER PERFORMANCE 63. Government performance – Rating Satisfactory: Government commitment at entry towards the issue of obsolete pesticides was evident through actions the government had already undertaken in view of the problem obsolete pesticides were posing. During project preparation and implementation, there was equally strong support for the project as demonstrated by active participation in preparatory and implementation activities. 64. Implementing Agency – Rating Satisfactory: Overall, NEMC’s performance was satisfactory, with gradual improvement throughout project implementation. Initially, the day-to-day management of project activities was hampered by NEMC’s project management capacity. The PMU coordinator position was only appointed in January 2008; a lack of specialized procurement staff further contributed to initial delays. However, following initial challenges, NEMC showed increasing capacity and commitment towards achieving the PDO. Project implementation was substantially improved, and a high level of effectiveness was maintained until project closure; performance was at a highly satisfactory level during the second half of project implementation. NEMC also complied with all World Bank loan covenants, and executed its fiduciary duties in a timely manner, including safeguards. 65. Overall borrower performance – Rating Satisfactory: Based on the above presented evaluation, the overall borrower performance is rated satisfactory. 6. LESSONS LEARNED 66. The most significant lessons learned are outlined in Section 6 of the program-level ICR, as the experience they are based on was often reflected across all country-specific projects. Lessons learned that were particularly relevant to ASP-P1 Tanzania include: 207 • The project’s effectiveness and development impact can significantly benefit from correct application of World Bank safeguard instruments. The CESA identified suitable collection centers, to which obsolete pesticide stocks were transferred to for safer storage and convenient transportation. The collection centers eventually represented a significant cost-saving benefit when final disposal activities were undertaken. • Participation of local NGOs in project activities can significantly enhance outcomes at the local level. The NGO network was substantially involved in project activities related to conducting the national inventory, and prevention. Awareness raising, independent monitoring of health and safety standards, and promotion of participatory community monitoring significantly improved the project’s impact at the local level. In addition, knowledge sharing platforms, implemented in form of the NSAF, proved to be a valuable venue to raise awareness for the issue of obsolete pesticides. • Local World Bank presence was important to getting implementation on track. The client struggled with getting guidance and approvals for many day-to-day issues. The presence of a locally based co-TTL was helpful in expediting these processes. • World Bank support for communications can be very useful. The country-based World Bank team provided important support to the client in communicating the important results obtained. The communications officer and co-TTL worked to convene key media outlets so that they could learn of the program's achievements. 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS 67. See Section 8.7 and 8.8. 8. ANNEXES 8.1 PROJECT COSTS AND FINANCING a. Project Cost by Component – All sources (in US$ million equivalent) – as of 05/30/2013 Actual/Latest Appraisal Estimate Actual as % of Component Estimate (in US$ million appraisal (in US$ million equivalent) estimate equivalent) 9. Disposal of obsoletes pesticides 5.83 5.89 101% 10. Prevention of accumulation 0.28 0.46 164% 11. Capacity building 0.15 0.19 127% 12. Project management 1.22 1.69 138% Total 7.48 8.23 110% 208 b. Financing – as of 05/30/2013 Actual/Latest Appraisal Estimate Actual as % of Type of Estimate (in Sources of Funds (in US$ million appraisal Co-financing US$ million equivalent) estimate equivalent) Borrower In-kind 0.39 0.65 167% Global Environment Facility Grant 6.87 6.87 100% (GEF) Parallel Co- FAO (Netherlands) 0.22 0.22 100% financing/Grant Parallel Co- CropLife International (CLI) 0 0.49 - financing/Grant Total 7.48 8.23 110% 8.2 OUTPUTS BY COMPONENT 1. Taking account of the project’s Level 1 restructuring in June 2011, the following table provides a comprehensive account of qualitative and quantitative information of outputs realized against both original (achievements at the time of restructuring in June 2011) and revised components (achievements at the time of project closing in May 2013). Table 8.1: Outputs by Original Components as presented in the GEF Trust Fund Grant Agreement Original Components as presented in the GEF Outputs achieved at the time of restructuring in June 2011 Trust Fund Grant Agreement Part A – Obsolete pesticides inventory: Achievements under this component are rated Moderately Satisfactory Carrying out of a detailed inventoried of the a. An inventory of publicly held obsolete pesticides was completed and recipient’s publicly held obsolete pesticides validated in November 2009, including 100% data entry into an stocks and associated waste through: inventory database, which has been under the management and a. Collection of data and compilation of a oversight of FAO, as part of their PSMS (Original PDO Indicator #1 database; – 100%). b. A risk-based prioritization of the stocks b. All identified sites in the country together with their respective identified in the database; and stocks of obsolete pesticides were subject to risk assessments based c. Preparation of the CESA. on the EMTKs. Data from each site was entered into the PSMS, which calculated risk factors related to the product and environmental conditions, and assisted in prioritizing the different sites. c. At restructuring, a CESA was still being finalized and not yet completed (Original PDO Indicator #2 – 0%). Part B – Disposal of obsolete pesticides: Achievements under this component are rated Moderately Unsatisfactory Treatment and/or disposal of publicly held a. A technological assessment of available treatment and disposal obsolete pesticides stocks and associated waste options was carried out by WWF in 2008. by: b. Due to substantial project implementation delays, the disposal of a. Carrying out of a technological assessment of publicly held obsolete pesticide had not begun at restructuring (PDO available treatment and disposal options; Indicator #3 – 0%). Only a draft disposal tender document had been b. Implementation of the treatment and/or prepared and submitted to the World Bank for review on May 26, disposal technology selected; and 2011. However, it should be noted that the project conducted an emergency safeguarding operation at a high-risk site in Morogoro in 209 c. Improving the operations of obsolete pesticide October 2010. The operation had been advised by the World Bank stocks collection centers, including improving and was conducted under application of an approved EMP. safety measures at those centers. c. Operations of obsolete pesticide stocks collection centers had not d. Container management. started by the time of restructuring. d. A container management review was not completed by the time of restructuring. Part C – Prevention of obsolete pesticide accumulation: Achievements under this component are rated Moderately Unsatisfactory Carrying out of activities aimed at preventing a. A communication strategy was developed in collaboration with obsolete pesticide accumulation, including: WWF, and its implementation was launched in 2009. Activities a. Awareness raising activities on the program, included awareness raising visits to all agricultural zones health, and environmental hazards of (particularly to accompany the inventory), development of briefing pesticides. materials, and press conferences. A network of local NGOs with b. A review and update of the legal and experience in health and environmental issues and formed in regulatory framework to control obsolete December 2002, had substantially supported these efforts through pesticide accumulation and enforcement sensitizing selected communities, and training community members mechanisms; and in independently identifying and monitoring chemical effects at the c. Preparation and implementation of an IPM local level. NSAF was established and held on July 31, 2009, to strategy. provide a forum of knowledge exchange and coordination of activities implemented by local NGOs. b. Tanzania’s legal and regulatory framework for pesticide management had not been strengthened at restructuring (PDO Indicator #4 – 0%); c. The envisaged Integrated Pest Management (IPM) strategy was neither developed, nor implemented (PDO Indicator #5 - 0%). It should be noted that in 2010, it was agreed by the World Bank and the PMU that such strategy would not be developed, but instead a national workshop on IPM would be held, however, the workshop had not taken place by the time of restructuring. Part D – Capacity building: Achievements under this component are rated Moderately Satisfactory. Carrying out of capacity building activities, a. In 2008, the PMU received training on conducting environmental including: and social assessments; the knowledge gained was subsequently a. Strengthening the recipient’s capacity to carry applied when preparing the CESA. out environmental assessments. b. Two consultative studies had been conducted by FAO to assess b. Reviewing the recipient’s pesticide contaminated soils and buried pesticides, and review pesticide management practices and providing training management in Tanzania. Representatives from the ministries and on improved methodologies. the PMU participated in a CLI-funded study tour to Kenya to learn c. Reviewing implementation capacity of the about a private-public program to identify, collect, and dispose PMU and providing requisite training. obsolete pesticides. c. Between 2005 and 2010, PMU members and representatives from MoA also received training in project management, conducting a national inventory of obsolete pesticides, entering data into the PSMS, safeguarding of hazardous waste, and procurement.. Capacity was strengthened as became evident through application of the acquired knowledge while conducting the inventory, and preparing safeguard instruments (PDO Indicator #6 - 100%). Part E – Project management: Achievements under this component are rated Moderately Unsatisfactory. Supporting the PMU in carrying out project An unclear partnership arrangement at the program-level hampered coordination, implementation, monitoring, and timely technical support, particularly during the first three years. 210 evaluation, through the provision of technical • The highly technical and specialized expertise required for advisory services, equipment, and operational implementation, supervision and monitoring of project activities the support. PMU was supposed to receive from FAO’s TSU was not delivered to the extent planned. Valuable support was given in terms of equipment through the provision of the PSMS, and associated training. However, limited technical advisory services were given on the pre-qualification, bidding, and execution of the specialized services associated with procuring the selection processes of the CESA and disposal service providers, and M&E. • The urgently needed operational support from the World Bank was not sufficiently provided prior to mid-term review; however, increased supervision, operational and technical support thereafter, and the subsequent deployment of an in-country co-TTL, improved the level of support. For technical support during disposal activities, a TAD was eventually recruited and financed by CLI, which positively affected project implementation. Table 8.2: Outputs by Revised Components as presented in the Level 1 Restructuring June 30, 2011 Revised Components (Level 1 Restructuring Outputs achieved at the time of project closing June 30, 2011) 257 Specific revisions are underlined Part A – Obsolete pesticides inventory: Achievements under this component are rated Satisfactory Carrying out of a detailed inventoried of the a. See outputs achieved under Original Part A. recipient’s publicly held obsolete pesticides b. See outputs achieved under Original Part A. stocks and associated waste through: c. Information on obsolete pesticides and associated waste obtained a. Collection of data and compilation of a through the completion of the national inventory of publicly held database; obsolete pesticides which had been completed in November 2009, was b. A risk-based prioritization of the stocks effectively used to inform the successful preparation of a CESA identified in the database; and (Intermediate Outcome Indicator #2 - 100%). The document was c. Preparation of the CESA. finalized on June 15, 2012, and disclosed to the World Bank InfoShop on June 27, 2012. Part B – Disposal and/or safeguarding of obsolete pesticides and associated waste: Achievements under this component are rated Satisfactory 257 Restructuring Paper on a Proposed Project Restructuring of Africa Stockpiles Program – Project 1 to the Republic of Tunisia, (Report No: 62797-TN), June 20, 2011 211 Safeguarding and/or disposal of publicly held a. See outputs achieved under Original Part B. obsolete pesticides stocks and associated b. All inventoried obsolete pesticides (575 tonnes) were disposed of waste by: overseas in an environmentally and technically sound manner (Revised a. Carrying out a technological assessment of PDO Indicator #1 – 100%). During disposal activities, it became clear available treatment and disposal options; that the total amount of obsolete pesticides was less than anticipated; b. Implementation of the safeguarding and/or only representing 575 instead of 665 tonnes. As a result, even though disposal technology selected; and the project disposed of less tonnes than originally targeted, the Revised c. Improving the operations of obsolete Indicator #1 was fully achieved as all inventoried obsolete pesticides pesticide stocks collection centers, including had been eliminated. The generated cost savings were subsequently store stabilization and safety measures at applied to the disposal of high risk contaminated soils; the project those centers. disposed of 392 tonnes of the inventoried heavily contaminated soils (Revised PDO Indictor #2 – 131%), more than targeted. c. Six major collection centers were rehabilitated, and low risk obsolete pesticide stocks were centralized at these centers. Part C – Reduction of obsolete pesticide re-accumulation: Achievements under this component are rated Satisfactory. Carrying out of activities aimed at reducing a. The project raised awareness on the health and environmental hazards obsolete pesticide and associated waste re- of pesticides and encouraged safe pesticide handling by having accumulation, including: implemented four different communication actions (Intermediate a. Awareness raising activities on the program, Outcome Indicator #3 - 100%). These included the development of two health and environmental hazards of TV and two radio spots, broadcasted in early 2013, raising awareness pesticides. through participation in five exhibitions, and outreach activities during b. A review and update of the legal and disposal activities in three different zones of Tanzania (Intermediate regulatory framework to control obsolete Outcome Indicator #6 - 100%). pesticide accumulation and strengthen b. At project completion, two draft legislations had been formulated and enforcement mechanisms; were awaiting cabinet clearance; they included a Plant Protection Bill c. Preparation and implementation of an IPM and a Pesticide Management Bill. strategy. c. A comprehensive strategy to better regulate and control the d. Preparation of pesticide and container registration, production, storage, and sale of pesticides in Tanzania was management strategies. adopted by the Steering Committee on April 22, 2013 (Revised PDO Indicator #3 – 100%). Overall, the strategy was composed of action plans for pesticide management and IPM including container management (Intermediate Outcome Indicator #4 – 100%), d. See above under c. Part D – Capacity building: Achievements under this component are rated Satisfactory Carrying out of capacity building activities, a. See outputs achieved under Original Part D. including: b. See outputs achieved under Original Part D. a. Strengthening the recipient’s capacity to c. See outputs achieved under Original Part D. Additional training was carry out environmental assessments. provided on International Maritime Dangerous Goods, Safety and b. Reviewing the recipient’s pesticide Health, and on Pesticide Life Cycle Management for District Plant management practices and providing Protection Officers and Storekeepers. training on improved methodologies. c. Reviewing the implementation capacity of the PMU and providing requisite training. Part E – Project management: Achievements under this component are rated Moderately Satisfactory Supporting the PMU in carrying out project In addition to outputs achieved under Original Part D, the World Bank coordination, implementation, monitoring, and provided effective operational support during the final disposal activities. evaluation, through the provision of technical For technical support, a TAD was eventually recruited and financed by advisory services, equipment, and operational CLI, which positively affected project implementation. support. 212 8.3 ECONOMIC AND FINANCIAL ANALYSIS 2. As outlined in Section 3.4, an economic and financial analysis is not available. 8.4 WORLD BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS a. Task Team Leaders Lending (Task Team, specialization, and unit as outlined in the PAD) Names Specialization Unit Responsibility/Specialty Adebowale, Modupe Senior Disbursement Officer LOAG2 Brackmann, Stefanie Environmental Specialist MNSRE Fye, Serigne Omar Senior Environment Specialist AFTS2 Jaya, Mel Team Assistant AFMZA Jordy, Denis Senior Agricultural Specialist AFTS4 Co-TTL Kaniaru, Muthoni Counsel LEGAF Kebede, Meseret Language Program Assistant AFTS1 Kibbassa, Jane Environmental Specialist Konare, Amadou Senior Environmental Specialist AFTS1 Kristensen, Peter Senior Environmental Specialist AFTS4 TTL Lagnaoui, Abdelaziz Senior Pest Management Specialist ESDQC Maber, Steven Senior Operations Officer MNSRE Mneney, Donald Procurement Specialist AFTPM Ndiaye, Marie-Jeanne Language Program Assistant AFTS4 Rashid, Aza Team Assistant AFMTZ Sabai, Mercy Senior Finance Officer AFTFM Tegwa, Pascal Procurement Specialist AFTPM Tran, Lucie Operations Officer AFTS4 Tynan, Ellen Senior Environmental Specialist ENVMP Warner, Christopher Senior Environmental Specialist AFTS1 Waters, Warren Regional Environmental and Safeguards AFTSD Advisor Supervision/ICR Names Specialization Unit Responsibility/Specialty Aryal, Dinesh Senior Operations Officer TTL Brackmann, Stefanie Environmental Specialist 213 Chande, Faustine Team Assistant Diallo, Bella Financial Specialist Doucouré, Djibril Safeguards Specialist Fodor, Martin Senior Environment Specialist Follea, Salimata Operations Analyst Glauber, Ann Jeannette Senior Environmental Specialist Co-TTL Guibert, Yves Consultant Ishemo, Sarah Team Assistant Kibbassa, Jane Environmental Specialist Co-TTL Kristensen, Peter Program Manager TTL Lagnaoui, Abdelaziz Senior Environmental Specialist Matumbo, Faith-Lucy Team Assistant Mengi, Hubert Financial Management Consultant Mneney, Donald Senior Procurement Specialist Mukuta, Leah Team Assistant Okuny, Michael Financial Management Specialist Peled Ben Ari, Ayala Environmental Economist Prevoo, Dirk Senior Operations Officer TTL Rechbauer, Gabriele Senior Environmental Operation Specialist Sabai, Mercy Senior Finance Officer Sakaya, Elizabeth Disbursement Assistant Schmidt, Veruschka Strategy Officer Shoo, Geoffrey Financial Management Specialist Tegwa, Pascal Senior Procurement Specialist Tynan, Ellen Environmental Specialist TTL Von Platen-Hallermund, Junior Professional Officer Tobias b. Staff time and cost – Staff time and costs associated with project preparation and supervision was not possible to determine as a result of poor recording in the system. 8.5 BENEFICIARY SURVEY RESULTS 3. Beneficiary survey results are not available, as no survey was conducted. 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS 4. An ICR mission including a stakeholder workshop was conducted from April 29-30, 2013 in Morogoro, Tanzania. Stakeholders highlighted the following key points: 214 • The most important prevention achievements included improved laws, strengthened capacity building, establishment of a pesticides management system, development of a communication strategy, development of IPM, pesticide and container management strategies, and establishment of a national NGO network. • The most important disposal achievements included disposal of obsolete pesticides and associated waste, improved collaboration between NGOs, research institutions, and the private sector, identification of obsolete pesticides locations, completion of a nation-wide inventory, and establishment of main collection centers for obsolete pesticides. • Key challenges included data verification, information gaps regarding obsolete pesticide stocks, high turnover of staff, short project timeframe, communities’ low awareness of obsolete pesticide issue, delay in disbursement of funds, and lengthy procurement processes. • Sustainability of project outcomes can be ensured through enforcement of existing legal instruments, additional funding for well-equipped laboratories to ensure quality control, awareness raising for the proper use or management of pesticides, an effective system for empty container management including the creation of collection centers, triple-rinsing, recycling, partnership between government and recycling industries, the creation of a multi-purpose disposing unit to cut across pesticides and other chemicals and industrial waste (even through a multi-regional approach), and careful management of subsidies including safety nets. • Unclear institutional set up at the program-level negatively affected project implementation in form of disbursement delays, a resulting bureaucratic arrangement that hindered timely decision making, and unclear procurement due to different procurement procedures conducted between World Bank and FAO. • Key lessons learned included: (i) involving target group from the beginning on is important; (ii) obsolete pesticide disposal was costly; (iii) strong collaboration was needed among stakeholders; (iv) in-country incineration should be explored; (v) Tanzania needed regulatory tools for prevention of accumulation of pesticides; and (vi) prevention was a cheaper alternative to overseas obsolete pesticide disposal. 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR 5. The borrower submitted a completion report; messages contained therein were integrated into this document. 6. Equally incorporated were comments received on the draft ICR. Additional comments included: a. It was suggested to upgrading the unsatisfactory rating for PDO achievement prior to the restructuring, as according to the borrower ASP-P1 Tanzania progressed well prior to the restructuring as was evident from PDO and IP ratings between February 2009 and November 2010. • World Bank response: The assessment took the project’s six PDO indicators into account, of which two were fully (100%), and four not (0%) achieved. Those not achieved included Original PDO Indicator #3 (Inventoried publicly held obsolete pesticides stocks disposed of, as a result of the completion of the disposal services contract(s) in accordance with national and international laws and regulations), which was closest linked to the PDO. 215 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS 7. No comments were received from co-financiers and other partners/stakeholders on the ASP-P1 Tanzania draft ICR. 8. Comments of co-financiers and other partners/stakeholders on the program-level ICR are summarized in Section 8.8 in the program-level ICR. 216 8.9 VERBATIM COMMENTS ON DRAFT ICR 217 218 219 TECHNICAL ANNEX VI: ASP-P1 TUNISIA 220 ABBREVIATIONS AND ACRONYMS ANGeD Agence Nationale de Gestion des Dechets ANPE Agence Nationale de Protection de l’Environment ASP-P1 Africa Stockpiles Program – Project 1 CAS Country Assistance Strategy CESA Country Environmental and Social Assessment CLI CropLife International EMP Environmental Management Plan EMTK Environmental Management Toolkit FAO United Nations Food and Agriculture Organization FFEM French Global Environment Facility FM Financial Management GDP Gross Domestic Product GEF Global Environment Facility GEO Global Environment Objective IPM Integrated Pest Management IVM Integrated Vector Management MDTF Multi-Donor Trust Fund M&E Monitoring & Evaluation NGO Non-Governmental Organization NIP National Implementation Plan PAD Project Appraisal Document PAN Pesticides Action Network PDO Project Development Objective PMU Project Management Unit POP Persistent Organic Pollutants PSMS Pesticide Stock Management System TAD Technical Advisor for Disposal TOR Terms of Reference TSU Technical Support Unit TTL Task Team Leader 221 UNEP United Nations Environment Program WWF World Wildlife Fund 222 TABLE OF CONTENTS 1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN ...... 232 1.1 CONTEXT AT APPRAISAL ............................................................................................................................................ 232 1.2 ORIGINAL PDO AND INDICATORS ............................................................................................................................. 233 1.3 REVISED PDO AND INDICATORS ............................................................................................................................... 234 1.4 MAIN BENEFICIARIES................................................................................................................................................... 235 1.5 ORIGINAL AND REVISED COMPONENTS ................................................................................................................. 235 1.6. OTHER SIGINIFICANT CHANGES .............................................................................................................................. 237 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES ..................................................................... 238 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY ............................................................................ 238 2.2 IMPLEMENTATION ........................................................................................................................................................ 239 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTILIZATION ........................ 240 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE ......................................................................................................... 241 2.5 POST-COMPLETION OPERATION/NEXT PHASE ...................................................................................................... 243 3. ASSESSMENT OF OUTCOMES ......................................................................................................................................... 244 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION ...................................................................... 244 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE ................................................................................... 244 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE ................................................................................... 248 3.4 EFFICIENCY .................................................................................................................................................................... 248 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING................................................................................................. 248 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS ............................................................................ 249 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS .................... 249 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME .......................................................................................... 249 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE .................................................................. 249 5.1 WORLD BANK PERFORMANCE .................................................................................................................................. 249 5.2 BORROWER PERFORMANCE....................................................................................................................................... 250 6. LESSONS LEARNED ............................................................................................................................................................ 251 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS ................ 252 8. ANNEXES ............................................................................................................................................................................... 252 8.1 PROJECT COSTS AND FINANCING ............................................................................................................................. 252 8.2 OUTPUTS BY COMPONENT ......................................................................................................................................... 254 8.3 ECONOMIC AND FINANCIAL ANALYSIS .................................................................................................................. 257 8.4 WORLD BANK LENDINGA AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS .............................. 257 8.5 BENEFICIARY SURVEY RESULTS .............................................................................................................................. 259 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS............................................................................................ 259 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR ................................................................. 259 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS ................................................... 259 8.9 VERBATIM COMMENTS ON DRAFT ICR ................................................................................................................... 260 223 A. Basic Information 258 Country: Tunisia Project Name: Africa Stockpiles Program - Project 1 Project ID: P075776 L/C/TF Number: TF055745 ICR Date: 06/28/2013 ICR Type: Core ICR Lending Instrument: APL Borrower: Republic of Tunisia Original Total US$4.0 million Disbursed Amount: US$3.78 million (03/19/2013) Commitment: Revised Amount: - Environmental Category: A Global Focal Area: CHEM Implementing Agency: Agence Nationale de Gestion des Déchets (ANGed) Co-financiers and Other External Partners: Government of the Republic of Tunisia, French Global Environment Facility (FFEM), Food and Agriculture Organization (FAO), CropLife International (CLI), Pesticide Action Network (PAN-UK and PAN-Africa), and World Wildlife Fund (WWF). B. Key Dates Process Date Process Original Date Revised / Actual Date(s) Concept Review: 07/17/2002 Effectiveness: 09/30/2005 11/21/2005 Appraisal: 04/19/2004 Restructuring: 11/25/2009, 06/30/2011 Approval: 09/08/2005 Mid-term Review: 03/2008 10/31/2008 Closing: 09/30/2009 06/30/2012 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Global Environment Outcome: Moderate to Low World Bank Performance: Moderately Unsatisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of World Bank and Borrower Performance World Bank Ratings Borrower Ratings Quality at Entry: Moderately Unsatisfactory Government: Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Satisfactory Agency/Agencies: Overall World Bank Overall Borrower Moderately Unsatisfactory Satisfactory Performance: Performance: 258 The information presented in the data sheet was inserted manually. Information provided in the system was incomplete resulting from coding issues experienced during implementation; these were not resolved prior project closing. 224 C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators - Performance (if any) Potential Problem Project at Quality at Entry Yes - any time (Yes/No): (QEA): Problem Project at any time Quality of Yes - (Yes/No): Supervision (QSA): PDO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total World Bank financing) Sanitation 33.3 33 Health 33.3 33 General Agriculture 33.3 34 Theme Code (as % of total World Bank financing) Environmental policies and institutions 50 50 Pollution management and environmental health 50 50 E. World Bank Staff Positions At ICR At Approval Vice President: Inger Andersen Christiaan J. Poortman /Gobind T. Nankani Country Director: Neil Simon M. Gray Theodore O. Ahlers Director (Regional Colin Bruce Mark Tomlinson Integration) Sector Manager: Charles Cormier Inger Andersen Project Team Leader: Dinesh Aryal Steven P. Maber ICR Team Leader: Dinesh Aryal ICR Primary Author: Veruschka Schmidt F. Results Framework Analysis - At project approval, no country-specific results framework existed; only six PDO Indicators had been defined. Following a Level 1 Restructuring in June 2011, a country- specific results framework was introduced. The original six PDO Indicators were revised and reduced to two PDO Indicators, and in addition, seven Intermediate Outcome Indicators were introduced. 225 Project Development Objective (PDO) – as presented in the GEF Trust Fund Grant Agreement: 259 To assist the recipient in: (a) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and (b) implementing measures to reduce and prevent future related risks. Revised PDO – as presented in the Level 1 Restructuring of June 2011 (Report No: 62797-TN): To assist the Government of Tunisia in: (i) eliminating and/or safeguarding inventoried publicly held obsolete pesticide stocks and associated waste; and (ii) implementing measures to reduce future related risks. Global Environment Objectives (GEO) - as presented in the PAD (Report No: 32232-MNA): Reduction of the effects of Persistent Organic Pollutants (POPs) on the global environment. Revised GEO – Not Applicable (a) PDO Indicators – Original PDO Indicators as presented in the GEF Trust Fund Grant Agreement are assessed against achievements realized at the time of the project’s Level 1 restructuring in June 2011, before revised PDO Indicators are assessed against achievements realized at project closing in June 2012. Original Target Formally Actual Value Achieved at Values Indicator Baseline Value Revised Target Completion or Target (from approval Values Years documents) Original PDO Indicators Inventory database of publicly held obsolete pesticide stocks in place and being Indicator 1: used by the PMU and government’s staff Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 09/30/2009 - 04/30/2007 Target fully achieved (100%): An inventory was completed and validated in 2007, Comments (incl. % including transfer of data into the FAO obsolete pesticide inventory database. The achievement) database was subsequently used for preparation of the CESA, and disposal activities. Indicator 2: Completion of CESA and implementation of the measures contained therein. Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 09/30/2009 - 06/30/2011 Target fully achieved (100%): Despite delays, a CESA was completed in February Comments (incl. % 2010, and found satisfactory by the World Bank. Recommendations contained achievement) therein were used for disposal bidding documents and safeguarding activities. Inventoried publicly held obsolete pesticide stocks disposed of, as a result of the Indicator 3: completion of the disposal services contract in accordance with national and international laws and regulations 259 The PDO, indicators, and components that are stated in the Project Appraisal Document (PAD) slightly differ from those stated in the GEF Trust Fund Grant Agreement. This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities; in addition, changes under a Level 1 restructuring were introduced to the GEF Trust Fund Grant Agreement. 226 Value (quantitative 0% 100% - 0% or qualitative) Date Achieved 04/19/2004 09/30/2009 - 06/30/2011 Target not achieved (0%): Disposal contract was in place. 560 tonnes of the Comments (incl. % inventoried publicly held obsolete pesticide stocks (2036 tonnes) had been achievement) repackaged but not yet shipped and disposed of overseas. Legal and regulatory framework for pesticide management improved, including Indicator 4: measures to strengthen compliance with the Basel Convention and the Rotterdam Convention Value (quantitative No Yes - Partly (70%) or qualitative) Date Achieved 04/19/2004 09/30/2009 - 06/30/2011 Target partly achieved (70%): Enactment of two legal regulations by the Ministry of Comments (incl. % Agriculture (100%). Only limited measures were introduced to strengthen achievement) compliance with Basel Convention; Rotterdam Convention was not ratified (40%). The training program under Part D 260 of the project is implemented and the Indicator 5: knowledge so acquired is being used by PMU and recipient’s government staff Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 09/30/2009 - 06/30/2011 Target fully achieved (100%): PMU members and representatives from the ministries Comments (incl. % of environment, agriculture, health, participated in six pesticide management achievement) trainings course. The acquired knowledge was applied during the inventory, and safeguard requirements. Indicator 6: PMU is functional and operating satisfactorily Value (quantitative No Yes - Yes or qualitative) Date Achieved 04/19/2004 09/30/2009 - 06/30/2011 Comments (incl. % Target fully achieved (100%): PMU was functional and operated satisfactorily. achievement) Revised PDO Indicators as per the Level 1 Restructuring in June 2011 Percent of publicly held inventoried obsolete pesticides and associated waste Indicator 1: disposed of or safeguarded by end of project Value (quantitative 0 100 - 85 or qualitative) Date Achieved 06/30/2011 06/30/2012 - 06/30/2012 Target partly achieved (85%): 1730 tonnes (85%) of the inventoried 2036 tonnes of Comments (incl. % publicly held obsolete pesticides and associated waste were disposed of or achievement) safeguarded. These include 1,595 tonnes of obsolete pesticides (against 1,280 tonnes inventoried), and 135 tonnes of associated waste (against 756 tonnes inventoried). 260 See Section 1.5 for the project’s Component D (in the GEF Trust Fund Grant Agreement referred to as ‘Part D’). 227 A strategy and/or action plan to sustainably improve management and reduce Indicator 2: accumulation of obsolete pesticides (including Integrated Pest Management) developed and endorsed by the Steering Committee Value (quantitative None Adoption - Adoption or qualitative) Date Achieved 06/30/2011 06/30/2012 - 06/30/2012 Comments (incl. % Target fully achieved (100%): A national action plan was approved by the Steering achievement) Committee and subsequently adopted by the Ministry of Agriculture in June 2012. (b) GEO Indicator(s) – The GEO was measured by the PDO and PDO indicators. (c) Intermediate Outcome Indicator(s) Original Target Actual Value Values (from Formally Revised Target Achieved at Indicator Baseline Value approval Values Completion or Target documents) Years Indicator 1: PSMS training completed Value (quantitative Training Yes - Yes or qualitative) in 2007 Date Achieved 06/30/2011 06/30/2012 - 06/30/2012 Comments (incl. % Target fully achieved (100%): Second PSMS trainings conducted by FAO in 2012. achievement) Indicator 2: Inventory completed Value (quantitative Yes - - Yes or qualitative) Date Achieved 06/30/2011 - - 04/30/2007 Indicator corresponds to original PDO Indicator #1 and was completed prior to Comments (incl. % restructuring. Therefore, the indicator is not taken into account for post-restructuring achievement) assessment. Indicator 3: CESA completed Value (quantitative Yes - - Yes or qualitative) Date Achieved 06/30/2011 - - 02/28/2010 Indicator corresponds to original PDO Indicator #2 and was completed prior to Comments (incl. % restructuring. Therefore, the indicator is not taken into account for post-restructuring achievement) assessment. Indicator 4: Drafting and piloting of a container management plan in two governorates Consolidation of Piloting of empty Value (quantitative the piloting pesticide container None - or qualitative) results in the two collection in two governorates. governorates. 228 Date Achieved 06/30/2011 06/30/2012 - 06/30/2012 Target partly achieved (100%): A container management plan was drafted, piloted in Comments (incl. % two governorates since 2009, and evaluated in December 2010. The consolidation of achievement) results was not completed. Indicator 5: Communication strategies drafted Value (quantitative 1 2 - 2 or qualitative) Date Achieved 06/30/2011 06/30/2012 - 06/30/2012 Target fully achieved (100%): Two awareness-raising strategies were drafted, of Comments (incl. % which one was drafted and subsequently implemented between April and December achievement) 2008; the other was finalized in June 2012 and implemented thereafter. Number of revised pesticide management regulation(s) drafted and submitted Indicator 6: for promulgation Value (quantitative 2 - - 2 or qualitative) Date Achieved 06/30/2011 - - 06/10/2011 Indicator corresponds to original PDO Indicator #4 and was completed prior to Comments (incl. % restructuring. Therefore, the indicator is not taken into account for post-restructuring achievement) assessment. Number of training courses provided on improved pesticide management Indicator 7: practices Value (quantitative 32 28 - 36 or qualitative) Date Achieved 06/30/2011 06/30/2012 - 06/30/2012 Newly introduced indicator but building on original PDO Indicator #5. Target Comments (incl. % exceeded (114%) by eight training courses provided on improved pesticide achievement) management practices. Indicator did not specify beneficiaries of training courses. G. Ratings of Project Performance in ISRs – Most DO and IP ratings and actual disbursement data are not available because ISRs reported on ASP-P1 as a whole, and only occasionally on country-specific project performance, including disbursement. Date ISR Actual Disbursements No. DO IP Archived (USD millions) 1 10/31/2005 - - 0.00 2 06/30/2006 - - - 3 12/27/2006 - - - 4 06/28/2007 Satisfactory Satisfactory - 5 12/18/2007 Satisfactory Satisfactory - 6 05/30/2008 - - - 7 12/18/2008 Moderately Satisfactory Satisfactory - 229 8 03/25/2009 Satisfactory Satisfactory - 9 12/29/2009 Moderately Satisfactory Satisfactory 0.68 10 06/29/2010 Satisfactory Satisfactory - 11 03/27/2011 - Moderately Satisfactory 0.99 12 12/25/2011 Moderately Satisfactory Satisfactory 0.99 13 06/29/2012 Satisfactory Satisfactory 2.69 03/19/2013 Final disbursement 3.78 H. Restructuring ISR Ratings at Amount Board Restructuring Disbursed at Restructuri Approved Reason for Restructuring ng Date(s) PDO Restructuring & Key Changes Made PDO IP in USD Change millions 11/25/2009 No Not Not Not • Extension of closing date by 22 months Available Available Available to compensate for delays (09/30/2009 to 07/31/2011). • Reallocation of proceeds to allow for the recruitment of a Technical Advisor for Disposal (TAD), preparation of specific risks mitigation plans, and training 06/30/2011 Yes Moderately Moderately 1.0 • Revision of the PDO and indicators to Satisfactory Satisfactory reflect a shift from complete disposal to disposal and safeguarding of stocks • Introduction of a results framework for better M&E • Extension of closing date by eleven months to compensate for delays (07/31/2011 to 06/30/2012) • Reallocation of proceeds to dispose additional pesticide stocks • Addition of OP/BP 4.12 on Involuntary Resettlement If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body), enter ratings below: Outcome Rating Outcome Ratings Against Original PDO targets Moderately Satisfactory Against formally revised PDO targets Satisfactory 230 Outcome Rating Outcome Ratings Overall (weighted) rating Satisfactory I. Disbursement Profile – A disbursement profile is not available (see explanation under G). 231 1. PROJECT CONTEXT, DEVELOPMENT AND GLOBAL ENVIRONMENTAL OBJECTIVES AND DESIGN 1.1 CONTEXT AT APPRAISAL 1. Country background: Despite the change and diversification observed in the Tunisian economy (industrialization, growth of service sector and expansion of tourism), the agricultural sector had remained economically and socially important for achieving national objectives such as food security, employment, regional equilibrium and social cohesion. At project approval in September 2005, agriculture employed more than one-quarter of the country’s workforce, and accounted for around 13% of GDP. 261 2. Sector background: Around 3000 tonnes of pesticides had been imported annually to boost agricultural production. The bulk was used on cereal crops to control insect pests, and diseases on food crops. The Ministry of Agriculture had been a major pesticide user during national pest control campaigns (e.g., locust and rodent control); in expectation of pest outbreaks, the Ministry had stored large quantities of pesticides. 262 3. Obsolete pesticide accumulation: Studies conducted in 1997 had revealed the existence of a number of stocks of obsolete pesticides, including Dichlorodiphenyltrichloroethane (DDT). These stocks had mostly accumulated through massive overprovision in relation to actual needs for public health vector control and combating migratory pests such as locusts. Some of these stocks had been stored for over 45 years. At appraisal, Tunisia had hundreds of sites containing obsolete pesticide stockpiles, which potentially posed serious health hazards to nearby populations. Product bans, uncoordinated donations, poor stock management, and inadequate storage had been the additional reasons for the stockpiling of pesticides in the country. 4. Institutional background: Prior to Tunisia becoming a party to the Stockholm Convention in June 2004, a wide process of dialogue and information exchange had been initiated by the Ministry of Environment with other ministries and agencies, which were concerned about the management of dangerous chemicals and wastes; a number of high level meetings among all stakeholders were held to effectively collaborate on developing a National Implementation Plan (NIP) under the Stockholm Convention to ensure ecologically sound management of Persistent Organic Pollutants (POPs). At appraisal, all pesticides imported or formulated in the country had to be registered with the Ministry of Agriculture. Pesticides used for public health purposes were subject to prior authorization by the Ministry of Health. However, the implementation of these regulations was not always rigorous, particularly the regulations that governed the labeling, conditioning, handling, transport and storage of these products. 263 5. Actions taken: At the time of project design, the Government of Tunisia had initiated preparatory activities under its NIP, 264 with the support of the United National Environment Program (UNEP) and 261 The Economist Intelligence Unit, Country Profile Tunisia 2005 262 Africa Stockpiles Program – IPM Research Project: Phase I Country Background Papers, Pesticide Action Network UK, February 2007 263 Tunisia Country Situation Report - Association for Environmental Protection in Kerkennah (APEK), October 2005, The International POPs Elimination Project 264 Plan d’Action National de la Tunisie pour la Mise en Œuvre de la Convention de Stockholm sur les Polluants Organiques Persistants, January 2007, République Tunisienne. 232 the United Nations Industrial Development Organization (UNIDO). These activities included completion of a preliminary inventory of obsolete pesticides across the country, which had identified 128 sites with estimated 1,200 tonnes of obsolete stocks. Subsequently, the inventoried obsolete stocks were pooled into 17 secured sites. In addition, the government officially established a Project Management Unit (PMU) within the Department of Solid Wastes at the National Environment Protection Agency. 6. Rationale for World Bank assistance: Please refer to Section 1.1 in the program-level ICR. 7. Higher-level objectives: The project intended to contribute to Tunisia’s national development strategies in the areas of public health, poverty alleviation, environmental protection, and strengthening the agricultural and rural development sector as outlined in Tunisia’s Country Assistance Strategy (CAS). 265 In particular, the project aimed to (i) improve the quality of life in poor communities, by reducing environmental health risks, (ii) improve environmental protection, and (iii) enhance the capacity of the agricultural sector to better manage crop pests. Contributing to international efforts to eliminate POPs, improving management of toxic chemicals, and promoting alternatives to pesticide usage that include Integrated Pest Management (IPM) and Integrated Vector Management (IVM) represented further objectives. 1.2 ORIGINAL PDO AND INDICATORS 8. The Project Development Objective (PDO), indicators, and components that are stated in the Project Appraisal Document (PAD) 266 slightly differ from those stated in the GEF Trust Fund Grant Agreement. 267 This ICR bases its evaluation on the PDO, indicators, and components presented in the GEF Trust Fund Grant Agreement, which served as the principal guiding document for the project’s implementation activities. 9. ASP-P1 Tunisia’s original PDO as presented in the GEF Trust Fund Grant Agreement: 268 To assist the recipient in: (i) eliminating inventoried publicly held obsolete pesticide stocks and associated waste; and (ii) implementing measures to reduce and prevent future related risks. 10. At appraisal, only an overall ASP-P1 results framework had been prepared (see program-level ASP ICR); results frameworks for the individual ASP-P1 countries, including Tunisia, were intended to be finalized during the individual ASP-P1 country project launches. However, country-specific PDO indicators were developed at appraisal. For ASP-P1 Tunisia, these were as follows: Table 1.2: ASP-P1 Tunisia Original PDO Indicators Original PDO Indicators Original PDO Indicators as presented in the PAD as presented in the GEF Trust Fund Grant Agreement a. Inventory database of publicly held obsolete pesticide a. Inventory database of publicly held obsolete pesticides stocks in place and being used by the PMU and the stocks in place and being used by the PMU and the Government of Tunisia’s staff. recipient's government staff. b. Completion of Country Environmental and Social b. Completion of CESA and implementation of the Assessment (CESA) and implementation of each CESA’s 265 Country Assistance Strategy for the Republic of Tunisia, (Report No. 20161-TN), March 28, 2000. 266 South Africa and Tunisia: Africa Stockpile Program – Project 1: PAD (Report No: 32232-MNA), August 19, 2005 267 GEF Trust Fund Grant Agreement between the Republic of Tunisia and the International Bank for Reconstruction and Development, (Number TF055745-TN), November 21, 2005 268 ASP-P1 Tunisia’s PDO as presented in the PAD: To eliminate inventoried publicly held obsolete pesticide stockpiles and associated waste, and implement measures to reduce and prevent future related risks. 233 measures. measures contained therein. c. Inventoried publicly held obsolete pesticides stocks c. Inventoried publicly held obsolete pesticides stocks disposed of, as a result of the completion of the disposal disposed of, as a result of the completion of the services contract(s) in accordance with national and Disposal Services contract(s) in accordance with international laws and regulations. national and international laws and regulations. d. Legal and regulatory framework for pesticide management d. Legal and regulatory framework for pesticide improved, including measures to strengthen compliance management improved, including measures to with the Basel Convention and the Rotterdam Convention. strengthen compliance with the Basel Convention and e. IPM Strategy developed. the Rotterdam Convention. f. Training program to be implemented and the knowledge so e. The training program under Part D of the Project is acquired being used by PMU and government staff. implemented and the knowledge so acquired is being used by PMU and recipient's government staff. 269 g. The PMU to be functional and operating satisfactorily. f. The PMU is functional and its operation is satisfactory to the World Bank. 1.3 REVISED PDO AND INDICATORS 11. The PDO and associated indicators for ASP-P1 Tunisia were revised through a Level I restructuring on June 30, 2011. 270 The original PDO had implied that all inventoried obsolete pesticides would be eliminated in high temperature incineration facilities overseas. With the discovery of higher than estimated stocks, including associated waste, it was preferable to adopt a blended approach of disposal overseas and lower cost safeguarding in-country based on risk profiling of the inventoried pesticides and associated waste, and availability of satisfactory storage facilities (i.e. obsolete pesticides and higher risk stocks of heavily contaminated soils being disposed of overseas while lower risk stocks of associated waste being safeguarded in-country). In addition, the original PDO had assumed no new accumulations of obsolete pesticides during the life of the project, which had been proven difficult to achieve in the context of agricultural intensification and food security. A focus on developing a strategy for sustainable management of future accumulations was pursued instead. 12. ASP-P1 Tunisia’s revised PDO: To assist the Government of Tunisia in: (i) eliminating and/or safeguarding inventoried publicly held obsolete pesticide stocks and associated waste; and (ii) implementing measures to reduce future related risks. 13. ASP-P1 Tunisia’s PDO indicators were reduced in number and revised to focus on the updated project objectives, i.e., disposal and/or safeguarding and reduction of future accumulations (see Table 1.4). In addition, a country-specific result framework was introduced for effective monitoring and evaluation of project progress (see Data Sheet). 14. ASP-P1 Tunisia’s revised PDO Indicators: (i) 100% of publicly held inventoried obsolete pesticides and associated waste disposed of and/or safeguarded by the end of the project; and (ii) a strategy and/or action plan to sustainably improve management and reduce accumulation of obsolete pesticides (including IPM) developed and endorsed by the Steering Committee. 269 See Section 1.5 for the project’s Component D, here referred to as “Part D”. 270 Revisions were approved on June 30, 2011, on the basis of: Restructuring Paper on a Proposed Project Restructuring of Africa Stockpiles Program – Project 1 to the Republic of Tunisia, (Report No: 62797-TN), June 20, 2011 234 1.4 MAIN BENEFICIARIES 15. ASP-P1 Tunisia was designed to eliminate a major source of risk to human health and environmental resources. Primary beneficiaries of the project were as follows: 16. Local beneficiaries: Primary beneficiaries were intended to be poor and farming communities. The project aimed to improve their quality of life by reducing environmental health risks: (i) Poor communities: Since obsolete pesticide stockpiles were often located in poorer communities where people had low awareness regarding the consequences of exposure to contaminated sites and storage equipment, the intended primary beneficiaries were the urban and rural poor; particularly those living in proximity to the 205 identified obsolete pesticide stockpile sites, spread over 24 governorates. (ii) Farming communities: A wide range of pesticides had been used for pest management and vector control in agricultural areas, with pesticide distributors and small-scale farmers often having low levels of awareness for exposure risks, little knowledge of safe pesticide handling and alternative pest control, inadequate storage facilities and limited alternatives. 17. National beneficiaries: The government, in particular the ministries of health, environment, and agriculture, were intended to be institutional beneficiaries. Benefits were planned to include cost- effective pesticide management in terms of reduced loss in capital, or improved effectiveness in appropriate planning, management, and monitoring regarding obsolete pesticides. 18. Global beneficiaries: The global community of people, species, and ecosystems was considered as global beneficiaries. Taking into account that obsolete pesticides are often characterized by high persistence in the environment, 271 the environmental impact in the form of damaged ecosystems had been considered serious. Extensive cleanup and contamination-prevention activities were intended to allow for substantial global environmental health benefits. 19. Beneficiaries were not modified during project implementation. 1.5 ORIGINAL AND REVISED COMPONENTS 20. The PDO of ASP-P1 Tunisia was to be achieved through the implementation of four distinct components (outlined in Table 1.3). The table also shows revisions that were made through a Level I restructuring on June 30, 2011. Table 1.3: ASP-P1 Tunisia Original and Revised Components Original Components Original Components as Revised Components (Level I as presented in the PAD presented in the GEF Trust Restructuring June 30, 2011) 272 Fund Grant Agreement Component 1.A – Disposal of obsolete Part A – Obsolete pesticides Part A – Disposal and/or safeguarding of pesticides (Total:US$3.90 million; inventory: obsolete pesticides and associated waste: GEF:US$3.67 million): a. Collection of data and a. Carrying out a technological assessment a. To complete detailed stock compilation of a database; of available treatment and disposal inventories; b. A risk-based prioritization of options; b. To produce a CESA including an the stocks identified in the b. Implementation of the safeguarding 271 For example, DDT persistence in soil ranges from 22 to 30 years, Toxaphene up to 14 years, and Dieldrin up to 7 years. 272 Changes outlined in more detail in the Restructuring Paper on a Proposed Project Restructuring of Africa Stockpiles Program – Project 1 to the Republic of Tunisia, (Report No: 62797-TN), June 20, 2011, were introduced to the original components as presented in the GEF Grant Agreement. 235 Environmental Management Plan database; and and/or disposal technology selected; and (EMP); and c. Preparation of the CESA. c. Improving the operations of obsolete c. To dispose of obsolete pesticides pesticide stocks collection centers, including contaminated containers. including improving safety measures at those centers. The changes reflected the addition of safeguarding as an outcome and clearly defined associated waste as an integral part of the stocks to be treated. Part B – Disposal of obsolete Part B – Disposal of obsolete pesticides: pesticides: No changes were introduced at restructuring a. Carrying out of a technological assessment of available treatment and disposal options; b. Implementation of the treatment and/or disposal technology selected; and c. Improving the operations of obsolete pesticide stocks collection centers, including improving safety measures at those centers. Component 1.B: Prevention of Part C – Prevention of obsolete Part C: Reduction of re-accumulation: accumulation (Total: US$0.40; GEF: pesticide accumulation: a. Awareness raising activities on the health US$0 million): a. Awareness raising activities on and environmental hazards of pesticides; a. Strengthen existing regulatory the health and environmental b. A review and update of the legal and system for pesticide control; hazards of pesticides; regulatory framework to control obsolete b. Promote ongoing IPM efforts, b. A review and update of the pesticide accumulation and enforcement particularly with small scale farmers; legal and regulatory mechanisms; c. Promote certified organic framework to control obsolete c. Preparation and implementation of an agricultural production; pesticide accumulation and IPM strategy; d. Develop a communication campaign enforcement mechanisms; d. A review of pesticide management to raise awareness about pesticide c. Preparation and practices and preparation and impact and opportunities created by implementation of an IPM implementation of a training plan; and IPM; and strategy; and e. Preparation and endorsement of a strategy e. Upgrade pesticide storage facilities. d. A review of pesticide and/or action plan to sustainably manage management practices and and reduce the re-accumulation of preparation and obsolete pesticides. implementation of a training The change reflected the transition from plan. prevention (zero new accumulation) to reduction of new accumulations. Component 1.C: Capacity building Part D – Capacity building: Part D – Capacity building: No changes (Total: US$0.23 million; GEF: US$0 Strengthening the capacity of the were introduced at restructuring million): PMU to implement the project a. Risk assessment of soil through the provision of contamination; training. b. Assessment of laboratory needs; and c. Training program in pesticide 236 storage management and Integrated Pest Management (IPM). Component 1.D: Project management Part E – Project management: Part E – Project management: No changes and Monitoring and Evaluation Supporting the PMU in carrying were introduced at restructuring (Total: US$0.60 million; GEF: out project coordination, US$0.32 million): The component was implementation, monitoring and intended to establish: evaluation, through the provision a. A PMU at ANPE; of technical advisory services, b. A National Steering Committee; and equipment and operational c. A Monitoring and Evaluation (M&E) support. system. 1.6. OTHER SIGINIFICANT CHANGES Table 1.4: ASP-P1 Tunisia Other Significant Changes Change Date Justification Approval Extension of closing date Extension of closing date by 22 11/25/ 2009 To compensate for initial delays. Country months from 09/30/2009 to Director 07/31/2011. Extension of closing date by 11 06/30/2011 To compensate for delays experienced during the Board months from 07/31/2011 to disposal tender process and delays that had been 06/30/2012. caused by the socio-political unrest in Tunisia. Reallocation of proceeds Reallocation of proceeds from the 11/25/ 2009 To accommodate (i) the recruitment of a Technical Country Unallocated category (- Advisor for Disposal (TAD); (ii) the preparation of Director US$400,000) to the Consultant specific risks mitigation plans for high risk sites to Services, Audit, Training and guide disposal activities; and (iii) training, workshops Workshops category. and M&E activities related to the project’s EMP. Reallocation of proceeds from the 06/30/2011 To accommodate the 100% disposal and/or Board categories Consulting Services (- safeguarding of the additional stocks and associated US$250,000) and Operating Costs (- waste inventoried. Contributions received from FFEM US$100,000) to the Disposal and the government were planned to cover the Services category (+US$350,000). budgeted consulting services and operating costs, respectively. Redefinition of activities Redefinition of the term “Disposal 06/30/2011 Addition of in country safeguarding as a possible Board Services” from services related to cheaper end result for lower risk stocks; this was due the treatment or disposal of obsolete to larger than budgeted stocks of obsolete pesticides pesticides and associated waste, to and associated waste. services related to the safeguarding and/or disposal of the same. Addition of safeguard policies Addition of OP/BP 4.12 on 06/30/2011 As part of the Environmental Assessment carried out Board Involuntary Resettlement to the list by the project, OP/BP 4.12 was triggered for a storage of triggered safeguard policies. site, requiring a succinct resettlement plan. 237 2. KEY FACTORS AFFECTING IMPLEMENTATION AND OUTCOMES 2.1 PROJECT PREPARATION, DESIGN, AND QUALITY AT ENTRY 21. ASP-P1 Tunisia was prepared between 2001 and 2005, as part of preparatory activities at the program-level. It took more than four years from GEF eligibility in July 2001, and more than three years from concept note in July 2002 to project approval by the World Bank Board in September 2005. 273 22. Soundness of background analysis: Project preparation was conducted in close collaboration between the government and the World Bank, the Food and Agriculture Organization (FAO), CropLife International (CLI), and FFEM, as was evident through joint missions, and pledged financial contributions from all partners. Key preparatory outputs such as the PAD and Operations Manual, however only broadly laid out the project’s plans; detailed country-specific information regarding M&E, risks and risk mitigation measures, Financial Management (FM) including disbursement plans, and procurement were missing. In addition, the political situation in Tunisia was not adequately reflected in the project’s design regarding Non-Governmental Organization (NGO) involvement, as became evident from the limited independence of local NGOs during project implementation. 23. During project preparation, Département des Déchets Solides (DDS) was identified as the implementing agency. DDS was anchored in the ANPE, whose overall capacity to implement the project was assessed adequate by the World Bank, particularly based on its experience in executing projects of similar magnitude. 274 However, it was noted that specialized staff would need to be identified before appraisal and that training and technical supervision in project management, FM, and procurement would be crucial. Shortly before negotiations, the DDS was separated from ANPE and reestablished as Agence Nationale de Gestion des Dechets (ANGeD) under the Ministry of Environment, resulting in a different legal status. A renewed assessment of FM and procurement capacity was subsequently necessary, but only conducted together with a procurement assessment a couple of months after project approval. The late assessment drew attention to some key FM elements that were missing. 275 24. Assessment of project design: The project’s design (as for all ASP-P1 participating countries) was preconfigured at the program level, and did not reflect any country-specific modifications. Design weaknesses at the program level that translated into implementation delays at the country-level, including ASP-P1 Ethiopia, are outlined in Section 2.1 of the program-level ICR. 25. Government commitment: Government commitment at entry towards the issue of obsolete pesticides was evident through preliminary steps the government had undertaken in view of the problem obsolete pesticides were posing. These included activities to reduce short-term risks through controlled storage, and activities targeted at reducing risks in the long-term through import regulation, legislation on registration of imported products, monitoring of wholesalers and retailers, and research 273 The long preparation time resulted from a lengthy partnership building/negotiation process at the program-level resulting from the complex implementation arrangement and project design (see Section 2.1 in the program-level ICR). 274 ANPE had been managing several environmental projects in collaboration with donors such as the KfW (German Development Bank), GEF, or the World Bank. 275 Missing elements included the development of an organizational manual, clearly assigned FM and accounting responsibilities, recruitment of an internal auditor, and presentation of a detailed budget plan. 238 and implementation of alternatives such as IPM, and organic farming. Further steps included the completion of a preliminary inventory of obsolete pesticides across the country with subsequent pooling of inventoried stocks into secured sites. 26. Assessment of risks: Program-level risks were adequately identified and outlined in the PAD. They were presented in the form of a matrix and represented a summary of risk assessments undertaken during identification, pre-appraisal and appraisal missions to ASP-P1 participating countries. Even though the assessment of risks was comprehensive and included a broad range of mitigation measures in generic terms, an assessment of risks and mitigation measures specific to Tunisia was lacking. At the same time, it should be noted that it was difficult to assess country-specific risks in the absence of inventories of obsolete pesticides and CESAs, which were only planned to be undertaken during project implementation. 2.2 IMPLEMENTATION 27. Throughout implementation, the project experienced long delays, eventually resulting in two extensions for a total of 33 months, and a Level 1 restructuring in 2011. 28. Key factors affecting implementation and outcomes included: • Delayed and insufficient technical support affected implementation performance. The PMU was supposed to receive highly technical and specialized expertise (required for implementation, supervision and monitoring of project activities) from FAO’s Technical Support Unit (TSU); however, this support was not delivered to the extent originally planned. Reasons were mainly collaboration issues between the World Bank and FAO, resulting from a complex and loosely defined stakeholder implementation arrangement at the program-level (see Section 2.2 in the program-level ICR). The limited support of procurement and M&E significantly affected the PMU’s project implementation capacity. The delayed recruitment of a Technical Advisor for Disposal (TAD), originally intended to be financed by CLI, further affected implementation.276 To fill the gap, the World Bank temporarily recruited an experienced obsolete pesticide and hazardous waste expert in July 2009, and together with the TAD’s eventual recruitment in September 2010, good implementation progress was consequently achieved. • Institutional challenges: The delayed institutional set-up had an effect on the project’s start-up activities. It took more than seven months before project activities began, because ANGed and the Ministry of Agriculture (DGPCQPA) had difficulties to agree on exact responsibilities. Their collaboration continued to be challenging thereafter and particularly affected implementation of prevention activities. Eventually in 2010, responsibilities associated with the management of the prevention component were handed over from ANGed to the Ministry of Agriculture.277 Collaboration improved gradually in the following. Further initial delays were caused by the PMU’s lack of specialized staff; until the mid-term review, the project coordinator was handling most of the project’s administrative tasks, including procurement and M&E related tasks. In 276 The recruitment of a TAD financed by CLI had been planned for February 2007. However, long negotiations between CLI, the World Bank, and the PMU on the financing modalities, had not led to an agreement. The eventual recruitment in 2010 was financed through reallocation of existing GEF funds. 277 Prevention activities were co-financed in parallel by the FFEM. As a result of the above mentioned initial delays, FFEM extended its financing after project closing by 12 months. 239 2010, the transition associated with the recruitment of a new project coordinator put project implementation temporarily on hold. • Higher than estimated obsolete pesticide stocks: At mid-term review in 2008, the project was facing a funding gap of around US$2.5 million as a result of higher than estimated obsolete pesticides, and an increase in unit cost for disposal.278 A subsequent estimate in 2009, determined the funding gap to be US$1.735 million. The government requested additional GEF financing in the amount of US$770,000, and possible co-financing by CLI was considered. However, in 2010, it was assumed that no additional funding would be needed as existing funds under the original GEF grant would be sufficient following ANGed’s decision to reallocate some of the funds. In addition, the competitive bidding process for the elimination contract had resulted in a lower disposal cost per unit in comparison to the one originally estimated. During disposal activities, nevertheless, further obsolete pesticide stockpiles were identified. • Weak NGO integration: The participation of local NGOs in prevention and information activities (intended to be strengthened through crosscutting support provided by PAN-UK and WWF) was weak throughout the project. Main reasons for the low involvement were (i) Tunisia’s political environment in which local NGOs were not allowed except for those mandated by the government, and (ii) a lack of agreements determining the extent of responsibilities, roles and extent of support between the PMU, and the NGOs. By recognizing the deficit, and to improve collaboration with the local NGO community, in 2009, the World Bank recommended carrying out a national workshop in order to jointly develop a framework for an improved and more effective involvement in project activities. Eventually in October 2012, such workshop took place; it was delayed as it was embedded in the second communication strategy that had only been completed in June 2012. • Restructuring: At mid-term review, the World Bank recommended revising the PDO, raising additional funding, and extending the project’s closing date. In addition and in view of the lack of technical support described above, the World Bank urged to strengthen the PMU with technical expertise during disposal activities, and administrative and Monitoring and Evaluation (M&E) support. Even though it took over two years as a result of political reluctance in Tunisia to restructure the PDO, 279 eventually the project was restructured in June 2011. 280 2.3 MONITORING AND EVALUATION (M&E) DESIGN, IMPLEMENTATION, AND UTILIZATION 29. Overall M&E - Rating Moderately Unsatisfactory. 278 The estimated unit cost for disposal had increased to US$3,500 as opposed to US$2,625 at appraisal, as a result of higher than estimated stocks, fluctuations in the exchange rate of the US dollar, and higher energy costs. 279 See Implementation Status and Results Report (ISR) – Sequence #9 280 Additional delays following the restructuring of the project resulted from a necessary three-months extension of the disposal contract caused by: (ii) the Spanish authorities that took five months to issue the necessary authorizations which were required for exporting the obsolete pesticides in accordance with the recommendations of the Basel Convention; (iii) changes to the maritime itinerary imposed by the shipping company; and (iv) the events associated with the Tunisian revolution that began in December 2010, and which were accompanied by various governmental strikes and blocked roads affecting accessibility of several sites. 240 30. M&E design – Rating Unsatisfactory: Despite the lengthy preparation time, an M&E system specific for Tunisia was not developed before or during appraisal. 281 A result framework, which was intended to be finalized by the TSU during project launch and included in the project’s Operational Manual, did not materialize as well. Only a set of PDO indicators linked to the different project components was developed in the PAD and GEF Trust Fund Grant Agreement, however, these did not allow for adequate monitoring of project implementation progress. The indicators focused on measuring outputs as opposed to progress towards project outcomes. Expected results under the prevention component, in particular, were not adequately considered by the indicators. 31. M&E implementation – Rating Unsatisfactory: Until the mid-term review in October 2008, there was neither an M&E specialist nor an M&E system in place, and several delays in submitting progress reports were noted. Eventually, in November 2009, the TSU provided the PMU with and trained it in using a Disposal-Project Tracking Workbook to monitor and evaluate project-related activities. However, the tool was complex, and introduced too late in the process to be used by the PMU as most of the activities the system should have monitored (e.g., the inventory) had already been completed. The PMU compensated for the lack of an M&E system by using self-created tables for progress monitoring. In 2011, following the restructuring of the project, a results framework specific to Tunisia was finally introduced and the PDO and associated indicators were revised to focus on the two main project achievements (disposal and/or safeguarding and reduction of future accumulations) which strengthened the M&E of the project, albeit late in the process. 32. M&E utilization – Rating Moderately Satisfactory: The inventory of obsolete pesticide stocks was conducted by using inventory forms and a manual provided by the FAO Environmental Management Toolkit (EMTK). 282 The inventory data was entered into a database of the FAO web-based Pesticide Stock Management System (PSMS) to record and monitor the pesticide stocks, and subsequently validated. The data was used by the PMU for risk profiling of the stocks 283 and preparation of the CESA. FAO had provided training to the PMU in how to best use the database, which took place in 2008 and 2012. The PSMS together with the EMTK proved to be very useful tools in managing obsolete pesticides stocks. 2.4 SAFEGUARD AND FIDUCIARY COMPLIANCE 33. Safeguard compliance – Rating Satisfactory: ASP-P1 triggered two safeguard policies: OP 4.01 on Environmental Assessment and OP 4.09 on Pest Management; ASP-P1 Tunisia triggered OP 4.12 on Involuntary Resettlement. 34. ASP-P1 Tunisia complied with OP 4.01 on Environmental Assessment: A CESA identifying risks and devising mitigation measures to address potential negative environmental and social impacts was completed to the World Bank’s satisfaction in February 2010, albeit with significant delays. The assessment included extensive public consultations, conducted between June and December 2009. 281 Lengthy discussions between the partners during project preparation on whether to have an M&E system at the program-level alone, or also at the country-level, may have contributed to the absence of an M&E system at the country-level at project launch. However, no information was available on whether it was a deliberate decision not to have an M&E system in place at the country-level. 282 The tool kit provides a practical set of methodologies to assist countries in the risk-based management of obsolete pesticides, including guidance on the management of collection/storage locations, based on international best practice. 283 Sites were evaluated by risk category (high, medium, and low). 241 EMPs and a Health and Safety Plan were equally completed. The assessment’s late completion resulted from: a) the project’s sequential design (the inventory was only finalized in 2007); b) a complicated and time-consuming consultant procurement process; and c) lengthy World Bank review and clearance processes. 284 Subsequently, project activities involving storage, transportation, disposal, and site remediation were implemented in full compliance with the recommendations and requirements of the CESA, the associated EMP, and the Health and Safety Plan. 285 With exception of the first two years (which were intended to focus on training local staff in conducting the inventory and risk assessment), World Bank safeguard specialists monitored and ensured compliance with safeguard requirements through participation in supervision missions and specific safeguard review missions. 286 Disposal operations were completed without reported incidents or accidents to personnel, or damages and/or pollution to the environment.287 35. OP 4.09 on Pest Management: Since project activities themselves met requirements of a pest management plan (i.e., through pesticide management, IPM, and capacity building), individual country projects, including ASP-P1 Tunisia, were not required to prepare self-standing documents. See Section 2.4 in the program-level for more details. 36. ASP-P1 Tunisia complied with OP 4.12 on Involuntary Resettlement: Based on the CESA’s recommendations, OP/BP 4.12 on Involuntary Resettlement was officially added to the list of triggered safeguard policies in June 2011 as part of the Level 1 restructuring. The policy was triggered for a storage site in Fathallah, a southern suburb of Tunis that was being used for business purposes by occupants. The high-risk site required a succinct Resettlement Action Plan (RAP) for the occupants of the site during disposal activities, as they were directly affected by the project. Despite smaller incidents and a difficult political environment resulting from the aftermath of the Tunisian revolution, the comprehensive resettlement action plan was implemented and completed to a highly satisfactory extent in June 2012. 288 37. Financial Management compliance – Rating Moderately Satisfactory: The responsible finance and accounting department was adequately staffed. Financial Monitoring Reports together with annual audits (conducted by an external auditor) were regularly prepared and submitted to the World Bank’s satisfaction, however often delayed. In addition, recommendations made during World Bank missions were often implemented with delays, the PMU and the financial and accounting department poorly collaborated with each other on several occasions, and an ineligible expenditure was noted. 289 284 The World Bank provided three sets of comments on the CESA over a period of 9 months (10/2008-06/2009), before it was submitted for final World Bank clearance and disclosure. 285 See Programme Africain relative aux Stocks de Pesticides Périmés (Phase I) – PASP Tunisie: Aide-mémoire – Mission de Supervision 5-8 juin 2012 286 In particular see Rapport de mission - Intégration des commentaires en vue de la finalisation de l’EIES PASP Tunisie 24–28 Juin 2009; and Mission de supervision du 05 au 08 juin 2012 – Aide-Mémoire. 287 Mission de supervision du 5-8 juin 2012 – Aide Memoire 288 A consultant was hired in October 2011, who prepared and oversaw necessary safeguarding activities until their completion in June 2012. Activities included a consultation campaign, financial compensation for the site’s occupiers, safeguarding activities, decontamination of the site, and the final return of the occupiers. During safeguarding activities, a scale (used by the occupiers to weigh the materials) and a metal gate owned by the occupiers broke but were subsequently compensated for by the World Bank. Also, the discovery of higher than estimated obsolete pesticides at the site delayed the estimated time frame of two months by four. 289 An unexplained discrepancy in the amount of US$1,128 between the balance of the special account listed on the statement of the Central Bank of Tunisia and the available balance shown in the counting records of ANGed in 2006 was considered as 242 38. Disbursement was low throughout implementation, with only 26% in June 2011, when the project was restructured. However, this was mainly attributable to the project’s design, which included the most costly activity (disposal) to take place at the end of the project. 39. Procurement – Rating Moderately Unsatisfactory: Procurement undertaken by ANGed suffered from a number of delays, particularly in selecting consulting firms to conduct the CESA 290 and disposal of obsolete pesticides. The delay was attributable to a lengthy negotiation process between FAO and the World Bank regarding the CESA consulting firm’s Terms of Reference (TORs), the PMU’s weak procurement capacity, and insufficient technical support that the FAO-TSU had originally intended to provide on the pre-qualification, bidding, and execution of the specialized services associated with procuring the selection processes. However, it should be noted that despite these challenges and delays, all services were eventually procured to the World Bank’s satisfaction. 2.5 POST-COMPLETION OPERATION/NEXT PHASE 40. Post-completion: Following the project’s closure in June 2012, the Government of Tunisia financed the elimination and safeguarding of 218 additional tonnes of obsolete pesticides and 44 additional tonnes of associated waste that had not been treated under ASP-P1 due to insufficient funds. The originally projected total of 1,280 tonnes had been exceeded by about 533 tonnes following the discovery of additional stocks during disposal activities; of those additional tonnes, only 315 tonnes had been eliminated under the project. The government’s commitment indicated the high priority given by the national authorities to address and resolve the issue of obsolete pesticides. 41. Next phase: FFEM co-funding of the prevention component was extended until March 2014 following the project’s closure. Achievements since project closure and expected outputs planned to be achieved prior to March 2014 include the following: • To raise awareness for sound pesticide management, two communication strategies had been drafted prior to project closing. The second draft was finalized in June 2012 and includes plans to raise awareness among approximately 4000 farmers located in five different governorates. Implementation is expected to be finalized by the end of 2013. Activities have been using communication tools such as posters, informational agricultural and scientific guide books, a video, three TV spots broadcasted on national television and three radio spots broadcasted on the local radio. To improve the collaboration with NGOs, ANGed called for proposals and, in May 2013, selected nine NGOs to support awareness raising activities. Specifically, they target raising awareness among primary school pupils to transmit information to their farming parents. Activities are expected to be completed by March 31, 2014. • Following the piloting of empty pesticide container collection in two governorates under the project, the government plans to establish a nation-wide network for the collection and processing of empty pesticide containers by 2015. A study on the network’s technical and economic feasibility is expected to be completed in the first half of 2014. Also planned by 2015 and in collaboration with the National Agency of Sanitary and Environmental Control Products ineligible expenditure at the end of 2011. The amount was subsequently refunded in March 2013, to the satisfaction of the World Bank. 290 An independent review of ANGed’s procurement procedures undertaken in June 2008 evaluated the processing time of 9.5 months for the selection of a consulting firm to conduct the CESA as too long. 243 is the establishment of a national vigilance system to control the risks associated with pesticides; a study regarding the system’s implementation is expected to be finalized in the first half of 2014. • In addition to the two pesticide management regulations that were promulgated in 2010 and 2011, the Ministry of Health is currently preparing an executive order for the establishment of a registration committee for the use of biocides and pesticides for public health purposes. • Lastly, following the acquisition of a mass spectrometer, which allows the government's laboratories to better analyze pesticides, a second mass spectrometer for Inductively Coupled Plasma analysis has been ordered and its delivery is expected at the end of 2013. 291 The additional machine will allow for fast identification and detection of trace metals content in samples, and to eventually analyze all types of imported pesticides. 3. ASSESSMENT OF OUTCOMES 3.1 RELEVANCE OF OBJECTIVES, DESIGN, AND IMPLEMENTATION 42. Relevance – Rating High: Despite having been too ambitious, the project’s original objectives were relevant when the project was prepared. The project’s revised objective, design, and implementation are considered highly relevant to the current national development priorities. The Government of Tunisia places strong emphasis on preserving the environment, reducing pollution, and supporting efforts to improve the quality of life in cities and rural areas, as evident from the country’s national development plan for the period 2010-14. 292 The project’s objective is also consistent with the World Bank’s Country Partnership Strategy for Tunisia FY10-13, 293 which supports efforts to reduce negative environmental and health impacts and to improve living conditions in urban areas; in particular, this includes consolidating improvements in solid waste collection, disposal, treatment, and management services with focus on cost-effectiveness and environmental sustainability. 43. The project has been supporting the Government of Tunisia in complying with the Stockholm Convention on POPs, and the Basel Convention on the control of trans-boundary movements of hazardous wastes and their disposal. In addition, the government has shown increasing commitment to eliminating obsolete pesticides and introducing measures to reduce future related risks, as demonstrated by its post-project efforts (see Section 2.5). 3.2 ACHIEVEMENT OF PROJECT DEVELOPMENT OBJECTIVE 44. Taking account of the project’s Level 1 restructuring in June 2011, the project has been evaluated against its original and revised PDO by measuring achievements at the time of the project’s restructuring (June 2011) and project closing (June 2012), respectively. 45. Original PDO achievement – Rating Moderately Satisfactory: The assessment took the project’s six PDO indicators into account, of which four were fully (100%), one partly (70%), and one not 291 The distributor was contracted on October 4, 2013, with a delivery time of 90 days. 292 Economic and Social Development in Tunisia 2010-2014 – Towards an Innovation and Creation Based Growth, Republic of Tunisia, September 2010. 293 Country Partnership Strategy for the Republic of Tunisia for the Period FY10-13, (Report No. 50223-TUN), November 23, 2009 244 achieved (0%). The three main project outcomes measured by indicators include: 294 (i) sound disposal and management of obsolete pesticides; (ii) prevention of accumulation of new stockpiles of obsolete pesticides; and (iii) enhanced capacity and institutional strengthening of pesticide management. 46. Sound disposal and management of obsolete pesticides. The level of risks emanating from publicly held obsolete pesticides and associated waste to adjacent and wider communities or critical natural resources was determined and quantified through the completion of a national inventory of publicly held obsolete pesticides in 2007 (PDO Indicator #1 – 100%). The hereby-obtained information (geo- location of the storage sites, whether the storage sites were protected or not, identity of the contaminant, its quantity, and general condition of the stockpiles) provided the government with the knowledge necessary to inform the successful preparation of a CESA (PDO Indicator #2 – 100%), and eventual disposal and safeguarding of stockpiles. Due to substantial project implementation delays, the disposal of publicly held obsolete pesticide stocks under the disposal contract had only started preparatory activities by the time of the restructuring (PDO Indicator #3 – 0%). The project had therefore not yet reduced the risks emanating from the inventoried obsolete pesticides and associated waste. 47. Prevention of accumulation of new stockpiles of obsolete pesticides. Better regulation and control of the registration, production, storage, and sale of pesticides was achieved by substantially strengthening the legal and regulatory framework for pesticide management. The project fully achieved the enactment of two legal regulations by the Ministry of Agriculture, which reduced the number of products approved by the ministry from 1300 in 2010, to only 400 in 2012. 295 The regulations’ preparation was based on an analysis of the regulatory framework governing the sector of agricultural pesticides and its alignment with international conventions. However, measures to strengthen compliance with the Basel Convention were only initiated; the Rotterdam Convention had not been ratified (PDO Indicator #4 – 70%). 296 The project also achieved raising awareness on the health and environmental hazards of pesticides and encouraging safe pesticide handling by drafting and implementing one out of two comprehensive communication strategies. 48. Enhanced capacity and institutional strengthening of pesticide management. Between 2006 and 2011, PMU members and representatives from the ministries of environment, agriculture, and health, participated in several pesticide management training courses. Capacity was strengthened as became evident through application of the acquired knowledge during the inventory, and when preparing safeguard instruments (Original PDO Indicator #5 – 100%). 49. Revised PDO achievement – Rating Satisfactory: The assessment took the project’s two revised PDO indicators into account, of which one was fully (100%) and one partly (85%) achieved. In addition, seven Intermediate Outcome Indicators framed the evaluation. Three of those were not taken 294 Achievements were identified based on intended outcomes presented in Section B.1 and Annex I of the PAD (Report No: 32232-MNA) 295 These include (i) Decree No. 2973-2010 which was enacted on November 15, 2010, and which established modalities and conditions for registration, production and storage of pesticides; and (ii) the Executive Order No. 42/2011 which was issued on June 10, 2011, and which set and revised modalities for tax collection related to phytosanitary control analysis, homologation and provisional permit for sale of pesticides. 296 70% achievement of PDO Indicator #4 consists of the following: Enactment of two legal regulations by the Ministry of Agriculture achieved (100%); and only limited measures introduced to strengthen compliance with Basel Convention and non- ratification of the Rotterdam Convention (40%). 245 into account as they corresponded with Original PDO Indicators that had been achieved prior to restructuring; of those taken into account, one exceeded its target, and three were fully achieved. Outcomes were slightly modified following the restructuring and included: (i) sound disposal and management of obsolete pesticides and associated waste including safeguarding; (ii) reduction of re- accumulation of obsolete pesticides and associated waste through regulations, pest management, and stakeholder involvement; and (iii) enhanced capacity and institutional strengthening of pesticide management. Figure 3.1: Publicly Held Obsolete Pesticides and Associated Waste 50. Sound disposal and management Inventoried and Eliminated/Safeguarded of obsolete pesticides and 2000 associated waste including Safguarded 1800 (reported/not verfied) safeguarding. The project 218 1600 June 2012 reduced risks emanating from 1400 publicly held obsolete pesticides Eliminated (or 1200 and associated waste to safeguarded) after 1000 project closure communities, natural resources, 800 1595 December 2012 and the global environment 600 1280 Eliminated (or through the sound disposal and 621 safeguarded) at project safeguarding of 85% of 400 756 closure June 2012 inventoried publicly held 200 44 135 Inventoried 2007 obsolete pesticides and 0 associated waste. In 2007, a total of 2,036 tonnes of publicly held Obsolete Associated Pesticides Waste obsolete pesticides and associated waste was inventoried, consisting of 1,280 tonnes of obsolete pesticides and 756 tonnes of associated waste. At project closure, 1,730 tonnes (i.e., 1595 tonnes of obsolete pesticides and 135 tonnes of associated waste) of the inventoried stocks had been disposed of overseas or safeguarded in country in an environmentally and technically sound manner (Revised PDO Indicator #1 – 85%). See Figure 3.1. The project had focused available funding towards eliminating obsolete pesticides and associated waste located at high-risk sites, while lower risk stocks of associated waste were safeguarded at a lower cost in-country. Specifically this included the disposal of (i) 1,595 tonnes of obsolete pesticides (including 18 tonnes that were safeguarded in country and 315 out of 533 tonnes that had been identified in addition to the inventoried 1,280 tonnes during disposal activities); 297 and (ii) 135 tonnes of associated waste. The remaining 621 tonnes of associated waste were safeguarded; 298 however, their final status could not be verified due to a lack of data verification and difficulties in measuring treatments. It should further be noted that the remaining 218 out of the 533 tonnes of obsolete pesticides that had been identified in addition to the inventoried 1,280 tonnes and which were not disposed of under the project due to time constraints, were disposed of under the government’s own funding after project closure, together with 44 tonnes of associated waste. 297 The additional 297 tonnes identified had not been inventoried in 2007 as a result of difficult site access. 298 According to the PMU empty containers were rinsed, crushed and recycled, contaminated soils located in storage facilities underwent superficial roughening or were delineated with restricted access, and contaminated building, equipment, and materials were cleaned with high-pressure air/vacuum. 246 51. Reduction of re-accumulation of obsolete pesticides and associated waste through regulations, pest management, and stakeholder involvement. The project implemented measures targeted at sustainably managing and reducing the re-accumulation of obsolete pesticides and associated wastes through the following activities: A national action plan including an IPM was adopted by the Ministry of Agriculture (Revised PDO Indicator #2 – 100%). 299 A diagnostic study on the management and use of pesticides in Tunisia was conducted beforehand and served as basis for the action plan.300 The project continued raising awareness on the health and environmental hazards of pesticides and encouraging safe pesticide handling by having drafted the second out of two comprehensive communication strategies (Intermediate Outcome Indicator #5 – 100%). Implementation of the second strategy is currently underway and focuses on training pesticide users and farmers in five different governorates (see Section 2.5 for details) in sound pesticide management. The project also improved the operations of collection centers through a piloting exercise of empty pesticide container collection in two governorates (Intermediate Outcome Indicator #4 – 100%). Plans to mainstream this pilot exercise are described in Section 2.5. 52. Enhanced capacity and institutional strengthening of pesticide management. In addition to the capacity building achieved prior to restructuring, the project further strengthened the government’s capacity in managing pesticides soundly. The government’s laboratory capacity was improved through the acquisition of a mass spectrometer which has allowed the government's laboratories to better analyze pesticides, and to improve the government's procurement capacity. Training courses on improved pesticide management practices were provided to government staff and national stakeholders (Intermediate Outcome Indicator #7 – 100%). Most notably, staff of the Ministry of Agriculture was trained in using and maintaining a container rinsing and crushing unit that was put in place and used for rinsing and crushing contaminated containers under the piloting exercise.301 53. Weighted average PDO – Rating Satisfactory: At restructuring in June 2011, the project had disbursed US$1.0 million, representing 26.3% of total disbursed funds of US$3.78 million. On this basis and as illustrated by the table below, the weighted average of the project’s PDO achievement is rated satisfactory. Annex 8.2 provides a comprehensive account of qualitative and quantitative information of outputs realized against both original and revised PDO and Intermediate Output Indicators. Table 3.1: Weighted Average PDO Achievement Against Original PDO Against Revised PDO Overall 1. Rating Moderately Satisfactory Satisfactory - 2. Rating Value 4 5 - 3. Weight (% disbursed before/after PDO change) 26.3% 73.7% - 4. Weighted value 1.1 3.7 4.8 5. Final rating - - Satisfactory 299 The national action plan was validated with stakeholders prior its approval by the project’s inter-ministerial Steering Committee and subsequent adoption by the Ministry of Agriculture in June 2012. 300 Diagnostic de gestion et d’utilisation des pesticides, AGRIFORUM – Tunisie/CA17 International (France), December 2009. 301 Trainings regarding use and maintenance of the container press were conducted by FAO in 2011 and 2012. 247 3.3 ACHIEVEMENT OF GLOBAL DEVELOPMENT OBJECTIVE 54. The project aimed to directly contribute to the GEO of reducing the effects of POPs on the global environment through: (i) improving the quality of life in poor communities, by reducing environmental risks; (ii) improving environmental protection; and (iii) enhancing the capacity of the agricultural sector to better manage crop pests. GEO achievement is measured by PDO achievement, and, thus, is also rated satisfactory (see Section 3.2). 3.4 EFFICIENCY 55. Efficiency - Rating Substantial. This rating represents a broad estimate, as financial and economic data to support this rating are scarce. At appraisal, an economic or financial analysis was neither undertaken for ASP-P1 at the program nor at the country-level due to non-measurable benefits of reducing the risk of contaminating the environment. 56. Based on the fact that approximately 91% (approximately US$3.44 million) of disbursed GEF funds went towards disposal activities under ASP-P1 Tunisia, the project’s efficiency was primarily assessed by its investment in the disposal of obsolete pesticides and associated waste. During appraisal, the unit cost for disposal was estimated at US$3,400 per tonne (based on approximately US$15 million spent on the disposal of 3,240 tonnes of obsolete pesticides between 1990 and 2004 across Africa). (See Section 3.4 in the program-level ICR for details). 57. At project closing, 1,730 tonnes of obsolete pesticides and associated waste had been disposed of at the cost of approximately US$1,986 per ton, which represents a much lower cost than originally anticipated. 302 Given the overall high cost involved, cleanup and safe disposal was prioritized on the basis of a detailed inventory of pesticide stockpiles and associated waste, determining the identity of the contaminant, its quantity, and its proximity to the people and environment. In comparison with the originally estimated US$3,400 per ton, the project achieved its goal with a much higher cost-benefit than originally estimated. In addition, Tunisia was the first ASP country that succeeded in disposing of its inventoried publicly held obsolete pesticides within allocated resources, and establishing a framework for sustainable pest and pesticide management. However, the US$ depreciation since 2004 and an increase in the price of fuel, which has increased the cost of disposal in US$ terms, is not taken into account by this calculation; neither are varying distances from collection centers to harbors, varying distances of shipping routes, and responses to call of bid by different companies. 3.5 JUSTIFICATION OF OVERALL OUTCOME RATING 58. Overall Outcome - Rating Satisfactory: On the basis of the project’s efficiency (substantial), relevance (high), and PDO achievement (satisfactory), the overall outcome is rated satisfactory. The overall outcome rating takes into account that despite substantial implementation challenges (see Section 2), the project successfully reduced the risks that emanated from publicly held obsolete pesticides and associated wastes, and significantly improved measures to reduce future re- accumulations by establishing a sustainable system of pest and pesticide management. 302 Costs include costs associated with repackaging, transport, and incineration of obsolete pesticides and other related activities such as for example the inventory, safeguarding efforts, associated studies, and advisory services. 248 3.6 OVERARCHING THEMES, OTHER OUTCOMES AND IMPACTS 59. Poverty impacts, gender aspects, and social development: The disposal of obsolete pesticides and associated wastes had a direct positive impact on poor and farming communities, species and ecosystems, located in proximity to the identified 205 sites. Baseline data on affected populations, species and ecosystems was not collected at appraisal; exact impacts are therefore difficult to measure. 60. Institutional change/strengthening: The project strengthened the government’s institutional capacity on several levels: (i) the regulatory and legal framework for pest and pesticide management was improved; (ii) the logistics of hazardous waste transportation have been internalized and the knowledge gained can inform future disposal activities; (iii) the experience and skills gained in managing the project, particularly in the areas of procurement and FM, can be easily applied to other projects and subsequently contribute to success; and (iv) knowledge gained on sound pesticide management through the various training courses. 61. Other unintended outcomes and impacts: No unintended outcomes and impacts were noted. 3.7 SUMMARY OF FINDINGS OF BENEFICIARY SURVEY AND/OR STAKEHOLDER WORKSHOPS 62. A specific completion, and/or ICR mission was not carried out, and thus no stakeholder workshop was held. However, the government provided a completion report, which did not specify any particular issues with stakeholders. 4. ASSESSMENT OF RISK TO DEVELOPMENT OUTCOME 63. Risk to the development outcome – Rating Low. The project substantially strengthened pesticide legislation and national capacity, and together with the various activities agreed upon in the national action plan, the government is well positioned to sustainably manage and reduce the re-accumulation of obsolete pesticides. The government’s commitment to enforcing these regulations and continuing to implement prevention activities has been demonstrated during the past two years of the project, its commitment to eliminating additional obsolete pesticides after project closure under its own financing is likely to continue. 5. ASSESSMENT OF WORLD BANK AND BORROWER PERFORMANCE 5.1 WORLD BANK PERFORMANCE 64. World Bank performance in ensuring quality at entry – Rating Moderately Unsatisfactory: ASP-P1 Tunisia was prepared in conjunction with preparatory activities at the program-level, which were lengthy (see Section 2.1 in the program-level ICR). While considerable time was invested in overall preparation, key project preparation documents for ASP-P1 Tunisia only broadly laid out project implementation plans, and lacked country-specific information. The project’s design was characterized by several weaknesses, which translated into substantial project implementation delays (see Section 2.1 in the program-level ICR). FM and procurement assessments were adequate. Noteworthy is the World Bank’s quick reaction in conducting a second FM and procurement 249 assessment, following the change in the legal status of the implementation agency late in the preparation process. 65. Quality of World Bank supervision – Rating Moderately Satisfactory: The project was supervised by the Task Team Leader (TTL) responsible for ASP at the program-level, and a co-TTL specifically assigned to ASP-P1 Tunisia – both based at the World Bank’s headquarters in Washington DC; supervision missions took place regularly, albeit only once a year. 66. During the first three years, World Bank performance was affected by a complex ASP-P1 program- wide supervision arrangement, which contributed to the project’s initial slow progress (see Section 2.2 in the program-level ICR). It should, however, be noted that despite these initial challenges, World Bank performance contributed to Tunisia having been the first country to successfully carry out disposal activities. The World Bank’s collaboration with ASP-P1 partners was mixed. While the World Bank team effectively worked together with representatives from FFEM throughout implementation, 303 the collaboration with FAO and CLI prior to mid-term review was challenging, mainly as a result of the loosely defined stakeholder implementation arrangement at the program- level (see Section 2.2 in the program-level ICR). Collaboration with other partners such as PAN-UK and WWF was adequate, however, limited. 67. Following the mid-term review, the World Bank team gradually addressed more effectively the project’s implementation delays more effectively. To compensate for the insufficient support received from the TSU and CLI, and to ensure achievement of the PDO, the team took corrective actions when needed. These included: (i) timely reaction to the mid-term review through the preparation of an additional GEF financing request, extension of the closing date, and the temporary recruitment of a pesticide expert; 304 (ii) reallocation of GEF funds to accommodate the eventual recruitment of a TAD (originally supposed to be financed by CLI); and (iii) the approval of a Level 1 restructuringinvolving revision of the PDO to respond to sector realities and country needs, which contributed to PDO achievement. 68. Overall World Bank performance – Rating Moderately Unsatisfactory: The project suffered from shortcomings in project preparation and design at the program level, but following a resulting slow performance during the first half of the project, the World Bank team gradually improved its performance through corrective actions, which contributed to the achievement of the PDO. 5.2 BORROWER PERFORMANCE 69. Government performance – Rating Satisfactory: Government commitment at entry towards the issue of obsolete pesticides was evident through previous steps the government had already undertaken. During preparation there was equally strong support for the project as demonstrated by active participation in preparatory activities. The government’s decision to modify the institutional set-up shortly before negotiations, however, delayed the project’s start-up activities. It took more than seven months before project activities began, as a result of ANGed and the Ministry of Agriculture (DGPCQPA) having had difficulties in agreeing on exact responsibilities. Collaboration difficulties continued thereafter and particularly affected implementation of prevention activities. However, the 303 Representative from FFEM joint all supervision missions. 304 Long negotiations between CLI, the World Bank, and the PMU at the country-level on the financing modalities of a Technical Advisor for Disposal (TAD) caused significant delay in recruiting a TAD. Eventually, negotiations did not lead to an agreement. The TAD was finally recruited in 2010, albeit financed through reallocation of existing GEF funds. 250 two entities recognized the importance of an effective partnership, and gradually improved their collaboration throughout implementation. Reluctance with respect to the World Bank’s recommendation to revise the PDO, contributed to the delayed restructuring of the project. However, thereafter, the government showed strong commitment towards achieving the revised PDO, and to sustain the project’s outcomes. This was evident through the government’s commitment to eliminate under its own financing after project closing 262 tonnes of obsolete pesticides and associated waste that had been identified during disposal activities, and that could not be financed by the project due to time constraints. 70. Implementing Agency performance – Rating Satisfactory: Overall, ANGed’s performance was satisfactory, with gradual improvement throughout project implementation. Initially, day-to-day management of project activities was hampered by the implementing agency’s lack of specialized staff. Until mid-term review, the project coordinator was handling most of the project’s administrative tasks, including procurement and M&E related tasks; the need for technical support was continuously raised. In 2010, the transition associated with the recruitment of a new project coordinator put project implementation temporarily on hold; a handover of responsibilities associated with the management of the prevention component from ANGed to the Ministry of Agriculture further contributed to delays. Particularly during the last two years of project implementation, ANGed demonstrated strong commitment towards achieving the PDO. ANGed complied with all World Bank loan covenants, and executed its fiduciary duties. Delays were noted in submitting Financial Management Reports and implementing World Bank FM recommendations, and the poor collaboration between the financial and accounting department in several occasions; however no mis-procurement was noted during project implementation. ANGed equally ensured safeguarding requirements. 71. Overall borrower performance – Rating Satisfactory: Based on the above presented evaluation, the overall borrower performance is rated satisfactory. 6. LESSONS LEARNED 72. The most significant lessons learned are outlined in Section 6 of the program-level ICR, as the experience they are based on was often reflected across all country-specific projects. Lessons learned that were particularly relevant to ASP-P1 Tunisia include: • Both, the implementing agency and the World Bank team need to be staffed with adequate technical expertise, which is crucial for sound project preparation and implementation of projects associated with pesticide disposal. As was evident during implementation, the disposal of obsolete pesticides and associated waste required highly specialized technical expertise, particularly in preparation of disposal activities (i.e., procuring the specialized services). The technical support the project received proved to be crucial for PDO achievement. • The project’s effectiveness and development impact can significantly benefit from correct application of safeguard requirements. OP 4.12 on Involuntary Resettlement was triggered for a storage site in Fathallah, a southern suburb of Tunis that was being used for business purposes by occupants. By complying with the policy, the project successfully promoted the participation of affected individuals in the planning and implementation of their temporary resettlement. The project compensated affected individuals financially for their loss in income during their 251 temporary resettlement, and allowed for improved standards of living following the successful disposal of obsolete pesticides at the site. • For projects classified as Category A, sufficient time and technical expertise need to be allocated for the preparation of required safeguards instruments. In the case of ASP-P1 Tunisia, the preparation of the CESA involved complex procurement procedures and resulted in being a lengthy process. A realistic timeline that plans for the required procedures needs to be in place to not delay overall project implementation. 7. COMMENTS ON ISSUES RAISED BY BORROWER, IMPLEMENTING AGENCY, AND PARTNERS 73. See Section 8.7 and 8.8. 8. ANNEXES 8.1 PROJECT COSTS AND FINANCING a. Project Cost by Component – All sources (in US$ million equivalent) – as of 06/30/2012 Component Appraisal Estimate Actual/Latest Actual as % of (in US$ million Estimate appraisal equivalent) (in US$ million estimate equivalent) 13. Disposal of obsoletes pesticides 3.70 3.96 107% 14. Prevention of accumulation 0.26 1.05 305 404% 15. Capacity building 0.16 0.17 106% 16. Project management 1.04 0.33 32% Total 5.16 5.51 107% b. Financing – as of 06/30/2012 Sources of Funds Type of Appraisal Estimate Actual/Latest Actual as % of Co-financing (in US$ million Estimate (in appraisal equivalent) US$ million estimate equivalent) Counterpart Borrower 0.53 0.67 126% Funding Global Environment Facility Grant 4.00 3.78 95% (GEF) French Global Environment Parallel Co- 0.63 1.05 167% Facility (FFEM) financing/Grant Total 5.16 5.50 107% 305 This is an estimate. Prevention activities funded by FFEM are still being implemented until March 2014. 252 253 8.2 OUTPUTS BY COMPONENT 1. Taking account of the project’s Level 1 restructuring in June 2011, the following table provides a comprehensive account of qualitative and quantitative information of outputs realized against both original (achievements at the time of restructuring in June 2011) and revised components (achievements at the time of project closing in June 2012). Table 8.1: Outputs by Original Components as presented in the GEF Trust Fund Grant Agreement Original Components as presented in the GEF Outputs achieved at the time of restructuring in June 2011 Trust Fund Grant Agreement Part A – Obsolete pesticides inventory: Achievements under this component are rated Satisfactory Carrying out of a detailed inventoried of the a. An inventory of publicly held obsolete pesticides in Tunisia was recipient’s publicly held obsolete pesticides completed and validated in 2007, including 100% data entry into an stocks through: inventory database which has been under the management and a. Collection of data and compilation of a oversight of FAO, as part of their PSMS (Original PDO Indicator #1 database; – 100%). b. A risk-based prioritization of the stocks b. The PMU and the government’s staff were trained by FAO in using identified in the database; and the system (2007), and have been using the information on stocks c. Preparation of the CESA. identified in the database for a risk-based prioritization of the different sites, and as basis for the preparation of the CESA. c. A CESA identifying mitigation measures to treat possible negative environmental and social impacts that were considered likely to occur during implementation of the project, was completed in February 2010, and found satisfactory by the World Bank (Original PDO Indicator #2 – 100%). Part B – Disposal of obsolete pesticides: Achievements under this component are rated Moderately Unsatisfactory Treatment and/or disposal of publicly held a. A technological assessment of available treatment and disposal obsolete pesticides stocks and associated waste options was completed in June 2008. by: b. Disposal activities, however, had only started at the time of a. Carrying out of a technological assessment of restructuring, thus implementation of the treatment and/or disposal available treatment and disposal options; technology selected was not completed (Original PDO Indicator #3 b. Implementation of the treatment and/or – 0%). disposal technology selected; and c. Operations of collection centers were improved through a piloting c. Improving the operations of obsolete pesticide exercise of empty pesticide container collection in two governorates stocks collection centers, including improving since 2009. The exercise was evaluated in December 2010. safety measures at those centers. Part C – Prevention of obsolete pesticide accumulation: Achievements under this component are rated Moderately Satisfactory Carrying out of activities at preventing obsolete a. General public awareness on the health and environmental hazards pesticide accumulation, including: of pesticides was raised through national media campaigns (e.g., a. Awareness raising activities on the health and television, radio). Awareness was also raised among journalists environmental hazards of pesticides; through a work shop held in June 2006. b. A review and update of the legal and b. The regulatory framework governing the sector of agricultural regulatory framework to control obsolete pesticides and its alignment with international conventions was pesticide accumulation and enforcement analyzed as part of a review of pesticide management practices (see mechanisms; below). Based on this review, the project subsequently achieved the 254 c. Preparation and implementation of an IPM enactment of two legal regulations by the Ministry of Agriculture strategy; and (Original PDO Indicator #4 – 100%). These include (i) Decree No. d. A review of pesticide management practices 2973-2010 which was enacted on November 15, 2010, and which and preparation and implementation of a established modalities and conditions for homologation, production training plan. and storage of pesticides; and (ii) the Executive Order No. 42/2011 which was issued on June 10, 2011, and which set and revised modalities for tax collection related to phytosanitary control analysis, homologation and provisional permit for sale of pesticides. c. An IPM strategy was prepared in 2011, and subsequently implemented between 12/2011 and 06/2012. d. A review of the management and use of pesticides in Tunisia was completed in 2009. In addition to a review of the legal and regulatory framework, it included an analysis of the data gathered through the inventory to better understand the causes for accumulation and possible prevention activities, and an examination of pesticide routes beginning with their import and ending with their usage. Implementation of a training plan to raise awareness for sound pesticide management included: (i) Two training sessions provided to NGOs in December 2005, and June 2009; (ii) 22 training session to 23 technicians from the public service sector and NGOs in April 2008; and (iii) 21 training sessions provided to approx. 360 regional technicians from the ministries of agriculture, environment, and health held between April and December 2008. Technical trainings to build capacity were provided to PMU members and representatives from the ministries of environment, agriculture, and health (see achievements under Part D). Part D – Capacity building: Achievements under this component are rated Satisfactory. Strengthening the capacity of the PMU to Between 2006 and 2010, PMU members and representatives from the implement the project through the provision of ministries of environment, agriculture, and health, received 6 pesticide training. management trainings. Capacity was strengthened as became evident through application of the acquired knowledge during the inventory, and safeguard requirements (Original PDO Indicators #5 – 100%). Part E – Project Management: Achievements under this component are rated Moderately Unsatisfactory. Supporting the PMU in carrying out project The highly technical and specialized expertise required for coordination, implementation, monitoring and implementation, supervision and monitoring of project activities the evaluation, through the provision of technical PMU was supposed to receive from FAO’s TSU was not delivered to advisory services, equipment and operational the extent planned. support. Collaboration issues between FAO and the World Bank hampered timely technical support, particularly in the beginning of implementation. Valuable support was given in terms of equipment through the provision of the PSMS, and associated training. However, limited technical advisory services were given on the pre-qualification, bidding, and execution of the specialized services associated with procuring the selection processes of the CESA and disposal service providers, and M&E. The recruitment of a TAD financed by CLI had been planned for February 2007. However, long negotiations between CLI, the World Bank, and the PMU on the financing modalities, had not led to an agreement. The eventual recruitment in 2010 was financed through reallocation of existing GEF funds, and represented a 255 valuable technical advisory service to the PMU. The PMU subsequently requested continued assistance for the remaining elimination phase which was granted. Table 8.2: Outputs by Revised Components as presented in the Level 1 Restructuring June 30, 2011 Revised Components (Level 1 Restructuring Outputs achieved at the time of project closing June 30, 2011) 306 Specific revisions are underlined Part A – Disposal and/or safeguarding of obsolete pesticides and associated waste: Achievements under this component are rated Moderately Satisfactory. Safeguarding and/or disposal of publicly held The changes reflected the addition of safeguarding as an outcome and obsolete pesticides stocks and associated waste clearly defined associated waste as an integral part of the stocks to be by: treated. a. Carrying out a technological assessment of a. See Outputs by Original Component A. available treatment and disposal options; b. In 2007, a total of 2,036 tonnes of publicly held obsolete pesticides b. Implementation of the safeguarding and/or and associated waste was inventoried, consisting of 1,280 tonnes of disposal technology selected; and obsolete pesticides and 756 tonnes of associated waste. At project c. Improving the operations of obsolete pesticide closure, 1,730 tonnes of the inventoried stocks had been stocks collection centers, including improving environmentally and technically sound disposed of overseas or safety measures at those centers. safeguarded in country (Revised PDO Indicator #1 – 85%). The project had focused available funding towards eliminating obsolete pesticides and associated waste located at high-risk sites, while lower risk stocks of associated waste were safeguarded at a lower cost in-country. Specifically this included the disposal of (i) 1,595 tonnes of obsolete pesticides (including 18 tonnes that were safeguarded in country and 315 tonnes that had been identified in addition to the inventoried 1,280 tonnes during disposal activities); and (ii) 135 tonnes of associated waste. The remaining 621 tonnes of associated waste were safeguarded; however, their final status is unclear due to a lack of data verification and the PMU’s suggestion that their treatment is difficult to measure. c. A consolidation of the pilot's results was not finalized prior to project closing (Intermediate Outcome Indicator #4 – 70%). However, plans to mainstream the pilot exercise are described in Section 2.5. Part C - Reduction of re-accumulation: Achievements under this component are rated Satisfactory Carrying out of activities aimed at reducing The change reflected the transition from prevention (zero new obsolete pesticide and associated waste re- accumulation) to reduction of new accumulations, which allowed the accumulation, including: project to achieve activities to a satisfactory extent. a. Awareness raising activities on the health and a. Awareness for sound pesticide management was raised through the environmental hazards of pesticides; drafting of 2 communication strategies: The first was implemented b. A review and update of the legal and between April and December 2008 through a workshop held with 23 regulatory framework to control obsolete technicians from the public service sector and NGOs, and 21 pesticide accumulation and enforcement training sessions provided to approx. 360 regional technicians from mechanisms; the ministries of agriculture, environment, and health. The second 306 Restructuring Paper on a Proposed Project Restructuring of Africa Stockpiles Program – Project 1 to the Republic of Tunisia, (Report No: 62797-TN), June 20, 2011 256 c. Preparation and implementation of an IPM includes plans to raise awareness of approx. 4000 farmers located in strategy; five different governorates (Intermediate Outcome Indicator #5 – d. A review of pesticide management practices 100%). and preparation and implementation of a b. See Outputs by Original Component C training plan; and c. See Outputs by Original Component C e. Preparation and endorsement of a strategy d. In addition to previous achievements (see Outputs by Original and/or action plan to sustainably manage and Components), representatives from the ministry of agriculture reduce the re-accumulation of obsolete received (i) an additional training in maintaining the PSMS pesticides. (Intermediate Outcome Indicator #1 – 100%), (ii) two training sessions on using and maintaining a container press that was put in place and used for pressing and rinsing contaminated containers at one of the sites (the knowledge acquired was further used to conduct the piloting exercise on the collection of empty pesticide containers), and (iii) one training in IPM (Intermediate Outcome Indicator #7 – 100%). e. Furthermore, a national action plan focusing on sustainably managing and reducing the re-accumulation of obsolete pesticides was validated with stakeholders prior to its approval by the project’s inter-ministerial Steering Committee and subsequent adoption by the Ministry of Agriculture in June 2012 (Revised PDO Indicator #2 – 100%). 8.3 ECONOMIC AND FINANCIAL ANALYSIS 2. As outlined in Section 3.4, an economic and financial analysis is not available. 8.4 WORLD BANK LENDINGA AND IMPLEMENTATION SUPPORT/SUPERVISION PROCESS a. Task Team Leaders Lending (Task Team, specialization, and unit as outlined in the PAD) Names Specialization Unit Responsibility/Specialty Abushakra, Hadi Chief Legal Counsel LEGMS Adebowale, Modupe Senior Finance Officer LOAG2 Allan, Christine Operations Analyst MNSRE Bakayoko, Siaka Senior Financial Management MNACS Specialist Brackmann, Stefanie Environmental Specialist MNSRE Chummun, Zakia Language Program Assistant MNSRE Di Leva, Charles Chief Counsel LEGEN Ebro, Kristyn Communications Officer ESDVP Gorman, Steve Lead Environmental ENVGC Specialist Kaniaru, Muthoni Counsel LEGAF Kranz, Frederick Senior Procurement MNACS 257 Specialist Krishnakumar, V.S. Manager ASPTC Lagnaoui, Abdelaziz Senior Environmental ESDQC Specialist Lotayef, Dahlia Senior GEF Operations MNSRE Coordinator Maber, Steven Senior Operations Officer MNSRE TTL Mani, Angeline Language Program Assistant MNSRE Melkonian, Hovsep Senior Finance Officer LOAG2 Newton, Murray Consultant ENVMP Olowo-Okere, Edward Lead Financial Management AFTFM Specialist Opsal, Knut Senior Social Development MNSRE Specialist Parish, Matthew T. Legal Counsel LEGMS Tynan, Ellen Environmental Specialist ENVMP Warner, Christopher Senior Environmental AFTS1 Specialist Supervision/ICR Names Specialization Unit Responsibility/Specialty Abudagga, Hussam Ahmedou, Hamed Lead Procurement Specialist Aryal, Dinesh Senior Operations Officer TTL Brackmann, Stefanie Environmental Specialist Charlier, Garry Senior Rural Development Specialist Co-TTL Clément, Viviane Dhouibi, Walid Procurement Specialist Doucouré, Djibril Safeguards Specialist Gregoire, Gael Senior Policy Officer Co-TTL Guibert, Yves Jordy, Denis Senior Environmental Specialist Co-TTL Kristensen, Peter Program Manager Lagnaoui, Abdelaziz Senior Environmental Specialist M’Baipor, Lucienne Senior Social Development Specialist Maber, Steve Senior Operations Officer TTL Mani, Angeline Language Program Assistant Mehdi, Mohamed Financial Management Specialist Motammedi, Alaleh Peled, Ayala Environmental Economist 258 Prevoo, Dirk Senior Operations Officer TTL Rechbauer, Gabriele Senior Environmental Operation Specialist Schmidt, Veruschka Strategy Officer Tynan, Ellen Environmental Specialist TTL b. Staff time and cost – Staff time and costs associated with project preparation and supervision was not possible to determine as a result of poor recording in the system. 8.5 BENEFICIARY SURVEY RESULTS 3. Beneficiary survey results are not available, as no survey was conducted. However, the government is planning an independent assessment of outcomes achieved under the prevention component, including an impact evaluation. 8.6 STAKEHOLDER WORKSHOP REPORT AND RESULTS 4. A specific completion, and/or ICR mission was not carried out before project closure, and thus no stakeholder workshop was held. However, the government is planning to conduct a stakeholder workshop in April or May 2014, following closure of the prevention component that is still being implemented under FFEM co-funding. 8.7 SUMMARY OF BORROWER’S ICR AND COMMENTS ON DRAFT ICR 5. The recipient submitted a completion report; messages contained therein were integrated into this document. 6. Comments and edits received on the draft ICR were equally incorporated. Additional comments included the view that the World Bank performed better than moderately unsatisfactorily between the mid-term review and project closing. The implementing agency referred to the support the PMU had received and which had been mobilized by the World Bank. This included additional safeguards support, the temporary recruitment of a technical expert to help procuring the disposal contract, the eventual recruitment of a TAD, and continuous effective communication with the World Bank including fast responses regarding non objection letters. 8.8 COMMENTS OF CO-FINANCIERS AND OTHER PARTNERS/STAKEHOLDERS 7. Comments on the draft ASP-P1 Tunisia ICR received from FFEM include: a. The PDO # 4 indicator refers to the non-ratification of the Rotterdam Convention. Efforts are underway towards the ratification but Tunisia does not yet have the mechanisms to apply this Convention (particularly regarding its customs). b. Revised PDO Indicator #1 on the disposal of obsolete pesticides and associated waste: Due to budgetary constraints, Tunisia prioritized the disposal of obsolete pesticides, and adjourned the disposal of the associated waste (which were mostly buildings and contaminated soil). Tunisia achieved to treat 125% of obsolete pesticides that had originally been inventoried. 8. Comments of co-financiers and other partners/stakeholders on the program-level ICR are summarized in Section 8.8 in the program-level ICR. 259 8.9 VERBATIM COMMENTS ON DRAFT ICR 260 IBRD 39963 TUNISIA MOROCCO ALGERIA LIBYA . ARAB REP WESTERN OF EGYPT SAHARA MAURITANIA THE NIGER GAMBIA MALI SUDAN ERITREA SENEGAL CHAD BURKINA GUINEA- FASO DJIBOUTI BISSAU GUINEA BENIN TOGO NIGERIA SOMALIA SIERRA CÔTE ETHIOPIA LEONE D'IVOIRE GHANA SOUTH CENTRAL SUDAN LIBERIA . AFRICAN REP CAMEROON EQUATORIAL GUINEA UGANDA CONGO KENYA SÃO TOMÉ AND PRÍNCIPE GABON DEM. REP . RWANDA OF CONGO BURUNDI TANZANIA SEYCHELLES COMOROS AT L A N T I C ANGOLA OCEAN ZAMBIA MALAWI MOZAMBIQUE MADAGASCAR ZIMBABWE MAURITIUS AFRICA NAMIBIA BOTSWANA STOCKPILES PROGRAMME SWAZILAND ASP-P1 PARTICIPANT COUNTRIES* LESOTHO INDIAN SOUTH OCEAN AFRICA *including ASP-P1 Nigeria which was not taken into account for this ICR. This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information shown on this map do not imply, on the part of The World Bank GSDPM Map Design Unit Group, any judgment on the legal status of any territory, or any endorsement or acceptance of such boundaries. JUNE 2013