Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) Report Number: ICRR0024151 1. Project Data Project ID Project Name P093775 INTEG NUTRIENT POLLUTION CONTROL Country Practice Area(Lead) Romania Environment, Natural Resources & the Blue Economy L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-48730,IBRD-85970 31-Dec-2013 118,839,696.89 Bank Approval Date Closing Date (Actual) 30-Oct-2007 31-Dec-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 68,100,000.00 0.00 Revised Commitment 118,228,014.55 0.00 Actual 113,372,814.84 0.00 Prepared by Reviewed by ICR Review Coordinator Group Katharina Ferl Chikako Miwa Avjeet Singh IEGSD (Unit 4) P099528_TBL Project ID Project Name P099528 INT. NUTRIENT POLLUTION CONTROL (GEF) ( P099528 ) L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-58040 31-Dec-2013 5466882.05 Bank Approval Date Closing Date (Actual) 30-Oct-2007 31-May-2017 Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 5,500,000.00 Revised Commitment 0.00 5,466,882.05 Actual 0.00 5,466,882.05 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Project Appraisal Document (PAD) (p. 11) the objective of the project was "to support the Government of Romania to meet the EU Nitrates Directive requirements by (a) reducing nutrient discharges to water bodies, (b) promoting behavioral change at the communal level, and (c) strengthening institutional and regulatory capacity”. According to the Financing Agreement of December 28, 2007 (p. 4) the objective of the project was “to reduce nutrient discharges into water bodies and to promote behavior change through strengthened institutional and regulatory capacity and demonstrated commune-based actions and, hence, support the Government of Romania to meet the EU requirements in the field of water protection”. When the project received Additional Financing (AF) in 2016 the objective was modified as stated in the Financing Agreement of April 14, 2016, “to support the Government of Romania towards meeting EU Nitrate Directive requirements at national scale”. This validation will use the objective as stated in the Financing Agreement of 2016 but will use three different areas (a) reducing nutrient discharges to water bodies, (b) promoting behavioral change at the communal level, and (c) strengthening institutional and regulatory capacity as stated in the original PAD objective to assess the achievement of the objective. Furthermore, this validation will not conduct a split rating since the essence of the objective was not changed and the revised PDO improved its outcome focus and reflected the increased scope and broader geographic coverage of the project b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 28-Mar-2016 c. Will a split evaluation be undertaken? No Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) d. Components The project included four components: Component 1: Investments in Local Communities for Reducing Nutrient Pollution (appraisal estimate US$57.8 million, AF US$36.1 million, actual US$88.0 million): This component was to finance a menu of investments focusing on the Nitrate Vulnerable Zone’s (NVZ’s) designated 91 communes in ten river basins. The menu of eligible investments from which communes were intended to prepare sub-project investments programs included the following activities: 1.1) communal storage and handling systems to promote better management of livestock and household waste; 1.2) planting of buffer strips and pastures’ rehabilitation to protect against nutrient discharges; 1.3) water and sanitation activities to rehabilitate or extend small-scale sewage collection and treatment at two to three sites; 1.4) promotion of the Code of Good Agriculture Practices to encourage farmers to adopt good nutrient management practices; 1.5) feasibility studies for improving water and wastewater services. When the project received AF in 2016, the component was modified and activity 1.4) was moved under component 2. Component 2: Institutional Strengthening and Capacity Building (appraisal estimate US$7.4 million, AF US$6.6 million, actual US$13.0 million): This component was to build capacity within the Ministry of Environment and Sustainable Development (MESD) and their National Administration for Romanian Waters (ANAR), as well as other national, regional, and county agencies involved in implementing the Nitrates Directive (Le., Public Health, Agriculture, etc.). Specifically, the component was to finance the following activities: i) technical assistance to ensure that legislation was fully in line with EU regulations; ii) capacity building for ANAR, the designated lead for inter-agency working groups at the river basin and county levels for the Nitrates Directive, including coordinating efforts of the different agencies, and reporting to the European Union (EU) through MESD on progress; iii) training program for relevant national, regional, and county level agencies that are member of the Nitrates Working Groups focused on M&E and EU reporting requirements. When the project received AF in 2016, the overall scope of the component remained the same with a focus on capacity building and providing knowledge and training to farmers through facilitator and on-farm demonstrations with the support of farmer organizations. Component 3: Public Awareness and Information Support (appraisal estimate US$3.1 million, AF US$4.42 million, actual US$7.27 million): This component was to finance a broad public information campaign of the project’s activities and benefits at the local, river basin, national and regional levels to achieve replication of project interventions in other similar areas within Romania (NVZ-designated communes in non-focus counties) as well as other Black Sea riparian countries and EU candidate countries. When the project received AF in 2016, the component promoted replication of activities in other counties as well as in other Black Sea riparian countries and EU candidate countries. Component 4: Project Management (appraisal estimate US$5.3 million, AF US$5.27 million, actual US$10.57 million): This component was to finance the Project Management Unit (PMU) within the Ministry of Environment and Sustainable Development (MESD). The Water Basin Authority in each of the ten river Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) basins was to provide one or two dedicated staff for supervising and coordinating the implementation of project activities at the commune level. When the project received AF in 2016, the financing was increased to reflect the related increase in scope of activities under this component. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project was estimated to cost US$73.6 million. The actual cost was US$118.8 million (including AF). Financing: The project was financed by an IBRD loan in the amount of US$68.1 million (of which US$60.3 million disbursed), an IBRD loan in the amount of US$52.4 million (of which US$53.0 million disbursed), and a Trust Fund (TF-58040) in the amount of US$5.5 million (of which US$5.4 million disbursed). Borrower Contribution: According to the PAD (p. 1-2) the National Administrator for Romanian Waters (ANAR) was to contribute US$1.1 million, local communities were to contribute US$4.7 million, and local governments were to contribute US$1.6 million. Dates: The project was restructured eight times:  On July 13, 2012, the project was restructured (level 2) to: i) expand the number of villages receiving wastewater investments, reduce the number of communal waste platforms being constructed, and cancel feasibility studies; ii) revise the Results Framework to reflect these changes; iii) reallocate financing between components in accordance with the modification of activities.  On August 5, 2013, the project was restructured (level 2) to extend the closing date by 23 months to November 30, 2015, to allow for the completion of project activities which had been delayed due to a substantially reduced budget allocation by the government due to budget scarcity.  On September 8, 2015, the project was restructured (level 2) to extend the project’s closing date by 18 months to May 31, 2017, to prepare the scale up of project activities related to the upcoming Additional Financing (AF).  On March 28, 2016, the project was restructured (level 1) to receive AF in the amount of US$52.4 million to increase the geographic scope of the project; ii) revise the PDO to “support the Government of Romania towards meeting EU Nitrate Directive requirements at a national scale”; and iii) revise the Results Framework to reflect this increase.  On November 3, 2016, the project was restructured (level 2) to reallocate funds between disbursement categories.  On October 15, 2019, the project was restructured (level 2) to the project was restructured (level 2) to add a new safeguard policy OP/BP 4.04 (Natural Habitats) that was triggered.  On October 22, 2021, the project was restructured (level 2) to extend the closing date by 15 months to June 30, 2023, to allow for the completion of project activities which had been delayed due to the COVID-19 pandemic.  On March 20, 2023, the project was restructured (level 2) to extend the closing date by six months to December 31, 2023, to ensure that the loan was fully disbursed and allow for a smooth transition to the project’s follow-on operation. Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) The project was approved on October 30, 2007, and only became effective on December 8, 2008, due to government changes and the implementation of a new policy on salaries which resulted in the departure of most Project Management Unit (PMU) staff. In January 2016, an interim ICR was authored before the AF was approved in March 2016. In total, the project was extended by 62 months resulting in an overall implementation period of 16 years and two months. The original closing date was December 31, 2013, and the actual closing date was December 31, 2023. 3. Relevance of Objectives Rationale Country/region and sector context. Over the past decades, the Black Sea experienced severe environmental damage due to eutrophication, resulting from increased nutrient runoff from agriculture, coastal erosion, insufficiently treated sewage, and inadequate resource management which resulted in long-term ecological deterioration. Romania had the largest land drainage area of the 13 countries comprising the Danube Basin (29 percent) and the largest population share (27 percent). The country’s location at the bottom of the Basin presented special challenges in terms of managing waterways with pollutant waste loads from upstream countries. Also, it meant that Romania’s land-based actions, particularly for nutrient management, had the most direct effects on the Black Sea. Therefore, actions taken in Romania to stem nutrient pollution flow into the Danube and the Black Sea were critically important and were to result in benefits to other riparian states. According to the PAD (para. 4), historically, rural water supply and sanitation were low priorities for the government. In 2007, at the time of project appraisal, water and wastewater service provision in Romania was low compared with other European countries. Of the 10 million people living in rural areas, approximately 33 percent had access to a piped water system. It was presumed that even fewer than 33 percent of rural population benefited from such service, as many systems did not function properly due to poor maintenance and/or lack of funds. Approximately half the rural population was served by public or private wells and the remaining 17 percent were served by public standpipes with varying travel distances to obtain potable drinking water. Many water systems did not comply with the European Commission Drinking Water Directive. The combination of underdeveloped sanitation, poor livestock management, and a large number of small farms resulted in significant nitrate and microbial contamination of shallow groundwater, which was the main source of potable water in rural areas. Alignment with the Government strategy. According to the PAD (para. 6), Romania had progressively started to invest in improved environment infrastructure with the support of European Union (EU) pre- accession funds. The government put in place strategies that supported solid waste service regionalization at the county level with transfer stations eventually extending out to reach the rural areas recognized as most significant to achieving targets for lowering organic content in the national waste stream. Also, the government strategy for water and sewerage provision emphasized regionalization as a priority to improve cost effectiveness and service quality. Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) However, environment investments require extensive capacity for inter-governmental coordination. The Romania Water Authority through its River Basins (Romania is divided into 11 separate river basins and watershed areas) was the designated leader to coordinate the Nitrates Directive implementation and reporting at both the River Basin and County levels through inter-agency working groups. In order to support rural areas to comply with the EU Nitrate Directive requirements, Romania needed further investments as well as strengthen its policies, regulations, administrative structures, services and competencies at the national, regional and local levels in the country. Alignment with the World Bank strategy. The objective of the project was in line with the World Bank’s most recent Country Partnership Framework (FY19-23) for Romania and its overarching goal of improving public service delivery by strengthening national and local institutional and regulatory capacity and its three focus areas. Focus area 1 “ensure equal opportunities for all” (by improving knowledge transfer and preventing and reducing rural pollution by which the poor are particularly affected) and focus area 3 “building resilience to shocks” (by supporting farmers to improve farming and nutrient management practices and positively impacting climate change mitigation and adoption). Furthermore, the objective of the project was in line with the CPF (FY24-28), which is currently being prepared. The new CPF aims to “improve human capital outcomes, enhanced environment for more and better jobs, increase resilience and accelerate the green transition while enhancing institutions to serve all people and businesses as a cross- cutting theme.” The objective addressed a critical development problem and was pitched at the appropriate level. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To support the Government of Romania towards meeting EU Nitrate Directive requirements at national scale. Rationale a) Reducing nutrient discharges to water bodies. Theory of Change: The project’s theory of change stated that inputs/activities such as developing communal manure storage and handling systems, planting plantation of buffer strips and rehabilitating pastures, as well as promoting the code of good agricultural practices and conducting feasibility studies for improving water and wastewater services were to result in several outputs. These outputs were to include platforms for improved management of livestock and household waste being completed, vegetative buffer strips and water bodies being protected, and studies to leverage EU investment funds being completed. These outputs were to result in the outcome of nutrient discharge into water bodies being reduced. Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) The theory of change was sound and logical. Outputs:  The percentage of cropped area in the project communes under relevant nutrient reduction measures increased from 9 percent in 2009 to 75.8 percent in 2023, exceeding the original target of 30 percent and the revised target of 60 percent.  174 platforms for animal waste management and storage were completed, exceeding the original target of 86 platforms and the revised target of 150 platforms.  86.23 percent of households, in targeted villages receiving sewage investments, had access and were connected to sewage systems, exceeding the original target of 30 percent and the revised target of 60 percent.  The number of ground water quality monitoring sites and sample analysis for the Nitrate Directive increased from a density of 7.50 groundwater stations per 1,000 km2 in 2015 to 8.12 groundwater stations per 1,000 km2 in 2023, achieving the target of 8 ground water stations per 1,000 km2. Outcomes:  70.1 percent of targeted project areas showed 10 percent reduction in nitrates discharge to water bodies, not achieving the original target of 80 percent of the targeted project areas but achieving the revised target of 70 percent of targeted project areas. When the project received AF in 2016, the target was revised to reflect the change of geographic scale, moving from just Nitrate Vulnerable Zones (NVZs) to the whole country.  In 2022, the nutrient load reduction achieved under the project was 698.68 tons per year, exceeding the original target of 300 tons per year and the revised target of 600 tons per year.  The European Commission’s Report to the Council and the European Parliament on the implementation of the Council Directive 91/676/EEF concerning the protection of waters against pollution caused by nitrates from agricultural sources based on Member States reports was disclosed in 2020 (for the period 2016-2019) and stated the achievements of the project through investments as well as awareness, education, training and support provided to farmers. While it seems that the EU assessment was favorable, it is not clear if progress towards meeting the EU Nitrates Directive was acknowledged and how “progress” was defined. The ICR (para. 48) stated that there are still specific hotspots with pollution level exceeding the quality of standard of 50mg/I (underground water) through GD no. 53/2009. However, the percentage of monitoring points above the threshold decreased substantially throughout the project’s implementation period (from 25.21 percent in 2009 to 11.43 percent in 2022 across the same areas with historically intensive agricultural practices). Overall, progress was acknowledged through EU comments on the 2020 report, therefore, the target of a favorable EU assessment of Romania’s progress towards meeting the EU Nitrates Directive, was achieved. b) Promoting behavioral change at the communal level. Theory of Change: The theory of change stated that project activities/inputs such as developing a public awareness and replication strategy were to result in a broad public information campaign being conducted and project interventions were to be replicated in other counties and riparian states. These outputs were to result in behavioral change at the communal level being promoted. The theory of change had a logical gap Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) and assumed that promoting the code of good agricultural practices was to result in farmers being willing to apply the code. Outputs:  6,504 farmers were trained by knowledge and training providers on Code of Good Agricultural Practices for three days, exceeding the target of 6,000 farmers. Of those farmers, 2,827 farmers were female, not achieving the target of 3,000 farmers. Also, out of the total number of farmers, 253 farmers received a fourth day of training on nitrogen pollution.  The percentage of farmers penalized by APIA for non-compliance with the Nitrate Directive decreased from 50 percent in 2015 to 14.47 percent in 2023, thereby overachieving the target of 25 percent.  48,300 land users adopted sustainable land management practices as a result of the project, exceeding the target of 30,000 land users.  202 project communes implemented at least one of the following nutrient reduction measures: communal platforms, pasture rehabilitation, or tree planting, exceeding the original target of 68 project communes and achieving the revised target of 200 project communes.  43,756 hectares of land area adopted sustainable land management practices as a result of the project, exceeding the original target of 18,000 hectares and the revised target of 38,000 hectares.  250 meetings /pubic consultations/workshops (at regional level as communal level) were organized, exceeding the target of 100 meetings/ public consultations/ workshops. In addition, over 369 discussion groups took place, supported by about 243 champion farmers, and 76 communal platforms hosted demonstrations of good practices for using manure for fertilizing land Outcomes:  The percentage of population in the project area adopting preventive and remedial measures to reduce nutrient discharge increased from 3 percent in 2009 to 76.5 percent in 2023, exceeding the original target of 50 percent and the revised target of 75 percent. c) Strengthening institutional and regulatory capacity. Theory of Change: The theory of change stated that project activities/inputs such as institutional strengthening and capacity building at MESD, ANAR, and relevant national, regional and county level agencies members of the Nitrate Working Group were to result in outputs such as legislation being in line with EU Nitrates Directive regulations and selected measures under Water Framework Directives as well as members of the inter-agency Nitrates Working Group being trained and the NWG being operational. These outputs were to result in the outcome of institutional and regulatory capacity being strengthened. The theory of change was sound and logical. Outputs:  Working Groups at Water Basin and County levels were functioning effectively to support EU reporting and coordinate other agencies' actions, achieving the target of doings. However, the project would have benefitted from defining “effectively” to allow for a proper assessment of achievement of the output. Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775)  709 technical staff (ANAR, EA, Health Directorates) were trained in water quality monitoring, achieving the target of 700 staff.  All questions, comments or complaints relevant to the project that were received and addressed within 30 days, achieving the target of 100 percent of questions comments or complaints being addressed.  Three surveys (baseline, mid-term and final survey) were conducted to collect feedback of about 3,750 households across 100 communes in the project area and report back the results to the communities, achieving the target of three surveys. Outcomes:  Inter-governmental coordination and capacity to assess, monitor and report on progress with implementation of the EU Nitrates Directive improved. According to the ICR (p. 40), the Inter- Ministerial Committee (IC) was functional throughout implementation and conducted regular meetings. During these meetings, the PMU actively participated in technical discussions and played an important role in developing the updated version of the Code of Good Agricultural Practices. However, this is an inadequate indicator to measure the achievement of this objective since it was not defined how “improved” would be measured. Also, the ICR did not provide sufficient evidence to allow for an assessment. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The project was able to contribute to several key achievements such as in nutrient load reduction, increase the percentage of population in the project area adopting preventive and remedial measures to reduce nutrient discharge. Also, while progress was acknowledged through EU comments on the 2020 report and the target of a favorable EU assessment of Romania’s progress towards meeting the EU Nitrates Directive was achieved, the PDO indicator could have been more specific with respect to the favorable assessment. The overall efficacy is rated Substantial with moderate shortcomings. Overall Efficacy Rating Substantial Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) 5. Efficiency Economic efficiency: The PAD (para. 64) conducted a cost-effectiveness analysis since it was not possible to quantify the project benefits. The analysis focused on determining reductions of nutrients (nitrogen and phosphorous) leakage into the environment that was achieved as a result of improved manure management and other agricultural practices, including afforestation, nutrient and grazing management, crop rotations, vegetation strips, etc. Cost effectiveness (CE) ratios were then calculated, as the ratio of total monetary cost of reduced leakages, over the total amount of nutrient reductions achieved. The costs taken into account included capital investment costs, maintenance and operation costs, and project management costs. The estimated CE ratios/kg for Livestock and Household Waste Management and Afforestation varied between US$10/kg and US$40/kg. The analysis compared these ratios with those achieved in the Chesapeake Basin of the USA and with similar unit costs achieved in EU countries and concluded that investments were cost- effective. For water and sanitation interventions, the cost effectiveness considered investment costs per capita. As such, it was expected that communes with at least 4,000 people were likely to be cost effective, as per capita costs in smaller communes were to be excessively high, and affordability and willingness of the consumers to meet the operational and maintenance costs unlikely. In 2016, when the project received AF, a benefit-cost analysis was conducted, which compiled data and information from the technical literature related to the economic impacts of nutrient pollution. Reduction of the nutrient pollution resulted in corresponding benefits that were quantified in the analysis. The analysis included the project portfolio of 80 manure management platforms and 20 composting, packing/ pellets stations, 20 sets of animal waste collection equipment, and 4 sewage plants. Applying a discount rate of 5 percent, the Net Present Value (NPV) was calculated at US$72 million, and the Internal Rate of Return (IRR) at 22 percent. This analysis indicated that the project was a worthwhile investment. At project closure, no updated benefit-cost analysis was conducted. A cost-effectiveness analysis showed that the expected cost per kg of nutrient discharge ranged from US$6-12 for a single platform and US$20-39 against the total cost of all physical investments financed by component 1, which was lower than the estimated US$10 to US$40/kg at appraisal. Operational efficiency: The project experienced several implementation delays. First, the original loan and the AF took over a year to become effective. In 2007/2008, the delay was a result of changes in the government and the implementation of a new salary policy, which resulted in 90 percent of PMU staff leaving. In 2016, the delay was related to the lengthy period for ratification and the summer recess of the parliament, as well as preparation and campaigning for the general elections. According to the ICR (para. 79), the project also experienced delays due to long and cumbersome internal approval procedures, austerity measures, slow permit approvals, and changes in permit requirements throughout the implementation period. In total, the project’s closing date for the original loan and AF was extended by 62 months. Given the lack of a traditional economic analysis at appraisal and closing and the significant implementation delays, the overall efficiency of the project is rated Modest. Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of the objective was High given its alignment with the World Bank’s most recent Country Partnership Framework. Efficacy was Substantial with moderate shortcomings and Efficiency was Modest. The overall outcome rating is rated as Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The main risk to the project’s development outcome is related to technical capacity. According to the ICR (para. 100) there is a risk of insufficient capacity to ensure the continued use of the manure platforms on the commune level since there is no formal enforcement and no entity in place which is being held accountable. There will be a need for continued monitoring and reporting of the operational use of the platforms against performance indicators. Furthermore, it will be critical to sustain and cultivate the adoption of good agricultural practices on the farm level, especially since the project experienced resistance to adopting new agricultural management practices. Therefore, it will be critical to continue to conduct outreach activities through knowledge management networks (e.g., farmers' organizations) to raise awareness of the importance of adopting good agricultural practices. This risk will be addressed through the World Bank’s follow-on preparation (Romania Rural Pollution Prevention and Reduction Project (RAPID), P179786, US$63 million) which was approved in March 2023 and aims to strengthen the institutional capacity of selected public entities to monitor pollution from Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) agriculture, and to transfer knowledge on agricultural pollution reduction. Also, Romania’s National Recovery and Resilience Plan (financed by the EU) aims to double the number of community-managed manure platforms in the country. 8. Assessment of Bank Performance a. Quality-at-Entry According to the ICR (para. 73), the project design was built on analytical work, a preliminary pilot phase, as well as lessons learned from a previous World Bank project (Agricultural Pollution Control Project (APCP) that had been implemented in Romania. The lessons learned included: i) importance of engaging local stakeholders to observe tangible results from mitigation measures to reduce nutrient discharge to ensure adoption and sustainability; ii) importance of raising awareness and disseminating information to the widespread adoption of new technologies and practices; and iii) importance of customizing project activities to local context to achieve sustainable outcomes. According to the ICR (para. 74), early and continuous engagement with local administrations and communities during project preparation and implementation was key for ensuring ownership. According to the PAD (p. 19), the World Bank team identified relevant risks and rated the following as Substantial: i) widespread corruption; ii) project pre-financing by the Borrower not being ensured appropriately; and iii) elderly, who comprised a large part of the rural population unwilling or unable to change former practices. Mitigation measures included a tighter internal control framework, borrower ensuring pre-financing in the legal covenant of the project, and including the elderly in all public participation activities. The mitigation efforts were adequate. However, the World Bank team did not identify several risks including: i) ongoing reforms in agriculture advisory services; ii) delays in project effectiveness; iii) weak implementation capacity due to high government staff turnover; iv) lack of farmer participation on communal waste management and unwillingness to adopt the Code of Agricultural Practices; and v) complex and lengthy permit approval processes. All these challenges resulted in significant implementation delays. The project’s Results Framework had some shortcomings such as some indicators not being sufficiently defined (see section 9a for more details). Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision According to the World Bank team (August 7, 2024), the World Bank team conducted a total of 32 supervision missions. The World Bank team restructured the project eight times to address capacity constraints and accelerate slow disbursement during the initial phase of project implementation. Also, the World Bank team received Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) AF to increase the scope of the project. According to the ICR (Annex 4, para. 6), the decision was made to proceed with an AF restructuring rather than prepare a new project as the development outcome was identical and implementation built on the previous project. Also, this allowed for the PMU to stay in place and continue and develop its technical skills and managerial capacity.According to the ICR (para. 81), the World Bank team provided candid reporting to management. However, while the indicators included in the Results Framework were updated following the mid-term review held in March 2011, these modifications were not reflected in the 2012 restructuring paper. Also, the World Bank team developed regular action plans to address implementation bottlenecks and provided training to PMU staff in different areas such as financing, accounting, procurement, monitoring and evaluation and safeguards. Overall, the Bank performance at Supervision is rated as Satisfactory. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design Both the original and revised objectives were clearly specified. The theory of change and how key activities were to result in the intended outcome were sound and reflected in the Results Framework. Most of the selected indicators were sufficiently specific, measurable, achievable, relevant and timebound. However, the definitions of PDO indicator 3 (Improved inter-governmental coordination and capacity to assess, monitor and report on progress with implementation of the EU Nitrates Directive) and PDO indicator 4 (Favorable EU assessment of Romania’s progress towards meeting EU Nitrates Directive) as well as intermediate outcome indicator 4 (Water groups at Water Basin and County levels functioning effectively and all staff working on the Nitrates Directive fully operational) were not specific making the assessment of the targets challenging. According to the ICR (para. 86), when the project received AF in 2016, new PDO indicators were added to the Results Framework and several modifications were made to better link project activities to outputs and outcomes. However, the Results Framework included two PDO indicators (“downward trend of nitrates concentration in waters” and “nutrient load reduction (Nitrogen (N)) achieved under the project”) whose achievements cannot solely be attributed to the project’s activities. The PMU was responsible for the project’s M&E activities, which had gained experience from implementing a previous World Bank project. Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) b. M&E Implementation According to the ICR (para. 87), the project’s M&E system was based on survey and administrative data sources. Also, data was also collected from ANAR to analyze water quality and estimate nutrient concentrations in water bodies. Beneficiary related data were collected using baseline, mid-term and end- of year project surveys. The ICR (para. 88) stated that in 2019, the PMU introduced ex-post monitoring. Activities included on-site visits and monitoring. Furthermore, the project conducted a comprehensive final evaluation. During project implementation, the Results Framework was modified twice to allow for a better monitoring of implementation progress towards achieving the objective. c. M&E Utilization According to the ICR (para. 90), M&E utilization was slow during the initial phase of project implementation. The M&E data showed slow implementation progress. For example, during the implementation of individual farm platforms, M&E data found that excessive customization resulted in delays of the installation of manure platforms, resulting in a reduced number of beneficiaries being reached. The project team addressed this issue by working with the government to shift the focus from individual farm level platforms to communal level platforms to accelerate implementation. This resulted in a more streamlined approach, prioritizing communal-level platforms to ensure a more efficient and effective implementation process. Overall, the quality of the M&E is rated as Substantial. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as category B and triggered, at appraisal, the World Bank’s safeguard policies OP/BP 4.01 (Environmental Assessment), OP/BP 7.50 (Projects on International Waterways). When the project received AF in 2016, OP/BP 4.04 (Natural Habitats) was triggered as a result of a new safeguard analysis and taking into account the environmental conditions and specific measures included in the permits issued by the local environmental protection authorities for each new instrument (ICR para. 92). According to the ICR (para. 92), site-specific Environmental and Social Management Plans (ESMP) were prepared for each sub-project. Also, the PMU distributed and prepared guidelines or the participating communes on ESMP structure and content. Furthermore, the PMU and World Bank environmental Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) consultant developed ESMP review checklists that were applied to evaluate the quality and completeness of site-specific ESMP prepared by local beneficiaries. The ICR (para. 92) stated that the project’s compliance with the World Bank’s safeguards was Satisfactory throughout implementation. b. Fiduciary Compliance Financial Management: According to the ICR (para. 94), the project’s financial management was mostly Satisfactory throughout implementation. The audit reports had a clean audit opinion and were submitted on a timely basis. Also, semi-annual Interim Financial Reports were submitted to the World Bank as per the agreed timeline. Furthermore, adequate accounting procedures and systems as well as internal controls were in place during implementation. However, the project experienced some Financial Management related implementation delays due to changes in the PMU’s financial management function, high staff turnover, legislative and operational constraints, as well as insufficient and/or delayed budget allocations to pre- finance the planned expenditures. According to the World Bank team (August 7, 2024), these challenges were eventually overcome through the government's commitment to provide adequate budget for the completion of the project implementation. When the project closed, Financial Management was rated Satisfactory. Procurement According to the ICR (para. 78), until 2016, the project encountered several procurement related delays due to long and complex internal approval procedures, changes in government, austerity measures, slow permit approvals, as well as changes in permit requirements throughout project implementation. In 2016, the requirement to obtain internal procurement approval was removed, improving the project’s procurement performance. However, the project experienced an additional delay of six months in preparing the communities’ investments when the project received AF. The ICR (para. 93) stated that even though procurement encountered several delays during the first half of project implementation, the institutional arrangements or procurement were continuously assessed as Satisfactory. Also, the project’s Procurement manager kept a fully functional filing system throughout implementation and the procurement documents were continuously assessed adequate. Furthermore, necessary measures were taken to ensure project compliance with the World Bank procurement rules. When the project closed, procurement was rated Satisfactory. c. Unintended impacts (Positive or Negative) NA Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) d. Other NA 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment There is no disconnect. The main text of the ICR rates the Outcome Satisfactory Moderately Satisfactory outcome as Moderately Satisfactory. Bank Performance Satisfactory Moderately Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR (p. 36-37) included several lessons learned, which were adapted by IEG:  Using standardized templates for investments in manure management or similar agricultural practices can ensure adequate quality. In this project, the original project used a standard template to control the design and technical specifications of the manure platforms. The AF changed the approach to a customized design to fit the needs and geography of each individual commune which resulted in poor quality designs and a lot of pressure on the PMU to support the communes in the preparation of technical designs, procurement documents, and to verify disbursement linked construction.  Including performance indicators and reporting requirements can stimulate beneficiaries to engage strongly in project activities. In this project, beneficiaries and communes’ officials, were not sufficiently incentivized to engage with farmers after the platforms were built resulting in an underutilization of the platforms. To address this issue, the PMU imposed contractual performance indicators on all beneficiaries and established an online reporting tool to periodically submit reports to the PMU.  The combination of raising awareness as well as implementing an incentive system can positively impact farmer’s resistance to implement sustainable farming practices. This project raised awareness through a Knowledge Transfer Network that allowed for building capacity and exchanging information and best practices among farmers and local communities. Also, payments to farmers incentivized compliance with sustainable farming practices. However, this was not sufficient and more direct involvement by e.g. a Social Development Specialist would have been needed to move from awareness to adoption of SLM practices at a higher level. Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review INTEG NUTRIENT POLLUTION CONTROL (P093775) 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provided an adequate overview of project preparation and implementation. It was also internally consistent and relatively concise. The lessons learned included were useful for future operations in this area. The ICR lacked a traditional economic analysis since it was not done at appraisal. The overall quality of the ICR was Substantial. a. Quality of ICR Rating Substantial Page 17 of 17