Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) Report Number: ICRR0021671 1. Project Data Project ID Project Name P088926 PH-Participatory Irrigation Development Country Practice Area(Lead) Philippines Water L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-77090 31-Mar-2015 58,015,956.52 Bank Approval Date Closing Date (Actual) 24-Jun-2009 31-Aug-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 70,360,000.00 0.00 Revised Commitment 58,015,956.52 0.00 Actual 58,015,956.52 0.00 Prepared by Reviewed by ICR Review Coordinator Group Maria Vanessa Christopher David Christopher David Nelson IEGSD (Unit 4) Corlazzoli Nelson 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the legal agreement of 2009, the objective of the Republic of the Philippine’s Participatory Irrigation Development Project was “to improve the irrigation service delivery of the Borrower so to have it provide a financially and technically sustainable basis in order to contribute to increased agricultural production and productivity among beneficiary farmers in irrigated areas” (Legal Agreement TF-7709 2009, pg. 5). Page 1 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) In January 2015, a restructuring changed the project objective to “improve efficiency of irrigation services delivered on a technically and financially viable basis” (Restructuring Paper, January 2015, pg. 6). The revised objective encompasses the spirit and aim of the original objective. While the revised objective deemphasized the final impact of the project from the objective, the revised objective and the indicators did not shift sufficiently to conduct a split evaluation. As a result, the ICRR will assess the project's achievements utilizing the revised objective. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 22-Jan-2015 c. Will a split evaluation be undertaken? No d. Components Component 1: Irrigation Sector Restructuring & Reform (Appraisal cost was estimated at US$31.96 million. At closing the cost was US$31.37 million). This component sought to provide support to implement NIA’s Rationalization plan, incorporate the program of severance payments, and the corresponding institutional strengthening and financial support for implementing the Irrigation Management Transfer (IMT). Activities that aimed to support this component would include: development and provision of educational material, organizational and technical trainings, purchasing of equipment, conducting regular review and planning workshops, conduct ongoing coaching, and field visits and monitoring (PAD, para. 21) The aim of these activities was to promote better operation and management of the transferred structures and facilities (PAD, para. 21). In addition, it sought to establish more sustainable financial and institutional mechanisms for improved participatory O&M and routine rehabilitation of national irrigation systems. Component 2: Irrigation Infrastructure Development (Appraisal cost was estimated at US$68.22 million. At closing the cost was US$65.31 million). This component aimed at improving the delivery of irrigation services in about 58 NISs through rehabilitation and modernization of existing irrigation systems in order to provide more reliable and flexible irrigation services. Component 3: Project Management & Coordination (Appraisal cost was estimated at US$4.81 million. At closing the cost was US$4.01 million). This component sought to provide support for the efficient Page 2 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) coordination, implementation, and management of the project, including strengthening financial management, procurement functions, and monitoring and evaluation. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost. At appraisal, the total cost of the project was estimated at US$113.59 million, of which US$70.36 million would be allocated by the World Bank (PAD, pg. 1). At closing, the total project cost was US$100,859,961 (ICR, pg. 2 and pg. 51). Financing. A total of US$70.39 million was financed through an IDA loan (Legal Agreement, TF-7709). The actual disbursement at the end of the project was US$58,015,957 (ICR, pg. 2). Borrower's Contribution. The PAD stipulated that the Borrower would contribute US$43.23 million or 38% of the project costs. The Loan Agreement also stated that the Borrower would contribute a front-end fee of 25% of the loan or US$180,000 (TF 7790, para.2.03). By the end of the project, the Borrower had contributed US$42,844,004 (ICR, pg. 2). Dates. The project was approved on June 24th, 2009 and became effective on November 03, 2009. It underwent a midterm review on May 10, 2013. The project underwent three additional financing restructurings. On January 22, 2015, the project made changes to the PDO,the results framework, components, and costs, loan closing dates, the financial plan, reallocation between disbursement categories, procurement, and the implementation schedule. On September 24, 2015, the loan closing dates were revised. On March 31st, 2016, there were changes made to the results framework, the loan closing dates, disbursement arrangements and the implementation schedule. Page 3 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) On February 13, 2017, the project shifted its results framework, components, and costs, changed loan closing dates, reallocated between disbursement categories, and changed the implementation schedules. The project was implemented over a period of 9 years. Its original closing date was March 31st, 2015, and it was extended for a total of 41 months. The final closing date was August 31st, 2018 (ICR, pg. 2). 3. Relevance of Objectives Rationale Country Context: At the time of appraisal, Philippines’ agriculture, fisheries, and forestry sector contributed to about 19 percent of the country’s economy and provided employment to almost one-third of its labor force. The country experienced an average economic growth of 5.1 percent between 2000 and 2008. However, the crop sub-sector was stagnant and rural-urban migration was contributing to declining investment in commercial crop production. Rice production was also facing declining farm sizes and the deteriorating condition of the irrigation infrastructure that provided a critical water source of rice production. Alignment with Government Development Priority: In 1997, the Government of the Philippines approved the Agriculture and Fishery Modernization Act (AFMA). The act aimed to improve the living conditions of farmers and fisher folks by improving agricultural production through efficient irrigation systems. The AFMA also sought to promote rehabilitation of irrigation systems and the irrigation management transfer (IMT) of secondary canals and on-farm facilities to Irrigator’s Associations (IAs). The National Irrigation Administration (NIA) would continue to maintain and operate the head works and main canal system of national irrigation systems (NISs) (PAD, para. 2). In the Local Government Code (LGC) enacted in 1991, the responsibility for communal irrigation systems (CISs) was given to local government units (LGUs) (PAD, para. 2). The project objective was relevant to the Philippine Development Plan (PDP 2017-2022) which made irrigation one of the central priority sectors for infrastructure development in order to expand economic opportunities. According to the ICR, the PDP included provisions of subsidies for capital investment and O&M of irrigation facilities, strengthening the implementation of the IMT program, and rehabilitation of the irrigation system. Alignment with Bank’s Assistance Strategy: While not a central component of the Philippines’ 2010- 2012 Country Assistance Strategy, the objectives contribute towards the achievement of outcomes “scale- Page 4 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) up provision of basic services through nationwide community-driven development program” (CAS, para. 74) and “increased investment and employment in rural and urban development.” (CAS, Annex 5, pg. 4). The objective contributed towards the 2015-2018 Country Partnership Strategy. It supports one of five priority engagement areas, which was “Engagement Area 3: rapid, inclusive, and sustained economic growth.” Within this engagement area, the project primarily contributed to “increased economic growth, productivity, and employment in rural areas.” (CPS, para. 83). The objective of the project was relevant to the country context and it aligned with the Government priorities. While improving irrigation service delivery was not a central theme or priority of the CAS at approval or the CPS at project completion, the objective supported economic growth related priorities. The relevance of the objective is therefore rated as substantial. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To “improve the efficiency of irrigation services delivered on a 'technically viable basis'. Rationale The original objective was “To improve the irrigation service delivery of the Borrower so to have it provide a financially and technically sustainable basis in order to contribute to increased agricultural production and productivity among beneficiary farmers in irrigated areas.” In January 2015 it was modified to “improve the efficiency of irrigation services delivered on a technically and financially viable basis.” Given that the objective was not radically changed during the restructuring, the efficacy section will use the January 2015 objective to assess the project. The aim of the project was to increase agricultural production and productivity by improving the irrigation service delivery, including its financial and technical capacities, which we have separated here. The theory of change stated that if the National Irrigation Administration (NIA) was restructured and strengthened, and if Irrigation Associations (IAs) increased their capacities, then there would be improved capacities to support irrigation service delivery and responsibility for operations and maintenance would effectively shift from NIS to IAs. Moreover, the project’s support for the repair, restoration, and modernization of key structures, canals, and drainage facilities would lead to improved irrigation and drainage services and reliable access to water. The theory of change of the project remained relatively unchanged despite the restructuring process and the Page 5 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) change to the objective. The references to agriculture production and productivity were dropped. Aside from that change, the revised objective is very similar to the original objective. At appraisal, the indicator selection focused on measuring the financial and technical sustainability of the NIA and the impact of improving the irrigation system on agriculture production and productivity. Unfortunately, there were no indicators to adequately measure improved service delivery. Capacity strengthening indicators related to the IA were also missing. Adjustments were also made to the indicators throughout the restructuring process. New indicators focused on measuring the irrigation sector and the viability of the system to provide irrigation service delivery. More capacity-related indicators related to IA could have been useful to adequately measure a change in the organization. Improved efficiency of irrigation service delivery Outputs  Work was fully completed in 56 out of the 58 NIS (Target not achieved, Target: 58, ICR, para. 49). The remaining works in the two NIS were included in NIA’s budget for 2019 and procurement was expected to be completed in 2019-2020 (ICR, para. 49).  648.22km of earth canal networks improved (Target met, Target: 647.48km, ICR, pg. 42).  181.09km of concrete lined canal installed (Target met, Target: 179.67, ICR, pg. 43).  626 of canal structures installed (Target not met, Target: 674, ICR, pg. 43).  1,088 of turn-out installed (Target met, Target: 839, ICR, pg. 43).  2 new division dams constructed and 15 rehabilitated (Target met, Target: 14, ICR, pg. 43).  2 pumping stations (Target met, Target: 1, ICR, pg. 43).  2 pumps (Target met, Target: 1, ICR, pg. 43).  4 intake structures (Target Met, Target: 4, ICR, pg. 43).  296.75km of service road (Target not met, Target: 307.87, ICR, pg. 43).  37km of access road (Target Met, Target: 37, ICR, pg. 43).  42.38km of drainage improvement (Target Met, Target: 42.38km, ICR, pg. 43).  71 drainage structures (Target not met, Target: 101, ICR, pg. 44). Outcome: A total of 126,184 hectares were provided with irrigation and drainage services (123,397 improved services and 2,787 hectares with new services) (Target partially achieved. Target total area 118,950 ha, Target new services 3,438, Target 115,512 improved services, ICR, para. 49). Rehabilitation and improvement of existing areas was prioritized due to unavailability of water and/or damaged facilities caused by natural calamities. Page 6 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) A total of 196,442 farmers benefited from the project (Target met, Target: 176,000, ICR, para. 47i) Of the farmer-beneficiaries, 17% were female (Target not met, Target 18% female beneficiaries, ICR, para. 47j). The ICR argues that “with infrastructure in good condition after rehabilitation and with improved water management, more water will reach the fields…and this, in turn, will result in more area under irrigation.” Between 2009 and 2017, the average patty rice yield in project areas increased by 17% from 4.27 tons/ha to 5.00 tons/ha in the wet season and 4.48 tons/ha to 5.26 tons/ha in the dry season (Target not provided, ICR, para. 43). The average yield in project areas was five percent higher than NIA’s national average of 4.73 tons/ha and 5.05 tons/ha during the wet and dry season in 2017, respectively (Target not provided, ICR, para. 43). The cropping intensity in the project’s system increased by 171 from 151 at Baseline (Target met, Target: 170, ICR, pg. 33) The rice cropping intensity increased from 151 percent in 2009 to 171 percent in 2017 (ICR, para. 44). The total productivity increased from 833,603 tons to 1,106,924 tons (ICR, para. 44). At project closing, 97% of IAs reported a satisfactory rating for irrigation service received, especially on the timely delivery of water, NIA’s technical support in maintaining the main infrastructure, and assistance to IAs in O&M activities (Target Achieved, Target: 85%, Baseline: 60% ICR, pg. 34). Technically Sustainable Outputs:  A total of 4,433 training courses were conducted for 172,402 IA participants (Target not met, Target: 5,389 courses and 209,489 participants, ICR, para. 38 and pg. 41). The courses focused on organizational development, leadership, skill enhancement, O&M, management of NIS, and financial management.  1,932 staff trainings were conducted for 100,408 staff (Target met, Target: 1,751 courses and 47,293 staff, ICR, pg. 41).  IA capacity building activities were extended to 2,100 IAs outside of the project’s 58 NIS (ICR, para. 38).  4 NIS were rehabilitated with increased disaster resilience (Target met, Target: 4, ICR, pg. 38).  11 policy studies or consultancy services (Target met, Target: 11, ICR, pg. 44). Outcomes: All 934 IAs in the project-assisted areas signed IMT model contract agreements (Target Met, Target: 90%, Page 7 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) ICR, pg. 37). This contract indicates their commitment to undertake O&M activities in their respective irrigation systems. Almost all (99%) of the IAs demonstrated the ability to undertake and fulfill their O&M contract obligations under the Irrigation Management Transfer (IMT) program (Target met, Baseline: 74%,Target: 85%, ICR, pg. 32). Contract responsibilities included clearing and desilting of canals, minor repairs of structures and facilities, and collection of ISF, amongst others (ICR, para. 38). Rating Substantial OBJECTIVE 2 Objective To “improve the efficiency of irrigation services delivered on a 'financially viable basis'." Rationale The same outputs relating to Improved efficiency of irrigation service delivery outlined above apply equally to the assessment of the project's objective of financial viability. Financially Viable A total of PhP1631.29 million were deposited in the National Irrigation Management Fund (NIMF) by December 2017 (Target met, Target: PhP690 million, ICR, pg. 36). The main source of the NIMF came from the savings generated due to the Rationalization Plans and other income such as the ISF collections, equipment rental, etc. The NIMF ensured payments of the IA’s share from the ISF collection, funding for O&M activities and more (ICR, pg. 37). Financial sustainability was also assessed through the ability to collect Irrigation Service Fees. The fees were needed to support operations and maintenance costs. Collecting Irrigation Service Fees also helped ensure that NIA’s would be financially viability. Between 2009 and 2016 the nationwide Irrigation Service Fee (ISF) collection efficiency increased from 55% to 60% (Target not Met; Target 86%,ICR, pg. 12). The ISF collection efficiency in project areas increased slightly more than the national average. That said, the baseline data was also higher for project areas than the national average. In project areas, between 2009 and 2016, the ISF collection efficiency increased from 61% to 67% (Target not met: Target: 85%, ICR, pg. 12). The growth in ISF revenues in project-assisted areas was initially slow because of delays in civil work procurement and implementation (ICR, para. 32). ISF collection efficiency was also impacted by damages to NIS brought about by a series of typhoons (ICR, para. 32). Page 8 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) The NIA financial viability index is defined as NIA’s revenue from ISF collections, amortization from communal and pump irrigation systems and equipment rental income vis-à-vis its expenses for salaries of regular personnel and operating costs (ICR, para. 33). In the early years of the project, the NIA financial viability index was positively influenced by the increased ISF collection and reduction of the number of staff. In fact, the NIA Financial Viability Index was 1.04 at baseline in 2009, 1.49 in 2011, and 1.36 in 2012 (Target not met, Target: 1.44, ICR, para. 34). From 2013 onwards, the index declined due to the implementation of the Salary Standardization Law (SSL) that increased NIA’s expenditures for salaries and the free irrigation service law decreased overall income (ICR, para. 34). By the end of the project, the NIA financial viability index based on cash and non-cash flows at the end of the project was 0.98 (Target not achieved, Target: 1.20, ICR para 40f). The ICR argues that the NIA financial viability index would have been met if other government funding sources were included in the calculation of the index and if the Salary Standardization Law had not negatively impacted it (ICR, para. 53). That said, the fact that the Government canceled the ISF collection led to increased risk of project achievements being sustained. The IAs were given a share from the ISF collections. A total of 71% of the IAs received their share from the ISF collections within one month (ICR, para. 39). This was an improvement given that before the project only 10% of IAs were receiving their ISF shares on time (ICR, para. 39). Financial viability also measured the transfer of funds from the Government to the NIA. The expectation was that the Government subsidy would decrease over time and the NIA would be more self-reliant. This expectation did not materialize. The government prioritized the rehabilitation of as many NIS as possible leading to increased funds transferred to NIA. From 2012 to 2016 the government provided significant funding to the NIA through the SFSSP project. At baseline, the Government was providing the NIA with PhP300 million. By the end of the project, it was providing PhP 2,639 million (Target not achieved, Target: PhP175 million, ICR, para. 30), considerably more than it intended. Based on NIA’s financial records, NIA continued to allocate 98% of the average annual budget towards O&M (Target met, Target 70%, ICR, para. 47h). The ratio to actual of desirable O&M costs at project closing was 70% (Target met, Target: 69%, ICR, para. 54). The financial sustainability of the NIA was also supposed to be supported by the staff separation through the NIA Rationalization Plans. By December 2016, 2,523 staff had retired (Target met, Original Target: 2,887, Revised Target: 2523, ICR, pg. 35). The NIA saved PhP390 million from the staff separation (Target not met, Target: PhP690 million, ICR, pg. 36). Page 9 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) By the end of the project, NIA personnel had been downsized by 3,414 staff (Target not met, Target: 3,819, ICR, pg. 36). Moreover, the total savings of about PhP771 million were realized in 2011, the amount decreased in 2014 due to the implementation of the SSL where the Rationalization Plan savings were partly used to pay for the salary increase of current and retired staff, including an increase in retirement benefits (ICR, pg. 36). By 2017, agricultural productivity and production was higher in project areas than the national average. At project closing, 97% of IAs reported a satisfactory rating for irrigation service received. With over 4,000 training courses aimed at IA participants, the performance had improved. The achievement of financial sustainability was mixed. On the one hand, the government canceled the ISF collection, which affected the NIA financial viability index and introduced risk to the sustainability of the project’s outcome. On the other hand, indicators related to the allocation towards O&M budget and actual vs. desirable O&M costs were met. Rating Modest OVERALL EFF TBL OLD Rationale The project underwent a restructuring that shifted the project’s objective in January 2015. Given that the objective and the indicators were not radically altered during the restructuring a split evaluation was not conducted. The ICR provided sufficient evidence across the different outcome areas to justify a rating of substantial for efficiency improvements on a technical basis. In particular, the ICR demonstrated that the project was able to contribute towards the IA’s technical sustainability, improved service delivery, and ensure that funds were allocated towards O&M. However, there were notable shortcomings regarding improvements in the financial viability of the model thus resulting in a Modest rating for Objective 2. However, even with the shortcomings experienced against Objective 2, overall efficacy is rated as substantial. Overall Efficacy Rating Substantial 5. Efficiency Economic Analysis: The project had two areas of focus (i) institutional reform and (ii) strengthening and irrigation infrastructure modernization. The economic analysis was only done on the irrigation infrastructure modernization. At appraisal, key tangible and intangible benefits were identified and they included (a) increase in total effective irrigated areas, (b) increase in annual cropping intensity, (c) less managerial efforts in water management with more effective water allocation and hydraulic regulation (d) prompt execution of optimal Page 10 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) maintenance and repairs in the systems with the establishment of NIMF, (e) more dependable and adequate water supply with the establishment of water augmentation and reuse the selected system, (f) reduced cost in NIA with the retirement of redundant staff, and (g) improved corporate viability of NIA with the adoption of organizational structuring, policy reform, and financial strengthening. The economic internal rate of return (EIRR) of the project in the PAD was estimated at 25 percent with a net present value (NPV) of US$74.06 million, using a 15 percent discount rate over a period of 30 years. The benefit-cost ration was estimated at 1.81. At project closure, the same methodology was applied as the PAD’s analysis. Modifications to the methodology included (a) irrigation data under the “with the project” was based on the average for wet and dry cropping season starting on the second year of the NIS rehabilitation up to yeah 2017, (b) yield was based on the average for wet and dry cropping season starting on the second year of rehabilitation up to the year 2017, (c) in the “without project” scenario, data of 2010 was used as a baseline due to project delay start-up, (d) benefits in the “without project” and the “with the project” were assumed to decrease by two percent annually to reflect NIS deterioration, (e) partial benefits were assumed to accrue on the year prior to the project completion based on the actual implementation, and (f) attribution of benefits of the project was assumed to be 80 percent for Core A NIS activities, and 30 percent for Core B NIS activities given that other there were other external organizations and donors that contributed towards improvement in rice production (ICR, Annex 4 para. 7). The ICR economic analysis showed that the economic internal rate of return of the increased irrigated area, while holding yield constant was positive for 19 out of the 26 NICs revaluated. The remaining 7 NICS registered a decrease in average irrigated areas which could be due to weather disturbances or other reasons (ICR, Annex A, para. 11). Based on the 26 NIC reviewed, at project completion the EIRR of the project was 32.9 percent with a net present value of (NVP) of PhP1,802 million, using a 15 percent discount rate. The benefit-cost ratio was 2.6 (ICR, Annex 4, para. 15). Additional benefits of the project included improved operation and maintenance capacity and improved IA’s capacity to undertake repairs and regain the functionality of damaged systems more quickly (ICR, Annex 4, para. 17). Unintended benefits included water saved or recovered from canal lining allowed for further expansion, recovery, and rehabilitation of irrigated areas for rice farming and other crops (ICR, Annex 4, para. 18). Financial Analysis: One of the project’s activities was to implement the NIA’s Rationalization Plan with the hope that there would be institutional sector reform. The goal was to raise NIA’s financial viability index from 0.78 in 2008 to 1.42 by 2015. NIA’s support of its operation would come from increased revenues from Irrigation Service Fee (ISF) collection and reduced operation and maintenance (O&M) expenditures due to the implementation of the Irrigation Management Transfer policy. In 2017, the government canceled the collection of ISF and allowed for free irrigation services. NIA’s expected income from ISF was replaced with an O&M subsidy from the national Government (ICR, Annex 4, para. 19). Page 11 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) The impact of the revised policy on Irrigation Association finances have been mixed. Consultation with IAs who are in Model 2-3 stated that they received higher ISF collections and received less O&M support from the government subsidy. Because of this gap, IA members were contributing between US$30 to US$50 per farmer per cropping season to supplement the Government Funds (ICR, Annex 4, para. 20). The government will be reviewing the standard formula for O&M subsidies to correct the negative impact of the policy on IAs. The financial viability index of NIA for the period of 2011 to 2016 before the Free Irrigation Service Law showed that NIA was performing beyond target but on a declining trend reaching 0.73 in 2016. In 2017 and 2018, the financial viability index improved to 0.88 and 0.98 respectively. Unfortunately, the financial viability index was still below the target of 1.2 to support sustainability (ICR Annex 4, para. 21). That said, NIA received additional funding that helped ensured that NIA was financially viable. The reduction in the index is associated with the implementation of the Salary Standardization Law, the launch of Staple Food Self Sufficiently Program that increased NIA’s operational expenses, and the announcement of free irrigation services in 2015 (ICR, Annex 4, para 21a-c). Operational Analysis: The project was extended several times throughout the life of the project. However, the net loss to the country if the project had not been extended were estimated at US$20 million (ICR, Annex 4, para. 16). Project management costs actually decreased from the approved amount to the actual at closing. The economic and operational analysis showed that despite several extensons the project was efficient. However, financial analysis demonstrated that government policies, in particular, the provision of free irrigation services had a negative impact on the ability of NIA to cover O&M costs and be financially viable. As a result, the efficiency rating is rated at substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. Page 12 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) 6. Outcome The objective of the project was relevant to the country context and it aligned with the Government priorities. While improving irrigation service delivery was not a central theme or priority of the CAS at approval or of the CPS at project completion, the objective supported economic growth related priorities. The relevance of the objective is rated as substantial. The economic analysis showed that the project was efficient. However, the financial analysis showed that government policies, in particular, the provision of free irrigation services, has had an impact on the ability of NIA to cover O&M costs and be financially viable. As a result, the efficiency rating is rated at substantial. The project underwent a restructuring in January 2015. That said, the objective and the indicators were not radically altered during the restructuring included new indicators. The ICR provided enough evidence to justify a rating of substantial for improving the efficiency of the sector from a technical perspective, but there were shortcomings in progress intended to ensure its financial viability. Thus, while the overall efficacy of the project is rated as substantial, the limitations in financial achievement result in an Outcome rating of moderately satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome There are several factors that may pose a risk to the development outcomes achieved throughout this project. They include: 1. Government Policy: The Government policy on ISF increased the level of risk, particularly in the event that the Government reduces or stops providing subsidies to the NIA and IAs for operations and maintenance. At the time that the ICR was written, the government was providing subsidies (ICR, para. 94). 2. Other stakeholder ownership: IAs started collecting contributions from their members to increase resources to cover operational and maintenance activities. While the contribution was agreed through the IA General Assembly, it is a volatile system that depends on farmers’ ability to make the contribution (ICR, para. 95). 3. Exposure to Natural Disasters and Climate Risks: The Philippines has been affected by climate change and intensified storm events. In fact, during the project implementation, the Philippines was affected by a series of typhoons of increasing intensity (ICR, para. 78). The project took steps to ensure some climate proofing in selected NIS. However, these actions were not applied nationwide. As a result, there is a potential risk in the sector in the future (ICR, para. 96). Page 13 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) 8. Assessment of Bank Performance a. Quality-at-Entry The World Bank team worked closely with Government counterparts during the preparation of the project. It provided technical guidance and worked with the NIA to justify the rationale for a five-year implementation period (ICR, para. 90). The project preparation was undertaken quickly given the NIA’s preliminary work on the NIS technical design and operational knowledge of the component implementation (ICR, para. 69). The project design was built on prior sector lessons learned and on the established systems. The memorandum circular was developed during the project preparation and it became NIA’s standard operating procedure (ICR, para. 69). The project schedule could have been designed taking into account the time that it would take to approve the Rationalization Plans. Similarly, the project’s result framework had insufficient indicators to adequately measure service delivery. The Bank provided technical training and workshops to the NIA prior to project commencement. The time lag in the Government’s approval of the Rationalization Plans led to the inability to adjust targets in some indicators (ICR, para. 90). Therefore, given these shortcomings, the quality at entry is rated as moderately satisfactory. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Through multidisciplinary teams, the Bank conducted implementation support missions and field visits. It also maintained regular interactions with the management of NIA, oversight agencies, LGUs, contractors, IAs, farmers, and project beneficiaries. The Bank worked towards finding solutions to issues related to finding alternative instruments for land acquisition to break an impasse, advising on additional structures and corrective measures to improve Page 14 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) occupational and community health and safety, helping improve the implementing of safeguard instruments, and addressing procurement issues. The project implementation was affected by the fact that implementation of different aspects of the project was not synchronized, in particular, the Rationalization Plans, IMT, and civil works. The Rationalization Plans started in 2008 while the O&M responsibilities were not transferred to the Irrigators Associations (IAs) until later. The project was understaffed particularly during the initial implementation phase (ICR, para. 73). The Department of Budget and Management approved initially only 32 out of the 59 proposed required positions (ICR, pg. 63). This affected various requests and supervision of contracts, particularly related to civil works activities (ICR, para. 73). The project experienced significant delays in the processing and approval of the Rationalization Plans. The delays were caused by various regional consultations, and the slow processing of the Department of Budget and Management (DBM), which needed to review the Rationalization Plans of all government agencies (ICR, para. 70). While waiting for the approvals, which took nearly 2 years, the project continued to implement several activities (ICR, para. 70). The project also experienced delays in procurement and the rebuilding of large civil works packages (ICR, para. 75). These delays had an impact on the project implementation period requiring project time extensions and led to the incompletion of 2 NIS (ICR, para. 75). The effects of climate change, including the extended rainy season with heavy downpour and flash floods, shortened the construction period which delayed the completion of civic works (ICR, para. 78). The project was also affected by the abolition of the Irrigation Service Fee and the implementation of the Salary Standardization Law and the new Staple Food Self Sufficient Project (SFSSP) (ICR, para. 72). There were programs and priorities of the succeeding government which was launched during the project implementation. All three of these new priorities affected the project. For example, the implementation of the Staple Food Self Sufficiency Project (SFSSP), meant that NIA had competing priorities. The project had to engage with additional temporary project staff, including seasoned irrigation engineers who acted as third-party monitors and provided technical advice on civil works implementation and quality assurance (ICR, para. 77). Page 15 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) The Bank used the restructuring process to modify and clarify the project objectives and adjust indicators and targets (ICR, para. 91). While the project was impacted by external effects and the Bank team responded accordingly, there were issues with procurement and unsynchronized activities that caused delays. As a result, the quality of supervision is rated as moderately satisfactory. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The appraisal documents granted monitoring responsibilities to NIA. Monitoring was expected to be carried out through existing NIA procedures, using existing indicators (PAD, pg. 45). A user satisfaction survey was supposed to be conducted semi-annually by an independent institution to monitor irrigator’s satisfaction. IAs would be involved in participatory assessments of their own performance, including water service delivery. The data collected as part of this project was supposed to be supplemented with government statistics. The design of the monitoring system included a baseline and end of project surveys (PAD, pg. 45). It also stated that at the end of the production cycle that IAs and NIA field units would undertake a joint IMT performance assessment to rate the accomplishments with respect to water delivery targets and schedule (PAD, para. 50). The intended user of the indicators was the IAs, Irrigation Management Officers, Irrigation Systems Operations Engineers and the Maintenance Engineers (PAD, para. 51). The results framework had an adequate theory of change which was supported by thoughtful activities. Unfortunately, at appraisal, there were not enough indicators to adequately measure service delivery or capacity strengthening. The lack of indicators was later rectified during the restructuring. Page 16 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) b. M&E Implementation According to the ICR, the NIA collected the required data. The Corporate Planning Group and the Systems Management Section managed the NIA’s database and information system on NIS (ICR, para. 80). The Bank provided technical assistance through the development of a detailed Performance Indicator Reference Sheet (ICR, para. 80). The ICR reflected that “the shifts in context on financial indicators regarding ISF collection efficiency and NIA viability index created challenges in tracking resources for O&M” This was eventually replaced with new indicators (ICR, para. 82). c. M&E Utilization The ICR did not provide a concrete example of how monitoring data was used for decision-making. The monitoring and evaluation system was adequately designed with a logical results framework that was well supported by the activities selected. While a bit vague, the monitoring system also designated responsibilities and sought to implement both a baseline and a participant survey. Unfortunately there were not sufficient indicators at appraisal to adequately measure all components of the project. This limitation was later rectified during the restructuring process. The implementation of the monitoring system, according to the ICR, was adequate with the responsible parties collecting data when required. Unfortunately, the ICR did not provide sufficient detail in the implementation of the monitoring system to reflect on whether there were sufficient staff and data was of high quality. The ICR did not adequately illustrate how data was used for decision-making. As a result, the monitoring and evaluation is rated as modest. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as Environmental Category “B” Partial Assessment. At appraisal, it triggered the following safeguards: Environmental Assessment (OP 4.01), Pest Management (OP 4.09), Involuntary Resettlement (OP/BP 4.12), Indigenous Peoples (OP/BP 4.10), and Safety of Dams (OP/BP 4.37). Page 17 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) Involuntary Resettlement and Indigenous Peoples: When it came to Involuntary Resettlement and Indigenous People safeguards, land acquisition was necessary for 10 National Irrigation Systems. There were 960 families that had encroached into the existing canal right of way (ICR, para. 84). While 35 families took advantage of resettlement sites provided by the local government units, the rest relocated on their own upon receiving cash compensation (ICR, para. 84). Of the 401 who had lost portions of their land, 102 opted to receive compensation at the prevailing market price, while the others donated their land (ICR, para. 84). The project ensured equal opportunities in project decision and activities for Indigenous Peoples, including membership to IAs (ICR, para. 84). Safety of Dams, Environment, and Pest Management: While the project activities would not involve work in the dam near the Magat River, it included repair works in the canal structures. Dam safety status and other features were assessed and found satisfactory (ICR, para. 86). While civil works were mostly rehabilitation of intake structures rather than major dam or construction works, all construction supervision included ensuring all required material testing and safety measures to be followed (ICR, para. 86). In addition, some civil works contract included an allocation for safety structures such as flood protection and other safety features as relevant. Environmental related mitigation measures were also included in bid documents and closely followed up during implementation (ICR, para. 86). The project did not include activities related to pest management, but NIA staff worked closely with the Department of Agriculture to deliver trainings to IAs on the proper use of fertilizers and pesticide (ICR, para. 86). Grievance Mechanism. The project also set up a System management committee (SMC), which was made up of representatives from NIA and IAs. The SMC handled complaints and suggestions. Each NIS established an SMC that met on a regular basis. (ICR, para. 87). Overall safeguard performance was rated as Satisfactory at the end of the project. The Bank’s implementation support missions always included a social and environmental safeguard specialist. The specialist provided a role in postponing civil works in areas where PAPs had not been paid yet or identifying required corrective measures to improve operational and community health and safety (ICR, para. 88). The lack of trained safeguard personnel at field level affected the timely preparation and updating of safeguard instruments (ICR, para. 88). b. Fiduciary Compliance Financial management: The project complied well with fiduciary covenants and agreements. During the first two years of project implementation, there were substantial delays in the generation of the required interim financial reports (ICR, para. 89). Timeliness improved as the project was implemented. There were delays in the submission of annual audited project financial statements due to the need to consolidate financial statements from various field offices (ICR, para. 89). At each of these offices, the respective auditors from the Commission on Audit also reviewed the statements. There were two Page 18 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) instances where independent auditors rendered a disclaimer audit opinion on the project financial statement. The issues were related to presentation issues and deficiencies in the disclosure on the notes to the financial statements (ICR, para. 89). Both of the issues were resolved by the project team (ICR, para. 89). Procurement: The project also experienced delays in procurement and rebuilding of large civil works packages (ICR, para. 75). These delays had an impact on the project implementation period and required project time extensions. Given that there is a limited time in the year that irrigation construction can take place, 2 out of the 58 NIS were not completed (ICR, para. 75). The project also found that contracts undertaken by local contractors were more effective than ICB contracts (ICR, para. 76). At the time of project closure, financial management and the procurement performance were assessed as moderately satisfactory. c. Unintended impacts (Positive or Negative) None d. Other None 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons The following three lessons were derived from the ICR: Page 19 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) 1. Select the procurement approach most suitable for the context. For example, in this project, the use of shopping and community participation methods undertaken by local contractors proved to be efficient in 71% of the contracts (ICR, para. 99). These contracts had lower estimated costs. The use of ICBs suffered significant delays and in some cases required re-bidding. The identification of which procurement regulation to use needs to be taken into account in the Project Procurement Strategy for Development (PPSD) (ICR, pg. 99). 2. Strengthen the awareness and importance of adequate funding for O&M, even when funds are available. For example, the project built the capacity and awareness of IAs for the need for adequate funding for O&M purposes from the beginning. The strengthening took place while the IAs were receiving funds directly from the collection of ISFs. When the government decided to provide free irrigation service, IAs sought to fill the gap in funding by requesting members provide additional funds to support operations and maintenance costs (ICR, para. 101). 3. Enable Irrigator Associations (IAs) to set their own contracting agreements. For example, the project gave IAs the opportunity to decide the type of Irrigation Management Transfer model contract. This enabled the IAs to pick the model based on their capacities and readiness to assume O&M responsibilities (ICR, 97b). 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR as well written and logical. The ICR could have been more upfront and candid about the challenges in entry and implementation. The ICR did not adequately explain some indicator related data in the narrative related to the process towards implementing the Rationalization Plans, retention of staff, and output level service delivery data. While the Safeguard section was very detailed, the M&E section was basic at best and failed to provide a reflection on what could have been improved. The ICR did not provide any examples of how the M&E system was used for reflective decision-making. a. Quality of ICR Rating Substantial Page 20 of 21 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PH-Participatory Irrigation Development (P088926) Page 21 of 21