Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) Report Number: ICRR0023185 1. Project Data Project ID Project Name P125999 PK:Punjab Irrigated Agriculture Producti Country Practice Area(Lead) Pakistan Agriculture and Food L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-87990,IDA-50810 31-Dec-2018 333,665,026.90 Bank Approval Date Closing Date (Actual) 20-Mar-2012 31-Dec-2021 IBRD/IDA (USD) Grants (USD) Original Commitment 250,000,000.00 0.00 Revised Commitment 380,000,000.00 0.00 Actual 333,665,026.90 0.00 Prepared by Reviewed by ICR Review Coordinator Group Hassan Wally Avjeet Singh Avjeet Singh IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDO) of the Punjab Irrigated Agriculture Productivity Improvement Program Project (PIPIPP) as articulated in the Financing Agreement (FA, page 5) was identical to the one stated in the Project Appraisal Document (PAD, paragraph 27) and aimed to:"improve productivity of water use in irrigated agriculture." Page 1 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) Parsing the PDO. The PDO will be parsed based on one objective: To improve productivity of water use in irrigated agriculture. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 30-Nov-2017 c. Will a split evaluation be undertaken? No d. Components The PDO was supported by the following four components: 1. Installation of High-Efficiency Irrigation Systems (appraisal cost: US$234 million, AF: US$60 million, actual cost: US$175.20 million). This component included the following two sub-components: 1.1. Installation of High-Efficiency Irrigation Systems (HEIS). This component would support the installation of drip, trickle, bubbler, or sprinkler irrigation systems at the field level for high-value horticulture, vegetables, floriculture, and other high-value crops. A service provider would install the irrigation systems on a shared cost basis. The farmers would provide 40% of the cost of works, and the project would provide the remaining 60% of the cost of works and the administrative and management costs. The drip units would include a pumping unit, fertilizer tank, delivery fittings, filters, main underground pipeline, and delivery lines. High-efficiency irrigation systems were expected to be installed over 120,000 acres. 1.2 Strengthening of Precision Land Leveling Services in the Private Sector. Un-leveled fields cause wastage of water, resulting in low irrigation application efficiency and much lower yields. Laser land leveling saves up to 30% of irrigation water, results in uniform seed germination, and increases fertilizer uptake efficiency, enhancing crop yields by up to 20%. Under this component, the laser leveling equipment would be provided to the service providers on a shared cost basis. The service providers would carry out laser leveling service for interested farmers on a charge-back basis as a business. An annual capacity for about two million acres of laser land leveling would be developed, for which about 3,000 laser leveling units would be provided. About 50% of the laser land leveling equipment cost would be provided by the service provider, who also owns a tractor capable of operating the LASER unit. 2. Upgrading of Community Irrigation Systems (appraisal cost: US$160.00 million, AF: US$44.5 million, actual cost: US$345.10 million). The component included the following three sub-components: 2.1. Watercourse Improvements in Canal Irrigated Areas. This sub-component would assist Government efforts to improve water courses (W/Cs), which were the tertiary level water distribution system where water losses were expected to be the highest. In Punjab 17,000 W/Cs needed improvement. Concrete parabolic channel sections up to 8 feet (or U sections namely canalets) would be placed on leveled compacted earth Page 2 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) with water tight joints, thus improving existing technology of brick lining. Where suitable and where farmers prefer, watercourses would be lined using traditional bricks with plaster. Water turnout structures would be replaced with adequately designed concrete structures. The earthen sections of the watercourse would be improved using clean compacted soil. Efforts would be made to have private contractors/service providers construct the canalets and then be installed by the Water Users Associations (WUAs). The project would provide technical assistance for layout and construction supervision to the WUAs. 2.2. Completion of Partially Improved Watercourses. Many W/Cs in Punjab were only partially (barely) improved in the early part of the program in the late 1970s. The improvement works on these W/Cs would have to be completed to realize the benefits fully. The project would cover the completion of about 1,500 W/Cs that have been partially improved. Farmers would contribute skilled and unskilled labor with the Project funding the remaining cost. 2.3. Improvement of Community Irrigation Systems in the Non-canal Commanded Areas. This sub- component would cover W/C improvements in the rain-fed (Barani) areas, i.e., areas that are not in command of barrage-controlled irrigation but have localized irrigation schemes. These were generally small W/Cs, and the improvement cost is less than in other areas. The project would cover about 2,000 W/Cs in Barani areas. The project would provide material costing up to US$2,950 for each scheme. Farmers would contribute skilled and unskilled labor and material cost above US$2,950—the cost-sharing arrangement results in about 40% by the farmers and about 60% by the project. 3. Improvement of Agricultural Technology and Practices, and Monitoring and Evaluation of Project and Environmental and Social Mana (appraisal cost: US$9.00 million, AF: US$11.00 million, actual cost: US$7.10 million). This component included the following two sub-components: 3.1. Improved Agriculture Technology and Practices. This component aimed to enhance the productivity of the irrigated lands through supporting: (i) effective research, extension, and agricultural information services; (ii) participatory training for farmers, involving the training of specific target groups in various agro- technical fields, farm management, and irrigation agronomy; (iii) demonstration and assistance in improved and modern technologies and methods to increase agricultural production through better agronomic practices; and (iv) the establishment of a Farmers Information Service Desk linked with internet and cell phone services. 3.2. Monitoring and Evaluation of Project Impact. This component would cover Monitoring and Evaluation (M&E) of the project's impacts. This would be done primarily by using a sampling technique and by conducting case studies, Geographic Information System (GIS), and satellite data. M&E activities would likely cover, but not be limited to (i) the impact of the irrigation improvements on water use efficiency, groundwater levels and quality, and soil salinity; on-farm water use; cropping patterns and yields; and livestock population, health, and production; (ii) socio-economic impacts and the impact on the level of employment, livelihood, and household incomes in the project area; estimation of the project's overall benefits and economic rate of returns. 4. Project Management, Supervision, Technical Assistance, Training, and Strategic Studies (appraisal cost: US$20.50 million, AF: US$14.2 million, actual cost: US$23.30 million). This component would cover the cost of (i) project implementation and management, including mobilization of farmers, surveys, engineering and designs, implementation supervision and assistance to the farmers and suppliers, and ensuring quality of the works carried out by farmers and suppliers/vendors; (ii) project supervision and spot checks, covering quality and quantity aspects, by third party consultants based on Page 3 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) which the funds would be disbursed; (iii) strategic studies and pilot projects that would be identified during project implementation, and technical assistance, training, in particular, training the project staff (i.e., training of the trainers) in crop diversification, shift to horticulture, vegetable and floriculture crops, operation and maintenance of the irrigation systems and the units installed under the project; and (iv) activities identified in the Operational Risk Assessment Framework (ORAF) and governance and accountability measures. Revised Components. The AF in November 2017 added additional activities to components 1 and 3 and scaled up component 2. These changes were as follows: Component 1:The allocation was increased by US$60 million (IBRD). Financing for constructing ponds in canal command areas underlain with saline groundwater and ponds for rainwater harvesting in the Pothohar region of Punjab was added (project contribution PKR 750,000/pond). Component 2: The allocation was increased by US$44.5 million (IBRD). The target for the number of improved watercourses and irrigation schemes constructed was increased to respond to the high demand for this activity from 9,000 to 11,550. Also, the length of the lining of watercourses increased from 30% to 50% based on analysis on optimizing the lining length for water saving. Component 3: The allocation was increased by US$11 million (IBRD). A new pilot was added on value chain improvements to assess the effectiveness of support to post-harvest processing of increased High- Value Agriculture (HVA) (sub-component 3.3). The sub-component included the provision of equipment (at a 50% cost-sharing basis), technical assistance, and training. This was to test possible solutions to the emerging market access challenge for farmers transitioning towards HVA due to weak value chains. Component 4: Financing was increased by US$14.2 million (IBRD) for continued project management. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost. The total project cost was estimated at US$423.50 million equivalent. According to the ICR (Data Sheet, page 2), the actual cost was US$543.97 million. The increase resulted from additional financing that the project received in 2017 (see below for more details). Financing. The project was financed through an IDA loan worth US$250.00 million. In 2017, the project received an IBRD additional financing (AF) worth US$130.00 million. The total financing amount was US$380.00 million. According to the ICR (Datasheet, page 2), the actual amounts disbursed were US$233.83 million and US$99.83 million for the IDA loan and IBRD AF, respectively. The total disbursed financing was US$333.67 million, or 88.6% of the expected financing amount. Borrower Contribution. The borrower was expected to provide US$173.50 million through beneficiary contributions. The actual amount was US$210.30 million. The increase was to match the increment in the project scope after the AF. Dates. The project was approved on March 20, 2012, and became effective about a month later, on April 26, 2012. The Mid-Term Review was conducted on January 15, 2016, compared to an expected date on October 31, 2016. While the MTR was conducted about 9.5 months earlier than expected, it was still adequate since it was about four years into effectiveness. The project closed on December 31, 2021, three years beyond the original closing date on December 31, 2018. While the ICR did not explicitly provide a reason for the extension, it was implied that the three years extension was needed as part of the AF Page 4 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) to accommodate the expansion of the project scope and allow enough time to complete the planned activities. The project received additional financing (AF, US$130 million) on November 30, 2017, with the closing date of December 31, 2021, when the amount disbursed was US$190.81 million. As part of the AF, the project had the following changes: revision of the Results Framework (RF), change in components and cost, and the closing date of the original IDA credit was extended from December 31, 2018, to December 31, 2019, and the implementation schedule was revised. The project was restructured once (Level 2) on December 17, 2019, when the amount disbursed was US$264.03 million, to extend the closing date of the original IDA credit from December 31, 2019, to November 30, 2020. According to the Restructuring Paper, the borrower requested this extension to allow enough time to spend all the funds in the designated account. 3. Relevance of Objectives Rationale Context at Appraisal. Agriculture remains the most critical source of employment and exports in Pakistan, accounting for two-thirds of employment and 80% of exports. In 2009, agriculture accounted for 22% of the country's GDP. Pakistan relies on the largest contiguous water system in the world, namely the Indus Basin Water System (IBWS), for basic food security and water supply for all sectors of the economy. Efficient use of available water resources is the key to meeting the uncertain water supplies in the future under changing climate. This project would be a significant contribution to that the goal, both in reducing water demand and helping to produce more crops per drop and creating a system that can convert the seven days turn system into a regular supply system. Previous Bank Experience. The Bank has a long history of partnership and collaboration with Pakistan in the water sector. As a key partner and principal donor, the Bank provided support to several main interventions in the development of the IBWS, including (i) facilitating the Indus Water Treaty negotiations between Pakistan and India; (ii) establishing the Indus Basin Development Fund that supported the construction of Mangla and Tarbela dams and several inter-river link canals and barrages; (iii) formulation of the Salinity Control and Reclamation Program (SCARP-1968); (iv) formulating the Revised Action Plan for Irrigated Agriculture in 1979; (v) assisting in the development of the Water Sector Investment Planning Study (WSIPS) in 1991; and (vi) providing guidance on the Drainage Sector Environmental Assessment in 1993 which contributed to the development of the Ninth Five Year Plan. The Bank also developed the Irrigation and Drainage Strategy of 1994 (grey cover Pakistan: Irrigation and Drainage – Issues and Options 11884-PAK) and the Pakistan Water CAS (Pakistan‟s Water Economy Running Dry, 2005 34081-PK) that led to a major shift in the I&D sector of Pakistan and the implementation of the current institutional reform agenda. Finally, the Bank has supported several On Farm Water Management Projects and has helped to introduce innovation in these operations in all provinces of Pakistan. Consistency with the Bank Strategies. At appraisal, the PDO aligned with the Bank's Country Partnership Strategy for Pakistan(CPS, FY2010-FY2013). The CPS was organized around four pillars to improve: (i) economic governance; (ii) human development and social protection; (iii) infrastructure to support growth; and (iv) security and reduce the risk of conflict. This project was guided by the strategic Page 5 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) principles of the third pillar by engaging in a program that would strengthen irrigation infrastructure and agricultural competitiveness. By improving Pakistan's water use efficiency and encouraging technology that would promote crop diversification and increase productivity, the project supported the Bank's emphasis on improving the efficiency of irrigation systems. At completion, the PDO continued to align with the Bank's Country Partnership Strategy for Pakistan (CPS, FY2015-FY2019), which was extended to FY21. The project directly contributed to Result Area 2: Private Sector Development, specifically outcome Outcomeich focused on improving farm competitiveness and productivity. The CPS emphasized farmers’ adoption of technology for efficient water use and improvement in irrigation and drainage systems. The project also contributed to Result Area 3 on Inclusion, specifically Outcome 3.3, which focused on increased disaster resilience. The PDO was also in line with the Bank's flagship report on Pakistan (Pakistan @100: Shaping the Future, 2019), a Bank-led analytical initiative designed to assess strategic development directions for Pakistan up to 2047. In the report, one of the four elements of the growth strategy proposed for Pakistan was ‘Environmental and Social Sustainability,’ which highlighted improved water management as a significant reform priority. Efficient water use and increased agriculture productivity were key focus areas to achieve water management outcomes. Consistency with Government Strategies. At appraisal, the PDO aligned with the Government's priorities for the irrigation sector. After 1994, the Government adopted a new approach to address the irrigation system issues. It started an institutional reform program to revamp irrigation and drainage (I&D) institutions to establish an efficient, self-sustaining I&D system. At project completion, the PDO continued to be in line with the government’s priorities, where the National Water Policy 2018, Vision 2025, Punjab Water Policy 2018, and Punjab Growth Strategy 2023 all commit to investing in water use efficiency and productivity (ICR, paragraph 17). According to the ICR (paragraph 17), "these documents highlighted agriculture growth and water use efficiency as priority investment areas for the government." Summary of Relevance of Objectives Assessment. The PDO statement was clear, focused, and pitched at an adequate level of ambition. At completion, the PDO aligned with the Bank's Strategy and Government priorities. Also, it is expected that improving irrigation efficiency will continue to be a top priority in the future for Pakistan at both the national and state level. Therefore, the Relevance of Objectives is rated High. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Page 6 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) To improve productivity of water use in irrigated agriculture. Rationale Theory of Change (ToC). To achieve the stated objective, the project would support the following activities: selection and lining of watercourse/Irrigation schemes; operationalization of and ongoing support to WUA; engagement with the private sector to provide equipment and after-sale support; identify beneficiaries and provide equipment after necessary field survey and receipt of beneficiary contribution; technical assistance to the farmers on the use of technology and modern practices for high-value agriculture (HVA); farmer field schools (FFS) and demo farms set up; irrigated agriculture research center set up; and Social mobilization of farmers to create awareness about the benefits of high-efficiency irrigation system (HEIS) and laser land leveling (LLL) (including FFS) and to set up WUAs. As a result of the activities above, the following outputs would be achieved: water courses and Irrigation schemes would be improved; LLL services would be made available to farmers through the provision of equipment to service providers; HEIS would be installed and operationalized on beneficiary farms; Farmers would be trained on efficient use of inputs, HVA, and improved agriculture practices like tunnel farming; and pilots would be conducted to inform advice given to farmers on improved agriculture practices. The expected intermediate outcomes included: (i) improved conveyance/delivery efficiency; (ii) improved application efficiency (irrigation practices); and (iii) farmers adopting improved agricultural practices to increase cropping intensity and change cropping patterns. All this combined was expected to contribute to the achievement of the PDO “improved productivity of water use in irrigated areas” measured through outcomes: (i) reduction in water loss; and (ii) improved crop output per unit of water. The achievement of the stated objective was underpinned by the following six critical assumptions: 1. Local farmers endorse the WUA, participatory decision-making is ensured, and WUAs remain operational for O&M; 2. Even if the supply side is initially weak, as demand increases, suppliers will also increase; 3. Farmers who opt for investment in technology are more likely to be convinced to become progressive farmers; 4. The OFWM department can manage demand-driven research, and farmers engage with them to seek advice; 5. Farmers and LLL service providers can access necessary maintenance services; and 6. Farmers adopt improved agriculture practices and can market their higher/more diverse produce. Overall, the activities reflected in the ToC were connected to the outputs, intermediate outcomes, and outcomes in a plausible causal chain. The stated critical assumptions were logical and realistic. Outputs/Intermediate Results (a) Reduction in water losses 1. 90,000 acres were brought under HEIS (baseline: no HEIS installed) below the target of 120,000 (75% achievement). 2. Number of companies supplying HEIS equipment increased from 7 to 17 between 2015 and 2021 (no target provided). 3. Proportion of respondents reporting that their post-HEIS installation complaints were not resolved reduced from 50% in 2015 to 13% in 2021 (no target provided). 4. Command area of 4 million acres was provided for improved community irrigation infrastructure (no target provided). 5. Average percentage of watercourse lined was 35% which was below the revised target of 44% but exceeded the original target of 29%. Page 7 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) 6. More than 60% of watercourses were lined with precast concrete parabolic segments (PCPS), a more efficient and effective material for lining. 7. 1.5 million acres were land leveled annually with estimated water saving of 1.13 million acre feet (MAF) water annually (no target provided). 8. 575 rainwater harvesting ponds with 14,042 acre-feet capacity were constructed to store water for irrigation (no target provided). 9. 17,300 community irrigation infrastructure were improved, exceeding the revised target of 11,550. 10. 12,825 water courses were improved, exceeding the target of 8,900, resulting in estimated additional 1.5 million acres brought under cultivation after improvement. 11. 4,475 irrigation schemes were improved, exceeding the revised target of 2,650, resulting in 33,545 acres of additional land irrigated 12. 9,577 HEIS units were provided, of which 70% of HEIS units size was below 12.5 acres 13. 5,000 LLL were provided on average level 300 acres per annum (target achieved). 14. 6,375 new WUA was set up, and 6,450 existing WUA were re-engaged (no target provided). 16% percent of WUAs had at least one female member, below the envisaged target of 30% female members of WUA. (b) Increasing productivity of water 1. Cropping intensity increased by 13% under HEIS, 8% in the watercourse command area, and 44% in the irrigation scheme command area (no targets provided). 2. More than half of the HEIS area is under orchards, and a quarter of cropped area under HEIS utilizes tunnel farming to get exceptional yields for high-value vegetables (no targets provided). 3. Land used from traditional major crops as a proportion of total cropped land fell by 56% under HEIS, 14% under watercourse, and 7% under the irrigation scheme (no targets provided). 4. 9,577 HEIS beneficiaries (100%) and about 20,000 farmers on watercourses (6.1% of 321 thousand) received training on improved agricultural practices (no targets provided). 5. Renala Kurd Water Management Research Facility was established to conduct pilots, research, and demonstrations for farmers. Pilots conducted included testing gated pipes to deliver water from the watercourse to the farm, using a moisture meter to track soil moisture, and developing remote farm management tools. Experiments to estimate water saving under different conditions (for instance, use of HEIS or leveling in different conditions and its impact on water efficiency). 6. Trainings were provided to build technical capacity including: (no targets provided).a. 156 professional courses were provided to 5,225 participants; b. 581 technical courses were provided to 26,015; participants;c. 28 specialized courses were provided to 501 participants; d. 86 refresher courses were provided to 1790 participants; and e. 307 farmer field days were conducted for 20,000 participants Outcomes The PDO (improving the productivity of water use in irrigated agriculture) would be assessed based on two elements: (i) reducing water loss during conveyance and application and (ii) Increasing agriculture output per unit of water. (i) Reducing water loss during conveyance and application. Reducing water loss in the project area was achieved through improvements to irrigation schemes (including lining watercourses), applying laser land Page 8 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) leveling technology to reduce application time, and installing high-efficiency irrigation systems (HEIS) to improve irrigation efficiency.  By project completion, the command area served by improved water courses reached 3.85 million acres, significantly exceeding the revised target of 1.78 million acres. The project improved 17,300 (new 6,375; partial 6,450; irrigation schemes: 4,475) watercourses and irrigation schemes, resulting in the project exceeding its target by about 50% (revised target: 11,550 (new 5,700; partial 3,200; irrigation schemes 2,650). The project overachieved the command area covered by 116% through watercourses alone. This is due to the achievement exceeding the targeted number of watercourses and the larger than-expected command area of watercourses (300 acres at end-of- project compared to the appraisal expectation of 200 acres per scheme). About 316 thousand farmers directly benefited from improved watercourses, with 67% of landholding less than 12.5 acres. The project survey measured water loss before and after lining 60 watercourses (a small but comparable sample). The results showed a 57% reduction in water loss exceeding the end target of 45%. Survey findings also showed that 84% of previously barren land on the watercourse command area was cultivated because of increased water availability. Also, 99% of survey respondents reported reduced groundwater pumping (indicating increased availability of surface water to fulfill needs) and reduced reliance on groundwater extraction. These findings confirmed the achievement of conveyance water loss reduction in the project area.  The project also achieved the target of providing 5,000 laser land leveling (LLL) equipment (intermediate indicator) to service providers to provide leveling services. It overachieved the 90,000 project beneficiaries expected at the time of appraisal. According to the ICR (paragraph 26), the project survey found that recipients of LLL services reported a 35% reduction in time taken to irrigate per acre, which means less water is lost in the application and is therefore available for irrigation. However, the total area that LLL serviced was not reported.  The project achieved 75% in the installation of HEIS, which according to the ICR (paragraph 24), was expected to result in a 35% reduction in on-farm water utilization on the 90,000 acres supported by the project. The ICR also explained that the expected savings were not a result of actual project measurements but based on experimental models conducted in Renala Khurd Water Management Research Farm (paragraph 24). The ICR (paragraph 22) noted that most HEIS units (73%) were installed in non-command/rainfed areas or at the tail end of watercourses indicating high uptake by users experiencing water scarcity. (ii) Increasing agriculture output per unit of water. The project overachieved improvement in agriculture productivity, measured as the value per unit of water, through (a) increased cropping intensity and crop diversification; and (b) increased yield. Changes in crop intensity and cropping pattern were achieved through access to water, technology, and training (ICR, paragraph 27). Assessment of results on productivity improvement between 2012-2021 based on survey data showed that in improved watercourse areas, agriculture output per unit of water used increased by 120% for the survey sample from a baseline of 7 Rupees per m3 for row Crops (wheat, maize, cotton, and sugarcane) to 16 Rupees per m3 against a target of 13 Rupees per m3 using 2012 constant prices. Also, in HEIS areas, agriculture output per unit of water used increased by 200% for the survey sample from a baseline of 12 Rupees per m3 for vegetables and orchards to 35 Rupees per m3 against a target of 17.40 Rupees per m3 using 2012 constant prices. Page 9 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) (a) Increasing crop diversification and cropping intensity. Assessment of results on cropping intensity and patterns between 2015-2021 based on survey data showed that in HEIS areas, cropping intensity reached 175% compared to a baseline of 155% in 2015. In watercourse areas, cropping intensity reached 181% compared to a baseline of 167.6% in 2015. Under irrigation schemes, cropping intensity reached 184% compared to a baseline of 128% (pre-improvement). The ICR (footnotes #24) noted that "the project cropping intensity was measured using total cropped area (area under orchards was multiplied by 2 to indicate land utilization in both seasons)/total area under cultivation. During this time, Punjab’s estimated cropping intensity changed from 154% to 157 %.Net sown area and total cropped was taken from statistical abstract from Punjab development statistics (PDS) 2021 and 2016". In HEIS areas, orchards represented 63% of the cultivated area compared to 13% at baseline. Also, vegetables, spices, tomatoes, and potatoes occupied 63% of the cropped area compared to 3% at baseline. In watercourse areas, orchards occupied 3% of the cultivated area compared to none at baseline; also, vegetables, spices, tomatoes, and potatoes increased to 18% compared to 0.5% of the cropped area at baseline. In irrigation schemes, orchards occupied 4% of the cultivated area compared to 2% at baseline, and fodder cropped area doubled from 6% at baseline to 12%, while vegetables, spices, tomato, and potatoes cropped areas slightly increased to 11% from 10% at baseline. However, the cropped area of major crops (cotton, sugarcane, rice, maize, wheat) decreased across all three areas; in HEIS, it decreased from 89% at baseline to 29%, and in the watercourse, it decreased from 89% at baseline to 75%, and in irrigation schemes, it decreased from 84% at baseline to 74% at completion. During the same period in overall Punjab % of the non-orchard area under cultivation under fodder increased from 11% to 14%, major crops decreased from 75% to 69%, and other crops increased from 15% to 17%, The ICR notes (Table 7) that the difference of the increase in crop yield compared to the average in Punjab over the same period was (i) 335% for Tomatoes (HEIS and Tunnel); (ii) 210% for Bitter gourd (HEIS and Tunnel); and 6% for Potatoes (HEIS). Farmers in command areas of watercourses and HEIS beneficiaries received training on improved agricultural practices like tunnel farming. However, adopting this training and changes in cropping patterns is most pronounced among users of HEIS. HEIS beneficiaries had more than half of the area under cultivation dedicated to orchards, a quarter of cropped area under the tunnel, and they show a significant transition away from major crops. (b) Increased yields. Wheat yields increased by 31%, 34%, and 14% in irrigation schemes, watercourses, and HEIS areas. About a quarter of cropped area under HEIS is farmed using tunnel farming to grow produce that was hardly grown by farmers before (therefore zero project baseline data) and achieving exceptional yields; for example, tomatoes produced 33 tons per acre, and bitter gourd produced 15.28 tons per acre. As for orchards, increased yields were also attributed to increased tree density which was made possible by designing higher-density orchards for HEIS. Comparing the number of trees per acre of citrus orchards (more than half of all new orchards designed for HEIS) between orchards designed for HEIS and those that do not have HEIS shows that tree density was increased by 52%, and in the case of orchards 40 months or older, yield increased by more than 8% (ICR, paragraph 29). The ICR (paragraph 14) reported that activities relating to sub-component 3.3 (strengthening value chain development activities) did not get implemented, and the resources for the subcomponent were utilized to line additional watercourses. Summary of Efficacy Assessment. The evidence provided in the ICR pointed to the project's success in improving watercourses and irrigation schemes and exceeding its end target by about 50%. In addition, the project fully met its target on laser land leveling and achieved 75% in the installation of HEIS. Overall, the Page 10 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) outcome of water loss reduction was largely achieved. However, the survey sample on laser land leveling was very small (11 service providers and 11 service recipients). It is worth noting that the project M&E should have captured that area serviced by LLL rather than just the number of LLL units distributed by the project. Also, it was not clear whether LLL serviced land was under a watercourse, irrigation scheme, or in HEIS areas. Finally, the evidence also pointed to the project's success in improving agriculture productivity, measured as the value per unit of water, through changes in crop intensity and cropping pattern achieved through better access to water, improved technology, and training. The project exceeded its PDO outcome target of increased agriculture output per unit of water used for improved watercourse and HEIS areas. Therefore, the Efficacy with which the PDO was achieved is rated Substantial but with minor shortcomings. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Overall efficacy is rated as Substantial. The PDO indicators relating to watercourse improvement, reduction in water losses, and improved water productivity were exceeded. While the project only achieved 75% of the target on HEIS installed, it significantly exceeded the target on watercourses which cover about 4 million acres compared to the missed target of 30,000 acres on HEIS coverage. Overall Efficacy Rating Substantial 5. Efficiency Economic and Financial Analysis (EFA) ex-ante  The overall project economic rate of return (ERR) was estimated at 32.6% and a benefit /cost (B/C) ratio of 1.9 at a 12% discount rate.  A simplified approach was used to estimate the incremental benefits of the project, and a cost-benefit analysis was carried out by determining a discount rate that equalizes the costs and benefits.  Sensitivity Analysis. The ERR was robust and not sensitive to reasonable cost overruns, reduced benefits, and a combination of both. For example, if cost increased by 20%, the ERR dropped to 25.7%; if benefits were reduced by 20%, the ERR dropped to 24.2%; if cost increased by 20% and benefits were reduced by 20%, the ERR dropped to 18.5%, if benefits were delayed by two years the ERR dropped to 19.6%. If costs increased by 90% and benefits decreased by 47%, the ERR dropped to 12%. Page 11 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999)  Overall, the PAD (Annex 7) included a comprehensive economic analysis that justified the project investments. ex-post  The ex-post analysis estimated the value of the economic internal rate of return (EIRR) of the combined interventions at 33.9% compared to 32.6% at appraisal and 30.2% at AF. The ex-post analysis applied the same methodology applied at appraisal to ensure consistency and comparability, except that updated prices and project costs were used.  Benefits were quantified to the extent possible for interventions under components 1 (HEIS and LLL) and 2 (watercourse improvements). However, the economic analysis did not include benefits related to establishing farmers' and water users’ representative bodies like WUA, training farmers on using equipment like HEIS and LLL and changes in cropping patterns, and enhanced employment opportunities for the rural population.  The sensitivity analysis (using a 20% increase in operating cost, a 20% decrease in overall benefits, and both simultaneously) also proved that the EIRR was robust.  Implementation Efficiency. The project was implemented over 9.5 years. It was scaled up after five years of implementation with additional financing due to the government’s request to increase coverage (ICR, paragraph 33). While some activities, like watercourse improvement and LLL, started quickly after approval and demonstrated early results, other activities that required mobilization, like the uptake of the new technology of HEIS, were relatively slow to start (ICR, paragraph 33). In 2018, the country was going through an election, and a caretaker government was in power. During this period, the project experienced implementation delays because the disbursement of budgeted funds for the project was delayed for six months. Some delays impacted HEIS activities due to COVID-19 restrictions over the last 1.5 years of implementation. While the project maintained overall implementation progress, the ICR (paragraph 33) noted that implementing LLL and watercourses activities demonstrated more implementation efficiency than HEIS investments.  The ICR (Annex 4) included a comprehensive economic analysis that justified the project investments. However, the analysis could have benefited from including a cost comparison with similar projects regarding the rehabilitation of irrigation schemes and water courses. Also, cost per beneficiary would have provided further insight into efficiency relative to similar projects. Summary of Efficiency Assessment. The ex-post EIRR at 33.9% slightly exceeded the 32.6% and 30.2% estimated at appraisal and AF, respectively. Implementation efficiency was mixed, as noted above. Overall, Efficiency is rated Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) Page 12 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) 0 Appraisal  32.60  Not Applicable 0 ICR Estimate  33.90  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of Objectives was rated High. Overall Efficacy was rated Substantial, with minor shortcomings. The PDO indicators relating to watercourse improvement, reduction in water losses, and improved water productivity were exceeded. However, the project only achieved 75% of the target on HEIS installed, and strengthening value chain development activities were not implemented. Efficiency was rated Substantial. The ex-post EIRR at 33.9% slightly exceeded the 32.6% and 30.2% estimated at appraisal and AF, respectively. Based on the assigned ratings for the three outcome criteria, the Outcome is rated Satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The ICR (paragraph 67) discussed three main risks that could potentially impact the Development Outcome: 1. Institutional risk. This is related to the sustainability of water user associations (WUAs) and their ability to manage Operations & Maintenance (O&M) of watercourses. The WUA is trained on their primary functions, including O&M of the watercourses improved under the project. They are the main focal point for coordination between on-farm water management (OFWM) field teams and farmers, so the likelihood that they will cease to exist is low. The willingness of beneficiary farmers to support WUA in O&M activities is likely to remain high as they highly value the benefits of improved watercourses. Therefore, it is expected that WUAs should continue to be able to coordinate O&M activities, provided its members remain actively engaged. OFWM field teams must keep these platforms active and relevant to farmers to ensure they maintain their capacity to coordinate O&M activities. 2. Technical risk. This relates to the possibility that service providers will no longer provide equipment and maintenance once the subsidy ends. Based on survey data and field visits, beneficiaries have been receiving post-sale services from service providers (free post-installation support is ensured for two years after installation, after two years, farmers pay for services), and access to spare parts has not constrained maintenance. Since part of the technology has been indigenized, it demonstrates the commitment of service providers to invest in this market. The government plans to continue supporting irrigation efficiency, so the likelihood of service providers discontinuing investment in this space is low, at least in the short term. Page 13 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) 3. Financial risk. This relates to the project farmers returning to traditional cropping patterns. The most significant change in cropping pattern has been observed in the high-efficiency irrigation system (HEIS) users because of the ability of this technology to lower input cost making high-value agriculture (HVA) even more profitable. Since most farmers engaging in the HEIS are likely to be progressive farmers investing in HEIS to engage in more profitable HVA, they will unlikely return to their traditional cropping pattern. This Review emphasizes the following risk: 4. Environmental risk. This relates to the potential of extreme weather events that could impact the project area and reverse the gains achieved under the project. Therefore, a comprehensive climate mitigation plan that emphasizes climate mitigation and adaptation efforts needs to be in place to address climate change and disaster risks. 8. Assessment of Bank Performance a. Quality-at-Entry  Strategic relevance and approach. The project was strategically relevant and aligned with Bank strategies and Borrower priorities (see section 3 for more details). The project aimed to improve irrigation efficiency by reducing water loss and increasing productivity. The Bank's Water Resources Assistance Strategy (2005) highlighted the declining per capita availability of water, making Pakistan one of the most water-stressed countries in the world.  Technical, financial, and economic aspects. The project was designed in the context of other ongoing engagements with the On-Farm Water Management Wing (OFWM) of the government of Punjab. Lessons from other World Bank engagements, including a Development Policy Loan (DPL) on irrigation policy reforms and an Investment Project Financing (IPF) for storage and significant canal delivery infrastructure improvement, were incorporated into the project design. The PAD included a comprehensive economic analysis that justified the project investments.  Poverty, gender, and social development aspects. The migration of people from rural areas continues to contribute to unplanned urban growth and urban poverty. Improving agricultural productivity in project areas was expected to reduce poverty. Women were expected to benefit from the project through increased farm employment and involvement in participative decision- making in WUA. On the social side, enhanced equity between tail and head users was expected to strengthen community relations and reduce tensions.  Environmental aspects. The project design included appropriate environmental and social risk assessments with adequate mitigation strategies. According to the ICR (paragraph 62), the project "allocated sufficient resources to monitor the implementation of an Environmental and Social Management Plan (ESMP)."  Fiduciary aspects. Financial management and procurement arrangements were adequate.  Implementation arrangements. The project was implemented by the On-Farm Water Management (OFWM) wing of the Punjab Government’s Agriculture Department, which was involved in implementing similar operations. The core Project Implementation Unit (PIU) team was in place soon after effectiveness activities like watercourse improvement, and LLL started quickly. However, other activities which needed more mobilization of farmers and service providers, like the uptake of the new technology of HEIS, were relatively slow to start (ICR, Page 14 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) paragraph 33). Also, the project experience showed that OFWM lacked the relevant capacity to strengthen value chains.  Risk assessment. Nine risks were identified at the appraisal stage with a Moderate overall rating. The identified risks are related to two main areas: Inherent Risk and Control risk. The PAD included relevant risk mitigation measures. However, the challenge of precisely tracking smallholders and medium-sized farmers remained throughout implementation. This risk is related to the targeting of project beneficiaries, which might have been underestimated at appraisal.  M&E arrangements. While the M&E design was adequate overall, it suffered from some weaknesses related to the lack of intermediate-level indicators to track changes in farmers' practices and measure the area that received LLL services. Summary of QAE Assessment. The project was strategically relevant with a clear PDO. While the design was technically simple, implementing activities related to supply chain strengthening was challenging. The design featured adequate environmental and fiduciary aspects. Implementation arrangements facilitated a smooth start after effectiveness. While the risk assessment was thorough, the risk-related beneficiary targeting was underestimated. Finally, M&E arrangements were adequate but with minor shortcomings. Overall, Quality at Entry is rated as Satisfactory. Quality-at-Entry Rating Satisfactory b. Quality of supervision  The Bank conducted 20 supervision missions over the 9.5 years of the project implementation. The project benefited from continuity in this task team leadership with only one change in Task Team Leadership throughout implementation. Also, with fiduciary and safeguards specialists supporting the project team based in Pakistan, this facilitated working closely with the PIU (ICR, paragraph 64).  According to the ICR (paragraph 64), the project ratings throughout implementation were candid and realistically reflected the project performance.  M&E implementation benefited from adequate resources, which allowed effective monitoring of the monitor implementation progress. However, the Bank supervision could have addressed the M&E design weaknesses at the AF stage or earlier. Also, strengthening value chain activity should have received more attention to ensure implementation. Summary of Quality of Supervision Assessment. The Bank team successfully guided the project implementation toward a successful outcome. However, the Bank Supervision should have attempted an earlier project restructuring to address M&E weaknesses, and more attention should have been given to strengthening value chains. Overall, the Bank supervision is rated Satisfactory, with minor shortcomings. Based on the assigned ratings, Bank Performance is rated Satisfactory. Quality of Supervision Rating Satisfactory Page 15 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design  The PAD did not include a Theory of Change (ToC), which the Bank did not require at the time of appraisal. Nonetheless, the ICR included a ToC constructed based on the project activities, outputs, and outcomes described in the PAD. The ToC also included the underlying critical assumptions underpinning the achievement of the PDO.  The achievement of the PDO was assessed throubyuring two key PDO outcome indicators: (i) Reduction in water losses in the project area; and (ii) Increased agriculture output per unit of water used. These indicators were directly connected to the PDO, measurable, included baseline data, and had reasonable targets.  The Results Framework (RF) included eight intermediate results indicators (IRIs) to track and measure the different activities supported by the project. The IRIs were measurable, had reasonable targets, and were directly connected to the supported activities. That said, the indicator measuring laser land leveling measured the number of laser units distributed by the project rather than the area of land that received service. Also, the RF lacked indicators to track changes in farmers' practices and adoption of new technologies.  Overall, the M&E design was adequate but had some shortcomings, as noted above. b. M&E Implementation  M&E implementation was handled by a private M&E firm hired by the PIU through a competitive process, but with some delay. M&E activities included: monitoring the implementation progress, carrying out spot checking and technical audits of works being implemented, and ensuring project beneficiary targeting, among other aspects; and (b) evaluating project outcome and assessing its physical, hydrological, environmental, social, and economic impacts (ICR, paragraph 51).  The ICR (paragraph 52) noted that US$3 million were specifically allocated to maintaining adequate capacity (both in-house and independent) to deliver regular and high-quality evaluation reports.  Adequate M&E capacity was maintained until project completion as the M&E firm remained on board until project completion to ensure consistency and continuity in the quality of reporting to PIU and other stakeholders. A baseline, mid-term evaluation, and final impact assessment report were prepared as planned.  M&E reports were shared with all stakeholders regularly. Also, technical reports on specific aspects of project implementation were provided on request, for example, reports on the impact of tunnel farming and the use of solar energy (ICR, paragraph 54).  According to the ICR (paragraph 55), "the methodology used for data collection was robust, and the M&E system used a good mix of triangulating data sources in its monitoring of the project." This was achieved through a mix of sample surveys, field observations, interviews, and research institute experiments to evaluate project implementation results and outcomes. Page 16 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999)  Project outcome surveys relied on random samples of beneficiaries of HEIS, watercourse improvement, and LLL. To ensure adequate representation, the sample size from each region was drawn based on the proportion of project beneficiaries in that area. Similarly, equal representation of beneficiaries from head, middle, and tail-end was selected to ensure representation of farmers in different sections of the watercourse (ICR, paragraph 55).  Overall, the evidence provided in the ICR point to effective implementation of the M&E activities. c. M&E Utilization  The M&E system generated much data from the field and service providers and was well connected with the implementation supervision firm to assess progress. This enabled the project management to develop insights and facilitated decision-making as required (ICR, paragraph 64).  According to the ICR (paragraph 56), "the project developed strong in-house M&E capability of OFWM, which enabled them to conduct quick tests and evaluations to inform changes in project implementation where required."  M&E evaluations informed management and resulted in an implementation change. This included evaluating optimum watercourse lining length, comparison of technical efficiency of brick and PCPS lining, field experiments, and sample surveys to estimate water savings through investment volume and impact of tunnel farming for HVA.  The findings of data gathered through M&E were also utilized to guide the improvement of outcomes, including specific guidance for using HEIS on orchards and the use of tunnel technology to encourage HVA. Summary of M&E Quality Assessment. The M&E arrangement benefited from an adequate design that was supported by suitable funding. M&E implementation relied on an independent consultant firm for project monitoring and evaluation, which was an effective arrangement. Finally, the project management utilized the M&E system-generated data and evaluation reports to inform decisions. However, the M&E design had some weaknesses related to intermediate indicators, and the hiring of the M&E firm experienced some delay. Overall, M&E Quality is rated Substantial, with minor shortcomings. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was an environmental category B. It triggered two environmental safeguards: Environmental Assessment (OP/BP 4.01) and Projects on International Waterways (OP/BP 7.50). Overall the project was expected to have positive environmental benefits; however, during project implementation, some low to moderate, short-duration impacts such as damage to assets, loss of land, and soil erosion were to be Page 17 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) expected. An Environmental and Social Management Plan (ESMP) was prepared as part of a full Environmental Assessment (EA). The project area was located on the Indus Basin, an international waterway; thus, international waterways safeguard under (OP 7.50) was triggered. According to the PAD (paragraph 99), "the project will not adversely affect the quality or quantity of water flows to other riparians, and it will not be adversely affected by other riparians' water use." Compliance with Environmental Safeguards. An exemption to the notification requirement was obtained for the project regarding OP 7.50. The ICR (paragraph 58) stated that the "project maintained an adequate institutional setup for compliance with safeguards requirements." However, no explicit statement on compliance with the Bank's safeguards policies was included. b. Fiduciary Compliance Financial Management (FM). According to the ICR (paragraph 60), the FM had an adequate internal control system, and the FM unit was adequately staffed with experienced professionals. The ICR (paragraph 60) stated that "financial reports, including Interim Financial Reports and Audited Financial Statements, were submitted to the Bank within the timelines stipulated in legal agreements." During FY 2017-18, the release of project funds from the Finance Department was delayed because the designated project account required re-validation. The project was audited by the Auditor General of Pakistan. This resulted in one qualified opinion in FY2019-20, highlighted in a subsequent management letter. The ICR noted that the observations were resolved by the project, and FM was rated Satisfactory at completion (paragraph 60). Procurement. The ICR (paragraph 59) reported that "staffing for fiduciary functions was adequate." The project's fiduciary functions benefited from being part of an existing government program, as existing teams were able to facilitate implementation. The ICR did not report any mis-procurement incidents and noted that "procurement ratings were rated Satisfactory at completion and during most of the implementation period." c. Unintended impacts (Positive or Negative) None. d. Other None. 11. Ratings Page 18 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR included six lessons. The following three are emphasized with some adaptation of language: 1. To increase the impact of infrastructure investments, a sustained dialogue between stakeholders and relevant institutions on irrigation tariffs is required. Project results show that water pricing is important to ensure the uptake of high-efficiency irrigation technology. The existing irrigation tariff structure provides no incentive to adopt water conservation practices. Water pricing reform is necessary to invest in maintaining the irrigation infrastructure, discourage inefficient water use, and increase demand for water-saving technologies. Policy dialogue on such reform was ongoing during project implementation, and the irrigation tariffs were doubled in 2020. However, these rates still do not reflect the cost of water delivery. 2. Increasing farmers' access to financing sources can facilitate the adoption of modern irrigation technologies. The high upfront investment cost and lack of access to financial services constrain adopting of high-efficiency irrigation systems (HEIS). It is important to mobilize funding to help farmers participate in the technology market and increase demand for productivity enhancing solutions. This requires supporting the financial sector to develop lending products/instruments suitable for the needs of farmers, particularly for productivity enhancement technologies (like HEIS). 3. To ensure the successful implementation of activities, operational design should carefully assess the strengths and weaknesses of existing institutional structures involved in implementing project activities. While On-Farm Water Management's (OFWM) strong capacity to implement water efficiency activities was well known, the support they needed to implement the next phase of strengthening value chains was not well understood. Because of this, the project could not implement the pilot on post-harvest processing and value addition as expected. Subsequent investments in irrigation need to include value chain development capacity within the implementing arrangements. 13. Assessment Recommended? No Page 19 of 20 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK:Punjab Irrigated Agriculture Producti (P125999) 14. Comments on Quality of ICR Quality of Evidence. The ICR benefited from the data collected by the M&E system, which enabled tracking the progress of activities and assessing the achievement of the PDO. Quality of Analysis. The ICR provided clear linking between evidence and findings and used the evidence base to serve the arguments under the different sections, particularly the discussion on outcomes. Lessons. Lessons reflected the project experience and were based on evidence and analysis. Results Orientation. The ICR included a comprehensive discussion on the achievement of the PDO. While the outcome discussion was slightly skewed towards achieving the PDO indicators, the ICR still adequately reported on what the project achieved on the ground. Consistency with guidelines. The ICR used the available data to justify most of the assigned ratings. However, the ICR lacked an explicit statement on the project's compliance with the Bank's safeguard policies. Conciseness. The ICR provided comprehensive coverage of project activities and candidly reported on shortcomings. Summary of the Quality of ICR Assessment. The ICR was well-written and benefited from the evidence base generated through the project's M&E system. It included a clear discussion on achieving outcomes and reflected relevant lessons. However, reporting on safeguard compliance was not explicitly stated. Overall, the Quality of the ICR is rated Substantial, with minor shortcomings. a. Quality of ICR Rating Substantial Page 20 of 20