Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) Report Number: ICRR0024333 1. Project Data Project ID Project Name P144336 NWRMP Country Practice Area(Lead) Kyrgyz Republic Water L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-16315,TF-B1013 31-Dec-2022 11,798,811.89 Bank Approval Date Closing Date (Actual) 04-Apr-2014 01-Apr-2024 IBRD/IDA (USD) Grants (USD) Original Commitment 12,240,000.00 12,240,000.00 Revised Commitment 12,239,032.07 11,798,811.89 Actual 11,798,811.89 11,798,811.89 Prepared by Reviewed by ICR Review Coordinator Group Craig Phillip Kullmann Dileep M. Wagle Avjeet Singh IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objectives in the PAD (p.4) are: i) to improve water resources management capability; and ii) to improve irrigation service delivery for the benefit of water users. The Financing Agreement for the grant (p. 5) has the same objectives. b. Were the project objectives/key associated outcome targets revised during implementation? Page 1 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) No c. Will a split evaluation be undertaken? No d. Components Component 1: Strengthening National Water Resources Management Capacity (estimated cost US$ 2.7 million). Activities financed under this component included: (i) installing a digital information exchange system between the Department of Water Resources and Land Improvement (DWRLI) offices at the central, regional and district level; (ii) development and implementation of a water information system based on data from different existing organizations; and (iii) strengthening the capacity of the Water Resource Management (WRM) division of DWRLI to do basin mapping, development of preliminary basin water management plans and a National Water Strategy on water resources, developing procedures for water and wastewater permits, and conducting a sector expenditure and benefits review as the basis for a financing plan for the water resources sector. This component also aimed to finance an information campaign on the critical water sector developments and develop reports on key transboundary water issues. Component 2: Improving irrigation service delivery to Water Users Associations (estimated costs US$ 2.0 million). This component financed activities to strengthen the capacity of the DWRLI to manage off- farm irrigation and drainage (I&D) systems including increased fee collection from water users; and increase the expenditure on system maintenance. This component financed inventories of I&D schemes, establishing procedures for determining annual maintenance requirements, and performance-based management, maintenance, and operation pilots are six I&D systems, upgrade of the IT hardware and software at the district irrigation offices, developing a training program for new I&D staff. Component 3: Improving irrigation management by Water Users Associations (estimated cost US$ 1.7 million). This component financed activities to strengthen local and federal water user associations (WUA and FWUA) as well as local Water Councils (WCs) and the National Union of WUAs to improve their capacity for coordination and management. The component financed TA for: (i) an assessment and strengthening of 481 WUAs; (ii) supporting and strengthening roughly 29 FWUAs managing off-farm systems; (iii) assessments of 70 WUAs; (iv) technical assistance to WUAs to prepare inventories and maintenance plans for their systems; (v) training programs for WUAs, WUA SUs, and FWUAs to address social dimensions of water management including training activities to ensure member engagement and participation, social accountability, and the role of women in water management. Component 4: Project management (estimated costs US$ 1.5 million). This component financed capacity building and project management, including administration, coordination of the project, procurement, financial management (FM) and monitoring and evaluation (M&E) in line with the procedures of the World Bank. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Cost: The estimated total project cost at appraisal was US$7.75 million, but the actual total cost was $10.91 million). Page 2 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) Financing: At appraisal, the IDA grant financing was estimated at US$7.75 million with additional grant financing of US$4.49 million for a total amount of US$ 12.24 million. The amount disbursed was US$10.91 million, which was 89 percent of the grant. Borrower Contribution: At appraisal, the Borrower did not have a planned contribution. Dates: The original project closing date was August 30, 2017. In total, the project was restructured six times along with the closing date to allow the completion of project activities, reallocation of loan proceeds, disbursement arrangements, modification of components, additional financing, and revision of results indicators. The project was restructured on August 26, 2019, for the fifth time to include additional financing and modify the results framework and PDO. The sixth restructuring modified the results framework and extended the closing date by 15 months. The project was approved on April 4, 2014, became effective on April 29, 2015, and closed on April 1, 2024, for a total of 10 years of implementation, including 81 months of extensions and additional financing. 3. Relevance of Objectives Rationale The Project Development Objectives (PDOs) are sufficiently outcome-oriented and appropriate for the development status of the country. Despite being well-endowed with water resources, only 54 percent of the rural population had access to safely managed drinking water supplies in 2020, and irrigation and drainage infrastructure are outdated and not resilient to climate change. (CPF FY24-28, p. 18) The rural population comprises 62 percent of the population, with a 40 percent poverty rate, which relies heavily on irrigation for agriculture. (PAD, p 3). Strengthening water resource management was considered essential for increasing agricultural productivity and poverty reduction as well as contributing to other critical sectors important for economic growth such as energy production and water consumption for industry and human consumption. The PDOs were relevant to the Country Partnership Strategy (CPS) FY14-17 at appraisal and its Strategic Area of Engagement 3: Natural Resource and Physical Infrastructure and specifically improving management of water resources. During implementation, the PDO remained relevant to the World Bank CPF for FY19–FY22 under Focus Area 2: Raise productivity and build connectivity. It contributed to the achievement of CPF Objective 4: Enhance growth of natural resource sectors, especially hydropower by improving water management within the framework of a changing climate. Under the new CPF for FY24– FY28, the PDOs were relevant most notably to two of the three CPF’s high-level outcomes (HLOs): Improved Access to Sustainably Managed Natural Resources (HLO2); and Enhanced Human Capital and Empowerment of Vulnerable Populations (HLO3). HLO2 specifically focused on energy, water, agriculture, and resilience to climate change, both to safeguard the livelihoods of rural populations and as a driver of economic growth. The PDOs remain relevant and aligned with government priorities and strategies. The PDO’s are aligned with the country’s Water Code, which was signed into legislation in 2005, and aims to modernize water resource management in the country. An important element of the country’s National Development Strategy Page 3 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) 2018–2040 is the development of energy- and water-efficient irrigation networks, with the improvement of agricultural production, which is considered key for food and nutrition security. In July 2017, the government adopted the National Irrigation Program for 2017–2026, which indicated that climate-resilient irrigation infrastructure should be modernized to (a) create new jobs, improve socioeconomic conditions, and reduce migration; (b) develop crop farming; and (c) increase tax revenues and GDP. The project and PDOs are based on the historical experience of the Bank in the sector. Prior to this project, the Bank invested in the Water Management Improvement Project (WMIP) 2006-2013, as well as the Irrigation Rehabilitation Project (IRP) 1998-2005, and the On-farm Irrigation Project (OIP-1 and 2) 2003- 2015. The key lesson learned from earlier projects was the need to focus on strengthening the capacity of existing institutions and implementation of the Water Code rather than focusing on the establishment of new ones. Secondly, the project proposed to pilot an approach to testing different models for operation and maintenance that would be scalable rather than assuming a predefined model will work. This project was highly relevant and strategically focused on strengthening the institutional capacity for water resource management. A historical gap with water reforms in the country was the lack of implementation of the Water Code that was established in 2005. This project aimed to close the policy-implementation gap where previous efforts had fallen short. Given the relevance at appraisal and closing of the PDOs to the CPFs, the lessons learned from prior operations, and that the PDOs are aligned with the country’s Water Code, the rating is considered High. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Improve water resources management capability. Rationale The theory of change (ToC) in the Implementation Completion and Results Report (ICR, p. 44) identified that the outcome of interest was to improve the recipient’s water resources management capacity. The mix of inputs identified is adequate to reach the outcome. These included the financing of interventions through various components: (i) enhancing and establishing water administration units at the national and basin level, (ii) developing and implementing water information systems at national, regional, and local levels, (iii) strengthening the capacity of DWRLI to do basin mapping, water management plans, and a national water strategy, and developing policies and procedures for water use permitting, and (vi) an assessment of investment planning in selected irrigation schemes. Page 4 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) As a result, the expected outputs would have been: (i) national and basin water administration units legally established and trained, (ii) a nationwide water information system installed, (iii) river basin management plans developed and approved, (iv) staff trained on the water information system, and (v) national water strategy approved, and (vi) and national irrigation, drainage, and land improvement policy is approved. As a result of these outputs, expected intermediate outcomes would have been: (i) national and basin water administration units are using water information systems to inform planning and decision-making about the management of water resources; and (ii) national water strategy and national irrigation, drainage, and land improvement policy are being implemented. The result of these intermediate outcomes would be the PDO outcome of improved recipient water resources management capacity, contributing to a higher level of outcome in terms of increased resilience to climate change and increased access to water for human and productive uses. The project was originally conceived as a two-phase program with each phase consisting of a three-year period. The implementation of the project did not follow separate projects but was rather implemented as one project with additional financing. The ToC’s activities, outputs, intermediate outcomes, and outcomes provide a logical and properly sequenced causal chain sufficient to reach the PDO outcome target. The ToC included explicit assumptions such as that: (i) the national water department would be financed through water user fees, (ii) the political environment would support the implementation of strategies and policies, and (iii) water users and farmers would be willing to pay full cost recovery levels. (ICR, p 44). The proposed activities appeared to be logical and adequate in scope and nature to strengthen the capacity to manage water resources. The results framework indicators were appropriate to capture key outputs, intermediate outcomes, and outcomes. While the ToC had an explicit assumption that the political environment would support implementation, it was less clear if there were other important power dynamics with the establishment of multiple institutional entities at different levels that would have supported or resisted the implementation of the reforms that typically accompany reform processes. OUTPUTS: The output level indicators for this objective in the results framework included:  Comprehensive River Basin Management Plans (RBMPs) would be prepared for all 5 river basins, approved by relevant River Basin Councils (RBCs), and endorsed by the National Water Council (NWC). (Original target was 5 RBMPs developed and approved): (Substantially achieved) (ICR, p. 47) The ICR reported that 5 RBMPs were developed through data analysis that was conducted at the central government level which was validated through stakeholder engagement at the local and regional level . (ICR, p 13) Development of the plans included 16 training and seminars for the RBC working groups along with basin webpages with data and analytics. The RBMPs included tailored development programs and projects based on local needs assessment. The drafts were completed in 2021, but formal approval is pending by the RBCs as they await their own legal authorization to be formally recognized.  National Water Strategy (NWS) would be formally adopted as a government strategy, endorsed by the National Water Council (NWC), including investment prioritization and financing plan (Yes/No). (Original target was Yes). (Achieved) (ICR, p. 12). This indicator was added during the restructuring for the additional financing. A national water strategy accounting for surface and groundwater based on the compilation of the five river basin plans was established using a bottom-up participatory approach and was endorsed in 2023 by the National Water and Land Council. Page 5 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336)  Water use permitting and fee system operational as percentage of total permits to be issued was zero: (Original target was 50 percent). (not achieved) (ICR, p 13-14) This indicator was added during the restructuring for the additional financing. The project financed the development of conceptual models for permitting and the calculation of fees for surface water extraction, which were approved by the Cabinet of Ministers in October 2021 and included in a new edition of the Water Code, but the revised Water Code has not been approved by Parliament. Implementation of the permitting and collection schemes for surface water extraction were not implemented by the time of the ICR nor at the time of the ICRR. (ICRR interview) The ICR reported that a Swiss Agency for Development and Cooperation project will continue to support water use permitting, which was expected to be approved by the end of 2024.  Nationwide Water Information System (WIS) would be operational, financially sustainable, online accessible, and Water Information Center established (Yes/No). (Original target was Yes). (Achieved) (ICR, p 12) This indicator was added during the restructuring for the additional financing. The ICR reported that the WIS is operational and is accessible through a website/portal and has been connected with all 53 irrigation units through a single digital network. The WIS has allowed water resources to be measured, quantified, and recorded in a digital data management system. At Project closure, five online thematic databases were operational, and 80 percent of the data had been digitalized. (ICR, p 12)  80 percent of water resources were measured, quantified and recorded in digital data management system. (Original target was 90%) (Substantially achieved) (ICR, p. 27). The WIS enabled water agencies to more efficiently manage and communicate water resource information for decision- making. The system and the digitalization of water information have reduced the irrigation agencies’ workload and improved data systematization efficiency. The WIS has improved the management of information from a paper-based system to an electronic one. The ICR reported that the operating costs of the WIS were partially covered through the central government’s budget, which allocated KGS 267 million for 2023–2025 under the Action Plan for implementation of NWS 2024. The functioning and further improvement of the WIS, including necessary training, data collection and updates, will continue to be supported under the ongoing World Bank-financed Climate Resilient Water Services Project (CRWSP) (P173734). (ICR, p 13) OUTCOMES: The outcome level indicators for this objective in the results framework were the following:  Department of Water Resources Management would be formally established under the State Water Resources Agency (SWRA), gradually financed through the water use fee (No/Yes). (Original target was Yes). (Achieved) (ICR, p 25). The Water Resource Services (WRS) was established in 2021 to oversee water resource management in the country. The WRS was originally established under the Ministry of Agriculture and was then moved to the Ministry of Water Resource, Agriculture, and Processing Industry in 2024. The integrated water resource management has been consolidated under the WRS. The ICR reported that 10 percent of the WRS costs are covered by industry water user fees. (ICR, p. 12)  Basin Water Resources Management offices and River Basin Councils (RBC) to be formally established and become operational. (Original target: 5). (Partially achieved) (ICR, p. 13) This indicator was added during the restructuring for the additional financing. At the basin level, the Project supported the delineation of the country’s five main river basins which were formally approved by the Page 6 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) NWLC in February 2023. In addition, the Project also supported the WRS in developing basic models and statutes of RBCs that include representatives of more than 10 different agencies. The standard regulation on RBCs was approved by the resolution of the Cabinet of Ministers on April 6, 2023. The RCBs have identified each of the basin representatives, but the ICR reported that the NWLC planned to formally legalize the five RBCs by the end of September 2024. The ICR reported the following additional outcome that did not have a target and was not included in the results framework but was related to inputs in the ToC that would contribute to the strengthened water resource management capacity.  The Basin Water Authority (BWA) is a government water agency responsible for water allocation, distribution, and water service to all water users in a given river basin. The ICR reported that one BWA was formally established and became functioning (meaning that financial, technical, and human resources were allocated), while for the remaining four BWAs, directors were yet to be elected by the respective RBCs. (ICR, p 13) The project partially or fully achieved its output, intermediate outcome, and outcome indicators as articulated in the results framework except for the water use permitting and fee collection for surface water. Notable achievements of the project included: (i) the establishment of the Water Resources Service in the Ministry of Water Resource, Agriculture, and Processing Industry enabling a central entity to oversee water resource management activities; (ii) development of RBMPs to detail critical information and investment needs for major basins, (iii) the designation of officials to the RBCs enabling them to carry out functions while they await legal ratification, and (iv) the implementation of the WIS enabling water resource management entities at the central, regional, and local level to exchange information to better manage water resources and make informed decisions. The indicators adequately measured progress against the Objective. The achievement of the results had minor shortcomings and therefore the rating is Substantial. Rating Substantial OBJECTIVE 2 Objective Improve irrigation service delivery for the benefit of water users. Rationale The ToC in the Implementation Completion Report (ICR, p. 44) identified that the outcome of interest was to improve irrigation service delivery for the benefit of water users. The mix of inputs identified was adequate to reach the outcome. These included the financing of interventions through various components: (i) training WUAs and federal water user associations (FWUA) on management, operation, and maintenance (MOM) of irrigation schemes as well as billing and collection, (ii) inventories of infrastructure for I&D schemes, (iii) procedures for developing costing requirements for maintenance, (iv) performance-based management, maintenance, and operation pilots in six I&D systems, (v) upgrading of information and technology (IT) hardware and software at district irrigation offices, (vi) capacity assessments of WUAs and FWUAs, (vii) training of WUA and FWUA on social engagement and accountability, and the role Page 7 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) of women in water management, and (viii) consultancy to develop National Irrigation, Drainage and Land Improvement (NIDLI) Policy. As a result, the expected outputs would have been: (i) 6 pilot irrigation service contracts would be signed for management, operation, and maintenance, (ii) irrigation offices at district and regional levels would have installed new IT systems to manage data, (iii) I&D inventories and maintenance plans would be developed, (iv) training for WUA and FUWAs would be conducted, (v) water management councils (WMCs) be established, (vi) NIDLI policy be developed. As a result of these outputs, expected intermediate outcomes would have been: (i) WUAs and district irrigation offices, regional and federal entities implementing management, operation, and maintenance plans and procedures, (ii) six pilot irrigation service contracts being implemented, (iii) district, provincial, reservoir management agencies, and DWRLI exchanging information on water resources, (iv) WUAs having adequate water to meet their demands, and (v) consumers being satisfied with the services provided by WUAs. The result of these intermediate outcomes would be achievement of the PDO outcome of improved irrigation service delivery for the benefit of water users contributing to a higher level of outcome of increased resilience to climate change and increased access to water for human and productive uses. The ToC’s activities, outputs, intermediate outcomes, and outcomes provide a logical and properly sequenced causal chain sufficient to reach the PDO outcome target. The ToC included explicit assumptions that are the same as objective 1. (ICR, p 44). However, a shortcoming in the assumptions was that there was no mention that the infrastructure and availability of water were adequate and sufficient to meet water demands. The results framework indicators were appropriate to capture key outputs, intermediate outcomes, and outcomes related to the activities financed by the project. OUTPUTS: The output level indicators for this objective in the results framework included:  6 Irrigation Service Contracts with MOM plans, irrigation service level, and Irrigation Service Fee (ISF) would be signed between Dep ILI and WUAs. (Original target 6). (Achieved) (ICR, p 14) This indicator was added during the restructuring for the additional financing. These contracts involved an inventory of current technical conditions of I&D infrastructure and an improved methodology for the estimation of annual MOM costs related to 106 WUAs. The ICR reported that this had helped to prioritize rehabilitation interventions for on-farm and off-farm irrigation and drainage infrastructure that were financed by the APNIP project (P132754) in the pilot areas. (ICR, p 14)  100 percent of system passports would be digitized and would provide information to the Water Information System (WIS) on-line. (Original target was 100 percent) (Achieved) (ICR, p 28). A system passport is a compilation of information about the characteristics of an irrigation scheme such as design capacity, dimensions, and assets that make up the schemes (PAD, pg. 26). Digitalization of the irrigation schemes’ assets enabled better asset management by WUAs, RBCs, and the central government.  53 district irrigation offices (RVKs), provincial irrigation offices (OVKs) and reservoir management agencies would be exchanging digital information with DWRLI Head Office and each other. (Original target 53) (Achieved) (ICR, p 27). The ICR did not elaborate on this indicator. Page 8 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336)  Increase in average annual maintenance expenditures for off-farm systems to 586 KGS/ha. (Original target 600). (Substantially achieved) (ICR, p 28) The ICR did not elaborate on this indicator.  ISF collection rates for 6 off-farm systems would cover at least 90% of the agreed tariff (based on annual MOM plans) after completion of rehabilitation works. (Original target: 6) (Achieved) (ICR, p 28) The ICR did not elaborate on this indicator.  WUA ISF would increase to 743 KSG/ha towards sustainable levels to meet MOM costs for on-farm systems. (Original target was 1,000 KSG/ha) (Partially achieved) (ICR, p 29). The ICR reported that as a result of the project, the WUAs demonstrated an increase in investment per hectare to cover O&M costs, which increased from a baseline of KGS 272 per ha in 2013 to over KGS 743.6 per ha in 2023. (ICR, p 15)  There were 486 water user associations (WUAs) with completed inventory and preliminary maintenance plans. (Original target was 315) (Exceeded) (ICR, p 28) The ICR did not elaborate on this indicator.  408 operational water user associations were created and/or strengthened. (Original target was 460) (Substantially achieved) (ICR, p 28). The ICR reported that the Project increased technical capacities of irrigation service providers at WUAs and provincial and district irrigation offices (Oblvodkhoz/Rayvodkhoz, OVK/RVK) levels by developing improved and joint MOM procedures, detailed asset management plans, and training and capacity-building programs. (ICR, p 14)  103 Water Management Councils (WMCs) were established, trained and applying training in off-farm and on-farm canal management. (Original target was 95) (Exceeded) (ICR, p 29) The ICR did not elaborate on this indicator.  10 Federations of WUAs would take responsibility for MOM of off-farm systems. (Original target was 15) (Partially achieved). (ICR, p 29) The ICR did not elaborate on this indicator.  36 RVK staff would utilize revised MOM procedures in preparing annual MOM plans. (Original target was 30) (Exceeded) (ICR, p 29). The ICR did not elaborate on this indicator.  There were 62,653 hectares of irrigated area with improved off-farm irrigation and drainage (I&D) services with water delivery more adequately meeting WUA water demand. (Original target was 60,000 hectares). (Substantially achieved) (ICR, p 28). The ICR did not elaborate on this indicator.  National Irrigation, Drainage and Land Improvement Policy (including Irrigation & Drainage Investment Plan) would be endorsed by the NWC (Yes/No). (Original target was yes) (Achieved) (ICR, p 29) The National Irrigation, Drainage and Land Improvement Policy was endorsed by the NWC in December 2023. As part of its development, the government was also equipped with an Irrigation and Drainage Investment Plan to facilitate the prioritization of infrastructure investment for the government as well as for the international donor community. The two strategic documents were developed for mid- and long-term interventions on the following aspects: (a) strengthening human resources and institutional capacity, (b) improving normative regulatory frameworks on dam safety, (c) improving water accounting and storage management, (d) improving investment prioritization and tariff setting procedures, and (e) enhancing cooperation between research and development.  There would be 5 percent female representation in WUA Representative Assembly and WUA Council in 60 percent of WUAs. (Original target was 15 percent) (Not achieved) (ICR, p 29) This indicator was added during the restructuring for the additional financing. The Project made some improvements to increase female representation in WUA Representative Assembly and WUA Councils; however, the Page 9 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) increase was negligible - from 4 percent at baseline to 5 percent by closing, despite explicit efforts to target women in project activities and in training sessions (ICR, p16) OUTCOMES: The outcome level indicators for this objective in the results framework are the following:  Irrigation and Drainage Management (IDM) divisions adopted revised procedures for Management, Operation and Maintenance (MOM) in 6 pilot schemes. (Original target was 6) (Achieved) (ICR, p 26).  97 percent of WUAs are supplied with adequate volumes of water for the six pilot schemes by Dep ILI. (Original target was 75 percent) (Exceeded) (ICR, p 15). The ICR reported that adequate demand was defined by whether 90 percent of the WUAs in a district had their water demand met.  92 percent of WUAs provided satisfactory services, as measured by the percentage of water users responding in surveys that they are satisfied with services provided by the WUA. (Original target was 83) (Exceeded) (ICR, p 26). This indicator was measured in the six pilot I&D districts where 106 out of the 440 registered WUAs operated. The pilot districts and associated WUAs benefited from rehabilitation efforts of irrigation systems funded by the World Banks APNIP, as well as the improved management, operation, and maintenance provided through this project. The ICR stated that the WUA Support Unit reported that in 2023 80 percent of the 440 registered WUAs were rated as ‘good’ or ‘satisfactory’ by water users. The ICR did not provide a source or report to substantiate this data point.  761,973 hectares were provided with irrigation and drainage services (ha). (Original target was 750,000 ha) (Exceeded) (ICR, p 26) These were the number of hectares managed by the WUA that benefited from institutional strengthening and capacity building. (ICRR interview)  The number of direct project beneficiaries was of the order of 278,313. (Original target was 298,000 people) (Substantially achieved) (ICR, p 26) The ICR reported that the Project did not provide new construction or rehabilitation of I&D infrastructure, but contributed to improved I&D services through capacity building of 408 WUAs serving about 278,313 water users (direct project beneficiaries). (ICR, p 14). The ICR did not provide a breakdown of how it calculated this figure.  8.70 percent of direct beneficiaries were female. (Original target was 10 percent). (Substantially achieved) (ICR, p 26). The ICR did not elaborate on this indicator. The project substantially achieved or exceeded its output, intermediate outcome, and outcome indicators as articulated in the results framework with the exceptions of the increase in the irrigation service fee and the percentage of female representation on WUAs. The ICR did not elaborate on various indicators that were included in the results framework, which would have bolstered and substantiated the ICR’s storyline given the high achievement of the indicators. Likewise, the ICR storyline could have more clearly articulated the distinction in interventions that the project provided between different geographic areas, and how those impacted the intermediate and outcome indicators such as with the six pilot schemes and the rest of the WUAs. Nevertheless, the project’s results framework tracked an array of interventions at the output, intermediate outcome, and outcome levels that captured the complexity of the project interventions that took place at central, regional, district, and local levels. Overall, the number of indicators and the quality of what Page 10 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) they were measuring - such as satisfaction of water users - adequately measured progress against the Objective. The achievement of the results had minor shortcomings and therefore the rating is Substantial. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The ICR documented the achievement of the output, intermediate, and outcome indicators with minor shortcomings in their achievement. The results framework adequately measured the theory of change. The rating for Objective 1 was Substantial and Objective 2 was Substantial, which resulted in an overall efficacy rating of Substantial with minor shortcomings. Overall Efficacy Rating Substantial 5. Efficiency Economic Efficiency: An economic and financial analysis was neither carried out at appraisal, the AF stage, nor in the ICR, on account of the challenges of calculating a net present value and an economic rate of return for technical assistance activities. The ICR reported that the project was the first 3-year phase of a two-phase program. The ICR did document that a moderate investment of US$1.7 million in improving irrigation management by WUAs increased the average annual WUA ISF nationwide (700,000 ha) from KGS 270 per ha (US$3.9 per ha) in 2017 to KGS 800 per ha (US$11.6 per ha) in 2023, generating additional contributions in MOM of US$5.4 million a year in 2023. (ICR, p 17) Operational and Administrative Efficiency: The project was designed to be implemented in two three-year phases for a total of 6 years; however, the second phase was incorporated through additional financing. The project was extended 6 times for a total of 81 months (almost 7 years) including the additional financing beyond the initial closing date of August 2017. The project was approved in April 2014. The reasons for these extensions were in largely due to: (i) initial delays in effectiveness and implementation on account of delays in government approval and parliamentary ratification of the grant (12 months), (ii) delays in approval of the additional financing (10 months), (iii) COVID-19 restricted travel of consultants, (iv) political crises in 2017 and 2020, and 8 ministerial changes that affected procurement approvals and implementation, and (iv) changes to the implementing agency (five times). The project was implemented over 10 years, of which six years were anticipated at design for project implementation. Conclusion. The project experienced operational and administrative delays of 48 months beyond the envisioned 72-month implementation period at appraisal for both Page 11 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) phases resulting in a total implementation period from approval to closing of 10 years. The efficiency rating is Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The project was aligned with the previous and current World Bank CPF as well as the government priorities and legislation. The project largely achieved its stated output and outcome indicators which were well articulated to support the theory of change, but the ICR could have elaborated on all of the results achieved. The project faced delays of 48 months for both phases. Nevertheless, taken together, a High rating for Relevance, a Substantial rating for Efficacy, and a Modest rating for efficiency results in a Moderately Satisfactory rating for the overall outcome. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The Project’s benefits will likely be sustained in the short to medium term. The project laid the groundwork for the design of a USD$ 100 million investment project – the CRWSP (P173734), which will continue to support the government with investments and reforms at least through 2028. The government has demonstrated its continued dedication to the sector through the adoption of its new National Water Strategy, with the main pillars focusing on promoting the rational use of water resources and scaling up the use of water-saving technologies based on principles of IWRM and river basin planning. The role of the autonomous WUAs has the potential to be weakened in the future if the government and its legislature make changes to the role of local governments and state basin water agencies at the on-farm level. This risk is considered moderate. The sustainability of irrigation service fees is considered moderate Page 12 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) and varied across the country with some areas paying increased fees with limited water availability and others demanding that water be provided for free through local governments, which is not unique to the country. The risk that the WIS will not be maintained and used is moderate. At the writing of the ICR, it was reported that the WIS was functioning and further improvement, including necessary training, will continue to be supported under the ongoing World Bank-financed CRWSP. Moreover, the government has indicated that the WIS will be integrated with the Kyrgyz national e-government system, which would require that all data entry and storage become a mandatory government service for citizens of the Kyrgyz Republic. If future political crises materialize, it is possible that related institutional changes could negatively impact the sustainability of project outcomes; however, the ICR (p. 19) indicates that the government has remained committed to water sector reforms and gradually implemented them during this project despite dealing with political crises. The overall risk to development outcomes is moderate. 8. Assessment of Bank Performance a. Quality-at-Entry The Project was strategically relevant and designed, based on lessons learned from previous projects such as the Water Management Improvement Project (WMIP) (P088671) that attempted to push for sector reforms that were not aligned with the prevailing power dynamics in the sector. WMIP attempted to set up an independent State Water Authority, which was not successful despite it being part of the Water Code. This project focused on implementing the functions of the State Water Administer within the existing institutional arrangement effectively going with the grain of the sector power dynamics. The Project took advantage of the grant opportunity provided by the Swiss Development Corporation (SCD) which had been engaged in the sector. The World Bank’s experience in the country and relationship with government counterparts supported the design of a technical assistance program with a clear Theory of Change aligned with the government's Water Code and existing institutional arrangements. However, the Bank team underestimated the capacity of the PIU to manage multiple projects, which should have been known given the Bank’s prior experience in the sector. Overall, the quality at entry is rated as Satisfactory. Quality-at-Entry Rating Satisfactory b. Quality of supervision The World Bank team provided regular implementation support to the Project with relatively minimal turnover of Task Team Leaders (TTL) over the 10 years of implementation. There were three TTLs and the Project benefited from having the TTLs and technical teams based in-country and the region, which proved critical to develop trust with government counterparts (ICCR interview). Over the course of the project, the Page 13 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) task team carried out 18 official implementation support missions that were well documented in the Aide Memoires. The task team contended with significant changes in the government and implementation unit but provided close support throughout (ICRR interview). The candor and quality of the Implementation Status and Results Reports of the World Bank were satisfactory, and the project was downgraded at appropriate times when implementation lagged. The task team was proactive in restructuring the project as needed to ensure that the PDOs could be achieved. The task team conducted joint supervision missions with SCD and ensured close coordination between the donor and government enabling the additional financing and achievement of results. The shortcomings during supervision were related to the monitoring and evaluation and the lack of reporting on the quality of the data provided by the PIU. The reporting improved over time after a dedicated M&E specialist was hired in 2015. Overall, the quality of supervision was considered Satisfactory. The World Bank worked with the client to design a project that was relevant and aligned with government priorities and brought the client the required expertise over the operation’s life. The World Bank team closely supervised the Project and proactively restructured it as needed to ensure successful implementation. The minor shortcoming was the overestimation of the PIU's capacity to manage several projects given the Bank’s experience in the country. Overall, the World Bank’s performance is rated Satisfactory. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The results framework indicators were adequate and sufficient to measure the project results at the output, intermediate, and outcome levels for the PDOs, and were aligned with governments roadmap to implement their Water Code. The objectives were clearly specified, and the indicators captured the components activities and outputs as well as the contribution to the outcomes of the PDO statement. The M&E design had some minor shortcomings. For example, while baseline information was reported for indicators, the ICR was not clear about the design and the methodology used to collect it other than mentioning that the baseline figures were collected from the previous World Bank projects. Additionally, the Project was designed to implement activities countrywide except for the six pilot schemes where additional or modified activities would take place such as improved management, operation, and maintenance plans. The M&E design could have been stronger to measure the difference in outcomes between areas that were receiving additional benefits such as in the pilot areas vs the non-pilot areas to make a stronger case to scale up or not to scale up pilot activities. During the Additional Financing, two new PDO and four intermediate results indicators (IRI) were added to reflect institutional changes that occurred during project implementation. The ICR stated that other minor weaknesses were that some of the IRIs had targets that were too ambitious and would require a longer time period to achieve such as increased irrigation service fee rates. However, Page 14 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) overall, the results framework indicators adequately captured the theory of change with 8 PDO and 18 IRI indicators. b. M&E Implementation The project M&E system was not put in place initially because the PIU did not have an M&E specialist until 2015. The PIU was responsible for monitoring the project’s progress. The ICR noted that progress reports and results framework updates were generally timely and made available by the PIU to the World Bank. (ICR, p 15). The World Bank’s implementation Status and Results reports (ISRs) document that the PIU was reporting against the indicators throughout the life of the project, showing gradual improvements signaling that the M&E was being used to inform dialogue on project implementation. What is not clear from the ISRs and the ICR is the quality and reliability of the data that were provided by the PIU. The PIU did undertake two surveys of water users in 2022 and 2023 within a sample of WUAs to measure water users’ satisfaction with the performance of WUAs enabling the PIU to report against the indicator. c. M&E Utilization The M&E results were used to inform project management, allowing course corrections and the justification for additional financing. The ICR reported that the M&E system was used as a tool to initiate proactivity when project implementation slowed. (ICR, p 21) The M&E system and results framework were adjusted accordingly during implementation to respond to institutional changes as well as reflect modifications at restructurings to capture additional activities. Overall, there were moderate shortcomings in the M&E systems design, implementation, and utilization, and were generally sufficient to assess the achievement of the project objectives and measure the theory of change. The M&E rating is Substantial. M&E Quality Rating Substantial 10. Other Issues a. Safeguards Environmental safeguards. The Project was classified as Category 'B' and only triggered OP 4.01 for Environmental Assessment, since the project was not expected to involve any large-scale, significant, or irreversible adverse environmental impacts. The project screened relevant activities before they were executed, and the screening procedures were identified in the Environmental Management Framework (EMF), which was published prior to appraisal. The EMF specified that terms of reference for technical assistance activities such as planning would follow World Bank Safeguard policies. The Project procured mostly goods, services, and works for establishing the water information system as well as consultancy services for capacity building and advisory purposes. There were selective small-scale Page 15 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) demonstrations on modern irrigation efficient technologies. (ICR, p 21) The Project closed with a Satisfactory environmental safeguards rating. Social safeguards. The Project established a grievance redress mechanism from the Project start. The PIU maintained a registry for all complaints and received four that were related to procurement procedures which were duly addressed. WUAs and RCBs played an important role in organizing agricultural communities across the Kyrgyz Republic. These organizations are facilitating broader consultations with community-level stakeholders allowing for local issues and knowledge to be considered and integrated into the basin plans. In addition, the project supported the establishment of public water management councils to help resolve conflicts between water users and WUAs. The Project financed training programs that included attention to the social dimensions of participatory water management under WUAs, including member participation and accountability, resolution of water-related conflicts, inclusion of vulnerable groups, and strengthening the role of women. (ICR, p 21) The Project closed with a Satisfactory social safeguards rating. b. Fiduciary Compliance Financial management. The ICR (p 21) noted that the financial management performance, including budgeting and planning, accounting and financial reporting, flow of funds, internal controls, and external audit, was rated Satisfactory throughout implementation. The ICR noted that there was no issue with the quarterly financial reports and annual audits carried out by private auditors. The Project closed with a Satisfactory rating for financial management. Procurement. The ICR noted that procurement had moderate shortcomings throughout the life of the project and particularly at the beginning when familiarity with World Bank procurement procedures was weak. The Project Implementation Unit (PIU) also suffered from complex decision making within the Department of Water Resource and Land Improvement related to procurement matters and the frequent changes to the institutional arrangements causing significant delays. The ICR reported that the procurement performance improved in the second half of Project implementation by replacing staff and hiring additional specialists. The World Bank procurement staff provided regular monitoring and capacity building to the PIU staff. Minor shortcomings remained concerning contract management, advertisement and disclosure arrangements, bidding processes, and incomplete information uploaded in the Systematic Tracking of Exchanges in Procurement (STEP) tool. The Project closed with a Moderately Satisfactory procurement rating. c. Unintended impacts (Positive or Negative) The ICR reported a positive unintended impact related to encouraging and fostering the interest of young professionals to work in the irrigation sector. The project organized a knowledge exchange event with four Kyrgyz universities (160 students) to learn about good practices in irrigation, two students went on to conduct research on the pilot schemes financed by the project which they used to defend their thesis. The Bank team and PIU were intentional about integrating academia into the project to support building a pipeline of future professionals. Page 16 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons Integrated Water Resource Management (IWRM) reforms are inherently complex, and development partners need to able to support clients with best-fit solutions rather than best practice allowing the human and social systems involved in IWRM reforms to gradually adapt to new realities. Because reforms and changes to social systems are conservative, development partners need to adopt realistic expectations about what can be achieved within specific timeframes. The Bank’s sustained engagement in the sector over the last 20 years, use of a mix of lending and grant-financed technical assistance, and a willingness to shift from best practice to fit-for-purpose interventions enabled the government to make changes at a rate acceptable to stakeholders. Whereas the predecessor project (WMIP) failed to achieve certain institutional reforms that mirrored best practice in IWRM such as having an independent state water authority, this project took a more incremental approach and focused on ‘function’ rather than ‘form’, and worked to embed the functions of a state water authority into existing institutional arrangements. Projects implemented on a national scale with activities scattered across the country can benefit from a specialized M&E firm to support M&E systems. While the project's M&E system was effective in measuring results and overall success, a dedicated M&E firm could be engaged for future projects to extend the client's capacity and bring in good practices to manage the complexity of dispersed project activities and ensure effective and efficient progress monitoring. (ICR, p 23) Effective donor coordination between the World Bank and bilateral organizations can leverage financial and technical resources benefiting government clients. This project demonstrated that solid coordination, knowledge sharing, and trust between the World Bank task team and bilateral donors such as SCD can maximize both financial and technical resources. The World Bank had deep experience in the country and was able to draw on its global experience in the sector, bringing that knowledge to the client with financial and technical support from SCD. The partnership and trust that was developed enabled the project task teams to adequately support the client when project implementation slowed and facilitated the additional grant financing. (ICRR interview) Page 17 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) To improve the sustainability of training activities, projects could consider strategic partnerships with local universities and vocational colleges to integrate project training materials into their curricula. This ensures that the knowledge is passed on to future generations and remains relevant and accessible. Projects could also work with local government bodies or sectoral agencies to institutionalize the training programs within their regular operations as well as encourage organizations involved in the training to develop succession plans that include knowledge transfer to new members or employees. (ICR, p 24) 13. Assessment Recommended? Yes ASSESSMENT_TABLE Please Explain Yes. While the project was a relatively small grant-financed project, the lessons learned from the stand-alone technical assistance grant have the potential to provide rich insights for other countries and World Bank operations. An assessment could explore the benefits of having a stand-alone technical assistance grant that is not directly embedded in a larger investment operation, but could be complemented by other investment projects. An assessment could explore how such a modality influences the dialogue between the Bank and the client, what aspects of this arrangement were unique to the Kyrgyz Republic, and what elements could be transferable to other countries. 14. Comments on Quality of ICR The ICR provided a detailed review of the project, a clear theory of change, and a narrative that largely supports the ratings. The ICR was candid and the lessons well thought through. One of the main shortcomings of the report was that it did not elaborate or incorporate all of the indicators into the storyline of the ICR. The results framework had numerous indicators, which may have limited the team from expanding on each one in depth; however, the richness of the results framework is what underpins the project’s theory of change. Not incorporating more of the intermediate results indicators into the storyline was a missed opportunity to fully appreciate the changes that the project supported. Nevertheless, these shortcomings were considered minor and the ICR Rating is Substantial. a. Quality of ICR Rating Substantial Page 18 of 19 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review NWRMP (P144336) Page 19 of 19