Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) Report Number: ICRR0023784 1. Project Data Project ID Project Name P145529 MA: Rural Water Supply Country Practice Area(Lead) Morocco Water L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-83670,IBRD-83680,IBRD-83970 30-Sep-2021 128,848,487.61 Bank Approval Date Closing Date (Actual) 29-Apr-2014 31-Mar-2023 IBRD/IDA (USD) Grants (USD) Original Commitment 158,615,000.00 0.00 Revised Commitment 158,615,000.00 0.00 Actual 129,883,749.37 0.00 Prepared by Reviewed by ICR Review Coordinator Group Craig Phillip Kullmann Fernando Manibog Ramachandra Jammi IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Loan Agreement (Schedule 1), the Project Development Objective (PDO) was to provide access to safe and reliable drinking water supply for rural communities in targeted under-served areas in the Project Area. The Project Appraisal Document (PAD, page iv) identified the Project Area to include the provinces of Nador and Driouch (Northern provinces), Bedouza, Skhour Rehamna, Sidi Bennour (Central Provinces), and Tiznit, Taghzout and Ait Baha (Southern Provinces). Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) The PDO did not change during the life of the project, and this review has assessed the PDO achievement in terms of the following two objectives: I. Provide access to safe drinking water supply for rural communities in targeted under-served areas II. Provide access to reliable drinking water supply for rural communities in targeted under-served areas A “safe water supply” is defined as a water supply service which meets the appropriate quality standards, as required by the applicable Moroccan regulation. A “reliable water supply” service is a continuous water service, provided twenty-four hours a day seven days a week. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components Component 1: Extension of access to water supply through standpipes in selected provinces within the Project area. (US$ 194.2 million at appraisal, and US$155.8 million actual). This component financed:  the construction of lateral mains from regional trunk lines, elevated or grounded storage tanks, pumping stations, and public standpipes (Loan Agreement, Schedule 1). A total of 1,400 villages within the identified Project Area were estimated to benefit. The August 2016 restructuring dropped water distribution works in the Province of Tiznit, but added 1,000 km of pipeline, 15,000 cubic meters of storage, and 1,500 standpipes in the Province of Safi;  the expansion of the Tiznit water treatment plant and associated infrastructure. The restructuring in August 2016 added the completion of the Safi water treatment plant, works and acquisition of goods to deliver water supply to selected villages through the construction, rehabilitation and strengthening of water production and transmission systems. These new activities were originally covered under the Rural Water and Sanitation Project (BIRD II) project (P086877) but were not finalized before that project’s closing and therefore added to this Project. Both projects share similar PDOs of increasing access to rural water supply. Component 2: Supporting the shift towards water service delivery through house connections in the Project area. (US$ 15.2 million at appraisal and US $9.5 million actual). This component financed:  the Pre-financing mechanism to pre-finance a share of the cost of households’ individual water connections  works and procurement of goods to increase the physical capacity of the regional water supply network to improve service delivery for household connections. The August 2016 restructuring added 4,200 household connections. Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) Component 3: Implementation Support and Capacity Building. (US$ 13.8 million and US $14.8 million actual). This component financed:  technical advisory services in the following areas: (i) project management and monitoring and evaluation, (ii) participatory approaches to water service provision, and (iii) identification of proper wastewater management solutions for villages and raising awareness on sanitation.  studies and strengthening capacity of the Borrower in the following areas: (i) rural water supply, (ii) management of the overall universal water access program, (iii) preparation of sustainable management models for extension and management of house connection service provision; (iv) performance of rural water supply standpipe service and household connection service operations, and (v) remote management pilots in rural water supply, and opportunities and conditions for scaling up successful approaches. The April 2021 restructuring dropped activities i-iv related to studies and capacity building. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The total project cost, inclusive of borrower contribution was estimated at US $223.60 million but the actual total cost was US $180.40 million. Financing: The International Bank for Reconstruction and Development provided an Investment Project Financing for this project in the amount of US$158.60 million. Due to currency exchange fluctuations between Euros and US dollars at closing, the total available financing was USD136.80 million. The amount used was US$129.90 million, which was 95 percent of the loan. Unused funds were returned to the Bank. Borrower Contribution: The Borrower committed and disbursed US $50.50 million in counterpart financing. Dates: The Project became effective on December 18, 2014, and was completed by March 31, 2023, following two extensions that added 18 months to the project period. A Mid Term Review (MTR) was conducted from October 31, 2018. There were five restructurings of the project that include the following:  On August 4, 2016, a level 2 restructuring modified Component 1 activities to incorporate subprojects that could not be completed by the Regional Potable Water Supply Systems Project (P100397) (BIRD II) that was closing. BIRD II had similar project development objectives and operated in the same provinces as this Project (P145529) (BIRD IV). The restructuring changed the geographical scope of the project slightly and resulted in more ambitious outcome targets.  On June 1, 2017, a level 2 restructuring modified the disbursement arrangements for Components 1 and 3 from direct payment to designated accounts to improve disbursements.  On June 29, 2018, a level 2 restructuring incorporated OP 4.37 on dam safety since the Project relied on two dams for water supply.  On April 6, 2021, a level 2 restructuring extended the closing date by 15 months to December 31, 2022, changed the implementation schedule, revised the results framework a second time, and reallocated resources between disbursement categories, and dropped activities under Component 3. Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529)  December 17, 2022, a level 2 restructuring extended the closing date a second time by three months to March 31, 2023. 3. Relevance of Objectives Rationale The PDOs remained relevant to the World Bank's Country Partnership Framework (CPF 2019-2024). The focus areas of the CPF are: (i) Promoting Job Creation and Private Sector Growth; (ii) Transforming Human Capital; and (iii) Promoting Inclusive and Resilient Territorial Development. The PDOs supported the third focus area and specifically Objective 9: Improving access to sustainable water resources in lagging regions and rural areas, because the Project would increase access to basic water supply services in underserved and poor rural areas. The PDOs also remained relevant to the country's National Program for Drinking Water Supply and Irrigation 2020-2027 (PNAEPI). Achieving universal access to safe and reliable water in rural areas is one of the five key axes of the PNAEPI. The Project builds on previous World Bank engagement in Morocco's rural water sector over the last 20 years and is the fourth investment project financing targeting rural areas. The outcomes under this Project are consistent with previous projects, focus on different geographic areas, and are more ambitious in terms of the quality of services delivered. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Provide access to safe drinking water supply for rural communities in targeted under-served areas. Rationale The theory of change (ToC) in the Implementation Completion Report (ICR) for this objective was explicit about the hardware aspects of the Project such as the expansion of water treatment plants, rehabilitation and expansion of networks and trunklines, along with expanding access to standpipes and household connections would lead to access to safe drinking water service delivery in participating villages and provinces and contribute to the government’s universal access program. While the theory of change also included a reference to technical assistance to support Project implementation, the ToC was not explicit that this assistance entailed a robust participatory and social mobilization activity to engage communities in the selection of the service type (standpipe or household connection). These participatory approaches are a vital Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) part of the Project to ensure user engagement and buy-in and could’ve been more explicit in the ToC. Likewise, the ToC did not explicitly include post-construction operation and maintenance arrangements, which is an indicator in the results framework and an important part of the ToC to achieve sustained results. The ToC would’ve been stronger by making these ‘soft’ elements more explicit. The ToC made four key assumptions: (i) no shift in political commitment, (ii) adequate technical capacity available, (iii) adequate sanitation solutions are verified in localities requesting household connections, and (iv) standpipes have reliable operational arrangements. These assumptions appeared to hold true during Project implementation. The results at the household level for PDO 2, Provide access to reliable drinking water supply for rural communities in targeted under-served areas, are reported in this section to reduce duplication. All other results related to PDO 2 are reported below. This theory of change also applies to PDO 2 since the same infrastructure and software investments deliver the safety and reliability outcomes. OUTPUTS:  2 Water Treatment Plants (Tiznit and Safi) constructed and operational (no original target) (ICR, page 15 and interview with TTL)  Constructed or rehabilitated 3,227 standpipes of which 2,589 had water services operational at ICR. (Original target was 1,600, and revised target at restructuring was 2,831. (constructed or rehabilitated standpipes exceeded revised target, but 91 percent of revised target was operational at ICR - almost achieved) (ICR, page 34)  3,219 km of pipeline installed or rehabilitated. (Original target was 2,600 km, revised target at restructuring was 3,000 km). (exceeded) (ICR, page 31)  22,444 household connections pre-financed. (Original target was 10,000, revised target at restructuring was 14,200). (exceeded) (ICR, page 28) OUTCOMES:  Water quality: In 2021, 2,048 water samples were taken in the areas served by the Project, and the national water operator (ONEE) reported 99 percent compliance with Moroccan Norm NMO3-7-002 (ICR, pg. 16) (Original target 100 percent, almost achieved). Additional information provided by the Task Team Leader after the November 27, 2023 interview showed that ONEE undertakes a water quality surveillance program that is ISO certified (2022) testing for a series of bacteriological and chemical parameters (email correspondence Nov 30, 2023). The additional information provided substantiated that the surveillance program is working, but did not substantiate testing results of water quality beyond 2021. The interview with the ICR author on December 1, 2023 confirmed that water quality information beyond 2021 was requested but not supplied by ONEE. The ICR author visited the treatment plants and confirmed the treatment processes, and the water quality monitoring program were sophisticated and functioning.  Access: 1,053,730 people had gained access to safe (and reliable) water through the Project. (Original target was 440,000 people, revised target at restructuring was 1,101,340). (Almost achieved). This includes: 621,360 people who gained access through 2,589 newly constructed or rehabilitated operational standpipes (Original target was 390,000, revised target at restructuring was 679,440 people, almost achieved). The total beneficiaries reached also includes 112,220 people who gained access through 22,444 household connections (Original target was 50,000 people, revised Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) target at restructuring was 71,000 people, exceeded); and 320,150 people who benefitted from the water production reinforcement activities of the Project. (No original target, added target at restructuring 350,900 people, almost achieved). (ICR, page 28) Evidence and data sources substantiating results in the ICR for people benefiting from system reinforcement efforts is limited. An interview with the Task Team Leader on November 27, 2023 confirmed that the Task Team monitored results in a disaggregated manner by service type and location, and between BIRD II and BIRD IV. The TTL stated that the monitoring information was triangulated through frequent monitoring of contracts (bi-monthly) and field visits. The TTL shared monitoring reports from the government with disaggregated data by service type, province, BIRD II vs BIRD IV on November 27, 2023 substantiating the TTLs claims.  Gender: Of the 1,053,730 people who gained access to safe and reliable water, 50.1 percent were female beneficiaries (Original target was 55 percent, almost achieved). Evidence presented for gender statistics was limited in the ICR. The project originally intended to have gender bias towards women and set a target of 55 percent of female beneficiaries. The implementation of how that would be achieved was not clear at design stage nor through implementation as project investments benefited both male and females (ICR pg 13).  Poverty: 6 percent of beneficiaries were poor (Original target 15 percent) (ICR, page 13). The poverty focus, while well intended at design stage was not monitored through implementation. The selection of provinces and villages in the PAD indicated that they were selected based on poverty data (PAD, pg 5); however, shifts in geographic targeting during implementation did not take this into account (interview with TTL). Nevertheless, the TTL shared demographic and poverty data by province substantiating the income and multidimensional poverty rates in the Provinces targeted by the Project. The provinces targeted had multi-dimensional poverty rates reaching close to 30 percent in rural areas (email correspondence). The ICR (pg 13) reported that the beneficiary survey confirmed that most of the beneficiaries were farmers with 68 percent literacy and 42 percent with primary education. These data points suggest that the project did serve poor rural households.  Reliability: Beneficiary survey results report an increase in the quantity of water consumed per household from 155 liters per day in 2018 to 229 liters per day in 2022. (ICR pg. 42). The ICR presented scant evidence on reliability of water services. The November 27 interview with the TTL indicated that ONEE’s policy is to operate services at 24 hours per day 7 days a week (minus service repairs), and that the systems are interconnected and operate on a SCADA system. However, there was no evidence provided on system pressure during implementation. The beneficiary survey of self- reported increased consumption is one data point that suggests the reliability and access to water services improved (see PDO 2 below). An interview with the ICR author on December 1, 2023 confirmed that there is no data from the SCADA system on pressures and continuity. ICR author confirmed that beneficiary villages moved from point sources such as wells to being connected to large bulk water infrastructure fed from large dams, water treatment plans, and reservoirs, thus improving reliability. The Project resulted in almost fully achieving or exceeding the original targets, resulting in Substantial efficacy rating. Rating Substantial Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) OBJECTIVE 2 Objective Provide access to reliable drinking water supply for rural communities in targeted under-served areas. Rationale See PDO 1 for the theory of change. OUTPUTS  98 reservoirs constructed with total capacity 44,540 cubic meters. (Original target was 17,000 cubic meters, revised target at restructuring 43,690 cubic meters. (exceeded) (ICR, page 35)  118 pumping stations constructed. (No original target) (ICR, page 35)  Studies for the design of secondary network for water supply supported in 410 villages. (No original target) (ICR, page 35) OUTCOMES  See PDO 1 for household level results.  More than two-thirds of standpipe operators were paying their water consumption invoices to ONEE regularly (ICR, paragraph 35).  2,219 standpipes had formal operation and maintenance arrangements in place post construction. (Original target was 1,400 standpipes, revised target at restructuring was 1,980). (exceeded) (ICR, paragraph 35)  Standpipe operators reported that 100 percent of the households they serve pay for the water they buy. (ICR, paragraph 36)  Interview with ICR author and TTL (November 27 and December 1, 2023) confirmed that standpipes and household connections are metered, and if standpipe operators do not pay their bill, ONEE will shut off water supply increasing accountability for operator to maintain services.  Results from the 2022 beneficiary survey reported that that 99 percent of the beneficiaries of standpipes were satisfied with the service with 48 percent of the standpipes being accessible more than 6 hours per day, 12 percent between four to six hours per day, and 40 percent between two to four hours per day. The survey reported that when a household needs water outside of the agreed times of operation, it can contact the standpipe operator to obtain water from the standpipe. About 98 percent of standpipe users were satisfied with the waiting times to purchase water at the standpipes. Survey results prior to Project interventions reported that water was provided on average for only two hours per day, often at random times. (ICR, paragraph 35)  The 2022 beneficiary survey reported that 23 percent of the standpipe operators said they had no service interruption. Operators who reported interruptions were during the first months of commissioning of the standpipe when testing was carried out. There were also service interruptions due to water use restrictions in response to the drought. The standpipe operators reported problems with functionality of the standpipes as the reason for the water cuts in only four percent of the cases. (ICR, paragraph 36).  While additional information from ONEE’s SCADA on system pressure would be the most objective data point on reliability, the evidence presented on additional storage capacity, pumping stations, and Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) increased sizing of mains to accommodate demand coupled with the beneficiary survey results suggest that reliability improved. The Project resulted in almost fully achieving or exceeding the original targets, resulting in Substantial efficacy rating. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale While noting the weaknesses in the availability of a longer time series of data for water quality and reliability, there was sufficient evidence through multiple sources that substantiates the project's overall efficacy rating as Substantial. Overall Efficacy Rating Substantial 5. Efficiency Ex Ante Economic Efficiency: At appraisal, the economic rate of return (ERR) was estimated at 16 percent with a Net Present Value of (MAD) 312.70 million for the investment based on two benefit streams: (i) reduction of the water-related burden of disease; and (ii) opportunity cost of time. For reference, this NPV calculation used a discount rate of 10 percent. Ex Post Economic Efficiency: The economic analysis at project completion in the ICR included the same benefits streams identified at appraisal. The economic rate of return (ERR) at closing was estimated at 21 percent with a Net Present Value of (MAD) 1,054.30 million. The ex-post analysis also used a discount rate of 10 percent. Since activities were transferred from BIRD II to BIRD IV, the Task Team confirmed that there was no double- counting since only the benefits and costs related to those activities were captured in the economic analysis in addition to the benefits and costs of investments originally planned under BIRD IV. (email correspondence with Task Team Dec 1 2023) Financial Efficiency: The ICR stated that the financial assessment of ONEE was carried out to analyze its operational efficiency, financial capacity, and long-term viability to support a project with negative financial returns. ONEE is national utility that provides water and electricity, and at appraisal and at closing it was not possible separate revenues and costs for the different services, and therefore the financial analysis of ONEE Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) was for both services. Key operational and financial indicators developed at appraisal to monitor ONEE financial performance during Project implementation were analyzed at ICR, including the ratio of total operating revenues to total operating expenses (Operational Cost Coverage ratio) and the ratio of current assets to current liabilities (Current ratio). The Operational Cost Coverage ratio of ONEE demonstrates that ONEE’s overall operating revenues exceed operating expenses, and the current ratio demonstrates a healthy financial position for ONEE. (ICR. Pg 43) The ICR stated that at appraisal a financial analysis of the project and select financial indicators of ONEE were evaluated, the results of which demonstrated that the projected operations and maintenance significantly exceeded the expected tariff. At closing this analysis was repeated to understand the financial evolution since appraisal. The financial NPV of the project at closing is MAD -885.9 compared with MAD -1702 at appraisal. The improvement in financial returns is driven by the increased number of project beneficiaries, resulting in greater revenues to ONEE for the same level of investment. (ICR, page 42) Operational and Administrative Efficiency: The project was designed to be implemented over 7 years. Delays in implementation were a result of procurement challenges, land acquisition, and payment arrangements in addition to the COVID-19 pandemic and Russian’s invasion of Ukraine which impeded works in various locations and had a negative impact on the financial position of contractors. As a result, disbursements reached only 34 percent by the time of the MTR in October 2018. These early challenges led to a number of actions including: (1) dropping activities that were not advancing and incorporating sub-projects from BIRD II that were ready to be financed and implemented, (2) changing disbursement arrangements from direct payment to a designated account which expedited contractor payments, (3) measures to improve land acquisition, and (4) citizen engagement processes that helped localities make final decision on the system designs. The participatory process allowed for the adjustment of sub-project designs to optimize the location of the infrastructure to prioritize public land, therefore avoiding compensation costs associated with private land acquisition. (ICR, page 43) ONEE with the support of the Bank developed a detailed methodology for land acquisition and a decision-tree- map describing the process to follow for each Project Affected People (PAP). ONEE also launched a large communication and sensibilization campaign in 2015, which resulted in the clarification of the legal status of the land plots, availability of land titles, and PAPs’ positions regarding the proposed compensation amount, among other themes. (ICR, page 18) A similar approach was followed for the implementation of any other new works in subsequent years of implementation. The project was extended twice for a total of 18 months to allow for the completion of works. Conclusion. The project's estimated EIRR at 21 percent was a significant result compared to a range of 7-10 percent discount rate suggested for Morocco in the World Bank economic analysis guidelines. This result, together with ONEE's financial efficiency with a positive operating cost coverage ratio of 1.10 and current ratio of 1.93, as well as the administrative measures taken to mitigate against further delays and costs related to land acquisition during implementation indicate that the efficiency of this project was substantial. Efficiency Rating Substantial Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  16.00  Not Applicable 0 ICR Estimate  21.00  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The PDO was highly relevant to the government and Bank strategy for Morocco. Efficacy of the two PDOs to provide access to safe and reliable drinking water supply for rural communities in targeted under-served areas in the Project Area - was rated as Substantial given the supporting evidence that the targets were almost fully achieved, and several activities exceeded the targets. Efficiency was rated as Substantial. The economic analysis showed that the project had a high EIRR. The project benefited from incorporating contracts underway from the BIRD II project and associated benefits, while also making improvements in administrative efficiency. Overall outcome was rated Satisfactory reflecting minor shortcomings in the Project's achievement of its objectives and in its efficiency. a. Outcome Rating Satisfactory 7. Risk to Development Outcome Based on the information in the ICR, the following issues posed risks to the development outcome:  Technical, Government ownership, Financing. Like the predecessor project BIRD II, the potential lack of post-construction operation and maintenance of standpipes in rural areas poses risks to the development outcome. The individual operator model using Gardien Gérants (GGs) appeared to be functioning at ICR for BIRD IV with a significant number of operators identified, trained, and agreements put in place with ONEE. Additionally, customers are paying the operators for water services, but the longer-term sustainability of the model will depend on ONEEs continued engagement with thousands of individual operators to ensure that services are being provided. Moreover, ONEE’s rural water services are subsidized by the central government and ONEE’s ability to finance operation and maintenance costs of its bulk infrastructure will depend on future government subsidies. (ICR, page 26)  Environmental. Morocco has faced a significant drought since 2021 impacting water services. To mitigate the impact on limited freshwater resources, the government has started augmentation efforts that include additional dams and desalination plants that will feed the rural water supply networks. Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) (ICR, page 26). If investments in augmentation to water production do not keep pace with demand then services could be curtailed. 8. Assessment of Bank Performance a. Quality-at-Entry The technical design of the Project incorporated lessons learned from previous projects such as BIRD II and BIRD I, and explicitly tried to address shortcomings of those projects, specifically, a more robust participatory and social engagement strategy at the pre-construction phase to select service levels, but also a strong emphasis on post construction with operation and maintenance models. The participatory and social engagement strategy also eventually proved critical to overcoming a structural issue around land acquisition and resettlement in the context where lands titles were often not available – a country wide problem (ICRR – BIRD II). The implementation arrangements were straightforward, and the Bank should be commended for building ONEE’s capacity and opting not to have a dedicated Project Implementation Unit. The Project had a strong poverty focus despite the shortcomings in monitoring. The main weakness at entry was the inadequate assessment of ONEE’s procurement capacity and their ability to deal with land acquisition and resettlement to ensure compliance with OP 4.12, which hampered the project in the beginning years. Given that BIRD II was implemented by ONEE and was closed with an unsatisfactory rating due to procurement and land acquisition these issues should have been salient to the Task Team at entry. Nevertheless, BIRD IV struggled with the same issues for the first few years of implementation. The ICR rated quality at entry Moderately Satisfactory. The Project had strategic relevance, a sound technical design, a focus on poverty, gender, and the social development aspects, clear implementation arrangements, and addressed technical shortcomings of the previous projects. However, this was counterbalanced with the lack of attention to procurement and land acquisition issues that hampered the project and caused the 18-month delay, which were significant shortcomings in the risk assessment given the Bank’s experience with ONEE under previous projects. Moreover, the lack of clarity on how the PDO indicators for quality, reliability, and poverty would be measured and monitored was also another significant shortcoming at Quality at Entry. As a result, the ICRR rates Quality at Entry Moderately Unsatisfactory. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision The quality of supervision was adequate and improved over time. A key ingredient to the success of the project as the relocation of the TTL to Rabat in 2019. This allowed for a more continuous dialogue with bi- monthly in-person meetings. The ISR ratings over the life project substantially improved after 2019, which can be attributed to the attention given by the TTL (ICR, page 26). The Project was complex with over 200 contracts requiring monitoring and supervision. The TTL worked closely with ONEE to simplify the Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) monitoring and increase the dialogue through more frequent meetings to diagnose and resolve implementation challenges (interview with TTL). In addition, the Bank was proactive in incorporating activities from BIRD II and removing activities from BIRD IV that were not advancing. This proactivity by the Bank helped the Project achieve its development objectives. The Bank team was proactive during implementation despite facing the challenges of the COVID-19 pandemic. The proactivity of the Bank team and its close supervision resulted in only minor shortcomings in the identification of opportunities and threats; therefore, the rating for Quality of Supervision is Satisfactory. The overall Bank performance rating is aligned with ICR rating of Moderately Satisfactory, because the Quality at Entry was rated Moderately Unsatisfactory, and the Quality of Supervision was Satisfactory. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project did not have an explicit theory of change in the PAD. The results framework ambitiously captured both water quality and reliability of services in rural areas in the PDO. The PAD made explicit the definitions of water quality and reliability, and the PDO indicators distinguished between these parameters; however, the methodology, frequency, and data sources of how these parameters would be monitored were not defined at entry nor during implementation. This was the major short-coming of measuring the PDOs. Given the ICR documented that ONEE has a sophisticated water quality surveillance program, it should have been possible to capture this in a more robust manner during implementation. Likewise, ONEEs rural water systems are interconnected via bulk infrastructure with reservoir levels and pumping stations managed via a SCADA; therefore, collecting periodic information on system pressure or reservoir levels throughout the year would have provided objective evidence on the reliability parameter coupled with household information from beneficiary surveys. The intermediate outcome indicator on post-construction operation and maintenance models was also instrumental to properly measure whether ONEE was fulfilling its responsibility to ensure sustainability of services. Adjustments were made at the 2016 restructuring to simplify some of the indicators that were duplicated between the PDO and intermediate outcomes. The restructuring also explicitly added an indicator on beneficiaries that would benefit from water system reinforcements; however, this indicator was vague on the improvements in the quality of service to be expected. The efforts to measure gender and the impact in poor areas were well intentioned; however, the gender indicator did not add value. The project intended to have a gender bias towards women beneficiaries, and set a target of 55 percent. However, there was no practical way to bias interventions towards women as infrastructure would be available equally to men and women. As such the indicator was meaningless. Likewise with the indicator on direct beneficiaries who are poor, the M&E system was not clear on how that would be monitored and tracked. Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) ONEE did not have a capability to track and measure whether poor people were benefiting (Interview with TTL). The other indicators on infrastructure such as number of standpipes, household connections, length of pipe installed, reservoir capacity were all easily tracked through construction contracts, and helped substantiate that the works were completed. b. M&E Implementation The ICR documented that ONEE prepared regular semi-annual reports on project progress including financial management, procurement, safeguards, and the results framework. The reporting and lack of progress against the PDO and downgrading of the project in 2019 resulted in increased frequency of reporting (ICR, page 23). This corresponded with the relocation of the TTL to Rabat. In 2019, the TTL simplified the reporting to make it a better management tool for ONEE and the Bank, and increased meetings to bi-monthly to track progress and troubleshoot issues (Interview with TTL). The 2018 mid- term review also included a beneficiary feedback survey that helped document the results at the household level and capture the status of services prior to works being completed. This survey coupled with the 2022 survey were able to capture the changes observed before and after the investments in terms of improvements in access to services, quantities of water consumed, time and distance traveled to access water, and satisfaction with services. This was not an official impact evaluation, but the survey at two different points in time was instrumental to document changes. The 2022 survey also captured the perspective of standpipe operators that provided insights on satisfaction with ONEE and the beneficiaries the operators served. The 2016 restructuring appropriately modified the targets to incorporate activities from the BIRD II. c. M&E Utilization ONEE and the Bank team utilized the M&E to diagnose and troubleshoot bottlenecks as they occurred in implementation. Tracking of contracts and land acquisition in a regular manner helped the Bank team guide the client on making course corrections (Interview with TTL). The M&E design to measure the PDOs for safety and reliability were significant shortcomings at design and it was not adequately rectified during implementation. Nevertheless, the beneficiary surveys provided critical information to substantiate the results but given the sophistication of ONEE’s water quality and SCADA system the Task Team could have made efforts to obtain objective times series data of the system to substantiate the impact in a more robust manner. Additional effort to measure whether poor people were being reached could have been done during implementation. The intermediate outcome indicators tracked outputs, but there were assumptions made to link those indicators with impacts. For example, water system reinforcements were assumed to equate to improved services. The positive aspects of the M&E system beyond the results framework were the TTL’s proactive monitoring of contracts and land acquisition issues along with progress of financial management. The TTL triangulated physical works progress with financial disbursements and linked it to the results being achieved with standpipes and household connections. The active monitoring of the project allowed for course corrections during implementation. (Interview with TTL) The implementation of the two beneficiary feedback surveys provided critical data points on the Project's impact. Given the significant Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) shortcomings in the design and implementation of measuring the PDOs, the overall rating for M&E quality is Modest. M&E Quality Rating Modest 10. Other Issues a. Safeguards The Project was designated as Category B. Environmental and social risks were deemed low and manageable, and consistent with the experience in previous projects of a similar nature. The safeguard policies triggered were Environmental Assessment (OP 4.01), and Involuntary Resettlement (OP 4.12). A subsequent restructuring triggered Safety of Dams (OP 4.37), but the safeguard category was maintained. (ICR, page 24) ONEE prepared a Resettlement Policy Framework (RPF) that was published before Project appraisal on January 28, 2014. (ICR, page 24). Some of the Land Acquisition Plans (LAPs) were pre-requisites for loan effectiveness since the BIRD II project suffered significant delays due to non-compliance with OP 4.12. The Project did not require any resettlement of people or production sites. The expropriation process followed the LAP provisions. ONEE withheld procurement and contracting activities until land acquisition processes were completed or fully under control to ensure full compliance with the OP 4.12 policy. ONEE had a fully functional Grievance Redress Mechanism (GRM) that was accessible to Project Affected People (PAPs). At Project closing, 95 percent of the 658 PAPs who could be compensated following a non- judiciary route had received compensation. The remaining 5 percent were in the process of being compensated. The ICR stated that for the majority of PAPs (2,206 in all), the finalization of the compensation process will take longer. This is due to land title disputes, non-agreement on compensation price, and non-responsiveness, among other reasons, which means that the compensation will need to follow a judicial process for which ONEE’s legal department is responsible for advancing. The full compensation of MAD 77.7 million for these PAPs is in an escrow account awaiting the final resolution of the cases. The Bank safeguards team will continue monitoring this issue until its resolution. (ICR, page 24) b. Fiduciary Compliance Financial Management: The project generally complied with the Bank's financial management requirements. During implementation, the FM arrangements allowed for the recording of all Project transactions and balances, supported the preparation of regular and reliable financial statements, and were subject to external audits which were found acceptable to the Bank after incorporation of Bank comments. Initially, there were shortcomings in the timeliness of the submission of project financial and audit reports by ONEE to the Bank, but this improved over time and the Project closed with a Satisfactory FM rating with no overdue audit or interim financial reports. (ICR, page 25). Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) Procurement: The Project followed national procurement regulations, which were deemed compliant with Bank norms. Initially the Project suffered from significant procurement delays due to the way ONEE was conducting the evaluation, but this was rectified after ONEE agreed to follow the Bank’s recommendations. (ICR, page 21). ONEE also improved its efficiency in contact administration by reinforcing coordination efforts within its various departments to reduce invoice processing times from 45 days to 12 days in 2017. At Project closing, ONEE had closed 189 of the 211 Project contracts (90 percent). The Project closed with a Satisfactory procurement rating. (ICR, page 25) c. Unintended impacts (Positive or Negative) None reported d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory The beneficiary feedback surveys coupled with the 2021 water quality report substantiated the results at PDO level; however, ONEE has robust systems for tracking Quality of M&E Substantial Modest water quality and reliability that were not utilized throughout the life of the Project. System data on water quality and reliability could have been provided over time. Quality of ICR --- Substantial 12. Lessons  A strong citizen engagement strategy is needed to increase the client’s ability to achieve the development outcomes. A lesson learned from the previous BIRD II project was that a strong citizen engagement strategy was needed in the pre-construction phase to assess the demands of the population related to service levels (standpipe or household Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) connections). This Project applied that lesson, which allowed for citizen feedback and selection of the type of service demanded by the population. Moreover, the engagement processes under this Project helped ONEE deal with land acquisition issues that were needed to install the bulk and distribution infrastructure. (ICR, page 26). Replicating processes or interventions from one country to another just because it worked does not mean it will work in another context. That said, rural water and sanitation projects worldwide suffer from not having an adequate citizen engagement strategy. This project offers a learning laboratory for other clients to learn about the key elements of why this approach worked well in the Moroccan context. (ICCR author)  The O&M model based on Gardien Gérants offers a potential model for piloting and replication in other projects. There are certain characteristics unique to the Moroccan context for why the Gardien Gérants (GGs) model may be working. The water systems serving the rural populations are not decentralized systems but centralized and managed by a large provider with technical capacity (ONEE). The agreement between ONEE and the local authority is that the local authority will select a GG that will be responsive to the population. This agreement between these two entities and the GG strengthens the compact between a central provider, a local government authority, the GG, and the population. Each standpipe is metered and if the GG does not pay their water bill to ONEE, ONEE will shut off the supply. (Interview with ICR author). This institutional commitment by ONEE to enforce its billing and collection increases the accountability of the GG to operate and maintain the standpipe and in turn charge for their services. Sustainability of rural water systems worldwide has vexed governments and financing institutions for decades. The GG model in Morocco offers an opportunity for others to learn what elements may be applicable to other contexts. (ICRR author)  The hiring of a technical assistance firm to support the project implementing agency could benefit supervision, operation and maintenance, and participatory approaches throughout implementation. ONEE has high technical capacity to design and build infrastructure but is less adept with managing social processes in rural areas. (Interview with TTL.) The decision to hire a technical assistance firm with qualified staff to help with pre- investment processes, construction and post-construction O&M processes was a key factor in ONEE achieving the development objectives. When client technical counterparts lack the skills on these social elements, this project can serve as a learning laboratory for Bank TTLs who are looking for potential ways to strengthen their client’s capacity. (ICRR author)  The Bank’s long-term engagement with the client in a specific sector could enhance prospects for implementation success. Previous projects such as BIRD II were not successful, but the Bank looked at what didn’t work and made explicit changes to the approaches and incorporated those into the design of this project. The GG model scaled up under this project was originally piloted under a predecessor project BIRD I. Although BIRD II was not successful, BIRD IV built on previous lessons from BIRD I and II and scaled them up. Without the learning from previous experiences in the Moroccan context, BIRD IV would not have benefited from the previous successes and failures in the Moroccan context. 13. Assessment Recommended? Yes Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review MA: Rural Water Supply (P145529) ASSESSMENT_TABLE Please Explain An assessment of the Project could yield important lessons on the sustainability for rural water provision and operation and maintenance models in similar contexts. 14. Comments on Quality of ICR The ICR was well-written, concise, and provided a good analysis; however, it did not provide disaggregated information about results at provincial level as was articulated in the PAD. Results were presented at a global level making interpretation of the project’s geographical impact more difficult. Subsequent information provided by the TTL demonstrated that the team did have disaggregated information, but it was not provided in the ICR. The ICR could have also expanded on the efforts to obtain information from ONEE related to the PDOs. The interview with the TTL and ICR author demonstrated that additional information on water quality was requested by the team to ONEE but was not provided. The ICR did not expand on the efforts to collect information reliability beyond the beneficiary feedback surveys. The constructed Theory of Change (ICR, page 8) in the ICR could have been stronger to reflect the activities of pre-construction social mobilization in selecting service levels and the post-construction efforts to install operation and maintenance models. The Theory of Change is focused on construction which are necessary but insufficient ingredients to sustainable services. Despite these weaknesses, the ICR provided sufficient evidence to substantiate the results and was also candid in areas of the M&E and assessments of quality at entry in assessing procurement capacity and dealing with the land acquisition issue. The ICR adequately captured how the Bank was proactive in addressing issues as they arose. While recognizing some of the shortcomings with presenting disaggregated data, and being more explicit about what information was requested but not provided by the client, the overall rating for the quality of the ICR is Substantial. a. Quality of ICR Rating Substantial Page 17 of 17