Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) Report Number: ICRR0024343 1. Project Data Project ID Project Name P149091 ZM-Lusaka Sanitation Project Country Practice Area(Lead) Zambia Water L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-56460 31-Dec-2020 65,270,024.11 Bank Approval Date Closing Date (Actual) 22-May-2015 30-Apr-2024 IBRD/IDA (USD) Grants (USD) Original Commitment 65,000,000.00 0.00 Revised Commitment 65,000,000.00 0.00 Actual 65,270,024.11 0.00 Prepared by Reviewed by ICR Review Coordinator Group Hassan Maher Amin Fernando Manibog Avjeet Singh IEGSD (Unit 4) Waly 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDO) of Zambia's Lusaka Sanitation Project (LSP) as articulated in the Financing Agreement (FA, page 6) was identical to the one stated in the Project Appraisal Document (PAD, paragraph 16) and aimed to: "Increase access to sanitation services in selected areas of Lusaka and strengthen the Lusaka Water and Sewerage Company’s (LWSC) capacity to manage sanitation services." Page 1 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) Parsing the PDO. The PDO will be parsed based on the following two Objectives: 1. To increase access to sanitation services in selected areas of Lusaka. 2. To strengthen the Lusaka Water and Sewerage Company’s (LWSC) capacity to manage sanitation services. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? Yes d. Components The PDO was supported by the following three components: 1. Sewerage Improvements (appraisal cost: US$40.50 million, actual cost: US$42.83 million). This component aimed to upgrade and expand sewerage systems in the Ngwerere and Manchinchi sewer sheds. Collection system upgrading and expansion would be based on top priority investments as identified in the Lusaka Sanitation Master Plan by LWSC. Year 1 investments would include sewer network expansion in the Emmasdale and Chaisa neighborhoods and Kafue Road and the upgrade of Ngwerere Western Interceptor. Year 2 to 5 investments would be network expansion in Chawama, Kuomboka and Garden, upgrade of Ngwerere downstream collector and upgrade and extension of Ngwerere sewage ponds. Resettlement costs and potential land purchase for extension of Ngwerere sewage ponds will be financed by the government. 2. On-site Sanitation (appraisal cost: US$14.00 million, actual cost: US$11.37 million). This component aimed to develop a comprehensive response to the on-site sanitation challenges facing Lusaka and support on-site sanitation services and systems in priority areas. It would focus on priority peri-urban areas that would not be sewered in the medium- to long-term due to technical and financial considerations. Investments would include a support fund for on-site sanitation facilities, support to the development of fecal sludge management (FSM) infrastructure and service providers, construction of a number of Decentralized Wastewater Treatment Systems (DEWATS) systems and sanitation and hygiene promotion. The areas to be covered under this component would be selected after the completion of the feasibility study. Land purchase for the FSM infrastructure would be financed by the government. 3. Institutional Strengthening (appraisal cost: US$9.00 million, actual cost: US$8.19 million). This component aimed to provide technical assistance (TA) to enhance LWSC’s capacity to implement the project, operate and maintain the facilities going forward, support the reforms needed to deliver sanitation services effectively and efficiently and to respond better to customer demand. This would include: (i) project management support to LWSC to implement the project, (ii) TA to strengthen the capacity of LWSC to Page 2 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) provide sanitation services and preparation funds for future investments and equipment for sewer maintenance and labs, and (iii) building capacity for monitoring program implementation and impacts in LWSC, Lusaka City Council, Ministry of Community Development, Mother and Child Health (MCDMCH) and the Ministry of Health (MoH). Revised Components. The first restructuring reflected a reduction in the scope of investment activities in all components. Specifically, the reduction included (a) dropping the sewer expansion works in Chawama and Kuomboka; sewer expansion works in Garden; sewer expansion works in Chipata; and upgrading and extending of Ngwerere sewage ponds; (b) downgrading from the original target of 10,000 household On- Site Sanitation (OSS) facilities to 3,500 facilities; and (c) dropping the updating of the Sanitation Master Plan, the preparation of additional sanitation investments, the performance contract audit, and LWSC credit rating. The component’s costs were adjusted accordingly. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost. The total project cost was estimated at US$68.50 million. The actual cost according to the ICR Data Sheet (page ii) was US$65.27 million. Financing. The project was financed through an IDA credit of US$65.00 million equivalent in Special Drawing Rights (SDR). The actual amount disbursed was US$65.27 million (ICR Data Sheet, page ii). Borrower Contribution. The borrower was expected to contribute US$3.50 million of counterpart funds. The actual amount of counterpart funding was not reported in the ICR. Dates. The project was approved on May 22, 2015 and became effective 13 months later on June 17, 2016. The Mid-Term Review was conducted on October 22, 2018 which was slightly later than the planned date as the PAD (paragraph 49) called for the MTR to be conducted no later than two years after effectiveness. The project closed on April 30, 2024 which was 40 months beyond the original closing date on December 31, 2020. The project was restructured twice, both were Level 2 restructuring as follows: 1. On August 19, 2020, when the amount disbursed was US$42.43 million, in order to: (a) reallocate funds between categories due to over/under expenditure in some categories and introduction of a new disbursement category for cash compensation for project-affected people (PAP); (b) extension of the project closing date by 22 months from December 31, 2020 to October 31, 2022 to complete all pending activities under the project; (c) changes to one legal covenant related to financial sustainability of LWSC; (d) update the implementation schedule, the disbursement estimates, and the economic analysis; and (e) reduce the total project financing (reducing Government of Zambia's contribution). Also, two new intermediate indicators were introduced under Component 3. 2. On June 20, 2022, when the amount disbursed was US$61.64 million, in order to: (a) adjust the number of sewer connections in Component 1 from 3,000 to 1,000 and revised the costs of each component; (b) extended the current project closing date by 18 months from October 31, 2022 to April 30, 2024 to complete all pending activities under the project; (c) reallocate funds between disbursement categories and project components; and (d) update disbursement estimates. Page 3 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) This Review will assess the outcome of the project based on a split rating methodology because the scope of the project was reduced consecutively with the first and second restructuring. While the PDO and the project costs remained the same, the project scope narrowed and it became less ambitious, hence the capacity of the project to deliver the PDO was reduced. Specifically, the PDO targets for two PDO outcome indicators pertaining to objective 1 were significantly reduced compared to their original targets. 3. Relevance of Objectives Rationale Context at Appraisal. In 2014, only 63% of Zambia's population had access to clean drinking water and 43% had adequate sanitation facilities. The deficiency in sanitation infrastructure adversely affected public health, increasing child malnutrition, illness, and premature deaths, and inflicted an economic cost, decreasing Zambia's GDP by 1.3%. In Lusaka, poor sanitation services resulted frequent cholera, typhoid, and dysentery outbreaks, and severe environmental pollution, especially in high-density, unplanned peri- urban areas where most residents relied on ineffective pit latrines. To tackle these challenges, this project aimed to enhance water supply and sanitation service quality, affordability, and sustainability through supporting water and sanitation infrastructure and improving the management of sanitation services. Previous Bank Experience. This project drew lessons from the Water and Sanitation Program (WSP) that supported the Kalingalinga condominial sewerage pilot project, which piloted among 45,000 people the use of condominial sewerage in Kalingalinga-a peri-urban area of Lusaka. However, the experience of the Kalingalinga project was not enough to inform LSP as the lack of information about the availability of the network connections, social barriers, pre-existing infrastructures, affordability and willingness to pay for service discouraged customers to request sewer connections. While the Bank has had significant global experience including in Africa in designing and overseeing the implementation of water and sanitation projects, the design of this project could have been better informed through further studies that reflected the conditions on the ground in Lusaka. Consistency with the World Bank Strategies. At appraisal, the PDO was in line with World Bank Country Partnership Strategy (CPS, FY2013–FY2016) for Zambia. The CPS focused on three pillars: (i) reducing poverty and vulnerability of the poor; (ii) improving competitiveness and infrastructure for growth and employment; and (iii) improving governance and economic management. The project was expected to contribute to the first pillar by supporting investments that would have positive effects on the health of poor residents in the beneficiary areas, and to the second pillar by enhancing economic development of the prioritized economic sectors through provision of improved infrastructure. The PDO was also in line with the World Bank’s twin goals of eliminating absolute poverty and promoting shared prosperity. At completion, the PDO continued to in line with the World Bank Country Partnership Framework (CPF, FY2019–FY2023) for Zambia. The CPF reflected three focus areas: (i) more even territorial development through opportunity and jobs for the rural poor; (ii) public services and social protection for jobs participation; and (iii) institutions for resilience. The project contributed to the CPF second focus areas through increasing access to sanitation services, which would help reduce child mortality and stunting, limit infectious diseases and though organizing and promoting FSM operators to allow more Page 4 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) opportunities for jobs and private sector development. In addition, the project contributed to CPF Objective 1.3: Access to and quality resilient infrastructures services through improving the capacity of relevant institutions. Consistency with the Government Strategies/Priorities. At appraisal, the PDO was in line with the Government’s strategy for growth and development as outlined in the Sixth National Development Plan (SNDP, 2011-2015). The Plan prioritized investments in sanitation in order to improve economic growth and quality of life. At completion, the PDO continued to be in line with Government's eighth National Development Plan (NDP, 2022–2026), specifically, the plan's strategic development area #2: Human and Social Development, which included a development outcome to improve water supply and sanitation. The PDO was also in line with Zambia's Water Investment Program (ZIP, 2022–2030), which aimed through its investment focus area number three, to ensure universal access to clean and safe water supply and sanitation services for all. This included improvement of sanitation services and hygiene promotion in urban areas, and promoting efficiency in the provision of water supply and sanitation services. Summary of Relevance of Objectives Assessment. While the PDO statement was clear and focused, the targets were pitched at a higher level of ambition in relation to the realities on the ground. At completion, the PDO continued to be in line with the Bank's CPF for Zambia and with Government strategies as noted above. Improving water and sanitation services was expected to continue to be a priority for the Zambian Government due to its economic and health impacts on the population. Therefore, Relevance of Objectives is rated High. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To increase access to sanitation services in selected areas of Lusaka. Rationale Theory of Change (ToC). To achieve the stated objective, the project supported the extension of the sewer network, construction of pumping stations, support of household sewer connections, promotion of sanitation marketing, construction of Fecal Sludge Treatment Plants (FSTPs), construction of On-Site Sanitation (OSS) facilities, and construction of Decentralized Wastewater Treatment Systems (DEWATS). Page 5 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) The expected outputs/intermediate outcomes included: sewer network constructed and/or rehabilitated, households connected to sewer service, waste water treatment plant built, good sanitation and hygiene practices adopted, improved OSS facilities constructed, and fecal sludge safely managed. All of the above was expected to increase access to sanitation services (PDO). Anticipated long-term impacts included inclusive access to quality social service, improved health, reduction of environmental pollution, and reduction of poverty and vulnerability of the poor. The achievement of the stated objective was underpinned by the following three critical assumptions: (i) the Government of Zambia supports the reforms of the water and sanitation sector, (ii) sewer connections and OSS facilities reach the most vulnerable households, and (iii) the macro-economic environment remains stable. This Review also finds that the availability of adequate implementation capacity on the ground is also a critical assumption. The ToC reflected relevant activities that were directly connected to the stated outputs/intermediate outcomes and to the PDO in plausible causal chains. However, the assumption that vulnerable households would connect to the service once it is available was not realistic because it ignored the costs of connections which might burden vulnerable households. Outputs/Intermediate Results  2,226 new household sewer connections were connected under the project which was below the original target of 4,100 (target not achieved).  117 km of sewer lines were rehabilitated or newly constructed under the project exceeding the original target of 82 km (target exceeded).  20 pumping stations were installed equipped with backup power from generators (no target provided). After project completion, 10 out of the 20 pumping stations were connected to the electricity grid (ICR, paragraph 31).  3,500 improved onsite sanitation facilities were constructed under the project which was below the original target of 10,000 (target not achieved).  One fecal sludge treatment facility was constructed and used which was below the original target of 2 (target not achieved).  300 tons per year of Biological Oxygen Demand (BOD5) were removed by the fecal sludge treatment facilities which was below the original target of 940 tons per year (target not achieved). Outcomes  By project completion 81,128 people were provided with access to improved sanitation facilities which was significantly below the original target of 213,000 (PDO outcome indicator #1).  263,898 people had access to improved fecal sludge management services* which was below the original target of 360,000 (PDO outcome indicator #2).  345,026 direct project beneficiaries which was below the original target of 393,000 (PDO Outcome indicator # 9). *According to the ICR (paragraph 32) "Improved FSM services is measured as fecal sludge collection and transportation services that meet an average annual score of 70 percent on the FSM performance scorecard, which is a performance management tool used on the performance-based contracts signed between LWSC Page 6 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) and FSM. The tool consists of three categories of indicators: (a) occupational health and safety, (b) customer service management, and (c) indicators on public safety during fecal sludge collection and transportation." Summary of Efficacy Assessment. The project did not achieve any of its original PDO outcome targets and all original targets for the intermediate results indicators were underachieved, except one. Therefore, the efficacy with which this objective was achieved is rated Modest. Rating Modest OBJECTIVE 1 REVISION 1 Revised Objective To increase access to sanitation services in selected areas of Lusaka. Revised Rationale Theory of Change (ToC). While the PDO did not change, the project scope was reduced with the first restructuring on August 19, 2020. Specifically, the scope of the sewerage component was reduced from the 8 originally planned sub-projects to 4 sub-projects. Also, support to onsite sanitation was scaled down from the original target of 10,000 household OSS facilities to 3,500 facilities. Finally, the project was no longer supporting wastewater treatment plants. All other elements and assumptions pertaining to the original ToC still applied to the revised ToC. Outputs/Intermediate Results  2,226 new household sewer connections were connected under the project which was below the revised target of 3,000 (target not achieved).  117 km of sewer lines were rehabilitated or newly constructed under the project exceeding the revised target of 108 km (target exceeded).  20 pumping stations were installed equipped with backup power from generators (no target provided). After project completion, 10 out of the 20 pumping stations were connected to the electricity grid (ICR, paragraph 31).  3,500 improved onsite sanitation facilities were constructed under the project fully achieving the revised target of 3,500 (target achieved).  One fecal sludge treatment facility was constructed and used which fully achieving the revised target (target achieved).  300 tons per year of BOD5 were removed by the fecal sludge treatment facilities supported by the project which was below the original target of 940 tons per year (target not achieved). Outcomes  By project completion 81,128 people were provided with access to improved sanitation facilities which substantially achieved the revised target of 90,000 (PDO Outcome indicator #1). According to the Restructuring Paper (Table 1), "access to an improved sanitation facility means a household has Page 7 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) access to a sanitation facility (either individual or shared) where human excreta is safely disposed in situ and/or transported and treated off-site."  263,898 people had access to improved fecal sludge management services* which exceeded the revised target of 230,000 (PDO Outcome indicator #2).  345,026 direct project beneficiaries which substantially achieved the original target of 393,000 (PDO Outcome indicator # 9). *According to the ICR (paragraph 32) "Improved FSM services is measured as fecal sludge collection and transportation services that meet an average annual score of 70 percent on the FSM performance scorecard, which is a performance management tool used on the performance-based contracts signed between LWSC and FSM. The tool consists of three categories of indicators: (a) occupational health and safety, (b) customer service management, and (c) indicators on public safety during fecal sludge collection and transportation." Summary of Efficacy Assessment. While the project did not achieve its original PDO outcome targets, it exceeded one revised PDO outcome target, substantially achieved the other two revised PDO outcome targets. The project also achieved most of its revised targets for the intermediate results indicators. The PDO outcome targets were revised as part of the August 19, 2020 restructuring, which was about three years and seven months prior to closing. Therefore, the efficacy of the project in achieving this objective is rated Substantial. Revised Rating Substantial OBJECTIVE 1 REVISION 2 Revised Objective To increase access to sanitation services in selected areas of Lusaka. Revised Rationale Theory of Change (ToC). While the PDO was not revised, the project scope was reduced further. The second restructuring on June 20, 2022 aimed to account for the unforeseen project implementation delays caused by the COVID-19 pandemic and protracted impacts, rocky ground conditions in Lusaka, and financing shortfall for household connections. The restructuring included the reduction of several project results indicators. The changes included: reducing the end target values for two PDO indicators and one intermediate indicator. This reduction in the final targets of indicators reflected what could be realistically be achieved within the project timeframe with the remaining project funds (Restructuring Paper, paragraph 14). All other elements and assumptions pertaining to the original ToC still applied to the revised ToC. Outputs/Intermediate Results  2,226 new household sewer connections were connected under the project which exceeded the revised target of 1,000 (target achieved).  117 km of sewer lines were rehabilitated or newly constructed under the project exceeding the revised target of 108 km (target exceeded). Page 8 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091)  20 pumping stations were installed equipped with backup power from generators (no target provided). After project completion, 10 out of the 20 pumping stations were connected to the electricity grid (ICR, paragraph 31).  3,500 improved onsite sanitation facilities were constructed under the project fully achieving the revised target of 3,500 (target achieved).  One fecal sludge treatment facility was constructed and used which fully achieving the revised target of 1 (target achieved).  300 tons per year of BOD5 were removed by the fecal sludge treatment facilities supported by the project which was below the original target of 940 tons per year (target not achieved). Outcomes  By project completion 81,128 people were provided with access to improved sanitation facilities which exceeded the revised target of 75,000 (PDO Outcome indicator #1). According to the Restructuring Paper (Table 1), "access to an improved sanitation facility means a household has access to a sanitation facility (either individual or shared) where human excreta is safely disposed in situ and/or transported and treated off-site."  263,898 people had access to improved fecal sludge management services* which exceeded the revised target of 230,000 (PDO Outcome indicator #2).  345,026 direct project beneficiaries which exceeded the revised target of 305,000 (PDO Outcome indicator # 9). *According to the ICR (paragraph 32) "Improved FSM services is measured as fecal sludge collection and transportation services that meet an average annual score of 70 percent on the FSM performance scorecard, which is a performance management tool used on the performance-based contracts signed between LWSC and FSM. The tool consists of three categories of indicators: (a) occupational health and safety, (b) customer service management, and (c) indicators on public safety during fecal sludge collection and transportation." Summary of Efficacy Assessment. The project exceeded all of its revised PDO outcome targets as noted above. The project also achieved most of its revised targets for the intermediate results indicators. Therefore, the efficacy of the project in achieving this objective is rated Substantial. It is worth noting that the revision of targets in the second restructuring was notably less than the drastic revision of targets as part of the first restructuring. Revised Rating Substantial OBJECTIVE 2 Objective To strengthen the Lusaka Water and Sewerage Company’s (LWSC) capacity to manage sanitation services. Rationale Page 9 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) Theory of Change (ToC). To achieve the stated objective, the project supported the FSM business model through training, developing an M&E system, supporting the carrying out of audit and surveys with the aim of enhancing financial viability and maintain the system functionality. The expected outputs included: FSM unit established, sewerage blockage incidents addressed in time, sewerage tariff increased, and integrated MIS functional. All this would contribute to strengthening the capacity of LWSC to manage sanitation services. Anticipated long-term impacts included reduction of environmental pollution. The achievement of the stated objective was underpinned by the following three critical assumptions: (i) the Government of Zambia supports the reforms of the water and sanitation sector, (ii) LWSC adopts an inclusive sanitation approach, and (iii) the macro-economic environment remains stable. The ToC reflected relevant activities that were directly connected to the stated outputs and to the PDO in a plausible causal chain. The stated critical assumptions were logical and realistic. Outputs/Intermediate Results  The O&M manual for FSM was developed and in use (target achieved).  Revenue collection efficiency reached 92% exceeding the target of 85%. Revenue collection efficiency was the proportion of billed amounts (for both water supply and sewerage services) that were collected (including arrears and advance payments made in the period) (target exceeded).  Sewerage blockages deceased from 57 per 100 km at baseline to 46 sewer blockages per 100 km at completion exceeding the original target of 52 (target exceeded).  An integrated MIS was established and in use (target achieved).  FSM unit was established (target achieved).  The annual average score on the FSM performance scorecard reached 90% exceeding the revised target of 70% (target achieved).  On-site sanitation Strategy approved by LWSC Board. Outcomes  By project completion 100% of sewer blockage complaints were addressed within 48 hours compared to a baseline of 28% and exceeding the original target of 96% (PDO Outcome indicator).  Although the project did not measure the planning/strategic capacity of LWSC to manage sanitation services, an OSS strategy was approved by the LWSC board on June 8, 2017, and the new LWSC Strategic Plan 2023–2027 that fully includes sanitation services and recognizes the growth of the FSM business model (ICR, paragraph 44). These strategic documents were attributed to the LSP and consequently concomitantly to the AfDB, the Bill and Melinda Gates Foundation (BMGF), whose financings were active on the ground for capacity improvement (ICR, paragraph 44). Summary of Efficacy Assessment. The project exceeded its PDO outcome target as noted above and all targets for the intermediate results indicators were achieved. Overall, and based on the evidence reported in the ICR, it is plausible to assume that the project contributed to strengthening the Lusaka Water and Page 10 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) Sewerage Company’s (LWSC) capacity to manage sanitation services. Therefore, the project’s efficacy in achieving this objective is rated Substantial. Rating Substantial OBJECTIVE 2 REVISION 1 Revised Objective To strengthen the Lusaka Water and Sewerage Company’s (LWSC) capacity to manage sanitation services. Revised Rationale The PDO was not revised and PDO outcome targets remained the same. Revised Rating Substantial OBJECTIVE 2 REVISION 2 Revised Objective To strengthen the Lusaka Water and Sewerage Company’s (LWSC) capacity to manage sanitation services. Revised Rationale The PDO was not revised and PDO outcome targets remained the same. Revised Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Overall Efficacy against the original targets is rated Substantial with moderate shortcomings. The project aimed to increase access to sanitation services in selected areas of Lusaka (objective 1).Out of three PDO outcome indicators pertaining to this objective, one was achieved at 88% (direct project beneficiaries), another at 73% (access to improved fecal sludge management services) and one was achieved at 40% (access to improved sanitation facilities). On the positive side, the project also aimed to strengthen the Lusaka Water and Sewerage Company’s (LWSC) capacity to manage sanitation services (objective 2). The project exceeded its PDO outcome target and all targets for the intermediate results indicators were Page 11 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) achieved. Overall, and on balance, the project efficacy in achieving the overall objectives is rated Substantial with moderate shortcomings based on the achievements mentioned above. Overall Efficacy Rating Substantial OBJR1_TBL OVERALL EFFICACY REVISION 1 Overall Efficacy Revision 1 Rationale Overall efficacy (first restructuring) is rated Substantial. The project aimed to increase access to sanitation services in selected areas of Lusaka (objective 1). The project exceeded one PDO outcome target, and substantially achieved the other two Outcome targets. The project also achieved most of its revised targets for the intermediate results indicators. The project also aimed to strengthen the Lusaka Water and Sewerage Company’s (LWSC) capacity to manage sanitation services (objective 2). The project exceeded its PDO outcome target and all targets for the intermediate results indicators were achieved. Overall Efficacy Revision 1 Rating Substantial OBJR2_TBL OVERALL EFFICACY REVISION 2 Overall Efficacy Revision 2 Rationale Overall Efficacy (second restructuring) is rated Substantial. The project aimed to increase access to sanitation services in selected areas of Lusaka (objective 1). The project exceeded all of its revised PDO outcome targets and achieved most of its revised targets for the its intermediate results indicators pertaining to this objective. The project also aimed to strengthen the Lusaka Water and Sewerage Company’s (LWSC) capacity to manage sanitation services (objective 2). The project exceeded its PDO outcome target and all targets for the intermediate results indicators were achieved. Overall Efficacy Revision 2 Rating Substantial 5. Efficiency Economic and Financial Analysis (EFA) Page 12 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) Ex-ante  The project economic analysis estimated the economic rate of return (ERR) at 11% over a period of 20 years at a discount rate of 10%.  The economic analysis aimed to examine the economic viability of components 1 and 2 of the project, by combining quantifiable direct benefits (land value increase) and indirect health benefits expected from sanitation interventions. However, benefits from improvements in environmental conditions and water quality were not included due to the lack of data.  The financial analysis of component 1 sought to determine the required sewerage tariff increase that would ensure LWSC collects sufficient incremental cash to cover all operation and maintenance expenses of the sewerage operations related expenses. The analysis showed that component 1 would be financially viable if the sewerage tariff was increased by 0.16 US$/m³ (i.e., from 0.81 US$/m3 to 0.97 US$/m3 on the average tariff) gradually over the project period. For component 2 the financial analysis showed that the initial operation would need to be subsidized, to enable a gradual increase in the charge of emptying a sewage on-site facility from US$50 per unit beginning in 2018 to US$75 per unit in 2021, which would then cover costs. Ex-post  The ex-post economic analysis estimated the overall economic internal rate of return (EIRR) of the project at 9.2% with an economic net present value (ENPV) of US$4.4 million. To ensure consistency and comparability, the ex-post EFA applied the same cost-benefit analysis methodology employed at appraisal to assess components 1 and 2. The analysis assumed that the benefits from the OSS investments would unfold over a 20-year period.  The ICR (paragraph 48) attributed the lower ex-post EIRR compared to the ex-ante results with an EIRR of 11% primarily due to "delays in implementation, restructuring of the project and significant reduction in the number of targeted beneficiaries, and several external economic factors, including the COVID-19 pandemic." Collectively, these factors negatively impacted economic activities, increased poverty, and led to substantial inflationary pressures.  Sensitivity Analysis. A sensitivity analysis indicated that the economic returns were sensitive to changes in costs and benefits, with the EIRR remaining above the discount rate of 10% only under mild scenarios.  Implementation Efficiency. The project closed on April 30, 2024 which was 40 months beyond the original closing date on December 31, 2020. This negatively affected implementation efficiency as benefit streams from the project activities were delayed. The actual implementation time was 8 years and 11 months, with 13 months spent between the Board approval and project effectiveness and 20 months between the MTR and the corresponding restructuring approval. The project also experienced cost overruns that led to reducing the scope of activities. These stemmed from an underestimation of works at appraisal. Summary of Efficiency Assessment. The ex-post EIRR at 9.2% was lower than the ex-ante estimate at 11%. The project also experienced extensive delays as not above which negatively impacted efficiency. Further, there were substantial cost overruns that resulted in reducing the scope of the project. Therefore, Efficiency is rated Modest. Page 13 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal  11.00  Not Applicable 0 ICR Estimate  9.20  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Pre-Restructuring Relevance of Objectives was rated Substantial. Overall Efficacy was rated Substantial with moderate shortcomings. Efficiency was rated Modest because the ex-post EIRR at 9.2% was lower than the ex-ante estimated at 11%, and the project experienced extensive delays, and substantial cost overruns that resulted in reducing the scope of the project. Based on the assigned ratings to the three outcome criteria, Outcome is rated Moderately Satisfactory. Post First Restructuring (August 19, 2020) Relevance of Objectives was rated Substantial. Overall efficacy was rated Substantial. Efficiency was rated Modest. Based on the assigned ratings to the three outcome criteria, Outcome is rated Moderately Satisfactory. Post Second Restructuring (June 20, 2022) Relevance of Objectives was rated Substantial. Overall efficacy was rated Substantial. Efficiency was rated Modest. Based on the assigned ratings to the three outcome criteria, Outcome is rated Moderately Satisfactory. Since the overall rating remained the same, a split rating methodology results in a final outcome rating of Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory Page 14 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) 7. Risk to Development Outcome The following risks could potentially impact the Development Outcome: 1. Financial risk. The lack of tariff adjustments since 2019 is leading to a cash shortage, causing routine maintenance operations to be deferred within many commercial water utilities. Without tariff adjustments or a viable alternative to be decided within the next 12 months, the project's results and outcomes could be seriously compromised. 2. Technical risk. Achieving a fully functional and effective fecal sludge management system requires the diligent construction of the waste water treatment plant downstream to expand the network and increase the number of connections. Also, implementing specific constructive measures and monitoring systems is needed to reduce vandalism and theft of the sewer network manhole covers This is critical because manhole cover thefts create numerous potential entry points for solid waste that could result in sewer blockages and compromise the network functionality. 3. Stakeholders risk. To sustain the development outcomes, it is important to continue awareness and training campaigns. This will help achieving a real behavior change through promoting good practices in the use of on-site sanitation facilities or network-connected latrines to maximize the benefits of the new sanitation services offered. 4. Institutional risk. It is critical to continue improving the operational performance of the Lusaka Water and Sewerage Company (LWSC) to ensure reliable access to services. 8. Assessment of Bank Performance a. Quality-at-Entry  Strategic Relevance and Approach. The project was strategically relevant and in line with the Government priorities. The PDO was also in line with the Bank strategies (see Section 3 for details). The project was expected to support the Government’s development plan by improving sanitation services of the poor in order to achieve improvements in public health and primarily benefit poor households in Lusaka.  Technical, Financial and Economic Aspects. The project design featured a clear PDO that was supported by relevant activities. The project design aimed to address multiple challenges including poor sewage facilities, contaminated groundwater, improper management of fecal waste, and poor connectivity to sanitation services in the project area. The project design featured infrastructure support combined with capacity strengthening to improve service delivery. However, the cost estimates for the sewerage component were higher than anticipated due to higher-than- expected technical assistance requirements for design and supervision, as well as challenging rocky ground conditions in Lusaka. The cost overruns resulted in the reduction of the scope of activities due to underestimation of the cost of works at appraisal. The PAD included a detailed economic and financial analysis that provided adequate justification for the project investments.  Poverty, Gender, and Social Development Aspects. The project was expected to benefit poor households since 85% of the direct beneficiaries of improved sanitation facilities were in low- Page 15 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) income and informal areas of the city of Lusaka. Women also represented 50% of the project beneficiaries. Women would benefit from sanitation connections or improved FSM services to household as this would decrease the women time burden needed for hygiene and sanitation.  Environmental and Fiduciary Aspects. The project design reflected adequate environmental and fiduciary aspects.  Implementation Arrangements. The project was implemented by LWSC. All funds flowed through LWSC, and implementation was coordinated by a Project Implementation Unit (PIU). However, LWSC was focused more on sewerage than on-site sanitation. Also, establishing the PIU with adequate staffing experienced delays. As a result, the project effectiveness took 13 months after the Board approval. Further, LWSC was not fully informed about the availability of the network connection, social barriers, pre-existing infrastructures, affordability and willingness of beneficiaries to pay for service. These factors combined negatively impacted customer demand for sewer connections.  Risk Assessment. Eight risk areas were identified at appraisal, and the overall risk was rated as Substantial. While the risk related to the project technical design was rated substantial, the implementation experience pointed that this risk was underestimated. The project featured relatively new technologies that were untested particularly at the scale included under this project.  M&E Arrangements. The PIU within LWSC oversaw M&E activities. The PIU included a dedicated monitoring and evaluation specialist to ensure all project monitoring and evaluation was implemented as expected. The project’s results framework was developed to track the progress in meeting the project’s objectives. However, the RF was complicated with nine PDO outcome indicators and ambitious targets, and some PDO indicators were out of the project control. (see section 9a for details). Summary of Quality at Entry (QAE) Assessment. The project was strategically relevant and its design included a clear PDO that was supported by relevant activities. However, the underestimation of costs for the sewerage component negatively impacted implementation and resulted in the reduction of the project scope. Environmental and fiduciary aspects were adequate. However, implementation arrangements were not adequate as the project faced challenges at the PIU level, which contributed to a significant effectiveness delay. Gender aspects were adequately covered by the project design. While risk identification was thorough and reflected relevant mitigation measures, the risk related to the project technical design was underestimated. Overall, Quality at Entry is rated Moderately Unsatisfactory due to significant shortcomings pertaining to the project design, over-ambitious targets, implementation arrangements, and to a lesser extent to risk assessment. The QAE shortcomings negatively affected implementation and directly led to Modest performance prior to restructurings. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision Page 16 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091)  The Bank conducted 22 implementation support missions over the course of the project implementation including virtual missions during the COVID-19 restrictions. On average, two implementation support missions were conducted per year.  Although the project was led by five different Task Team Leaders (TTL), continuity and stability were maintained because transitions primarily occurred between Co-TTLs and TTLs. Also, over the entire implementation period at least one TTL or co-TTL were based in neighboring countries which ensured a smooth transition within the World Bank team.  While the Bank team maintained constant dialogue with other Lusaka Sanitation Program partners within the Steering Committee, there was limited coordination with international development partners along the entire sanitation chain. This was evident with the European Investment Bank (EIB) and Credit Institute for Reconstruction (KfW) financing components associated to the downstream waste water treatment plant that were just starting and were expected be completed by 2027.  The Task Team punctually conducted the MTR and prepared the restructurings to align projected results. However, the slow decision-making of the counterpart sometimes antagonized the World Bank's proactivity. Summary of Quality of Supervision Assessment. The Bank supervision experienced challenges during the project implementation that stemmed from design weaknesses. Also, the borrower exhibited slow- decision making combined with slow release of counterpart funds, which contributed to delays. Despite the afore-mentioned challenges, the Bank restructured the project twice to align the project and adapt activities to realities on the ground. However, the project could have benefited from a restructuring at an earlier stage of implementation. The project also could have benefited from further coordination between the Bank and the other international development partners providing support along the sanitation chain. Overall, the Bank managed to guide the project towards completion despite various challenges. Therefore, Quality of Supervision is rated Moderately Satisfactory. The overall rating of Bank Performance is Moderately Satisfactory based on the assigned rating to QAE and Quality of Supervision. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design  The PAD did not include a ToC since it was not yet required by the Bank at the time of appraisal. Nonetheless, the ICR (Annex 8) included a ToC that reflected the relationship between the activities, outputs and outcomes in plausible causal chains. This Review reconstructed a ToC for each objective using the detailed project description in the PAD and the ToC reported in the ICR. Page 17 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091)  The PDO was to be assessed through measuring nine PDO outcome indicators. However, some PDO indicators were out of the project control, for example increasing domestic sewer tariff, while others were pitched at the intermediate level outcomes, for example, Tons of BOD5 removed. Finally, some PDO outcome targets were too ambitious.  The Results Framework included twelve intermediate outcome indicators to track the project activities and assess the progress towards achieving the PDO. Most intermediate results indicators were measurable and connected to the project activities.  Overall, M&E design was complicated with nine PDO outcome indicators, overly ambitious targets, and indicators that were outside the control of the project. The project could have benefited from a simpler RF design given that it was the first innovative and stand-alone sanitation project in Zambia. b. M&E Implementation  The PIU oversaw M&E implementation. According to the ICR (paragraph 72) "the PIU and LWSC effectively monitored the impact and performance of the project in collaboration with the National Water Supply and Sanitation Council."  The project developed an integrated M&E-MIS for LWSC, LCC, and MoH. This system was operational, and service providers were using it to safely manage sewer services and enhance data management. The project also supported sanitation monitoring through the provision of vehicles, motorbikes, testing kits, and tablets. Monthly, quarterly, and annual reports were regularly reviewed and validated by the established Steering Committee.  As part of the two restructurings, the M&E framework was revised twice to account for reduction in the scope of activities and match the expected results. Also, some indicators were refined and updated to better track the project activities. c. M&E Utilization  The project indicators were used throughout the implementation to assess the project progress, identify the need for adjustments, and make informed decisions. For example, cost overruns were reflected in reducing targets and dropping some activities during the first restructuring.  In addition, M&E data informed a high-level dialogue with the client and within the World Bank, resulting in developing scenarios for scaling up the project to enhance public health impact, in terms cholera risk reduction should additional financing become available (ICR, paragraph 73). Summary of M&E Quality Assessment. While M&E design was complicated with nine PDO outcome indicators, most design weaknesses were identified and addressed during implementation. M& E implementation was adequate and generated enough data to track most project activities and assess the PDO. Utilization was also evident as the M&E data informed project decisions and were used as part of the ICR. Overall, except for some over-ambitious PDO outcome indicators, the M&E system was adequate for assessing the project achievements. Therefore, the Quality of M&E is rated Substantial. Page 18 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) M&E Quality Rating Substantial 10. Other Issues a. Safeguards  Environmental Category and Safeguards. This project was classified as environmental category B - Partial Assessment based on the minimal environmental impacts associated with the small-scale sanitation civil works which would entail rehabilitating and upgrading sanitation infrastructure. Two environmental and one social safeguards were triggered: Environmental Assessment (OP 4.01), Projects on International Waterways (OP/BP 7.50), and Involuntary Resettlement (OP 4.12). A Resettlement Policy Framework (RPF), a Resettlement Action Plan (RAP) and an Environmental and Social Management Framework (ESMF) were prepared in compliance the World Bank’s safeguards policy.  Compliance with Environmental and Social Safeguards. According to the ICR (paragraph 75) "the project was in compliance with World Bank safeguards policies." All sub-projects were subject to the Environmental and Social Impact Assessment and Environmental and Social Standards and Resettlement Action Plans, which were approved by the World Bank, and implemented before the start of physical works, in accordance with the ESMF and RPF. Two fatal accidents occurred during implementation. Compensation payments were made, and mitigations and corrective actions were implemented. The ICR (paragraph 75) reported that both cases were closed. Also, 551 Project- Affected People (PAP) were compensated where 521 cases were related to informal traders and business disturbances and 15 were for land acquisitions. The GRM remains functional with a cumulative number of 306 cases and only one outstanding case waiting for judgment in the court. b. Fiduciary Compliance  Financial Management (FM). FM interim financial reports were submitted consistently by the PIU. However, there were a few delays and errors in their submissions. While the financial analyses by the PIU initially showed savings, the actual project savings were smaller than estimated. This resulted in a cost overrun of US$1.1 million above the IDA financing. The ICR (paragraph 78) noted that two contracts had remaining payments which were expected to be included in the 2025 national budget. The FM performance was rated Moderately Satisfactory in the final ISR.  Procurement. The project signed and completed 55 contracts, and one contract was terminated. During the first years, there were procurement-related delays that stemmed from inadequate procurement staffing. This contributed to the inaccurate initial cost estimations and resulted in delays in the processing of procurement activities. Procurement performance improved with recruitment of a new Procurement Specialist. The project procurement performance was rated Satisfactory in the final ISR. Page 19 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) c. Unintended impacts (Positive or Negative) The ICR (paragraphs 59 & 60) noted two notable positive impacts: First, 850,000 people benefited from improved hygiene and sanitation awareness. Furthermore, every LWSC customer with a water connection gained from better groundwater protection and avoided additional water treatment expenses that LWSC would have had to charge water supply customers. Second, during the cholera outbreak in 2023–2024, the Government supported a cholera emergency emptying initiative that began in January 2024. This initiative leveraged the existing LSP emptier organization. Under this emergency effort, 14,261 m3 of sludge was removed from 3,803 toilets in the cholera hot spot areas of Lusaka, including Kanyama. This resulted in 64,454 additional beneficiaries not included in the project results achieved. d. Other None. 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory QAE had significant Bank Performance Satisfactory Moderately Satisfactory shortcomings and QoS had moderate shortcomings Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR includes three lessons which are emphasized below with some adaptation of language: 1. Implementing a successful citywide inclusive sanitation approach requires timely and targeted sanitation marketing combined with improved customer services. During the first four years, Lusaka Sanitation Project relied on enforcement for connecting households to sewers, while the connection costs, as well as those of the OSS facilities, were not affordable for the peri-urban targeted population. The project needed to change its approach with door-to-door communication instead of community-level engagement only. Also, adding subsidies for both types of infrastructures and improving FSM services through a transitional performance-based subsidy associated with the coaching of private operators, a new call center, and an FSM unit among other, dedicated to sanitation customers. Page 20 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) 2. In countries with power supply challenges, it is crucial for projects supporting sanitation systems that require power for their operation to secure affordable backup or alternative power sources. Despite have been fully paid from the beginning, the national electricity company failed for over a year to connect 10 out of the 20 pumping stations installed as part of this project. As a result, LWSC has been forced to operate these stations at a minimum capacity and using backup generators, incurring additional fuel costs. Starting mid-March 2024, ZESCO implemented gradual load shedding due to drought, resulting in more than 20 hours of announced power interruptions per day. Despite the recent connection of the pumping station sites, LWSC is still required to operate the stations sparingly using generators. 3. The sustainability of water and sanitation utilities require regular tariff adjustments to maintain the sanitation chain functionally. Despite a good revenue collection efficiency (92% for 2023), LWSC's capacity to cover essential O&M costs could be negatively affected due to low tariffs, which have not yet been adjusted since 2019. Operational improvements provide leverage for the utilities to make a case with the government and the customers of water services that tariffs need to be increased. Increasing tariff is a critical step for the sector's financial equilibrium and the sustainability of services. The longer the decision is deferred, the wider the financial gap, the higher the probability of the water and sanitation system collapsing, and the higher the next tariff adjustments would need to be. 13. Assessment Recommended? No 14. Comments on Quality of ICR  Quality of Evidence. The ICR reflected an evidence base that was adequate to assess the project outcomes. The ICR effectively triangulated evidence to confirm some project data, for example, using the African Development Bank (AfDB) Implementation Progress Reports to confirm that sewerage blockage complaints were addressed satisfactorily.  Results Orientation. The ICR used the available data to the extent possible to discuss the achievement of the two elements of the PDO.  Quality of Analysis. The ICR assessment of outcomes was adequate and included a good balance between achievement of indicators and what the project achieved at the country and sector levels.  Internal Consistency. Various parts of the ICR were internally consistent and logically linked and integrated.  Lessons. Lessons reflected the project experience and were based on evidence and analysis.  Consistency with guidelines. The ICR used the standard structure defined in the Guidelines and used available evidence to justify the assigned ratings.  Conciseness. Overall, the ICR was well-written and provided a concise coverage of project activities to the extent possible, and candidly reported on most shortcomings. Page 21 of 22 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ZM-Lusaka Sanitation Project (P149091) Summary of the Quality of ICR Assessment. The ICR included an adequate discussion on the achievement of the PDO. The lessons drawn by the ICR were relevant. Most sections were concise and reflected relevant information. Overall, the Quality of the ICR is rated Substantial. a. Quality of ICR Rating Substantial Page 22 of 22