Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Safety Net Program (P131305) Report Number : ICRR0022636 1. Project Data Project ID Project Name P131305 National Safety Net Program Country Practice Area(Lead) Additional Financing Kenya Social Protection & Jobs P161179 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-52870,IDA-59810 31-Mar-2018 231,235,921.94 Bank Approval Date Closing Date (Actual) 23-Jul-2013 31-Dec-2020 IBRD/IDA (USD) Grants (USD) Original Commitment 250,000,000.00 0.00 Revised Commitment 243,080,819.41 0.00 Actual 231,235,921.94 0.00 Prepared by Reviewed by ICR Review Coordinator Group Hjalte S. A. Sederlof Salim J. Habayeb Eduardo Fernandez IEGHC (Unit 2) Maldonado 2. Project Objectives and Components a. Objectives The Program Development Objective (PDO) for the National Safety Net Program for Results (NSNPR) operation as set out on page 5 of the Financing Agreement was to establish an effective National Safety Net Program for poor and vulnerable households. The PAD (page 15) had the same formulation for the program objective. b. Were the project objectives/key associated outcome targets revised during implementation? Page 1 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Safety Net Program (P131305) Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Components The PDO was to be achieved by a Program for Results operation supporting the government’s efforts to create a coordinated National Safety Net Program (NSNP) around its five principal cash transfer programs: for orphans and vulnerable children, for the elderly, for urban food subsidies, for persons with severe disabilities, and hunger safety net program. To this end, the operation focused on achieving results in three areas: • Results area 1: expansion of selected cash transfer programs to promote more comprehensive and equitable coverage; • Results area 2: strengthening program systems to ensure good governance; and • Results area 3: harmonizing cash transfer programs to increase the coherence of the safety net sector. The operation was supported by nine Disbursement Linked Indicators (DLIs, listed below in Section 3b) that express what results are aimed for in each results area, including key steps in the results chain for the NSNP or critical activities to strengthen operational systems. Funds were to be released once progress was assessed against specific targets set for each DLI. SIGNIFICANT CHANGES DURING IMPLEMENTATION Additional Financing of US$50 million was approved in April 2017 in response to a drought emergency in early 2017 to build the resilience of drought-affected households. At the same time, a first project restructuring was undertaken. It included an extension of the closing date by two years; a minor adjustment to the PDO and three DLIs that did not involve substantial revisions. Some outcome targets were revised to take into account the additional funding and the project extension. A second restructuring extended the closing date by another six months. US$50 million remained undisbursed at closing and were reallocated to the COVID-19 Emergency Response. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates Program Cost. The overall program cost at appraisal was estimated at US$904.4 million, while the cost of the operation was estimated at US$250 million. With the introduction of Additional Financing, the estimated cost of the operation rose to US$300 million, and the cost of the program rose to US$954.4 Page 2 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Safety Net Program (P131305) million. Actual program costs at project closing were US$1,908.4 million, and the cost of the operation was US$231.3 million Financing. The PforR program was supported by two IDA loans of US$250 million and US$50 million, respectively. In addition, the British government through the Department of International Development contributed US$235.4 million (actual spending US$167.8 million), and UNICEF contributed US$7.8 million (actual spending US$8.2 million. Borrower contribution. The Borrower contribution at appraisal was estimated at US$411.2 million. The actual contribution was US$1,501.1 million. Dates. The operation was approved by the Board on July 23, 2013 and became effective on October 16, 2013. Additional Financing was approved on April 27, 2017 at which time also a first restructuring took place. Two restructurings followed - on June 23, 2020 and December 3, 2020. The original closing date March 31, 2018 was extended twice with the project closing on December 31, 2020. 3. Relevance of Objectives Rationale The PDO is aligned with country needs, government policy and Bank strategy in Kenya. While the government’s spending on safety nets has risen and coverage has expanded, both were still at modest levels compared to the population’s needs at the time of project preparation. Government policy as articulated in its National Safety Net Program (NSNP) aimed at better coordination among the country’s five main cash transfer programs in order to strengthen benefit delivery and raise benefit coverage among poor and vulnerable households, ultimately seeking to expand the coverage of cash transfers in an equitable manner. The Bank-funded operation under review – the National Safety Net Program for Results, NSNPR – aimed at helping the government achieve results that would respond to the objectives of the NSNP. To ensure that the operation effectively supported those objectives, the Bank chose a Program-for-Results instrument, plausibly arguing that it was more likely to ensure sustainable government commitment to a modernized safety net system than a Development Policy Loan or a Specific Investment Loan (PAD, page 7). The operation was also fully aligned with the Bank’s Country Partnership Strategies (CPSs) at the time of project preparation and implementation (2010-2013, 2014-2018) and, drawing on the 2020 Systematic Country Diagnostic for Kenya, remains so today, continuing to pursue NSNP effectiveness and efficiency goals, albeit with an increased focus on broader human resource development outcomes. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Page 3 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Safety Net Program (P131305) Objective Supporting the Republic of Kenya’s efforts to establish an effective National Safety Net Program for poor and vulnerable households. Rationale The objective was to be achieved by progress in each of the three results areas. Results area 1: expansion of program coverage The number of NSNP beneficiaries rose from 1.65 million in 2013 to 4.08 million in 2020, against a target of 4.28 million. Of the new households enrolled in the NSNP, 1.2 million were paid for by the government compared to a baseline of 114,000 in 2017 and a target of 792,000. Results area 2: strengthening program systems – covered targeting, registry, payment processes, grievance procedures and beneficiary outreach A new harmonized targeting methodology was adopted that applies to all the NSNPs; and a review of targeting performance indicates that 82.4 percent of NSNP beneficiaries fell below the cut-off point of the targeting method (a proxy means test). The single registry of beneficiaries is operational and supported by an integrated MIS that provides a consolidated source of information on the cash transfer programs, permitting cross-checking and reconciliation. A recertification plan was adopted and piloted, but budget constraints delayed its rollout in some areas. A technology-based payment system is in place which allows beneficiaries to receive payments under a two- factor authorization procedure. Still, timeliness of payments remains a challenge, affected by awkward budget approval procedures and by the effects of COVID-19. By end-2020, only 19 percent of payments were on time compared to a target of 90 percent. A fully functional grievance mechanism is in place at the central government level, but COVID-19 restrictions have delayed the implementation of decentralized case management. A beneficiary outreach strategy has been developed, adopted and rolled out. However, since the rollout in 2018, awareness of the NSNP has deteriorated. For instance, awareness of the correct transfer amount has gone from 86 percent of beneficiaries to 75 percent. Results area 3: harmonizing programs for greater coherence – scaling up against short term risks, consolidating programs, predictable government financing The Hunger Safety Net Program (HSNP) has developed procedures for scaling up payments to households during times of drought. Measures include pre-registering vulnerable households, setting up payment systems and defining emergency triggers. An emergency fund remains to be set up: its regulations have been approved, but it still needs seed capital to be functional. According to the ICR, that is expected to take place in the FY21-FY22 budget. A consolidation strategy was introduced that allowed harmonized mechanisms – for targeting, payment and grievances- to be applied, improving coherence between programs. While the government took over the funding of all the programs included in the NSNP in FY 19-FY20, budget constraints in FY 20/21 have resulted in budget shortfalls that have affected the cash transfer programs Page 4 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Safety Net Program (P131305) Rating Substantial PHREVDELTBL PHOVRLEFFRATTBL Rationale Overall Efficacy Rating --- PHREVISEDTBL 5. Efficiency --- Efficiency Rating --- a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of the PDO was rated high, noting its continued alignment with country and government priorities in safety nets, and Bank strategy in the area. As the PDO was mostly achieved with minor shortcomings mainly due to budgetary challenges and the COVID-19 pandemic, efficacy was rated substantial. a. Outcome Rating Satisfactory Page 5 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Safety Net Program (P131305) 7. Risk to Development Outcome The IEG rates the risk to development outcome as moderate. The government has shown commitment to building a national safety net system. This is apparent in its planning, its engagement with the Bank in building a comprehensive system, and its readiness to finance it despite fiscal constraints that at least in part have arisen due to the effects of the COVID-19 pandemic. The follow-up engagement to this project – the Social and Economic Inclusion Project (P164654) – reinforces some of the features of this project towards further strengthening delivery systems for better access to social and economic inclusion and shock-responsive safety nets. 8. Assessment of Bank Performance a. Quality-at-Entry The program was strategically relevant and aligned with country needs, and government and Bank priorities. The Bank team opted for a Program for Results approach that was likely to closely engage the government in the preparation and implementation process in a program that already was strongly favored by the government. This turned out to be correct, judging from subsequent results with a government that remains committed to the process even after the project being completed. The program’s buy-in by the government was further advanced by the Bank’s familiarity with the policy environment, having the experience of a preceding project and close collaboration with other development partners; factors that all facilitated the preparation of this program. The results framework and the DLI matrix were coherent with a systematic logic, the M&E design was straightforward and implementation arrangements drew on existing structures. The risk assessment was initially cautious, rating project risk as high at startup. At closing, the risk had been moderated by the team based on considerations noted above, and the ICR ended up rating overall risk as moderate, an assessment with which the IEG agrees. Quality-at-Entry Rating Satisfactory b. Quality of supervision Reflecting thorough preparation, initial implementation challenges initially appeared to have been modest. General elections and a ministerial reshuffle did not seem to disrupt implementation progress, and over the early years the operation made steady progress: 44 percent of the credit amount had been disbursed by mid-term review. Challenges did arise, notably in the form of a severe drought in late 2016/early 2017 that evolved into a national disaster. The operation was partly restructured by revising some parts of the results framework and the DLI matrix; and Additional Financing was introduced. Project closing was delayed by over two years, largely because of the COVID-19 pandemic that brought about travel restrictions and, with an accompanying slowdown in the economy, constrained Page 6 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Safety Net Program (P131305) budget resources. Implementation support was timely, results focused and well documented, both in Aide Memoires and ISRs (ICR, p. 35). Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The operation based its M&E on the NSNP’s framework for monitoring for results. Progress towards the development objective was assessed twice yearly against 39 results indicators selected from the framework that were considered particularly relevant for the operation. Some of the indicators were also selected as DLIs to assess progress in key areas. The government’s M&E arrangements for the NSNP were deemed credible and, as they were also being used by other development partners as well as the government, they were likely to create unified demand for information that helped create capacity building in the data-generating institutions. b. M&E Implementation A Social Assistance Unit in the Ministry of Labor and Social Protection provided routine monitoring based on sub-national MISs. An integrated registry ensured consistency between beneficiary lists for the participating programs. The M&E system faced some challenges during implementation in terms of field work and analysis: quality of reporting was initially less than satisfactory, and revisions led to delays in reporting. Still, the ICR (page 32) indicates that the M&E system produced valuable analysis for substantiating the operation’s outcomes. c. M&E Utilization Reports were routinely used by joint review and implementation support missions to discuss results and plan corrective actions; the Additional Financing with adjustments to some indicators and targets, is a significant example of using the M&E system and its findings. M&E Quality Rating Page 7 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Safety Net Program (P131305) Substantial 10. Other Issues a. Safeguards The ICR (page 33) noted that, according to OP/BP 9.00, the Environmental and Social Systems Assessment (ESSA) should consider applicable and relevant environmental and social issues. In this regard, a review of social safeguards was carried out in 2016 to assess the NSNP’s compliance with the Bank’s social safeguards policies and the status of the ESSA. The review found no evidence of any consistent pattern of exclusion of marginalized groups, although low program coverage in remote areas was a challenge. Consequently, operating manuals for the five cash transfer programs were revised to include guidance on how to ensure that marginalized communities would be considered equally during the targeting process and participate during program implementation. b. Fiduciary Compliance Financial management. The NSNP’s FM reviews and internal/external audits periodically identified routine internal control and systemic weaknesses that were addressed. Immediately after the mid-term review, the government and the development partners initiated a broader stream of work on preventing, detecting, and deterring error, fraud, and corruption in NSNP. A draft strategy and action plan was prepared and capacity building was incorporated in the NSNP technical assistance plan. Then, a forensic audit was conducted by the Kenya National Audit Office (KENAO) raising several fiduciary issues, including those related to the payment service providers (PSPs) and refunds that had not been processed and reconciliations that were still pending, and also several operational issues related to the MIS. A management response to the audit’s observations was prepared with the assistance of the Bank. In addition, critical FM issues were identified in a Comprehensive Action Plan, with which the government has complied. c. Unintended impacts (Positive or Negative) None noted d. Other None noted. 11. Ratings Page 8 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review National Safety Net Program (P131305) Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory --- Bank Performance Satisfactory Satisfactory --- Quality of M&E Substantial Substantial --- Quality of ICR Substantial --- 12. Lessons The following lessons are drawn from the ICR: Institutional strengthening is facilitated when a confirmed source of financing is readily available. This project included a significant number of time-bound institutional reforms (release of project funds triggered by policy reforms) where timely availability of technical assistance financing was a key feature for moving forward. External shocks create opportunities for program reform. The experience of a drought drew attention to the need for, and served as a catalyst for introducing, a safety net buffer against similar future events 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provided a thorough description of the project, its key achievements, and the challenges it faced during implementation. The discussion was sufficiently results-oriented and provided enough evidence to allow a fair assessment of the results and ratings of the operation as presented in the ICR. That said, while the ICR provided sufficient clarity in its messaging, it could have been more concise. a. Quality of ICR Rating Substantial Page 9 of 9