Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) Report Number: ICRR0023449 1. Project Data Project ID Project Name P154875 Somalia Recurrent Cost Country Practice Area(Lead) Somalia Governance L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-D3780,TF-A0534,TF-A6971,TF- 30-Jun-2022 168,691,340.02 A8244 Bank Approval Date Closing Date (Actual) 25-Jun-2015 30-Jun-2022 IBRD/IDA (USD) Grants (USD) Original Commitment 118,000,000.00 118,000,000.00 Revised Commitment 118,000,000.00 110,899,990.27 Actual 111,112,902.27 110,899,990.27 Prepared by Reviewed by ICR Review Coordinator Group Mees Daniel van der Clay Wescott Jennifer L. Keller IEGEC (Unit 1) Werf 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The Project Development Objective (PDO) for the Somalia Recurrent Cost & Reform Financing Project - Phase 2 was stated in the Financing Agreement as "to support the Recipient to provide credible and sustainable payroll, and to establish the foundation for efficient budget execution and payment systems for the non-security sectors in the Federal Government States (FGS), Eligible Federal Member States and Interim and Emerging Administrations." Page 1 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) This project was the second project in a Series of Projects (SOP), which had the objective to "enable the Somali Government to achieve reliable and transparent financing of critical civil service operations to help strengthen the legitimacy of the state." The Somalia Recurrent Cost and Reform Financing Project – Phase 2 (RCRF II) came after RCRF I, whose project development objective (PDO) was to finance and support an operational payroll and payment system for the non-security sectors. RCRF II is followed by RCRF III, whose objective is to support the Federal Government of Somalia and Eligible Federal Member States to strengthen resource management systems, the inter-governmental fiscal framework, and service delivery systems in health and education. RCRF II made use of the flexibility of the IPF instrument by taking advantage of reasonable restructuring to provide fiscal space and stabilize the federal state. With the FGS facing a fiscal and security crisis, the Bank restructured the project in 2018 to include additional financing (AF), strengthen eligibility criteria, added Disbursement Linked Indicators (DLIs), made changes to the PDO and project components, and extended project closing to June 2022. The new PDO simplified the desired outcomes and brought service delivery to the fore. The restructuring and AF strengthened inter-governmental fiscal transfers and provided foundational support for education and health services. The PDO was revised to " support the Federal Government of Somalia and Eligible Federal Member States to strengthen resource management systems, the inter-governmental fiscal framework, and service delivery systems in health and education." Objective: For this review, the PDO will be parsed into three objectives that will be referred to as Objectives 1, 2, and 3, as follows:  Objective 1: support the Federal Government of Somalia and Eligible Federal Member States to strengthen resource management systems  Objective 2: support the Federal Government of Somalia and Eligible Federal Member States to strengthen the inter-governmental fiscal framework  Objective 3: support the Federal Government of Somalia and Eligible Federal Member States to strengthen service delivery systems in health and education Split rating: A split evaluation was considered due to the dropping of the word ‘sustainable’ of the PDO and the downward revision of some of the outcome targets, which constituted a reduced ambition for those aspects of the project. However, since the project substantially achieved its original targets, the split evaluation would not have any implications on the rating. Hence, and in the interest of clarity, no split rating has been applied. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 22-Nov-2021 Page 2 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) c. Will a split evaluation be undertaken? No d. Components Component 1: Support to core government functions of ministries, departments and agencies in FGS (Appraisal US$81.0 million, Actual US$ 99.41 million). The primary objective of this component was to cover the salaries of qualified civil servants in non-security sectors, the costs associated with staff hired through the Capacity Injection Mechanism (CIM), and other operational expenses not related to salaries. Sub-Component 1:1: Financing eligible civil service salaries in non-security sectors in the Federal Government of Somalia (FGS) (US$49.0 million). This sub-component aimed to fund the salaries of civil servants in non-security sectors from fiscal years 2015 to 2020, to phase out this financial support by the project's end. An independent agent would thoroughly review civil service salary expenditures to ensure compliance with agreed-upon procurement and control procedures, including processing through the Somalia Financial Management Information System (SFMIS). Starting in January 2016, salary payments would transition from cash to direct bank deposits. The project would also support the Central Bank of Somalia (CBS) to establish the necessary payment systems. Sub-Component 1:2: Financing salaries and allowances to government staff and young graduates recruited under the capacity injection mechanism (CIM) in FGS (US$17.0 million). This sub-component would finance the salaries and allowances of 525 critical advisory, managerial, and technical staff to fill urgent staffing gaps identified by the FGS through the CIM. This includes salaries and allowances of several young graduates expected to become part of the civil service. The government would make all salary payments for government contracts through the regular payroll process and directly to the staff's bank accounts. Sub-Component 1:3: Financing eligible non-salary recurrent costs in MDA in the FGS (US$15.0 million). This sub-component would allocate funds for essential non-salary operational costs required for the efficient functioning of Ministries, Departments, and Agencies (MDAs). These costs include rent, fuel, maintenance of vehicles and equipment, office supplies, civil servant travel, and utilities. Component 2: Inter-governmental fiscal transfers to core government functions of eligible federal member states and interim and emerging administrations (Appraisal US$34.0 million, Actual US$ 99.41 million). RCRF sub-national financing would reward accountable and transparent government in line with RCRF's broader principles with matching funds for targeted service delivery expansion. To receive financing, interim and emerging states would need to comply with clearly defined readiness criteria, which would be reviewed and updated on an annual basis to ensure adaptation to the country context and to balance delivering basic public services against maintaining minimum fiscal and fiduciary management practices and systems in recipient governments. Essentially, these criteria serve as the foundation for fiscal transfers under the project. Financing would be through an inter-governmental fiscal transfer, reflected in the FGS budget appropriation. Sub-Component 2:1: Transfers to interim and emerging states for core government functions (US33.0 million). The sub-component through the transfer grants from FGS to Puntland, Jubbaland, and other interim and emerging states will finance (i) reforms to meet the readiness criteria, (ii) a pilot program of financing salaries and allowances of civil servants (excluding elected officials) in selected MDA; salaries and allowances to government staff and young graduates recruited under the CIM in interim and emerging Page 3 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) states; (iii) systems-strengthening and the establishment of basic accountability systems; and, (iv) eligible non-salary recurrent costs for these selected MDA. Sub-Component 2:2: Inter-governmental fora on transfers and information exchanges (US$1.0 million). This sub-component supports the government in establishing a framework for inter-governmental discussions and corresponding resource transfers. The aim is to enhance trust between different government levels and ultimately create fiscal arrangements that encourage a more equitable distribution of resources among federal and sub-national entities. Component 3: Support of recurrent costs in health and education sectors in FGS and eligible member states, interim and emerging administrations (Appraisal US$20.0 million, Actual US$ 16.77 million). The project would support the FGS, Puntland and Jubbaland to expand their service delivery systems by improving accountability systems and supporting recurrent costs in the education and health sectors. The specific activities include financing: (i) the establishment of basic accountability systems in both sectors (US$1.5 million); (ii) eligible salaries and allowances for teachers and health workers from 2015 to 2020 (US$15.0 million); and (iii) eligible non-salary recurrent costs in the sectors (US$0.5 million). The project would finance the salaries and allowances of eligible health workers and support strengthening the payroll system. In addition to financing salaries in the education sector, the project would support efforts to strengthen the public education delivery system to strengthen access, equity and quality. In addition, the project would support the implementation of key steps, including preparing a teacher recruitment, remuneration, and career progression policy in line with the Education Sector Strategic Plan, linking the system strengthening with the financing of salary recurring costs. This would ensure that all new teachers are recruited based on criteria defined in the policy. The project would also provide technical assistance to strengthen the Education Management Information System in Puntland and Jubbaland. Component 4: Project management and coordination (Appraisal US$9.0 million, Actual US$ 8.90 million). This component supported various administrative functions such as coordination, communication, management, procurement, monitoring and evaluation (M&E), impact assessment, auditing, and the dissemination of project activities across the FGS, Puntland, and Jubbaland. Restructurings: The project underwent 6 level two restructurings, with the most significant restructuring in August 2018, when FGS accessed US$60 million in IDA financing for the first time. This restructuring to include AF strengthened eligibility criteria and DLIs, changed the PDO and project components, and extended project closing by two years to June 2022. This restructuring aimed to simplify the PDO and prioritize service delivery. It also fortified inter-governmental fiscal transfers and laid the groundwork for improvements in education and health services. Seven new DLIs replaced the previous performance benchmarks in alignment with existing UK and EU supported operations and close coordination with the IMF. The revised components were as follows: Under Component 1, the component was renamed "Recurrent Cost Finance to reform resource management systems." Reform-minded DLIs were introduced in addition to input financing and included a benchmark for NSRC at FGS and discontinued NSRC input financing at FGS. Under Component 2, now called "Strengthening Inter-Governmental Fiscal Relations," this component introduced forums for fiscal, health, and education sectors. All activities related to core government functions moved to Component 3. Page 4 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) Under Component 3, renamed "Transfer for core government functions and foundational education and health service delivery," saw a significant expansion in scope and funding for foundational education and health services. In education, the project helped set up basic accountability systems to include teacher assessment, education recurrent costs, and systems strengthening, and in health, the component now covered the development of a cadre of female health workers, foundational health service delivery mechanisms, and strengthening government capability for contract management, monitoring, and coordination. Under Component 4, the allocated budget for this component was reduced. In subsequent restructurings, the Bank introduced additional DLIs (8-15) to bolster Human Resource Management reforms, wage bill management, fiscal reporting, and service delivery. In 2021, when five Disbursement Linked Results (DLRs) under DLIs 1 and 6 were not met, they were cancelled and their funds worth US$5.7 million were reallocated to cover civil servant salaries. By the project's end, US$60 million from IDA and US$118 million from trust funds had been committed, resulting in an estimated total project cost of US$178 million, of which US$168.69 million was disbursed. A split evaluation is not called for because the revised PDO targets and DLIs were more ambitious. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The Project's original cost at appraisal was US$ 144 million, to be committed through the Somalia Multi-Partner Fund. However, the project cost was revised to US$ 178 Million following a restructuring that facilitated an additional financing of US$ 60 million from IDA with an actual commitment of US$ 118 million from the Multi-Partner Fund. The Bank disbursed US$ 59.2 million, and US$ 107.11 million was disbursed by the Multi-Partner Trust Fund. Actual project cost at closure was US$ 168.69 million. Financing: An IDA credit of SDR 42,800,000 million (equivalent to US$60.0 million as additional financing) and a US$ 118 million commitment from Somalia Multi-Partner Trust Funds financed the project. Borrower Contribution: The Federal Government of Somalia did not contribute financially to the project. Dates: The Bank approved the Project on June 25, 2015, and became effective on July 2, 2015. The original closing date was June 30, 2020; however, this was revised, and the project closing date was extended by two years to June 30, 2022. The Project Mid-term Review was completed on November 27, 2017. The extension was due to the additional financing from the World Bank and project restructuring. 3. Relevance of Objectives Rationale The ability to pay civil servants' salaries and provide health and education services would address the need of the Somali government to build legitimacy among its people. The Somali Government's PFM systems were rudimentary, and intergovernmental fiscal transfers were non-existent; salaries were paid in cash, and monthly payroll processing involved submitting physical pay lists, with each submission subject to multiple manual checks. Budget control by individual ministries was also inadequate, and risks of Page 5 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) corruption were omnipresent as the government lacked the mechanisms to investigate or detect fraud. Complicating matters, at appraisal, revenues, and expenditures were de facto highly devolved in Somalia, and de jure revenue and expenditure assignments were unclear. Both the FGS and subnational authorities collected taxes simultaneously. Most federal member states (FMS) were heavily reliant on taxes collected at ports and airports. The power-sharing arrangements put in place in 2012 were fragile and contested and posed significant challenges to the institution-building process. Informed by the Somalia Risk and Resilience Assessment and Systematic Country Diagnostic, the theory of change emphasized that strong PFM systems for staff salaries would boost state credibility and create the space for service provision, and the focus on inter-governmental fiscal transfers would boost the legitimacy of and incentives for a federal system. The project would also support platforms for dialogue between ministers of finance at FGS and FMS levels and, most importantly, the sector-specific platforms for health and education. The Project Development Objectives, both at appraisal and after the 2018 restructuring, were highly relevant and well-aligned with WB-supported strategies. The FY2014-2016 Interim Strategy Note (ISN) focused on two areas: (i) strengthening core economic institutions and (ii) expanding economic opportunity. RCRF II's focus on stabilizing the budget through salary payment and the establishment of a core payroll system to provide a financial and institutional platform for the government corresponded with Strategic Objective I and was included in the ISN. The project contributed to the Somalia Multi-Partner Fund's objective of fostering socio-economic recovery and stabilization by supporting the payment of civil service salaries and the running costs of government institutions, financing technical assistance, and supporting the building blocks of larger public investment programs across a range of sectors. The FY2019-23 Country Partnership Framework (CPF), launched when the project was active, included a 'Strengthening Institutions to Delivery Services' focus area, reflecting the increased focus on service delivery reflected in the revised PDO indicators. RCRF II contributes to and is mentioned under pillar 1.1. Improve public finance management and institutional effectiveness, pillar 1.2. Enhance domestic revenue mobilization and resource sharing and pilar 1.3. Establish basic delivery systems for more inclusive social services. Additionally, the strengthening of the Somali state that the project would deliver is a crucial contribution to addressing the drivers of conflict and Somalia's main development constraints as identified by the SCD and RRA. DLIs incentivized reforms impacting RCRF II as well as other projects in the portfolio, including CIP, PFMCSP II, EU, and UK-supported projects, and the IMF Staff Monitored Program. For FGS and FMS to continue to receive financing under the project they needed to meet certain eligibility criteria. These DLIs, later Performance Based Conditions (PBCs) in accordance with IPF regulations, promoted basic system strengthening such as inter-governmental relations, fiscal transparency, and basic payroll and PFM controls. They focused on areas needed to reach the Heavily Indebted Poor Countries (HIPC) Decision Point. DLIs added in 2021 were primarily focused on ensuring robust and transparent FMS and FGS human resource management systems, and inter-government fiscal transfers. These DLIs promoted system building and strengthening. The DLIs provided a guaranteed advance and payment despite the declining scale of the recurrent cost financing. This incentivized continued reform and anchored the project’s results chain across all three components. This was particularly important given the documented capacity constraints in Somalia. Introduced across two restructurings, the DLIs disbursed US$50 million of the project’s commitments. In 2021, when five Disbursement Linked Results (DLRs) under DLIs 1 and 7 were not met, they were canceled and their funds worth US$5.7 million were reallocated. When additional DLIs under DLI 8 to 15 were not met, their money was returned. IEG agrees with the justification provided for the choice of instrument as the IPF provided the right set of incentives to support the achievement of the Page 6 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) desired outcomes, although additional technical assistance would have been warranted and an opportunity was missed to address revenue generation through the DLIs. The project was consistent with the Somalia National Development Plan 2017-2019 priority of effective and efficient institutions. The NDP recognized the need for further engagements in the re- organizing/streamlining of public sector structures as well as the re-building of the operational effectiveness and capacities through reforms of PFM, revenue generation, and Human Resource Management. The RCRF II was therefore designed with an adaptive approach of supporting and incentivizing reform processes associated with establishing core government functions, providing inter-governmental fiscal transfers, and developing the foundational mechanisms for service delivery in health and education. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To strengthen resource management systems Rationale Theory of change: The overall theory of change (TOC) was that increased funding through Financial Management Systems will increase confidence in all levels of government, increase the cohesion of the federal system, and provide more predictable financing for service delivery. Intermediate results and outputs  Increased the monthly payroll reports published on the Ministry of Finance (MoF) website from 4 to 12, achieving the target of 12.  Integrated the 570 cadres recruited under CIM to support 10 MDAs (office of the prime minister, finance, the Central Bank of Somalia, planning, agriculture, and labor) on the SFMIS for transparent payment of staff.  Electronic payroll systems for payment of salaries and vendors established.  Established payment systems by the Central Bank of Somalia/ Dahabshiil International Bank.  Capacity of staff at central regional and district level in FMS developed and feedback from the Monitoring Agent.  Operations manual – more specificity on procedures updated and improved. DLIs DLI 1: Strengthen customs administration as per customs reform Action Plan Page 7 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) Activity Complete Comments 1.1: Customs modernization roadmap & Successfully completed within the specified time Yes action plan frame 1.2: Implementation of interim IT systems at No Cancelled Mogadishu, Kismayo & Bosaso Ports 1.3: Functional specifications for the new Successfully completed within the specified time Yes customs IT system frame DLI 2: Strengthen Tax Administration systems & staffing 2.1: TIN & TCC mandatory when bidding Successfully completed within the specified time Yes for government contracts frame 2.2: Review existing business processes & Successfully completed within the specified time Yes adopt common practices frame Cancelled. Will be supported through the Somalia 2.3: Targeted Automation of Tax collection No Enhancing Resource Management Project (Excise duty, sales tax, road tax) (177298) DLI 3: Strengthen cash management & TSA 3.1: SFMIS to incorporate end-of-day Successfully completed within the specified time balances of all TSA accounts & monthly Yes frame balances of all MDA accounts 3.2: SFMIS incorporates end-of-day Successfully completed. Currently awaiting reconciled balances & all outstanding Yes disbursement only. commitments made Currently underway. Addressing challenges in 3.3: SFMIS incorporates cash forecasting No developing the cash planning system & the new model CoA to be implemented. DLI 4: Strengthen payment process for operational expenditures 4.1: Reduce cash advances to MDA’s for Successfully completed within the specified time non-salary recurrent costs from 56.4% to Yes frame 26.4% 4.2: Reduce cash advances to MDA’s for Successfully completed within the specified time non-salary recurrent costs from 56.4% to Yes frame 16.4% 4.3: All government payments for salaries & Successfully completed within the specified time capital expenditures to be paid directly to Yes frame vendor accounts DLI 8: Reforming Civil Service Legislation 8.1: The cabinet has endorsed the first set Successfully completed within the specified time of the admin regulations for the civil service Yes frame bill 8.2.1: Federal parliament has enacted the No Unable to complete within specified time frame. Civil Service Law 8.2.2.: The cabinet has endorsed the Successfully completed within the specified time second set of the admin regulations for the Yes frame civil service bill Page 8 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) 8.3.1: MoLSA has issued two quarterly Unable to complete within specified time frame. reports covering all non-security MDA’s No Consequence of missed DLR 8.1 applicability of new civil service Law 8.3.2: The cabinet has endorsed the No Unable to complete within specified time frame. ‘Organizational Ordinance’ DLI 9: Monitoring staff attendance with Biometrics 9.1: One non-security ministry has Successfully completed within the specified time implemented biometric attendance Yes frame monitoring Currently underway to create links with Payroll through SFMIS. Delay is due to previous efforts to 9.2.1: 15 non security MDA’s have collect data & record attendance that were implemented biometric attendance No affected by the pandemic & govt. directives to monitoring (including links to Payroll, work remotely. performance & disciplinary actions Deferred to next verification cycle. Currently underway. Unable to report on lessons 9.2.2: NCSC has adopted action plan learnt as the activity is still in process. No based on lessons learnt Deferred to next verification cycle. DLI 10: Promoting opportunities for women in the recipient’s civil service & leadership 10.1: NCSC issues guidelines documenting number of women in various govt roles and Successfully completed within the specified time Yes measures to achieve 30% women in senior frame leadership roles. 10.2: NCSC has an annual report covering number of women in senior roles, progress Successfully completed within the specified time Yes made and action plan for next reporting frame period DLI 11: Reforming Civil Service Pay & Grading 11.1: The govt has approved the ‘Pay Unable to complete within time frame. There is a No Grade’ reform PBC regarding this activity in RCRF III. 11:2: The govt has approved a plan for Unable to complete within time frame. There is a transitioning CIM staff into new Pay & No PBC regarding this activity in RCRF III. Grading system DLI 12: Reforming Pensions Policy & Administration 12.1: Cabinet has approved pension policy Successfully completed within the specified time Yes & submitted draft pension law to Parliament frame 12:2: The Recipient has adopted the Unable to complete. However, RCRF III has a No pension regulations PBC on Pensions. 12.3.1: The recipient has granted pensions Unable to complete. However, RCRF III has a No and started paying them in Jan 2022 PBC on Pensions. 12.3.2: The MoF has established a pension Unable to complete. However, RCRF III has a No fund department PBC on Pensions. DLI 13: Improving Wage Bill Management & Transparency Page 9 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) Unable to complete. Introduced as a new IMF 13.1: Cabinet approved Action Plan to No Extended Credit Facility for Somalia Project improve transparency and management of benchmark and will be addressed through that non-formal wage bill project. Outcomes against targets The total amount of the civil service wage bill financed by the FGS budget was raised from 40% in 2018 to 86% in January–December 2021, exceeding the revised (reduced) target of 40%. This value represented the total amount of civil servants' salaries deducted by the amount of 'baseline' civil service salaries and CIM (Capacity Injection Mechanism) salaries paid by the RCRF and divided by the total amount of non-security sector civil servants. The value does not include the DLI disbursements. If these are to be added, the indicator value changes to 53 percent in 2019, in 2021, and 45 percent in FY2022 (July 2021–June 2022). The percentage of eligible civil servants' salaries paid on time was raised from 8% to 75%, exceeding the revised (reduced) target of 22%. Before the intervention, payment of civil servants was in cash and would take up to four months. After this reform, payments were ready with the Central Bank of Somalia (CBS) or the commercial Bank for timely payment before the 28th of each month in line with the government's statutory pay period. Progress across the DLIs was mixed. Under the first set of DLIs (1-4) most targets were met although DLI 4 was the only one under which all targets were met. Under DLI 8-13 the majority of targets were not met. The project contributed to strengthening the resource management system. All PDO and intermediate indicators achieved or exceeded their target. The extent of the achievement was well captured by the PDO indicators. Rating High OBJECTIVE 2 Objective To strengthen the inter-governmental fiscal framework Rationale Theory of change: to achieve the stated objective, the project's national financing would reward accountable and transparent government in line with RCRF II's broader principles. Interim and emerging states will need to comply with clearly defined readiness criteria before receiving financing. Intermediate results and outputs  The number of inter-governmental forums held annually was raised from 3 to 4, meeting the revised (downward) target of 4. Page 10 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875)  Strengthen FGS transfers to FMS was not met for 2021. The execution rate for fiscal transfers to FMS for the months (Jan.-June) of 2021 is 18.48% (US$10.45m/56.5million), which is less than the outturn rate for domestic revenue of 37.1% (US$100million/$269.7million).  US$34.4 million was transferred to FGS from the FMS.  Through sustained dialogue, the fiscal federal technical committee successfully harmonized sin tax rates and reached an agreement on tax laws and the Revenue Act.  The Finance Ministers' Fiscal Forum created a policy note that led to an agreement on a fiscal transfer system. On the topic of tariff harmonization, an ad Valorem-based taxation system was implemented between FGS and FMS with ports (Puntland and Jubbaland).  An intergovernmental forum for the education sector was established to negotiate national exams, a common curriculum framework, and a syllabus for primary education.  FGS and FMS MoHs conducted quarterly health technical coordination meetings that brought together the health sector where harmonization of salary scales was discussed. DLIs DLI 5: Strengthen intergovernmental fiscal policy framework Activity Complete Comments 5.1: Policy paper outlining policy framework Successfully completed within the specified time Yes for intergovernmental fiscal transfers frame 5.2: Draft policy paper including principles Successfully completed within the specified time to guide FGS transfers, comparative Yes frame analysis and long-term views 5.3: Fiscal transfer allocations in draft 2021 Successfully completed within the specified time Yes FGS budget reflect policy paper principles. frame DLI 6: Strengthen FGS transfers to FMS 6.1: FGS fiscal transfers execution rate to Unable to complete within specified time frame. FMS equal or greater than outturn rates for No There is a similar PBC in RCRF III. revenue (by Aug 2019) 6.2: FGS fiscal transfers execution rate to Successfully completed within the specified time FMS equal or greater than outturn rates for Yes frame. There is a similar PBC in RCRF III. revenue (by Aug 2020) 6.3: FGS fiscal transfers execution rate to Unable to complete within specified time frame. FMS equal or greater than outturn rates for No There is a similar PBC in RCRF III. revenue (by Aug 2021) DLI 14: Harmonizing CoA and reliable & timely fiscal reports 14.1: MoF has issued a policy prescribing the harmonization of budget codes and Successfully completed within the specified time CoA, standardized reporting formats, Yes frame generation of budget reports and creation of Financial Reporting Unit 14.2: The new Financial Reporting Unit has Yes Successfully completed within the specified time adopted a harmonized budget code and frame Page 11 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) CoA and control mechanisms to manage expansion 14.3.1: Budget for FY2022 prepared with Unable to complete within specified time harmonized CoA and issued instructions for No frame. There is a PBC regarding this activity in its use on SFMIS RCRF III. 14.3.2: MoF publishes monthly fiscal Unable to complete within specified time reports using the new CoA & reporting No frame. There is a PBC regarding this activity in format RCRF III. Outcomes against targets Inter-governmental transfers to eligible FMS rose by 109% vis-à-vis the baseline year of 2018, exceeding the target of 45%. Total transfers to eligible FMS in 2018 were US$15 million. Total transfers to eligible FMS by the end of 2020 were US$59.6 million or 296% compared to the baseline year of 2018 (US$15 million). Total transfers to eligible FMS in FY2021 were US$31.6 million or 109.9% compared to the baseline. The enhanced fiscal transfers significantly improved salary payments for teachers and health workers. DLI 5 met all of its targets but many of the targets under 6 and 14 went unmet. The project contributed to strengthening the inter-governmental fiscal framework. The PDO and one of the intermediate indicators achieved or exceeded their target. One intermediate indicator was not met. The extent of the achievement was well captured by the PDO indicator. Rating Substantial OBJECTIVE 3 Objective To strengthen service delivery systems in health and education Rationale Theory of change: to achieve the objective, the project would support the FGS, Puntland, and Jubbaland to expand their service delivery systems through the improvement of accountability systems and support of recurrent costs in the education and health sectors. It would specifically finance (i) the establishment of basic accountability systems in both sectors, (ii) eligible salaries and allowances for teachers and health workers from 2015 to 2020, and (iii) eligible non-salary recurrent costs in the sectors. Intermediate results and outputs  School supervision was strengthened; school supervision reports were prepared for 1,554 primary and secondary schools in five states and the capital. No school supervision was conducted in 2021 due to COVID-related school closures.  Contract Management Unit established and fully staffed according to organogram, Page 12 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875)  A quarter of women of childbearing age (15-59 years) and children 0-59 months were referred to health facilities by Female Health Workers, exceeding the target of 10%.  Female Health Workers selected according to guidelines, trained, and actively providing services at the target of 750.  3,372 teachers were recruited, of which 80 percent were male.  The Federal Ministry of Education conducted a national teacher proficiency test in the Benadir region and four FMS that participated 1,816 teachers to measure their understanding of both pedagogical and content knowledge in Mathematics, Somali Islamic Studies, and the English Language.  In March 2021, a Health Technical Partner was recruited and provided technical assistance and capacity building.  681 Female Health Workers (FMS) were identified, verified (360 existing cadres), and trained.  The FGS team set up an Open Data Kit for each FMS on a single server where FHW could upload different registers, namely family register, daily activity tracker, and weekly supply tracker, to the real- time server data started to come. DLIs DLI 7: Strengthen teacher proficiency training Activity Complete Comments 7.1: Teacher proficiency testing in at least Successfully completed within the specified time Yes one FMS or BRA frame 7.2: Teacher proficiency testing in at least Successfully completed within the specified time Yes two FMS or BRA as part of annual exercise frame 7.3: Teacher proficiency testing in at least Successfully completed within the specified time three FMS or BRA as part of annual Yes frame exercise DLI 15: Enhancing Health Service Delivery Systems 15.1: MoH has issued instructions on the Successfully completed within the specified time use of Mobile Phone infrastructure for Yes frame improved service delivery 15.2: MoH has piloted data gathering & Successfully completed within the specified time Yes training of FHW/FHS in 2 FMS’ frame 15.3.1: MoH has recruited, trained, and Successfully completed within the specified time Yes deployed at least 700 FHW. frame 15.3.2: At least 300 FHW report daily Successfully completed within the specified time activities and additional 300 report activities Yes frame fortnightly Unable to complete within the specified 15.3.3: MoH has reached out to at least timeframe. No 500 pregnant women using mobile phones Deferred to next verification cycle. 15.4: MoH has conducted a stock-taking Yes Successfully completed within the specified time exercise on lessons learned from the rollout frame of IT-based data gathering & has adopted Page 13 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) an approved action plan for calendar year 2022 Outcomes against targets Four eligible governments started conducting proficiency testing of teachers as an annual exercise in 2020, establishing minimum competency level of teachers (DLI 7), exceeding the target of 3. Performance across the DLIs was strong with just one target missed. The project contributed to strengthening service delivery systems in health and education. The PDO and all the intermediate indicators achieved or exceeded their target. The extent of the achievement was well captured by the PDO indicator, but the project lacked a PDO indicator on the outcome of health activities. Rating High OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The PDOs were substantially relevant to the project's overarching objective and all PDO targets were met and surpassed. Seven out of eight intermediate indicators were met. The only intermediate indicator not met was also a DLI related to the intergovernmental transfers. The DLIs supported achieving the overarching objectives pursued by the PDOs and substantive progress was made against them. Thirty out of 44 DLRs were met within the project timeframe. Four DLRs were canceled, with the funds reallocated to other components. Of the 21 DLRs agreed during the 2018 restructuring, 16 were achieved, one was not, and four were canceled. Twenty-seven DLIs (8-15) were added in March 2021. These were intended to help Somalia through a liquidity crisis by frontloading support from the outer month of RCRF II and to capture a window of opportunity for agreeing on a more complex set of reforms related to the fiscal sustainability of the wage bill. With less implementation time and more political upheaval, of these 27, 14 were met. Overall Efficacy Rating High 5. Efficiency Page 14 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) Analysis: The usual economic and financial analysis for this project was not carried out, given the paucity of economic data, the ongoing conflict and fragility in southern Somalia, and the uncertainty in determining costs and benefits. Consequently, no traditional measures of efficiency are available. Instead, this ICRR includes a qualitative analysis of the impact of the project on state-building, stability, federalism structure, and service delivery and spillover effects in an FCV context, in comparison with the hypothetical counterfactuals. Result/reform Impact Counterfactual The recurrent cost had the strategic objective of financing the civil service to maintain a government The FGS payroll would have for the vital purpose of stability. continued to include errors, duplications and inaccuracies As a result of the ‘sliding scale’ of and lack the formalization to incrementally reduced financing of correct them. This would have the FGS payroll, while also grown over time and as the FGS maintaining a monthly Monitoring added new employees without Agent review of the entire FGS Civil service salary having put in place proper payroll during the life of the project, at the FGS level on payroll systems. Ultimately the the FGS has established credible a declining scale integrity of the FGS payroll, payroll systems and saved up to which comprises half of public 15% of payroll payments in expenditure, would be so low efficiency terms through capturing that HIPC Completion Point and correcting errors in the wider would not feasible/justifiable, (non-RCRF financed) FGS payroll. and the state would ultimately Compounded over time these be a less effective vehicle for amounted to significant efficiency delivering public services. savings for FGS as many of the payroll deficiencies were addressed. The federation would have Establishment of 4 out of 5 federal remained highly asymmetrical, member states in Somalia. with a federal government in Mogadishu, a relatively strong Building stronger public trust in the administration in Puntland, but Civil service government as the decision-making no meaningful local salaries and non- and basic public services were administration above District salary recurrent brought closer to citizens and level throughout the rest of the cost financing at the businesses. country. Higher levels of FMS level instability and insurgency would Efficiency gains through the ensue, as for example the monitoring of FMS payroll by the Jubbaland FMS was Monitoring Agent, similar to the instrumental in limiting Al argument on the FGS payroll above. Shabab in the South. RCRF established a cadre of FGS and FMS would have The establishment accountants in FGS MoF and in lacked capacity to implement and support to every FMS MoF and a culture of and ensure financial External Assistance stricter financial accountability. This accountability of the whole Page 15 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) Finance Section improved the efficiency of RCRF World Bank US$1.8 billion (EAFS) staff financed expenditures and had portfolio in Somalia. spillover effects in the entire FGS and FMS public expenditure and throughout the World Bank portfolio. HIPC decision point reached in Support to March 2020 opened-up access to achieving HIPC the expanded IDA grant financing decision and and IMF Extended Credit Facility. anticipated HIPC completion point On reaching the Completion Point, (DLIs on revenue HIPC decision point would not Somalia will qualify for full and mobilization, PFM, have been achieved at this time, irrevocable debt relief, which could the creation of the depriving Somalia from much reduce the debt-to-GDP ratio from intergovernmental needed debt relief. 41% in 2022 to 6%, meaning that fiscal forum, debt is sustainable in a forward- intergovernmental looking sense, contingent on the full agreements in the delivery of HIPC, Multilateral Debt education sector, Relief Initiative, and beyond-HIPC etc.) assistance. Implementation of DLI benchmarks: Without this reform, the Ministry Ministry of Finance of Interior would have continued to take over This helped to break a deadlock on to collect these revenues and responsibility for this topic between various ministries refuse to remit them to the collection of and while also demonstrating that Treasury to finance the general revenues at benchmarks could be used to budget, creating lasting Immigration and advance their reform objectives. distortion in resource allocation Commerce through at FGS level. SFMIS collection system. Implementation of DLI Without this reform, progress on benchmarks: Adopti Helped to lay the foundation for revenue generation through on and enforcement non-customs revenue generation at taxation would have been of use of Taxpayer FGS level. challenging. Identification Numbers (TINs) Implementation of Lasting efficiency gains for the FGS, Without this reform, the DLI benchmarks: which will have compounded over government’s financial viability Suspend all subsequent years, especially when would have remained undercut practice of off- the precedent (and underlying by inefficiencies. payroll cash systems) established in the civil payments at the service was later extended to the Ministry of Finance, 24,000+ security forces. allowance payments to be Page 16 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) made directly into individual bank accounts Implementation of Without this reform, the DLI benchmarks: Efficiency gains in FGS undoubtedly opportunities for inefficiency and Reduction of cash as a result of this. It was also part of waste would not have been transactions and the HIPC process, keeping the reduced, undercutting the transition to results/progress moving even in financial viability of the Somali electronic funds midst of political setbacks state. transfer RCRF was the foundation for an inter-governmental fiscal settlement In the absence of the between FGS and the FMS by a) intergovernmental forum, trust supporting the creation of these between federal entities would Intergovernmental entities, and b) bringing them deteriorate. This would lead to dialogue together in a facilitated forum for the lack of agreements on key dialogue and negotiation, and c) reforms in accounting, revenue structuring results around the sharing, harmonization of incremental advancement of a standards and regulations. positive fiscal settlement. The revenue sharing agreements reached through the intergovernmental component enabled predictable transfers of budget grants through a 60/40 Without RCRF, there would formula, whereby for instance FMS have been no functioning FMS received 40% of budget support systems, no intergovernmental funds in 2022, 50% of which was fiscal forum, no benchmarks to directed at service delivery in health anchor incremental deal-making and education. progress, and no inter- governmental customs and Agreements on customs tariff Revenue sharing other revenue harmonization harmonization has potential future settlement. Moreover, there efficiency gains in terms of would be arguably no HIPC debt predictable trading environment, relief, certainly much slower and reduction in frictions/distortions more contested customs tariff between ports allowing traders to harmonization, trade frictions select the economically desirable and associated costs to traders port, compounding over time of continued at far higher levels. marginally higher GDP growth across Somalia, fundamental to the international community benchmarks for HIPC Decision Point and Completion Point. Onboarding teacher After decades of off-budget non- The school enrollment rate cadre governmental financing of schools would have been lower by at the RCRF helped to establish a least 10% and drop-out rates Page 17 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) government financed teacher cadre. higher, as in the absence of RCRF II financed the salaries of government financed teachers, 3,720 teachers, that delivered parents would need to pay education to approximately 338,520 school fees (15-20$ a month) students (each teacher recruited per child. covers approximately 91 students see: https://moe.gov.so/wp- content/uploads/2022/06/Annual- Statisitics-Yearbook-2021- Final.pdf). This corresponds to 25% of children currently enrolled in schools. There would be limited or no Female Health Workers delivering health service delivery by the Female Health services, healthy behavior change government, undermining citizen Workers messages and medication to trust and nascent social approximately 85,000 households. contract. The Introduction of ICT innovations to deliver services and engage Use of ICT beneficiaries in an extremely harsh Limited use of ICR for service innovations environment and related savings on delivery. travel costs and monitoring and supervision costs. Other aspects of design and implementation also indicate substantial efficiency. Actual costs by component did not exceed the costs as approved through additional financing in 2018 and 2021. The project was extended by two years from June 2020 to June 2022 to reflect the additional financing and allow for activities that were added in the 2018 restructuring. RCRF was completed on time and on budget. Component 4 Project Management costs closed under the estimated amount of US$9 million or 5.5 percent of the project's total actual cost. Additional financing allowed the project to do more as evidenced by the project outcomes and additional outcomes. Cancelled DLR amounts did not represent a significant reduction, they have been reallocated to other project activities and the reforms included in the RCRF III. Changes of TTLs throughout the life of the project were efficient, as indicated under Bank Performance. The use of a Monitoring Agent to make Ex Ante and Ex Post review requests for financing by the FGS contributed to program efficiency. In the case of ex-post requests, the expenses could still be declared ineligible and recovered by lowering replenishments if necessary. The Monitoring Agent supported FGS in building the technical skills necessary to make quality withdrawal requests that were accurate and evidence- based. Their use of real-time feedback to identify problems, facilitate agreement on potential solutions, and build institutional capabilities strengthened pre-existing systems so they could meet World Bank fiduciary standards. These standards remained robust over time, for example, payroll integrity. RCRF's approach of using country systems was likely not the most efficient but achieved important state-building objectives. RCRF's approach of channeling funds through Somali systems was key to the project's goal of building state capacity and represented a major break with Somalia's history of aid dependence and minimal government agency. The project also played a vital role in fostering dialogue and trust between FGS and FMS, crucial for Somalia's future. The project's financial incentives encouraged FMS participation in technical meetings with FGS, even during political conflicts. However, improving governance proved challenging Page 18 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) as political differences between federal and state governments were still being resolved. These political settlements resulted in obtuse governance outcomes. The governance systems set up in the states were, at times, not designed for service delivery but to accommodate political clan composition and rent distribution. Furthermore, the newly created ministries were very rudimentary, especially in the weakest states, lacking even the most basic staffing and office resources. As a result, time and money were lost, making this approach likely not the most efficient in the short term (discussed further in Bank Performance). However, such settlements preserved peace in the short term and represent an important step in Somalia’s state building. Similarly, to increase service provision, the project opted to invest in quality service delivery and not just inputs, adding activities like implementing school supervision to strengthen the quality of education outcomes. Thus, RCRF II involved government agencies in the service delivery for the first time, but only in a very limited manner. RCRF II was only to have a modest impact on overall education and health service access but did start important intergovernmental conversations around the role of the state on each government level. As such, it was only a first step, opening the door for the federal government and federal member states to define more clearly their role in ensuring basic services across the country for its people. Overall The efficiency of this project is challenging to assess due to a lack of data and the hard-to-quantify benefits of the state-building impacts of the project. The ICR cites studies that found investments of FCV countries in conflict prevention and in health can be highly cost-effective, yet whether that was the case here cannot be assessed. While the reduction in the PDO Indicator targets could be taken to indicate less-than-perfect efficiency, there is little certainty that better outcomes could have been achieved in Somalia's unpredictable FCV environment. The qualitative analysis of project impacts and the potential counterfactuals present sufficient evidence to rate efficiency substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome RCRF II achieved significant state-building gains in a difficult environment. It strengthened resource management, built a system for fiscal transfers, and improved intergovernmental relations. It scaled up transfers to FMS for education and health services delivery. The RCRF helped establish a common approach to financial Page 19 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) management. The technical nature of the RCRF also helped enable continued inter-governmental dialogue on reforms, even at times when political dialogue was deadlocked. Most importantly, RCRF showed that it was possible to work with government systems in Somalia rather than the off-budget approach taken by most international donors. This helped to strengthen the Somali state and increase its legitimacy, thus tackling some of Somalia's drivers of conflict. The RCRF II introduced disbursement-linked indicators as part of an adaptive management approach to create incentives for reform and support FGS liquidity issues. The DLIs provided a guaranteed advance and payment despite the declining scale of the recurrent cost financing (see section 3. Relevance of Objectives). While the DLIs helped to incentivize reform and helped Somalia complete the HIPC process they were not all achieved. Some Disbursement Linked Results were canceled under DLIs 1, 2, and 6, with the funds reallocated to other components. Additional DLIs were added in March 2021. The main objective of this package of reforms was to help Somalia grapple with a liquidity crisis by frontloading support from the outer month of RCRF II and to use this window of opportunity to agree with the government on a more complex set of reforms related to the fiscal sustainability of the wage bill. The shorter implementation time (15 months) and the 2020-2022 political crisis and related dysfunction of parliament caused a higher share of these DLIs not to be met, and money was returned. These reform areas were continued under RCRF III. The Relevance of the PDO is rated High, the overall efficacy is High, the efficiency is rated Substantial, and the overall outcome is rated Highly Satisfactory. a. Outcome Rating Highly Satisfactory 7. Risk to Development Outcome The risk to the project's development outcomes is significant and spans most risk categories, primarily due to Somalia's status as a fragile state marked by weak institutions, limited state capacity, and ongoing conflict. These challenges pose significant obstacles to project implementation and long-term sustainability. Escalating tensions with Al Shabab could erode the government's effectiveness, potentially draining resources required for civil servant salaries. Additionally, any breakdown in relations between the Federal Member States (FMS) and the Federal Government of Somalia (FGS would undermine the progress made in state-building efforts. The political landscape in Somalia is characterized by competing interests and power struggles, further jeopardizing the sustainability of development outcomes. Moreover, Somalia's significant financial constraints limit the government's ability to maintain the project's achievements. Reliance on financial support adds to the risks. Overall, it should be noted that while the last decade represents a positive break from Somalia’s recent past and a hopeful sign of a potentially more peaceful future, the improvements should not be overstated. The Somali central and state governments remain largely in an embryotic state. Though the RCRF project was initially designed with financial sustainability in mind, the political realities in Somalia meant the government's wage bill grew quickly and outpaced revenue generation. Accommodating the competing interests of different clans and political factions often meant adding more individuals to the government's wage bill (as discussed above in efficiency). As government revenue is low and likely to remain low for the foreseeable future, new commitments present a burden for the Page 20 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) future state and should be avoided. For now, it remains unclear if the government could ever fully pay its staff without outside support. The scale of the problem grew as the project moved into the service delivery areas of government. Wanton civil service expansion could put an unsurmountable fiscal burden on the Somali state. Stronger conditionality on the FMS could have helped to curtail the expansion of cabinets and ministries. 8. Assessment of Bank Performance a. Quality-at-Entry The project design was sensible, focusing on key state-building activities. The project design recognized that the success and sustainability of any development intervention in Somalia required effective governance arrangements and capable public administration, as also argued by the Somalia risk and resilience analysis and systematic country diagnostic. Financing of recurrent costs enabled the government to function and continue strengthening core public administration. RCRF II support was implemented through government systems rather than creating a parallel system, which was essential for building institutional capacity. The project design was flexible and adaptable to respond to the emerging federal system, the fiscal crisis, and the country's FCV context. At the time of project preparation, the team had limited information on the institutional capacity of the FGS and FMS due to a lack of baseline information and analytical work to inform project preparation. Although the project was expected to support multiple FMS, several of them had not been created at the time of appraisal, adding to the complexity of the project design. These difficulties prevented the team from preparing a traditional detailed design and appraisal for the project. Instead, the team made the project design flexible by introducing readiness criteria, implementing performance benchmarks, conducting annual reviews, and frequently restructuring the project to adapt to changing circumstances. These measures allowed the team to mitigate implementation risks and steer the project amid uncertainties toward achieving the PDO. Readiness Criteria and DLIs, later PBCs, incentivized government reforms and provided macroeconomic stability. The RCRF II Readiness Criteria, explicit requirements to be met by FGS and FMS prior to receiving recurrent cost financing, empowered those in the government who were interested in reform and promoted system building and strengthening. The RCRF II reform benchmarks, followed by more formalized DLIs, provided a guaranteed advance and payment despite the declining scale of the recurrent cost financing. This dispelled the perception of 'endless free money' while still incentivizing the government to think about the future and pursue reforms. Some DLIs were further amplified through alignment with EU budget support triggers. The DLIs also incentivized reforms impacting other projects in the portfolio, such as CIP and PFMCSP II. Collectively, they contributed to the reform "track record" needed to reach the HIPC Decision Point. Early on, there was significant buy-in and interest in reform among ministers, supported by dialogue at the highest level. As more DLIs were introduced, this became increasingly challenging fewer reforms out of the second set of DLIs, were implemented, indicating that the provided parallel technical assistance was not sufficient to maintain engagement on reforms. Though the RCRF had a strong initial financial sustainability goal the operation did not sufficiently pursue this over the course of the project. The operation was designed with the strong aim to increase government capacity to cover recurrent costs, as captured by the use of the word Page 21 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) ‘sustainable’ in the PDO. Indeed, the World Bank anticipated that the FGS would take sole custody of recurrent costs during the operation's lifetime and included this PDO level indicator with a target of 100%. However, in 2018, the PDO was simplified including by dropping the word ‘sustainable.’ At the same time, the scope of the project was expanded, through the inclusion of seven DLIs and increased in the sectors. The target of the declining scale calculation was reduced from 100% of eligible recurrent costs covered by the government to 40%. The baseline was at that moment adjusted from 0% to 40%, effectively meaning that no further improvement was targeted. The 2021 restructuring did introduce DLIs 11-13 which supported effective and transparent wage bill management for enhanced fiscal sustainability. However, given the significant risk that an expanding civil service wage bill without accompanying improvements in revenue generation poses to the viability of the Somali state, a stronger focus on sustainability should have been maintained. The Afghanistan Reconstruction Trust Fund provides a cautionary example in this regard. It intended to stop financing recurrent costs by 2004 but continued to until the fall of Kabul. A combination of stricter adherence to set targets, more support for revenue generation, and stronger conditionality on the FMS to curtail the expansion of the wage bill would have been warranted. The PAD was overly optimistic in some of its core assumptions. The project faced the difficult challenge of designing a project and setting targets that were ambitious and would constitute a marked improvement if achieved while taking into account the challenging environment and limited capacity of the FGS and FMS. The PAD was overoptimistic in assuming that investments in salaries would result in health and education outcomes. Engaging the government in service delivery was important for improving state legitimacy but proved difficult to execute effectively (as described under efficiency). The assumption that the project could build state capacity at the rate needed to reach project outcomes and withstand external shocks also proved overly optimistic. The World Bank involved key stakeholders from FGS and FMS in the preparation of RCRF II and ensured that the project was aligned with government priorities, creating strong ownership. The project was important to both the FGS and FMS. Not only because they needed the resources of RCRF II to pay the salaries of civil servants, teachers, and health workers but also because the project would grant the government international credibility by having the World Bank channel funds through their systems on an ongoing basis. FGS and FMS stakeholders were highly motivated and actively participated in the project's preparation, including in multiple field consultations. Despite that, several coordination challenges were to emerge during implementation (see Quality of Supervision). Quality-at-Entry Rating Satisfactory b. Quality of supervision High-level political support and dedicated technical staff enabled the implementation of difficult reforms. RCRF II experienced strong ownership from the government. High-level government officials, including the prime minister and the ministers of finance and planning, were accessible and committed to the success of the project. They played a vital role in building consensus amongst stakeholders and Page 22 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) mobilizing constituents. The close engagement with high-level government officials allowed the World Bank to have a strategic dialogue with the government on reforms and continue supporting the reform process. Constructive collaboration and coordination with the donor community increased development impact. RCRF invested significantly in coordination. The Bank participated regularly in donor coordination meetings to provide updates on project implementation progress and flag risks. The Bank kept the donor communities informed on the design of DLIs, which generated constructive collaboration with other donor- supported projects, such as the PREMIS project and the EU budget support. The PREMIS project supported FMS that were not supported by the World Bank's FM and CI operations. PREMIS supported systems building in Galmudug, Hirshabelle, Jubbaland, and South West, while the World Bank's FM and CI operations supported Puntland. The EU budget support was informed by RCRF II. It adopted several RCRF II benchmarks, which increased the associated financial incentives and reduced transaction costs for the FGS. RCRF II was also closely aligned with IMF structural benchmarks and reform dialogue. Communication and coordination with the FMS stalled implementation progress. Initially, the World Bank relied on FGS to engage with FMS, and the FGS did not consistently consult FMS before making all decisions. This led to delays in implementation. For example, in the female health worker program, the World Bank and FGS had agreed on the parameters for selecting the health technical partner without sufficient and timely communication with the FMS. When FMS sent their inputs later, the parameters needed to be revised, causing further delay. After being made aware of the situation, the Bank integrated health director generals from FMS into weekly meetings and encouraged FGS to improve communication with FMS. Severe capacity constraints of the FGS contributed to administrative bottlenecks and delays. Capacity constraints in Somalia are extensive. Few people in the workforce have completed primary education. The institutional capacity is also weak due to the lack of defined processes and responsibilities. It took a year to prepare FMS systems for teacher payroll. To reduce opportunities for corruption and fraud, the World Bank needed to provide No Objection for every expenditure request. As a risk mitigation measure, almost every decision of the project required a No Objection from the World Bank. The World Bank team was small compared to the complexity of the project and the substantial number of transactions that required World Bank approval. At times, the World Bank was not able to respond in a timely manner to the multitude of requests from the government, causing administrative bottlenecks and delays in implementation. Furthermore, security constraints increased the difficulties for implementation and supervision. It was difficult to organize in-country missions due to security threats. Missions were postponed with short notice due to security alerts. Supervision required significant engagement from the Bank. Supervision of RCRF II required substantial dedication from the task team. The team held weekly virtual meetings and conducted at least two supervision missions a year in Somalia or from Kenya. The weekly meetings and day-to-day engagement provided substantial technical assistance to the government. Having two Co-Task Team Leaders based in Nairobi also facilitated the engagement with the client. However, given the demanding environment and complex operation, the considerable number of consultants, and No Objections to manage, the team indicated that it could have benefited from more operational support to expedite the process and reduce bottlenecks. Coordination and capacity challenges were reflected in the delayed hiring of the health technical partner. The procurement process –from defining the terms of reference to signing the contract– took almost two years. FGS and FMSs grappled with how to provide health services. The FMSs were nervous Page 23 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) about channeling resources through the treasury and outsourcing the supervision work to the health technical partner. It took a significant amount of time for the World Bank, the FGS, and the FMS to reconcile. The procurement process was led by the PIU housed in the MOF, while the MoH was less involved in developing the Terms of References. The PIU did not have the technical expertise and competence on health matters, while the MoH had not managed any project through the government budgetary system. Administrative mistakes made in the procurement process also contributed to the delay. An independent monitoring agent was instrumental in implementation support and risk management. Abyrint, a global management consultancy that specializes in rebuilding governments in post-conflict states, was hired by the Multi-Partner Fund to be its monitoring agent. In that capacity, it provided third-party monitoring, assurance, and advisory support to ensure that MPF funds engaged by Somali authorities are being used according to the MPF grant agreements. This arrangement has been in place since RCRF I. The responsibilities of the firm regarding RCRF II encompassed three areas: i) monitoring every transaction of RCRF II, identifying ineligible expenditure, and performing other monitoring and verification work for the project as needed; ii) providing strategic advice to the World Bank team on country context, payroll trends, Human Resources Management practices and policies, and risks; iii) ensuring that the World Bank procedures were understood by the government and helping the government create the instruments and processes needed to implement those procedures. The Monitoring Agent's role was critical for the implementation of RCRF II. It ensured the integrity of RCRF II transactions, provided invaluable supervision support to the World Bank, and provided tremendous technical assistance to the government. However, as the Monitoring Agent has no permanent presence in Mogadishu, they could not provide accessible daily support to the PIU, which would have been beneficial. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design While the project’s theory of change was largely clear, the thinking on its sector impact evolved over time. The project's original theory of change was clear, except with respect to revenue for the sustainability of the outcomes. As discussed under Components, the 2018 restructuring revised the PDO project components and expanded the sector reform agenda of the project, reflected through the intermediary indicators. The expanded scope also included the inclusion of seven DLIs. After restructuring extensive DLI support in sectors and investments in component 3 shifted, the project's focus shifted from inputs for salaries to foundational investments in health and education. The most significant design shortcoming was the lack of a PDO indicator on the outcome of health activities, which was a key aspect of the project's contribution to service delivery. While there were two health intermediate results indicators these are no substitute for a DPO results indicator. As the project was progressively restructured the focus on sustainability of government expenditure deteriorated. Page 24 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) b. M&E Implementation The project faced FCV challenges in collecting data and long lacked an M&E specialist. The data for most of the indicators come directly from government financial management systems, while the rest can be obtained from regular monitoring of project activities. The project initially encountered challenges in the collection of data related to the work of female health workers and resolved the problem by implementing mobile phone-based solutions. Shortcomings include the lack of clarity on the calculation of the PDO indicator on the share of civil service salary financed by the government, which was not clarified until the last Implementation Status and Results Report (ISR). Its original target was revised from 100% to 40%, while achievement was 86%. That value represents the total amount of civil servants' salaries deducted by the amount of 'baseline' civil service salaries and Capacity Injection Mechanism salaries paid by the RCRF and divided by the total amount of non-security sector civil servants' salaries. However, this calculation did not treat DLI disbursement as WB financing. If DLI disbursement is considered World Bank financing and added, the indicator value changes to 45%. That significantly reduces the growth from the baseline of 40% to a fraction of the currently used achievement, although still meeting the final target, which itself was set at baseline. Another shortcoming is that the project did not hire a dedicated M&E specialist until late 2021, depending mostly on the PIU for data collection. However, the project reported timely data in the ISRs. The Monitoring Agent provided regular analytical reports. M&E design, implementation, and utilization were adequate to the task and robust in the areas with the highest risk. c. M&E Utilization M&E data was used to provide evidence on the achievement of results and form the basis for discussion on course correction, but additional reform could have been incentivized. The Monitoring Agent's analytical work was used by the Bank and development partners alike. The frequent restructurings reallocated investments between PBCs based on their performance and the country's reform needs. M&E data created positive knock-on effects. For example, the data collected in health feed into their information systems, rendering them operational. Additionally, the targeted share of civil servants' salaries financed by the government was reduced from 100 percent to 40 percent, and the targeted percentage of eligible civil servant salaries paid on time was reduced from 80 percent to 22 percent. By the project closing, achievements far exceeded the revised targets, leaving the question of whether this reduction in ambition was necessary. Especially the reduction in ambition on the recurrent cost covered by the FMS should have been considered more carefully given its significance to the sustainability of the Somali state. M&E Quality Rating Substantial 10. Other Issues a. Safeguards Page 25 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) The project was classified as Category C, and no environmental or social compliance issues occurred during implementation. No specific environmental or social safeguard instruments were required, and no safeguard policies were triggered. b. Fiduciary Compliance The project had a longstanding Moderately Unsatisfactory (MU) staff self-evaluation rating in Financial Management because DLIs were introduced without a formal restructuring. The project was originally designed to be a traditional Investment Project Financing. Performance benchmarks were introduced during implementation to incentivize the government to undertake reform. Although this contributed significantly to the reform progress, it also caused a longstanding MU rating in Financial Management in the initial years because proceeds disbursed against the performance benchmarks are not linked to specific activities as in a traditional IPF. Furthermore, the project was misclassified in the government budget, undermining the transparency of budget execution and financial reporting. The issue was resolved in 2019 following a thorough review of the supporting documents submitted by the government. In a capacity and credibility-building measure, the Monitoring Agent provided ex-ante review and verification of the entire payroll, not just World Bank-covered expenditures. Throughout the life of the operation, the Monitoring Agent identified US$5.4 million in payroll exceptions, of which only US$280,000 was declared ineligible. World Bank and Ministry of Finance management were consulted and involved in finding solutions, thereby addressing systemic issues and improving the system overall. Subsequently, the remaining US$5.12 million was either not paid by the World Bank, refunded to the designated account, or was resolved by the government. c. Unintended impacts (Positive or Negative) The reforms implemented under the RCRF had wide-reaching impacts. The processes and reforms built by the RCRF enabled other reforms within the government. For example, the RCRF demonstrated payroll reform was possible in non-security government ministries. This inspired similar reforms in the security sector, where the electronic registration of security personnel was implemented. Those payments were not channeled through the RCRF, but they mirrored the reforms that were developed through the RCRF. A further unintended impact of RCRF II has been spillovers from RCRF II project staff into the civil service. Senior management positions at technical and political levels are being filled by former project consultants familiar with good budgeting and accounting practices and the practical implementation of fiscal federalism. For example, one RCRF project consultant later became Director General of Finance. They represent a cadre of trained financial experts entering the FGS and FMS ministries of finance. Constructive collaboration and coordination with the donor community increased implementation efficiency and development impact. RCRF II created a platform for donors to coordinate on the state- building agenda. The Bank participated regularly in SMPF meetings to provide updates on project implementation progress and flag risks. The Bank kept the donor communities informed on the design of Page 26 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) DLIs, which generated constructive collaboration with other donor-supported projects, such as the UK/EU Public Resource Management project and the EU budget support (discussed above). d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Highly Satisfactory Highly Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons 1. Successful state-building in fragile contexts requires a long-term, multi-phased approach that fully engages the government. Full government engagement in project design led to strong ownership and alignment with the Federal Government of Somalia's (FGS) strategies, allowing for flexibility in resource allocation and enabling the project to build capabilities where the government was prepared to act. This engagement was complemented by a focus on strengthening core Human Resource Management (HRM) and Public Financial Management (PFM) systems, which proved fundamental for the sustainability of development interventions. The use of country systems not only built institutional capacity but also served as a testing ground for other projects, thereby paving the way for future operations like RCRF III and sectoral projects in health and education. Recognizing that state-building is a long-term endeavor, the project wisely adopted a multi- phased approach, allowing for significant progress even within a five-year timeframe. This approach underscored the need for sustained, long-term support from development partners as the government continues to rely on external funding for some recurrent costs, emphasizing the necessity for a longer-term commitment from development partners for ongoing reforms and state-building. 2. The use of focused DLIs with eligibility criteria can be a powerful combination to ensure ownership and incentivize reforms, but coordinated technical assistance is needed. DLIs and eligibility criteria allowed the government to make progress at a realistic pace while maintaining pressure for reform. However, consistent technical assistance was required. Some DLIs were intended to incentivize reforms supported by other projects. In such cases, stronger Bank internal coordination and with different government counterparts would be essential to facilitate the achievement of such DLIs. 3. The use of a Monitoring Agent is an innovative approach to buttress credibility in country systems while building institutional capacity. The monitoring agent created trust in nascent fiduciary systems. The frequency of communication between the Monitoring Agent Page 27 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) and the government helped build state capability and explain how to work with partners like the Bank. The Monitoring Agent also provided regular strategic and options papers, which benefited the CMU and development partners alike. This approach could be beneficial in other fragile and conflict-affected settings to ensure credibility and facilitate state-building efforts. 4. Coordination amongst World Bank projects and with donor projects can increase impact. In Somalia, donors provide financing and support in different regions and different sectors, and the government's capacity to coordinate these interventions was low. Therefore, the World Bank can play a proactive role in coordinating with and informing donors of implementation progress and risks, ensuring synergy in project design. 5. Bank Communication and coordination with subnational governments is essential. In a conventional IPF, the World Bank engages mostly with the center and relies on the national government to coordinate with their subnational counterparts. In a context where there is a lack of trust between the national and subnational governments and an unstable security situation, this could worsen intergovernmental relationships or result in frequent delays. Therefore, the World Bank should maintain close engagement with state or subnational governments and include them in discussions and decision-making whenever possible. A communication and change management plan could further these efforts. 6. Intensive implementation support in FCV projects in Somalia should be adequately supported by both human and financial resources. Due to the security challenges and the huge need for technical assistance, which is not easy to acquire, RCRF II required a supervision budget much higher than that of a typical IPF. The resources provided by the SMPF were key to ensuring that the Bank could carry out regular supervision missions and dedicate sufficient time and talent to provide intensive implementation support. For a project with such a broad scope and high complexity as RCRF II, it could have benefited from additional staff members to support operational tasks, M&E, and training. 13. Assessment Recommended? Yes ASSESSMENT_TABLE Please Explain This a rare example of recurrent cost financing with DLIs through an IPF and therefore merits a deeper analysis. 14. Comments on Quality of ICR The ICR offers an in-depth summary of the project, substantiating its ratings with available data and evidence. It maintains a straightforward, precise, and largely consistent alignment with the project's developmental objectives. While adhering to most of the established guidelines, the report endeavors to cross-reference data to arrive at its conclusions and maintains a results-oriented focus. Although the quality of the evidence and analysis is generally robust, there are gaps in data completeness and information. The report includes a Page 28 of 29 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Somalia Recurrent Cost (P154875) discussion of the project's theory of change, aiding the reader in comprehending the basis for the given ratings. However, the ICR could have gone further in depth as to the basis of the TOC, addressing what analysis informed it and if a similar approach had been successful in other FCV countries. The description of the design, selection, and monitoring of the DLIs in the ICR was limited. The ICR’s risk section could have more strongly acknowledged the existential risks to the sustainability of the project outcomes. Lessons drawn from the project are articulated clearly, are practical, and are substantiated by the evidence presented in the ICR. a. Quality of ICR Rating Substantial Page 29 of 29