Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) Report Number : ICRR0022984 1. Project Data Project ID Project Name P152736 TZ-Strengthening PHC for Results (FY15) Country Practice Area(Lead) Tanzania Health, Nutrition & Population L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-56430,TF-A0261,TF-A0270,TF- 30-Jun-2020 244,998,461.33 A1567,TF-A9831 Bank Approval Date Closing Date (Actual) 28-May-2015 30-Jun-2021 IBRD/IDA (USD) Grants (USD) Original Commitment 260,000,000.00 74,521,343.00 Revised Commitment 270,177,362.00 58,760,294.00 Actual 244,998,461.33 58,760,294.00 Prepared by Reviewed by ICR Review Coordinator Group Salim J. Habayeb Judyth L. Twigg Eduardo Fernandez IEGHC (Unit 2) Maldonado 2. Project Objectives and Components a. Objectives The objective of the Program was to improve the quality of primary health care services nation-wide in the Recipient's territory with a focus on maternal, neonatal and child health services (Financing Agreement, August 25, 2015, p. 5). The operation consisted of a Program for Results (PforR) operation that would support the Recipient's overall primary health care program under the Fourth Health Sector Strategic Plan. Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) On January 16, 2020, six months before the closing date (at which time disbursements aggregated at 81 percent), four out of five outcome targets were revised upward, and all were achieved or exceeded. A fifth outcome target was revised downward, and neither its original nor revised target were achieved. This ICR Review did not apply a split rating because it was clear at the outset that the above revision would not impact the assessment of achievement of objectives. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Components Program scope and boundaries The Program planned to support the government’s primary health care (PHC) program for a period of five years (2015/16 – 2019/20) under the Fourth Health Sector Strategic Plan (HSSP IV), including a key government initiative in primary care titled as “Big Results Now in Health” (BRN in Health) for 2015-2019. The BRN aimed at accelerating the reduction of maternal and neonatal mortality through improvements in performance, governance and accountability in PHC. It was developed as part of Tanzania’s Development Vision 2025. The financing share of the PforR operation within the larger national PHC program was small at about 11 percent (financing details are shown in section 2e, below). This ICR Review notes that the scope of the government’s program and that of the PforR Program were the same, constituting an optimal framework that would reduce the risks of manipulating Program boundaries. Results Areas The PforR Program aimed at advancing results in six areas, some of which were key national results areas under BRN: 1. Health financing and public financing management aimed at ensuring adequate levels of spending and transparent financial management in health. 2. Performance management aimed at improving overall performance with a primary focus on improving health workers' performance. Interventions included: (i) a stepwise performance of health facilities as an accreditation scheme for quality improvement titled as “Star Rating” initiative; (ii) implementation of the Decentralization by Devolution Policy through the empowerment of health facilities to plan, budget and manage revenue in line with set guidelines; (iii) performance contracts and targets at individual health worker levels; and (iv) social accountability mechanisms. Explanation of the Star Rating Assessment (ICR, p. 9): This consisted of a quality improvement approach designed to assess performance of health facilities in a stepwise manner. Performance scores assessing the quality of reproductive, maternal, neonatal and child health at the facility level included four domains: (i) facility management and staff performance; (ii) service charters and accountability; (iii) safe and conducive facilities; and (iv) quality of care, including interviews with patients and spot checks of medical records to Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) verify the quality of content and recording. The Star Rating Assessment Scale ranged from 1 to 5 stars, with 5 being the best quality and 3 being the minimum acceptable. 3. Human resources for health aimed at improving the distribution of skilled PHC workers with a focus on nine regions that had critical shortages in human resources (in comparison with national averages). Interventions included: (i) increasing PHC employment permits for these regions; (ii) engaging the private sector to provide skilled personnel for public health facilities through public-private partnerships; (iii) redistributing health care workers within regions; and (iv) optimizing the pool of new recruits through a bonding policy or compulsory attachments. 4. Supply chain management: This result area aimed at improving the availability of health commodities, notably essential medicines in PHC facilities. Interventions would tackle key issues along the supply chain and included: (i) the introduction of new governance and accountability mechanisms; (ii) development of a new finance and business model for the Medical Stores Department; (iii) engagement of the private sector in procurement and distribution; (iv) implementation of quality improvement initiatives for inventory management; and (v) using innovative information and communication technology to report stock-outs. 5. Maternal, Neonatal and Child Health Continuum: This results area aimed at improving coverage and quality of maternal, neonatal, and child health (MNCH) along the continuum of care. Interventions included: (i) ensuring that dispensaries and health centers meet basic emergency obstetric and neonatal care requirements; (ii) expanding comprehensive emergency obstetric and neonatal care (CEmONC) to selected hospitals and health centers; (iii) strengthening the corresponding satellite blood banks; and (iv) extending MNCH services to communities through the use of community health workers and awareness campaigns. The Program intended to provide additional attention to five regions with poor performance in terms of maternal and neonatal mortality indicators. 6. Monitoring and evaluation, supervision and capacity building to address capacity constraints and foster accountability. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates Cost and financing National PHC program: The entire cost of the PHC program under HSSP IV was estimated at US$2.6 billion that represented 55 percent of the government health sector budget over five years. The government would finance about 78 percent of the national PHC program cost. PforR Program: The cost of the PforR operation that would support the larger PHC program was estimated at US$274.5 million (about 11 percent of the total PHC program), consisting of the following: i. IDA Credit of US$200 million equivalent (SDR 145 million); ii . US$40 million Grant from the Multi-Donor Trust Fund for the Global Financing Facility in Support of Every Woman and Every Child; iii . Grant of US$20 million from the “Achieving Nutrition Impact at Scale” Multi-Donor Trust Fund; and iv . United States Agency for International Development Grants in the amount of US$14.5 million. Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) Other parallel financing: Other development partners were expected to contribute an amount estimated at US$290 million in support of the overall PHC program. Actual Cost and fund flow issues: The actual cost of the PforR Program was US$245 million. The moderately incomplete fund utilization (9 percent) was explained by the ICR as being due to a slow-down in results-based financing (RBF) implementation during the last 2-3 years of the operation resulting from turnover of senior officials at the Ministry of Finance with new leadership that was not in favor of the payment of RBF bonuses to health workers (Task Team clarifications, April 19, 2022). The ICR (p. 26) also reported that prioritization of civil works within the health sector posed a detriment to RBF and the payment of bonuses to health workers. The ICR noted that turnover of senior officials at the Ministry of Health and Social Welfare, Community Development, Gender, Elderly and Children (MOHCDGEC) made it difficult to maintain technical and policy dialogue, particularly from 2018 onwards. Subsequently, in 2020 and 2021, these issues were further exacerbated by COVID-19 pandemic disruptions. Other factors contributing to underutilization of funds were related to the non-achievement of disbursement-linked results on employment permits, share of the health budget, and Star-Rating assessment. Dates The Program was approved on May 28, 2015 and became effective on November 5, 2015. A Mid-Term Review was carried out on May 18, 2018. A Level 2 Program restructuring was carried out on January 16, 2020 at which time disbursements aggregated at 81 percent (US$199 million out of US$245 million). The restructuring introduced changes to the results framework (upward revisions of four outcome targets, and one downward revision of a fifth outcome target), reallocations between disbursement-linked indicators (DLIs), and a one-year extension of the original closing date of June 30, 2020, to align the last DLI payments with a full annual implementation cycle, and to provide the government with sufficient time to complete data verification (ICR, p. 15). The operation closed on June 30, 2021. 3. Relevance of Objectives Rationale Responsiveness to the country context and strategy. Despite progress made over the past decades, Tanzania’s health outcomes were lower than expected for its level of economic development. The maternal mortality ratio remained high at 556 deaths per 100,000 live births (2015), the neonatal mortality rate was 25 per 1,000 live births, and stunting remained persistently high at 34 percent among children under five years of age. Low quality of care was a major bottleneck to improving sector performance (ICR, p. 5). According to the 2012 Service Availability and Readiness Assessment, only 32.3 percent of dispensaries and 50 percent of health centers had the capacity to provide basic emergency obstetric and neonatal care services. A range of factors contributed to the low quality of care. Tanzania spent less public money on health than comparable countries, and its health sector was highly dependent on external support (48 percent of total public expenditure on health in 2011/12). Service delivery was constrained by a shortage and inequitable distribution of skilled human resources. Decentralization in the health sector had not fully materialized, hindering operations of health facilities that had limited financial autonomy. Most primary care facilities did Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) not even have a bank account (PAD, p. 4), and funding was channeled to local government authorities (LGAs), often further constraining resources to reach lower levels. According to the ICR (p. 7), the Mid-Term Review of Tanzania’s Third Health Sector Strategic Plan (HSSP III) for FY09-FY15 concluded that, while the health sector had made progress in all strategic areas, the pace had been slower than anticipated. The HSSP III Mid-Term Review identified priority directions, notably for making optimal use of available resources. The findings led the government to embark on a high-profile PHC initiative, the BRN in Health that was described in section 2d above. The BRN was embedded in HSSP IV that was to guide Tanzania's health sector development during the FY16-FY20 period. Program objectives are relevant to Tanzania's current national development agenda and health sector priorities. Program objectives support the 2021-2026 Fifth Health Sector Strategic Plan (HSSP V) “Leaving No One Behind” that envisions a healthy and prosperous society contributing to the development of individuals and the nation. A specific goal for HSSP V is to provide sustainable health services with standards that are acceptable to all citizens and based on geographical and gender equity. An increased focus on primary health care, maternal, neonatal, and child health in particular, and improved governance and accountability are central elements of HSSP V. Relevance of the PforR instrument. The rationale for PforR was based on the following justifications: (i) its focus on sector-wide results rather than on specific inputs; (ii) consistency with the government commitment to results in the health sector; (iii) increased flexibility and responsiveness to the country’s needs though the provision of non-earmarked funds; (iv) utilization of country systems and their strengthening; and (v) motivation of implementers to find locally relevant and sustainable solutions to overcome operational bottlenecks (ICR, pp. 8-9). According to the PAD (p. 6), PforR would also provide a performance-based framework for other Development Partners to further align their support with national plans. Alignment with Bank strategy. In the past, the World Bank played a key role in supporting the Tanzania Health Basket Fund that was initiated in 1999 with coordinated donor support to government efforts in strengthening primary health care services under a pooled mechanism. At entry, the PDO was aligned with two objectives set out in the Country Assistance Strategy (CAS) for FY2012-2015 (this period was extended): (i) strengthening human capital and safety nets aiming at improving access to services and quality of health services; and (ii) promoting accountability and governance. The CAS committed to addressing gender concerns and identified the high maternal mortality rate as a critical issue for increased attention. Specific health indicators under the CAS included reducing the overall maternal mortality rate and increasing the proportion of births attended by skilled health personnel. At Program closing, objectives remained consistent with Bank strategies. The Program PDO continued to be closely aligned with the Country Partnership Framework (CPF) FY18-22. Specifically, the PDO is consistent with CPF Focus Area 2 -- Boost Human Capital and Social Inclusion -- with a lifecycle approach to human development challenges, giving particular emphasis to investing in the early years, including health and nutrition (Objective 2.1) and improving the quality of health care and education (Objective 2.2). The operation’s emphasis on enhancing performance management is also consistent with Focus Area 3 -- Modernize and Improve the Efficiency of Public Institutions – that assigns a high priority to reinvigorating public sector reform, with particular emphasis on strengthening public sector accountability and financial efficiency in delivering services (Objective 3.1). The focus of development objectives on maternal and child health remains central to the attainment of Sustainable Development Goal 3 (SDG 3 -- to ensure healthy lives and promote well-being for all, at all ages). Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Improve the quality of primary health care services nationwide in the Recipient's territory with a focus on maternal, neonatal and child health services Rationale Theory of change The theory of change was adequately illustrated by both the PAD and ICR and captured the operation’s pathway to improve the quality of PHC services with a focus on maternal, neonatal and child health services. The theory of change held that: (i) ensuring adequate and more transparent funding at local and facility levels; (ii) improving the performance at LGA and health facility levels; (iii) augmenting the availability of human resources in PHC; (iv) enhancing the availability of essential medicines in PHC; and (v) improving the Maternal, Neonatal, and Child Health continuum of care would all contribute to improved quality of primary health care services with a focus on maternal and child health services. Intended improvements (outcomes) would be measured mainly through health facility quality ratings, ante-natal care, attended deliveries, preventive treatment for malaria, and vitamin A supplementation. Intermediate results by Results Area 1. Health financing and public financing management: • The share of health in total government budget receded from a baseline of 8.5 percent in 2015 to 7 percent in 2020, short of the target of 9.75 percent. This indicator was outside the direct influence of the Program and its inclusion was not realistic. • The proportion of Councils (LGA level) with unqualified opinion in the annual external audit report increased from a baseline of 80 percent in 2015 to 95 percent in 2020, exceeding the target of 92 percent. 2. Performance management: • The “Star Rating" Assessment of PHC facilities as per the two-year cycle was not completed. The final assessment cycle was not undertaken (see outcomes below). • RBF facilities receiving timely RBF payment on the basis of verified results every quarter was not achieved. The ICR (p. 38) reported that payments were consistently delayed and not made within the stipulated time. • LGAs with functional Council Health Service Boards increased from a baseline of 86.3 percent in 2015 to 100 percent in 2020, achieving the target. Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) 3. Human resources for health: • Dispensaries with skilled human resources for health increased from 91 percent in 2015 to 99.6 percent, essentially meeting the target of 100 percent. 4. Supply chain management: • Health facilities with continuous availability of tracer medicines in the past year based on an established list of medicines increased from a baseline of 30.6 percent in 2015 to 89.4 percent in 2020, exceeding the target of 80 percent. 5. Maternal, Neonatal and Child Health Continuum: •The number of health facilities with functional Emergency Obstetric and Neonatal Care increased from a baseline of 79 facilities in 2015 to 160 facilities in 2020, achieving the target. 6. Monitoring and evaluation, supervision and capacity building: • The completeness of quarterly Health Management Information System data that were entered in the District Health Information System by LGAs by the end of the month after the end of the quarter increased from 89.5 percent in 2015 to 97 percent in 2020, exceeding the target of 96 percent. • The Regional Health Management Team’s required biannual Data Quality Audits for LGAs that meet national standards for Data Quality Audits increased from a baseline of zero in 2015 to 91 percent in 2020, exceeding the target of 80 percent. • Regional Health Management Teams conducting quarterly supportive supervision visits for LGAs meeting national supervision standards and that were reported in the District Health Information System increased from a baseline of zero in 2015 to 93 percent in 2020, close to but short of the target of 100 percent. • The completion of annual capacity building activities compared to agreed annual plans was 81 percent in 2020, short of the target of 100 percent. Corporate Results Indicators: • People who have received essential health, nutrition and population services through the operation reached 37.4 million people in 2020, exceeding the target of 30 million people. • The number of children immunized through the operation reached 10.9 million children in 2020, exceeding the target of 10 million children. • The number of women and children who received basic nutrition services reached 19.5 million, exceeding the target of 10 million women and children. Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) • The number of deliveries attended by skilled health personnel reached 7 million attended deliveries, short of the target of 10 million deliveries. Outcomes The intended outcome pertaining to PHC facilities with 3- Star Ratings and above was not achieved. Actual recorded achievement reached 19% in December 2018, short of both the original target of 50 percent and the reduced revised target of 30 percent. The ICR (p. 18) noted that that the operation planned for three assessment rounds. The first round (baseline) in 2015-2016 covered 6,993 health care facilities, and the second round assessed 7,289 health care facilities in 2017-2018. In the first round, almost one-third of all PHC facilities were rated 0-star, and only 2 percent scored 3 stars. The second round showed that 19 percent of health facilities scored a 3-star rating, reflecting an important improvement. The planned third round for 2019-20 was not completed due to various delays, including delays in fund release and key staff transfers. Nevertheless, as noted in section 3b, the ICR reported that this star accreditation was overambitious. The percentage of pregnant women attending four or more ante-natal care visits increased from a baseline of 41.2 percent in 2015 to 90 percent in December 2020, exceeding both the original target of 60 percent and the upward revised target of 68 percent. The percentage of antenatal care attendees receiving at least two doses of intermittent preventive treatment for malaria increased from 42.5 percent in 2015 to 79 percent in December 2020, exceeding the original target of 60 percent, and almost achieving the revised target of 80 percent. The percentage of institutional deliveries increased from a baseline of 44.7 percent in 2015 to 83 percent in December 2020, exceeding both the original target of 60 percent and the upward revised target of 70 percent. The proportion of children aged 12-59 months receiving at least one dose of Vitamin A during the previous year increased from a baseline of 51 percent in 2015 to 100 percent in December 2020, exceeding both the original target of 65 percent and the upward revised target of 90 percent. In addition to the above, the ICR (p. 18-19) reported on post-natal care and reproductive health improvements that were not included in the results framework. This ICR Review notes that several program outcomes reflected increased utilization rather than direct measures of quality. Nevertheless, it is reasonable to expect that increased utilization was likely to be associated with improved service quality. Rating Substantial PHREVDELTBL PHOVRLEFFRATTBL Rationale Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) Overall Efficacy Rating --- 5. Efficiency --- Efficiency Rating --- a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Point value (%) *Coverage/Scope (%) Available? 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance is rated high, encompassing a full alignment between Program objectives and the Country Partnership Framework at Program closing, and between DLIs and PDO. Efficacy is rated substantial, as objectives were almost fully achieved. These findings are consistent with a moderately satisfactory rating for overall outcome. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The main risk that targeted improvements may not be maintained is related to financial sustainability challenges resulting from declining government spending on health, low budget execution, and dependence on external funding to sustain payments to health care facilities. Other risks include staffing challenges faced by the health sector as a whole. Institutional capacity gaps remain at the peripheral level. There are political and governance risks related to increasingly centralized budget management affecting fund flows, and variable commitment to the payment of bonuses to health workers under the RBF scheme. Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) 8. Assessment of Bank Performance a. Quality-at-Entry The overall design of the Program was relevant and technically sound. The government PHC program was well identified and the PforR Program was well defined. Preparation adequately identified DLIs, with lapses in setting some DLRs. Program assessments in terms of technical, fiduciary, and social and environmental aspects were satisfactorily undertaken along with the identification of opportunities for building capacity. The technical assessment adequately identified key risks and mitigation strategies (PAD, p. 30). The fiduciary systems assessment that included the assessment of governance and anti-corruption mechanisms, procurement, and financial management systems concluded that the legal and regulatory framework for the Program’s fiduciary systems was comprehensive and in line with principles and standards for public procurement and financial management, but that there were salient weaknesses at the LGA and facility levels, and appropriate measures were taken to address them. Institutional and implementation arrangements were well identified and prepared (PAD, pp. 19-21). Implementation arrangements were based on the current institutional arrangements for the delivery of PHC services. MOHCDGEC was responsible for health policies, regulations, coordination and oversight. PMORALG, through the LGAs, was responsible for providing administrative support and allocating resources for the delivery of PHC services. Health facilities at the LGA level were responsible for delivering PHC services at the grassroot level, in line with Tanzania’s decentralized health service delivery system. The PforR operation relied largely on existing government information systems and adopted indicators that were embedded in the government’s systems. Disbursement arrangements and verification protocols were well prepared (PAD, p. 23), and the Internal Auditor General’s office was assigned as the independent verification agency for the Program with defined terms of reference that were satisfactory to the World Bank. The Bank Team was proactive in engaging with national and sub-national agencies and Development Partners in conducting PforR information sessions. Quality-at-Entry Rating Satisfactory b. Quality of supervision Program supervision was reportedly thorough and consistent throughout the operation's lifetime, with regular supervision missions. Findings were adequately reported and recorded in detailed Aide Memoires and Implementation Status and Results Reports (ICR, p. 30). The Program had three Task Team Leaders, two of whom were based in the country, facilitating communications with government counterparts and Development Partners. The Bank Team reportedly played a proactive technical assistance role throughout the operation's lifetime, providing labor-intensive technical assistance in substantive topics, operational aspects, and the implementation of fiduciary, environmental and social safeguards. For example, the Task Team ensured the provision of technical assistance for developing the Star-Rating database and technical support to Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) PMORALG. The Task Team also took the lead in coordination activities among Development Partners, including in the Health Basket Fund Performance Committee, the Audit and Finance Sub-Committee, and the Performance Monitoring Sub-Committee. The Team helped in organizing "Operations Clinics" that were helpful for government agencies, non-governmental organizations, and Development Partners in various implementation aspects, such as in LGA performance scorecards. Task Team members extended their engagement with main nutrition stakeholders. The Task Team also contributed to fostering policy dialogue with Tanzanian authorities, including during the Mid-Term Review of HSSP IV and the development of HSSP V. The Task Team organized a Mid-Term Review in 2018 that identified the need to address several challenges, including the need for revising DLI payment framework, some DLRs, and the results framework. Examples of the main issues that needed to be addressed were discussed in the 2020 Program Restructuring Paper; fund reallocations among DLIs were needed to support the functionality of refurbished facilities. The original DLRs under DLI 1 on foundational activities were completed, and there was a need for a new DLR to reimburse additional refurbished health facilities that were CEmONC functional. Under DLI 2 on the achievement of annual results in institutional strengthening, there was a need to drop the "all or nothing approach" and to introduce scalability. Under DLI 3, the payment framework needed to exclude the 28-day time frame for RBF payments. Under DLI 4, there was a need to revise and simplify the payment calculation formula to reward health facilities that maintained high performance. The results framework and its targets required revisions, and there was a need to extend the operation’s closing date to align DLI payments with a full annual implementation cycle. In this context, the ICR (p. 31) reported a noticeable shortcoming related to Program restructuring that was delayed by about 18 months, partly due to lengthy internal government discussions: the ICR appropriately argued that an earlier downgrading of the operation's ratings would have helped to convey a strong message on Program challenges, and on the need for urgent action to resolve them. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design Program objectives were clearly specified. Quality improvement was inbuilt in several DLIs and was directly reflected by intermediate results in various Results Areas. Most PDO outcomes under the national program reflected utilization that is likely to be associated with improved quality. Indicators were measurable, and baselines were available. The contribution of DLIs and DLRs toward the attainment of intended outcomes was captured by the theory of change. M&E arrangements were well embedded institutionally, as the operation adopted the government’s own indicators and used national routine information systems, namely: (i) the District Health Information System that provided information on service delivery in PHC facilities; (ii) Logistics Management Information System that provided information on the supply chain for health commodities; (iii) Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) Human Resource Health Information System for staffing information; (iv) planning and reporting that provided information on council health plans and related implementation progress; and (v) the Integrated Financial Management Information System that included LGA financial management information. The first three information systems would be maintained by MOHCDGEC, while the fourth and fifth systems would be maintained by PMORALG and the Ministry of Finance and Planning, respectively. b. M&E Implementation M&E implementation proceeded as planned, and disbursements were made upon the presentation and verification of evidence on the attainment of Program DLIs. The Program revised the results framework and two DLIs under a Level 2 restructuring on January 16, 2020. Targets for four outcome indicators were revised upward, and the target for the fifth PDO-level indicator (PHC facilities accredited with 3-Star Ratings and above) was reduced from 50 to 30 percent of facilities to reflect realistic expectations. DLI and DLR adjustments were discussed in section 7b. Targets for 11 intermediate results indicators were increased, two were reduced, and eight others were maintained. c. M&E Utilization According to the ICR, DLIs contributed to holding stakeholders accountable, building a culture focusing on results and targets, reviewing progress regularly, identifying lagging indicators, and improving performance (ICR, p. 29). This pattern was well noted at the LGA level, where Program-related discussions gradually evolved from being centered on facility refurbishment to planning and budgeting of MNCH and nutrition activities and overall accountability. M&E findings helped in detecting bottlenecks; for example, the nutrition team used M&E data to identify low-performing councils. According to the ICR (p. 29), the PforR framework was used by Canadian authorities to tie their assistance to results; it contributed to further strengthening information systems in the health sector, and helped in enhancing coordination of donor assistance to the health sector. The ICR (p. 30) also noted that this Program was the first such operation in the health sector in the Africa region, and that it served as a precedent for other health operations in the region, such as in Benin and Ghana. M&E Quality Rating Substantial 10. Other Issues a. Safeguards An Environmental and Social Management System Assessment (ESSA) was undertaken to ensure consistency with core principles outlined in the World Bank’s Operational Policy 9.00, Program-for-Results Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) Financing (PAD, p. 32), with a focus on two areas: health care waste management and social accountability. The PforR Program used the Environmental and Social Management Framework and a Healthcare Waste Management Plan prepared under previous Bank-assisted projects. The ICR noted that prior to the Mid-Term Review of May 2018, related implementation was slow, but subsequently improved. The ICR reported that performance was rated moderately satisfactory and that the inclusion of environmental and social measures during annual verifications was noted to have contributed to strengthening the monitoring of ESSA implementation. The ICR (p. 29) also noted that, for future operations, it would be important to institutionalize the implementation of ESSA at an early stage. b. Fiduciary Compliance The ICR (p. 29) reported that the overall assessment of the Program’s fiduciary systems was rated moderately satisfactory. The Program maintained appropriate financial management arrangements, including staffing, and was in compliance with legal covenants. Interim unaudited financial reports and external audit reports were submitted in a timely manner and found acceptable to the Bank. Initially, fiduciary capacity was quite weak at the level of LGAs and health facilities. Capacities improved with training, application of necessary guidelines, recruitment of 500 accountants by LGAs, and the use of the electronic financial management reporting system countrywide. During the last two years of implementation, RBF payments to health facilities experienced considerable delays that resulted in unutilized funds (see section 2e). The annual procurement audits commenced in the third year of implementation, and a "value for money audit" was conducted in the final year of implementation (ICR, p. 30). Audit findings helped in strengthening fiduciary capacity, notably for procurement functions at the peripheral level. c. Unintended impacts (Positive or Negative) The ICR (p. 24) reported on the unanticipated negative impacts of COVID-19 pandemic. The actual extent of the impact of COVID-19 on the health sector was unknown because of the lack of official data, but anecdotal evidence suggested that the pandemic disrupted both access to essential health services and their provision, especially for reproductive, maternal and child health care. The ICR (p. 25) noted that there was macroeconomic stability until 2020, when Tanzania’s economy contracted sharply due to the pandemic, with GDP growth dropping to about 2 percent. Although the government did not impose stringent mobility restrictions, the pandemic prompted firms and consumers to adopt precautionary behaviors that resulted in an understandable slow-down of domestic economic activity. d. Other -- Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome --- Satisfactory Satisfactory Moderately Moderately Bank Performance --- Satisfactory Satisfactory Quality of M&E Substantial Substantial --- Quality of ICR High --- 12. Lessons The ICR (pp. 31-33) offered several lessons and recommendations, and the following lessons were partially re- stated by IEG Review: The PforR operation fostered an increased focus on results and coordination between government agencies and development partners. The operation motivated stakeholders to focus on results and facilitated the coordination of donor assistance through its results-oriented framework. There are tradeoffs between a program's scope and its sustainability. The demands imposed on the system by an ambitious Star Rating accreditation initiative in terms of technical capacity and financial resources proved to be difficult to achieve and sustain, indicating that a less resource-intensive approach may be more appropriate in the future. Nutrition support can be effectively integrated into broader maternal and child health care development efforts. In the past, support to nutrition services in Tanzania tended to be compartmentalized. Under the PforR operation, the nutrition agenda benefited from (i) a strengthened primary health care system that is essential for the delivery of nutrition interventions; and from (ii) improvements in maternal and child care that synergistically contribute to progress in the nutrition agenda. Financial sustainability of results-based financing is vulnerable to changes in government commitment to the payment of financial incentives to health workers. The lack of reliable financial flows to RBF, exacerbated by the turnover of senior government officials, slowed down Program implementation during the last years of the operation and negatively impacted Program sustainability and the trust of stakeholders who were previously motivated by noteworthy RBF results that were observed during the preceding years of the Program. 13. Assessment Recommended? No 14. Comments on Quality of ICR Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TZ-Strengthening PHC for Results (FY15) (P152736) The ICR was clearly and tightly written. The theory of change in the context of the national PHC program was well illustrated, and the ICR linked its narrative with the evidence and ratings. The ICR was results oriented and its analysis was thorough. The ICR’s discussion of the contributing role of DLIs to improve the quality of primary health care services was noteworthy. The evidence presented on intended improvements was adequate. The ICR provided a complete and candid critique of the PforR Program, supported by a clear narrative on how the PforR instrument contributed to improved PHC results. The ICR followed guidelines and was internally consistent. The lessons were derived from Program experience. The ICR had no visible shortcomings other than a minor lapse in fully explaining the variable government commitment to the payment of RBF financial incentives to health workers, and that was promptly addressed by the Task Team. a. Quality of ICR Rating High Page 15 of 15