Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792) Report Number: ICRR0022477 1. Project Data Project ID Project Name P145792 GH-Maternal, Child Hlth & Nutr. (FY14) Country Practice Area(Lead) Ghana Health, Nutrition & Population L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-54630,TF-16617 30-Jun-2020 63,538,642.37 Bank Approval Date Closing Date (Actual) 20-May-2014 30-Jun-2020 IBRD/IDA (USD) Grants (USD) Original Commitment 68,000,000.00 5,000,000.00 Revised Commitment 70,407,203.16 2,500,000.00 Actual 63,538,642.37 2,500,000.00 Prepared by Reviewed by ICR Review Coordinator Group Judith Hahn Gaubatz Salim J. Habayeb Joy Maria Behrens IEGHC (Unit 2) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Financing Agreement (page 5) and the Project Appraisal Document (PAD, page 9), the project objective was as follows: To improve utilization of community-based health and nutrition services by women of reproductive age, especially pregnant women, and children under the age of two years. Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792) b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components 1. Community-Based Maternal and Child Health and Nutrition Interventions (Appraisal: US$ 63.0 million; Actual: US$ 52.1 million) This component aimed to strengthen the supply of maternal and child health services, create demand for such services, and increase ownership and accountability for delivery of services among stakeholders. Activities included: support to community authorities to plan and implement maternal and child health and nutrition activities through the provision of district-level grants (to be used for registration of pregnant women, antenatal care and delivery package, counseling, management of childhood illness at home, and mobilization of community members for growth monitoring activities); capacity building support to central, regional and district authorities to plan and supervise maternal and child health and nutrition activities; training for community health officers; procurement of critical commodities (including weighing scales, motorbikes, bednets, vitamin A capsules, and oral rehydration salts); and piloting of a community performance-based financing program, which would finance a fee-for-services approach according to achievement of maternal health indicators. 2. Institutional Strengthening, Capacity Building, Monitoring and Evaluation, and Project Management (Appraisal: US$ 8.0 million; Actual: US$ 6.3 million) This component aimed to develop more effective inter- sectoral coordination, ownership, and accountability for health and nutrition; strengthen Ministry of Health (MOH) capacity; and evaluate the impact of the project. Activities included: technical assistance on project implementation; updating of health and nutrition strategies and policies; tools for quality improvement; knowledge sharing; implementation of community-based monitoring tools; and project impact evaluation. 3. Unallocated (Appraisal: US$ 2.0 million; Actual: US$ 5.2 million) This component provided additional funds to ensure flexibility of the project activities. At the time of project restructuring in July 2015, Component 3 was named "Epidemic Preparedness and Control" in response to the Ebola outbreak in the region, in order to enhance the government's public health emergency response capacity. Activities included: establishment of treatment facilities in each region, training for health personnel and development of standard operating procedures for Ebola response, establishment of the Ghana Center for Disease Control and Prevention, and renovation of the isolation unit at the hospital in Accra. At the time of restructuring in December 2018, the scope of Component 1 was expanded as follows: coverage of facilities expanded to health centers, private maternity homes and physician assistant led private clinics credentialed by the National Health Insurance Authority (NHIA); and the package of services expanded to complete antenatal care and delivery. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792)  The appraised project cost was US$ 73.0 million. The actual project cost was US$ 63.5 million. The lower than appraised project cost was due to a partial cancellation in the Health Results Innovation Trust Fund (HRITF) grant from US$5.0 million to US$ 2.5 million, as well as an exchange rate loss of US$ 6.88 million during the project period.  In response to the Ebola virus outbreak, Component 3 (original allocation of US$2.0 million) was provided with an additional reallocation of US$1.5 million from Component 2. Financing  The original project cost was US$73 million consisting of an IDA Credit of US$ 68.0 million, of which US$ 61.0 million disbursed, and a Health Results Innovation Trust Fund (HRITF) grant of US$ 5.0 million, of which US$ 2.5 million disbursed, thus aggregating at a total actual cost of US$63.5 million.  The HRITF grant was reduced from US$ 5.0 million to US$ 2.5 million due to a shortage in HRITF funds.  There was no planned direct Borrower contribution. Dates  July 29, 2015: The project was restructured to reallocate project funds. A third component (originally "Unallocated") was added on epidemic preparedness and control, in response to the Ebola outbreak in the region, and an intermediate results indicator for this component was added to the results framework. The unallocated amount of US$ 2.0 million and US$ 1.5 million from Component 2 were assigned to the new component.  December 30, 2015: The project was restructured to amend expenditure categories to give distinct responsibilities to the Ministry of Health (MOH) and the Ghana Health Service (GHS) for project implementation, with the aim of improving institutional coordination and flow of funds.  December 17, 2018: Following the Mid-term Review in March 2018, the project was restructured to expand the project scope to health centers and maternity homes (in addition to the Community- based Health Planning and Services or CHPS zones) and to offer a more comprehensive package of services. The National Health Insurance Agency (NHIA) was also added as an implementing agency to address financial bottlenecks, and the fund disbursement modality of the performance- based pilot was more clearly aligned with the country's service payment mechanism under the NHIA to avoid double payments. Lastly, the results framework was also revised to better capture results at the primary health center level (see Section 9).  June 30, 2020: The project closed as planned. 3. Relevance of Objectives Rationale At the time of project appraisal, Ghana was experiencing rapid economic growth, leading to an increase in per capita income and a decrease in the number of people living below the poverty line. However, recent global and domestic macroeconomic instability has challenged these gains and contributed to significant disparities in access to economic, social and political opportunities. Similarly, while health outcomes had Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792) seen steady improvement over the last two decades - with declines in total fertility, under-5 child mortality, and maternal mortality - these outcomes were still worse than other countries at similar socio-economic levels. In particular, there were large disparities in health outcomes and in access to basic services according to geographic location (as reported in the PAD, drawn from Demographic and Health Survey data). The government had introduced the Community-based Health Planning and Services (CHPS) Program in 1994, which aimed to reduce barriers to access and disparities by bringing essential health and nutrition services closer to patients - the designated "CHPS zones" represented the lowest tier of the Ghana Health Services delivery structure. The project objectives aimed to support this program by improving utilization of essential services at the community level, thus bolstering the country's primary health care system and strengthen the health system as a whole. The project also supported the National Health Insurance Scheme (NHIS), which aimed to improve affordability, and thus utilization, of services among the poor and vulnerable (for example, pregnant women were eligible for free maternal care at all NHIS-accredited facilities). The objectives were fully relevant to the country's Medium-term Health Sector Development Plan for 2014- 17, in place at the time of appraisal, which aimed to contribute to socio-economic development by promoting health and ensuring access to quality health and nutrition services. Similarly, the Bank's Country Partnership Strategy (CPS FY13-18) at the time of appraisal, identified protection of the poor and vulnerable as one of three key pillars, specifically through improving access to health services, with improved maternal health (as measured by births attended by skilled health personnel) as a key outcome. The new Bank Country Partnership Framework (CPF FY21-26) is still under preparation, and the ICR (page 12) reported that the draft CPF includes a sub-objective to improve the utilization of health and nutrition services by the poor and vulnerable. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To improve utilization of community-based health and nutrition services by women of reproductive age, especially pregnant women, and children under the age of two years. Rationale The theory of change was clear overall, with interventions intended to increase availability, accessibility and affordability of essential health and nutrition services, while also increasing demand for services and improving fund flows from central to district entities to finance and deliver those services. The project aimed to expand geographical coverage of essential services (antenatal care and delivery, counseling of new mothers, management of childhood illness at home, growth monitoring), while also providing training to Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792) service providers, essential drugs and equipment, upgraded health facilities, and monitoring systems. These interventions were to be selected according to district health plans and disbursed directly to community health teams. Capacity building interventions focused on strengthening health systems to improve the quality and quantity of services provided, including by improving uptake and accountability, as well as improving access and affordability through the national health insurance program. These project outputs were likely to lead to the intended outcome to increase utilization of health services, while the health and nutrition interventions would also plausibly be expected to contribute in the long run to reducing maternal and child undernutrition, morbidity, and mortality. The project supported primary health care (PHC) facilities in all 216 districts nationwide. The community performance-based financing (cPBF) pilot initially targeted remote and underserved areas (75 CHPS zones in the most vulnerable regions) in order to improve equity in access to services. The project subsequently expanded its cPBF targets from eight districts in four low-performing regions to 43 districts over the five years of project implementation. Outputs Community-based service delivery  The project provided grants to Community Health Teams (CHTs) to support day-to-day operations of health facilities, including operating costs, procurement of basic equipment and essential commodities (followed by an analysis of consolidated district procurement plans), community outreach, and payment of performance bonuses. Support to the CHTs also included management meetings and workshops for annual planning and budgeting and fiduciary training. The project team clarified (3/25/2021) that project financing supported the implementation of prioritized activities that directly contributed to the project objectives, including community outreach, household visits, training, and supervision.  Training was provided to 114,556 community health personnel in both an in-service and field module. The Community Health Officer, a trained health professional, was provided with the required training in clinical skills to provide first-line health care at the community level. CHPS zones have been effective in last mile outreach through home visits and community outreach by community health officers and volunteers. However, according to the ICR (page 22-23), there remained significant gaps in human resources at the service delivery level. Only about 10 percent of CHPS facilities met the standard of three staff members. 55 percent of CHPS zones had not regularly trained active Community Health Volunteers (CHVs). The retention of trained staff and maintenance of motivation among CHVs also remained a challenge in human resources at the community level.  Supervision visits to health facilities and CHPS zones by district level supervisors were undertaken. According to the project team, the standard operating procedure of the Ghana Health Service was to conduct twenty visits over the project period (four times over five years) to health facilities and sixty visits (twelve times over five years) to CHPS zones. The project funded 14 out of the 20 health facility visits, with the remaining self-funded by the district health teams, and 42 out of the 60 CHPS zone visits, with the remaining self-funded by the sub-district health teams. Institutional capacity  Piloting of community performance-based financing mechanism. Under the cPBF pilot, 69 CHPS zones received base funding (to restock basic equipment and commodities and refurbish health Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792) facilities) while another 75 CHPS zones received base funding as well as incentive payments upon achieving targets in eight maternal and child health and nutrition indicators (number of pregnant women registered for service delivery by CHTs; children under 5 years old registered for service delivery by CHTs; number of pregnant women making the first antenatal visit in the first trimester; number of pregnant women receiving at least four quality antenatal visits; number of pregnant women referred to a health facility for skilled delivery; number of mothers receiving at least a quality postnatal visit by the end of the seventh day after delivery; number of newborns receiving at least a quality postnatal visit by the end of the seventh day after delivery; number of children under two years old monitored continuously for the last three months). Both "control" and "treatment" CHPS zones were in the same targeted districts in order to compare outcomes in improving equity in access (see below for results).  Roll-out of electronic NHIS claims software in 268 facilities, as well as provisional credentialing of 1,700 CHPS (out of 2,500 CHPS) which enabled the CHPS to submit claims to NHIS directly. The ICR (page 19) reported that this effectively resolved one of the biggest bottlenecks of financing to health facilities, although the scheme resource gap remained over 6 months in debt to service providers.  Training was provided to 20 headquarters and regional finance officers in financial management and 980 subnational financial managers, with training of 65 district and hospital directors in management.  Support was provided for the development of a new national policy for PHC package. Emergency preparedness  Furnishing of the largest intensive care unit in the country (Military Hospital in Accra).  Strengthening of the Emergency Operation Center and integrated disease surveillance and response system at GHS.  Construction and equipping of the Ghana Center for Disease Control. Outcomes Utilization of community-based services  140 million people, cumulatively, utilized the package of essential health and nutrition services during the project period. Of these, 40.8% accessed the services at the PHC level.  The cumulative number of home visits reached 6.0 million by project end, with a steady annual increase from 529,547 in 2014 to 1,667,102 home visits in 2019.  The cumulative number of births attended by skilled health personnel was 3,328,828 by December 2019, surpassing the original target of 1,925,000 as well as the revised target of 2,100,500 attended births. Of these, 32.14% took place at the PHC level, achieving the target of 30.0%. The 2017 Ghana Maternal Health Survey findings supported this trend, showing that home deliveries decreased from 45% in 2007 to 20% in 2017.  The cumulative number of new female acceptors (age 15-49 years) of modern family planning methods was 2,577,617, surpassing the original target of 1,400,000 and achieving the revised target Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792) of 2,405,810 new female acceptors. This increase was also reflected in a steadily increasing annual trend in the number of women from 2014 to 2019.  The cumulative number of women receiving postnatal care within 48 hours of delivery at the PHC level was 999,422, achieving the original target of 932,180 and the revised target: 947,500 women. This increase was also reflected in a steadily increasing annual trend from 2014 to 2019. Of these, 35.1% took place at the PHC level, achieving the target of 33%. The ICR (page 15) attributed this outcome to the proximity of PHC facilities to women in communities, as well as to extensive home visits by CHTs.  The cumulative number of children under two years old who received at least one counseling visit per month was 1,152,336, surpassing the target of 875,000 children. This increase was also reflected in a steadily increasing annual trend, with 195,704 children in 2016 to 392,461 children in 2019 (no data for 2014 and 2015).  The number of children receiving Measles 2 vaccinations increased annually from 2014 to 2019, with the share of children receiving them through PHC facilities also increasing from 69.0% in 2014 to 80.9% in 2019. The national dropout rate for the second dose decreased from 24.1% in 2014 to 11.5% in 2019, which the ICR (page 16) attributed to extensive community outreach by the CHTs.  The cumulative number of NHIS clients (insured patients) utilizing essential health and nutrition services at the PHC level was 40 million people by project end, although the annual number decreased for the initial project period and only increased during the last two years of the project period, which, according to the ICR (page 13) "coincided with the period after the MTR when the project expanded its support for the NHIA and financial flows to PHC facilities." The ICR also suggested that the use of insurance "protected the poor from further impoverishment due to ill health and related health expenditures" and that "these results were directly attributable to the project because the resources from [the project] were used to directly pay for services provided by the CHPS zones, maternity homes, and health centers through NHIS."  The UHC Service Coverage Index Score, which measures coverage of essential health services that includes not only reproductive, maternal and newborn and child health, but also infectious diseases, non-communicable diseases, and service capacity and access. The Index score for Ghana improved from 45.8 in 2015 to 47 in 2019, ahead of other sub-Saharan Africa and lower-middle income countries. Community performance-based financing  While CHPS facilities in both the control and treatment groups (the latter receiving incentive payments upon achievement of indicators) showed "incremental progress" in all of the eight indicators, "those in treatment group made far more significant improvement than those in control group." Capacity building impacts included enhanced motivation for service providers, enhanced autonomy on the use of resources, increased community participation and ownership. However, no specific data on these capacity building impacts were provided in the ICR.  A client satisfaction survey showed high level of overall satisfaction (80%) on the quality of care in 2018-2019, measured by indicators on staff attitudes, waiting times, out-of-pocket expenditures, satisfaction with treatment received, and likelihood of revisits. However,  The cumulative number of pregnant women making the first antenatal visit before the fourth month of pregnancy was 1,226,074, achieving the target of 1,132,500, but the annual trend in the number of Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792) antenatal showed very slight increases from 2014 to 2018, with a significant jump in 2019. Of these, 45.0% took place at the PHC level, falling short of the original target of 55.0% and the revised target of 52.0%.  Quality of services as reflected in availability of essential commodities at the PHC level had shortcomings. The ICR (page 51) also noted some implementation inefficiencies due to constant shortage of essential drugs, medical supplies and equipment at health facilities due to disruption or gradual reduction of external support, triggering a significant commodity gap. There was no Vitamin A and iron folate supplements for children and adolescent girls in stock (ICR, page 23). In an assessment of availability of equipment, infrastructure, and human resources, less than 30 percent of facilities attained the minimum mark at the primary level and as much as 74 percent did not have the full complement of equipment to work with (ICR, page 27).  Also, as noted above in the outputs section, there were significant gaps in human resources at the service delivery level. Only about 10 percent of CHPS facilities met the standard of three staff members, and the retention of trained staff and sustaining motivation among CHVs remained a challenge. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale Achievement is rated Substantial due to evidence of increased utilization of essential health services among both pregnant women and children under age two, with moderate shortcomings related to antenatal visits and quality aspects. Overall Efficacy Rating Substantial 5. Efficiency The project design prioritized several maternal and child health interventions that have been identified in international academic literature as cost-effective and with high impact. These include home visits by community health volunteers, growth monitoring, antenatal care, and Vitamin A supplementation. An economic analysis was prepared for the project at appraisal (PAD, Annex 6 ), which considered the direct benefits of reduced maternal Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792) and child mortality versus the project costs. The analysis estimated 32,000 lives saved, or US$ 171.0 million in monetary terms, based on the estimated reduction in maternal mortality and child mortality rates by the end of the project. Given the expected project cost of US$ 73 million, the benefit-cost ratio would thus be 2.3. An updated economic analysis was prepared at completion (ICR, Annex 4). Project benefits were estimated as 2.4 million disability-adjusted life years averted, amounting to a monetary value of US$ 5.14 billion. Actual project costs, adjusted for deflation and discounted at 3.0%, were calculated as US$ 49.6 million. Thus, the benefit cost ratio was calculated as 103:1 and the cost per DALY averted was US$ 20.33. With a GDP per capita (in 2015) of US$ 1,766 - indicating a relatively high DALY value in Ghana - the cost per DALY averted appeared to be cost-effective. A sensitivity analysis, based on changes in the discount rate, also showed that estimated benefits exceeded costs even under the most conservative assumptions (a discount rate of 3% resulted in a cost per DALY of US$ 20.33, while a discount rate of 20% led to a cost per DALY of US$ 14.61). With regards to operational efficiency, there were initial implementation delays and slow disbursement levels in the first two years of the project. In particular, the innovative cPBF pilot activity experienced significant delays due to weaknesses in implementation arrangements (i.e., multiple layers of implementing units). However, following the mid-term review, the project was able to implement all planned activities within the original project period. There were also some efficiency gains due to efficient procurement although there were exchange rate losses of US$ 6.88 million during the project period. Funds were reallocated to support additional activities due to unforeseen events, though not at the expense of achieving original objectives and targets. The ICR (pages 22-23, 51) noted some implementation inefficiencies due to frequent changes in counterpart leadership, low retention of trained staff, and a constant shortage of essential drugs, medical supplies and equipment at health facilities due to disruption or reduction of external support. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of the objective is rated High due to strong alignment with country conditions, and country and Bank strategies. Efficacy is rated Substantial due to evidence of increased utilization of essential maternal and child Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792) health and nutrition services by pregnant women and children under two but with some moderate shortcomings. Efficiency is rated Substantial due to positive values in benefit cost ratios and cost per DALYs averted, all within the original project time period, but with moderate inefficiencies in implementation. a. Outcome Rating Satisfactory 7. Risk to Development Outcome Risks include financial uncertainty and the impact of the COVID-19 crisis on the health sector. With Ghana's transition to lower middle-income status, a number of development partners began decreasing support. However, the ICR noted that as the government still heavily relied on external support, this triggered a significant commodity gap, for example, there was no Vitamin A and Iron Folate for children and adolescent girls in stock. Also, the projected financial needs for CHPS are US$ 9-10 million per year, but the government was only expected to increase its on-budget expenditures for CHPS from US$3 million in 2018 to US$5 million by 2025 (ICR page 23). In addition, the COVID-19 pandemic severely hit Ghana, negatively impacting an already strained fiscal space. At the same time, the project contributed to substantial institutional strengthening including improved dialogue among MOH, Ministry of Finance, Ghana Health Services, and the NHIA; strengthening of decentralized service providers; improved financial flows; and improved leadership capacity at the district and sub-district levels. 8. Assessment of Bank Performance a. Quality-at-Entry The project had high strategic relevance given the country conditions and the government's health sector priorities. It built upon prior Bank work in Ghana's health sector, particularly the Nutrition and Malaria Control for Child Survival project (P105092), which supported an initial scale-up of the essential health and nutrition services for pregnant women and children through the CHPS program. The project was clearly aligned with the existing flagship CHPS program, in terms of its design, institutional arrangements, as well as the data collection systems, and also included significant poverty and gender elements. However, some aspects of the original implementation arrangements needed adjusting through project restructuring (assigning of distinct responsibilities to MOH and GHS, addition of NHIA to address financial bottlenecks) and the multiplicity of stakeholders for the cPBF pilot required a series of consultation and agreements, which contributed to implementation delays. The risk assessment rated the project as moderate, primarily due to the introduction of the innovative cPBF mechanism. While data collection arrangements were robust, there were some shortcomings in the results framework: some indicators such as the coverage of exclusive breastfeeding and the diversity of young child nutrition required population surveys and proved difficult to report quarterly; there was lack of clarity on tracking cumulative vs. annual figures, and on disaggregating data at the PHC level. Lastly, the classification of the project as Environmental Category "C" was inconsistent with the triggered Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792) Environmental Assessment safeguard policy (OP 4.01) due to medical waste (see PAD (p. 29) and ISDS (No. ISDSA5374, August 21, 2013, p. 3). Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision Bank supervision was proactive and focused on development impact, enabled through consistent conducting of implementation support missions, candid supervision ratings reports, and the presence of a country-based task team leader. While the overall policy environment remained strongly supportive for the project objectives, there were frequent changes in senior management in government (four Ministers of Health, four Chief Directors of MOH, four Chief Executives of NHIA, and three Director Generals of GHS). The Mid-Term Review was well-utilized as an opportunity to address implementation bottlenecks and refine the results framework to better attribute outcomes to the project. The task team was proactive in working with the government in responding to the Ebola and COVID crises. Lastly, the Bank team provided effective hands-on guidance and training to ensure satisfactory fiduciary management. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The M&E design built upon the existing GHS routine service data collection system generated from all districts and sub-districts, including service registers at all service delivery points. The M&E design also included evaluative activities such as an impact evaluation of the cPBF pilot and client satisfaction surveys. However, there were some shortcomings in the results framework: some indicators such as the coverage of exclusive breastfeeding and the diversity of young child nutrition required population surveys and proved difficult to report quarterly; there was lack of clarity on tracking cumulative vs. annual figures and on disaggregating data at the PHC level (a subsequent project restructuring noted that revisions to the results framework enabled monitoring of changes at the community level, which was more relevant to the PDO, rather than broad national level changes). Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792) b. M&E Implementation Shortcomings in the results framework were addressed during the project period, as indicators were revised on several occasions to improve tracking of progress at different levels. For example, two PDO indicators on skilled birth attendance and first ANC visit were disaggregated to monitor progress of PHC service delivery at the three different levels (health centers, maternity homes and CHPS), an indicator on postnatal care was added, the indicator on child growth monitoring was revised to measure progress in community-based infant and young child feeding, and the indicator on exclusive breast feeding was dropped due to difficulties in data collection and monitoring. The revisions also aimed to strengthen attribution of outcomes to the project. The MOH and GHS consistently reported data and the project implementation unit was adequately staffed with full-time M&E experts. The Bank team provided extensive support in the analysis of these routine data to inform decisions of the MOH and GHS officials for course correction at the decentralized level. However, the ICR (page 25) noted that discontinuity of support from a development partner and the resultant shortage of the Maternal and Child Health Handbook and service registries hindered some intermediate indicators from being monitored in 2019, such as the number of supervision visits to CHPS zones. c. M&E Utilization The ICR (pages 24-25) reported that monitoring data were used on a frequent (quarterly) basis to inform project implementation decisions, including for course correction at the decentralized level (no specific examples reported in the ICR). It is notable that the project implementation unit was staffed with full- time M&E experts and that the Bank team provided strong support in the analysis and use of project monitoring data. M&E Quality Rating Substantial 10. Other Issues a. Safeguards According to the PAD, the project was classified as an Environmental Category "C" project; however, the safeguard policy on environmental assessment (OP/BP 4.01) was triggered due to the potential for medical waste. The central implementing agencies were considered to have strong capacity in managing medical waste and also experience in implementing Bank safeguard policies; however, district level implementing entities were still to be provided training in handling medical waste. The ICR (page 25) reported that the project was in compliance with medical waste management requirements and no environmental issues arose during the project period. b. Fiduciary Compliance Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792) Fiduciary roles and responsibilities were refined and clarified through project restructurings, including establishing separate Designated Accounts to improve transparency and accountability, clarifying lines of reporting between MOH and GHS and fund flows from national to district levels. These actions all contributed to satisfactory fiduciary compliance. Financial management: Interim financial reports and audit reports were submitted on time. 90% of audit recommendations were fulfilled. The ICR did not report on whether there were any qualifications to audit reports, though the project team subsequently confirmed there were none. Procurement: Procurement performance improved during the project period, with significant hands-on support from the Bank team, including in the quality of documentation and delivery. The ICR did not report on whether the project was in compliance with procurement guidelines nor whether there were any procurement irregularities, though the project team subsequently confirmed there were no irregularities. c. Unintended impacts (Positive or Negative) None noted. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Quality at Entry is rated Moderately Satisfactory due to shortcomings in implementation Bank Performance Satisfactory Moderately Satisfactory arrangements and M&E design, and inconsistent environmental classification. Shortcomings noted in M&E Quality of M&E High Substantial Design and Implementation. Quality of ICR --- Substantial 12. Lessons Lessons drawn from the ICR (pages 28-29), adapted by IEG:  Holistic investment in the lowest tier of the health system can lead to increased service utilization of basic health services. In the case of this project, the project increased Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792) autonomy of community- and district-level decision making and improved fund flows from central to district agencies. These investments led to increased coverage of maternal and child health and nutrition services, although the impact on health outcomes (i.e. maternal mortality) remained unclear given that the project did not include interventions in the continuum of care above the primary health care level.  For a highly decentralized implementation approach, monitoring arrangements that are robust and well-considered can ensure quality of data and thus contribute to achievement of outcomes. In the case of this project, the monitoring plan made use of existing electronic routine service data collection systems, along with manual service registries. In addition, the project implementation unit was staffed with full-time M&E experts and the Bank team provided extensive M&E support. These all contributed to the quality and timeliness of data reporting.  Flexibility in fund utilization can facilitate response to emergencies. In the case of this project, "unallocated" funds as well as reallocations from other components were well-utilized to aid the government's response to the Ebola outbreak and the COVID pandemic, without jeopardizing achievement of project objectives. 13. Assessment Recommended? Yes ASSESSMENT_TABLE Please Explain To verify whether service utilization outcomes and institutional capacity gains have been sustained, particularly in light of COVID. 14. Comments on Quality of ICR The ICR was well-organized and results-oriented. The theory of change was clear for the specific and succinct project objective to increase utilization for basic maternal and child health services. The quality of data and evidence was substantial. Aside from thorough reporting on indicators at project end, the ICR also showed annual trends to confirm achievements, triangulated data from external sources (Ghana Maternal Health Survey), and reported results from the cPBF approach. The ICR was overall consistent with guidelines, although it did not report complete information on fiduciary compliance such as audit results. a. Quality of ICR Rating Substantial Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review GH-Maternal, Child Hlth & Nutr. (FY14) (P145792) Page 15 of 15