Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721) Report Number: ICRR0024456 1. Project Data Project ID Project Name P163721 Sri Lanka: PSSP Country Practice Area(Lead) Sri Lanka Health, Nutrition & Population L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) 197,572,462.59 Bank Approval Date Closing Date (Actual) 27-Jun-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 200,000,000.00 0.00 Revised Commitment 197,572,462.59 0.00 Actual 197,572,462.59 0.00 Prepared by Reviewed by ICR Review Coordinator Group Judith Hahn Gaubatz Denise A. Vaillancourt Susan Ann Caceres IEGHC (Unit 2) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Loan Agreement (page 4), and the Project Appraisal Document (PAD, page 17), the project objectives were as follows:  To increase the utilization and quality of primary health care services, with an emphasis on the detection and management of non-communicable diseases in high-risk population groups, in selected areas of the country. Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721) At the time of project restructuring in January 2021, the following project objective was added:  To provide immediate and effective response to an Eligible Crisis or Health Emergency. In May 2022, when the project was 49 percent disbursed, the composite outcome indicator for assessing the achievement of Objective 2 (establishment of the required capacities for providing comprehensive and quality care in the 550 targeted PMCIs) was relaxed. However, a split rating will not be undertaken because achievement of the original, more rigorous requirements and achievement of the relaxed requirements received the same rating. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? No c. Will a split evaluation be undertaken? No d. Components (Note: The amounts reported below are Bank contributions, not total project costs. The ICR (Annex 3) did not report figures on actual costs.) 1. Implementation of the PHC System Reorganization and Strengthening Strategies (Appraisal: US$ 185.0 million; Actual: n/a): This component aimed to support implementation of the primary health care (PHC) system reorganization and strategies to strengthen routine health sector planning and budget execution systems. The funds for this component were to be disbursed upon achievement of specified results (disbursement-linked indicators (DLIs). The five results areas were as follows: 1) Definition and endorsement of policies, guidelines, and standards for reorganizing the PHC system (including population empanelment and risk stratification processes, a comprehensive PHC service package, a PHC network, functionality and service standards for primary care facilities, requirements in terms of human resources, infrastructure, equipment, essential medicines, diagnostic capacity, and information system) and also clinical protocols for defined priority health conditions; 2) Strengthening of primary medical care institutions' (PMCIs) capabilities and services (for more comprehensive and patient-friendly care); 3) Implementation of a personal health record system and an improved national procurement and supply chain system; 4) Implementation of a community engagement mechanism to increase accountability of the health sector (including communication campaigns and "Friends of the Facility' committees); and 5) Utilization of PHC services focused on NCD detection and active follow-up (including screening for cervical cancer and high risk patients). The five results areas and relevant DLIs were as follows:  PHC policy and standards defined to support implementation o DLI 1: Endorsement by the MoH and by the provinces of policies and standards for reorganizing PHC system o DLI 2: MoH adopts and updates clinical protocols for selected health conditions . Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721)  PMCI capabilities and services strengthened for more comprehensive and quality care o DLI 3: PMCIs have required capabilities for providing comprehensive and quality care. o DLI 4: Enhanced patient-friendly services provided at PMCIs.  Support to personal health record system and improvement of the national procurement and supply chain system o DLI 5: Personal health records are used to coordinate patient care over time and through the referral chain. o DLI 6: Procurement and supply chain management efficiency is improved.  Development and implementation of a community engagement mechanism o DLI 7: Community engagement mechanisms for health sector operational, including grievance redressal mechanism and community committees at the PMCIs.  Utilization of PHC services, with a focus on NCD detection and active follow-up, is increased o DLI 8: Number of women at age 35 and at age 45 years who are screened for cervical cancer at a network of public health facilities. o DLI 9: Adults determined to have high risk for non-communicable diseases registered and actively followed up at PMCIs. 2. Project Implementation Support and Innovation Grants (Appraisal: US$ 14.2 million; Actual: n/a): This component aimed to provide direct support to the Ministry of Health and the provinces for project management, project monitoring, results verification, operations research, and capacity building. Activities also included a competitive small grant mechanism to support innovations in priority areas such as patient- friendly services, technology, civic engagement, and public-private partnerships. 3. Contingent Emergency Response Component (Appraisal: US$ 0; Restructuring: US$ 9.0 million; Actual: n/a): This component aimed to improve the country's capacity to respond to an emergency. In anticipation of such an event, the component would be activated and provide rapid reallocation of undisbursed funds from other operations to address immediate financing needs. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost  The project cost at appraisal was US$ 4,419.0 million. The ICR did not report actual project costs. Financing  The project was financed by an IBRD loan of US$200.0 million, of which US$197.6 million disbursed. These funds were to be used to reimburse the Borrower up to US$185.0 million of certain defined expenditures under the total project cost, based on the verified achievement of the disbursement-linked indicators. The remaining US$15.00 million of the IBRD loan was to support direct investment in activities of Component 2. Borrower contribution Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721)  The Borrower was expected to provide US$4,219.0 million in parallel counterpart financing. The ICR did not report actual counterpart contribution. Dates  June 27, 2018: Project approval.  January 23, 2019: Project effectiveness.  January 13, 2021: Project restructuring. Due to the onset of the COVID-19 pandemic, the project development objectives were revised to support an immediate response. US$ 9.0 million was reallocated from Component 2 to a common CERC pool created by the government. The following revisions were made to the results framework: a PDO indicator was added for the CERC component; the intermediate results indicator on nutrition counseling was dropped due to the decision to utilize existing PHC staff rather than recruit additional nutritionists; the intermediate results indicator on food safety was dropped due to the limited role of the MOH in this area.  May 30, 2022: Project restructuring. The definition and verification protocol for the DLI on PHC minimum capabilities (human resources and drug availability) was revised .  December 17, 2023: Project restructuring. The project closing date was extended from December 2023 to June 2024 due to COVID-19 delays.  June 30, 2024: Project closing. 3. Relevance of Objectives Rationale Sri Lanka is a lower middle-income country that has experienced rapid economic growth since the end of its civil war in 2009. The economy has been transitioning to a more urban-based one centering around manufacturing and services. Social indicators are among the highest in the region, having met most targets for the Millennium Development Goals. Coverage of maternal and child health (MCH) services is nearly universal, and communicable diseases such as tetanus and measles are nearly eliminated. The main health sector challenge remains to effectively address non-communicable diseases (NCDs), as the share of the population over 60 years old will nearly double from 2015 to 2040, comprising over one-fourth of the total population. NCDs account for 81 percent of total deaths and 77 percent of disability-adjusted life years (DALYs). Despite a strong MCH services base, there is limited capacity in the primary care system to provide comprehensive primary health care (PHC). This is due to challenges with supply shortages, human resource gaps, drug supply chain management, coordinated continuous care, and health information systems. The Ministry of Health developed a position paper to reorganize and strengthen the PHC system, identifying thematic areas of reorganizing PHC to meet Sri Lanka’s needs, improving information management to improve health services in real time, and strengthening the health sector through key system improvements. Components of this PHC system strengthening paper are also reflected in the National Health Policy (2016–2025). The Bank's Country Partnership Strategy for FY 2017-2020 identified the improvement of health systems to address the challenges of an aging population as a key objective, with the percentage of people over 40 Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721) years old screened for selected NCDs as a key indicator. The more recent Country Partnership Framework for FY 2024-2027 similarly identified the sustaining and strengthening of health systems as a key objective, with the number of primary medical care institutions with required capabilities as a key indicator. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To increase the utilization of primary health care services, with an emphasis on the detection and management of non-communicable diseases in high-risk population groups, in selected areas of the country. Rationale The theory of change was sound. The project intended to support both demand and supply side activities, including proactive outreach on NCD awareness, strengthened PHC capabilities, and strategic deployment and in-service training of human resources. Challenges to providing comprehensive primary health care services at the PCMI level to cover NCDs included: supply shortages (and inefficient distribution), human resource gaps in physicians trained in chronic NCD management, limited availability of laboratory services, drugs and equipment for NCD screening, lack of models for continuous and coordinated care, poor "consumer experience" at public health facilities; and absence of an electronic medical information system. The MOH’s strategy for reorganizing PHC was integrated in the Government’s health sector development plans but not defined as a separate or distinct Government program; therefore, the project was structured primarily as a results-based financing approach. Disbursement-linked indicators (DLIs) allowed for identifying the important benchmarks of progress in the implementation of the strategy and to release the funds to reimburse program expenditures. It is noted that specific attributability to the Bank's support is limited in terms of direct contribution to project outputs. Outputs The Bank supported the following areas:  Results Area 1/ DLI 1: Definition of PHC policy and standards for reorganization of the PHC system. These included: population empanelment and risk stratification processes to link all populations at risk to a particular PMCI ensuring that those PMCIs were responsible for screening, treatment, and follow- up services (including extensive ground verification alongside Global Positioning System (GPS) mapped secondary data), definition of a comprehensive PHC service package and a PHC network; definition of functionality and service standards for PMCIs; and identification of PMCI requirements in terms of HR, infrastructure, equipment, essential medicines, diagnostic capacity, and information Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721) systems. The project team confirmed that these actions were completed within the first year and a half of the implementation period.  Results Area 1/ DLI 2: Definition of protocols for preventing, detecting, and managing NCDs with a focus on hypertension, diabetes mellitus, and cervical cancer. These included: definition of the risk stratification thresholds based on population risk factors; screening guidelines that are both facility- and community-based; management and referral guidelines for NCDs at the PMCI level; and requirements for essential drug and diagnostic services. These actions were achieved by June 2019.  Results Area 2/ DLI 3: Strengthening of PMCI capabilities and services, originally defined as follows: having at least two trained medical officers and one nurse (public health) officer; having minimum set of operational diagnostic equipment and tools; having minimum availability of essential drugs and lab test capacity (on-site or through a networked/contracted pharmacy or laboratory); and undergoing quarterly supportive supervision facilitated by a checklist to ensure adherence to national quality and safety standards. The project team clarified that training was provided on essential health services in addition to NCD screening and treatment, and that the newly created cadre of public health nurses played a key role not just in providing services PCMIs but also reaching out to communities, with the help of the outreach committees, to ensure follow-up care was delivered. Minimum standards were relaxed under the May 2022 restructuring, as described under Objective 2 outcome reporting. Nevertheless, progress in moving toward improved quality of services contributed to the achievement of Objective 1 (increased utilization of services), though it fell short of outcome expectations under Objective 2 (increased quality of services).  Results Area 2/ DLI 4: Provision of at least four patient-friendlier services, including provision of NCD drugs through one-month supply blister packs, transportation arrangements with laboratories (including with private laboratories) where patients can drop off specimens and pick up results at the PMCI, pharmacy partnerships (including private pharmacies) where patients can pick up prescriptions for publicly-financed drugs free of charge, and an operational appointment system (web-based, phone, and other) where patients can register for specific appointment times. The ICR (page 19) indicated that this was achieved in all 550 PMCIs and that verification confirmed continuity of services.  Results Area 3/ DLI 6: Development of improved supply chain management system to ensure timely and efficient supply of drugs. 550 PMCIs were utilizing the Medical Supplies Management Information System (target: 550), which automated medical supply management. This system enabled real-time monitoring of drug shortages at PMCIs and facilitated inventory sharing between hospitals.  The number of PMCIs with lab access increased from 143 to 400 by project closing. However, as noted in the ICR (page 10), challenges persisted including unclear guidelines for sample transportation, leading to ad hoc transport methods, and a shortage of trained lab technicians.  Results Area 4/ DLI 7: Establishment of direct links between the PMCIs and the communities they serve through ‘Friends of the Facility’ committees and a Grievance Redress Mechanism (GRM). The newly created cadre of public health nurses played a key role in working with these committees to reach out to their communities, identify high risk individuals, and ensure follow-up care was delivered. However,  Results Area 3/ DLI 5: The key activity of an electronic health management information system, essential for coordinating patient care through personal health records and a web-based referral network, was not completed. Although the MoH rolled out an initial version, it lacked critical functionalities, such as fields for effective patient follow-up, focusing only on patient registration, screening, and risk stratification. Networking issues further complicated information management Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721) across facilities. Instead, patient records were maintained manually at healthy lifestyle centers but as noted in the ICR (page 5), this raised concerns about the quality and reliability of the data through a manual system. The parallel use of a manual and digital record system, as well as the development of a parallel Hospital and Health Information System, led to confusion. The lack of a unique ID system also posed challenges to implementing a more effective coordinated, continuous care model, although the project team clarified that the comprehensive services were still being pursued through the use of manual written records. Outcomes  DLI 8: The number of women at age 35 and at age 45 years who were screened for cervical cancer increased from 107,551 in December 2017 to 238,000 in June 2024. This surpassed the target of 213,000. As noted in the ICR (Annex 1), this baseline was established only for those aged 35, as the information system of the program at that time did not capture the data for the age 45 cohort. Also, while the target was achieved for screening of women at age 35, the lack of robust data systems prevented tracking of repeat screenings at age 45 (ICR, page 8). While the indicator was not specifically measuring repeat screenings (to reflect coordinated and continuous care and capacity for follow-up), evidence that this was in fact happening would have significantly strengthened this result.  DLI 9: The percentage of screened adults at high risk for non-communicable diseases who are registered and actively followed-up (defined as having been followed up within the last three months of the reporting period) at PMCIs reached 68% by project closing. This surpassed the target of 25%. As noted above, the reliability of data from manual records at healthy lifestyle centers was a concern. While the indicator was not specifically measuring the nature of the active follow-ups such as repeat screening or initiation of treatment (to reflect coordinated and continuous care and capacity for follow up), evidence that this was in fact happening would have significantly strengthened this result. Achievement is rated Substantial due to evidence of increased utilization of NCD services, including meeting of targets, although there are shortcomings noted in the robustness of the indicators and concerns about data quality due to reliance on manual records. Rating Substantial OBJECTIVE 2 Objective To increase the quality of primary health care services, with an emphasis on the detection and management of non-communicable diseases in high-risk population groups, in selected areas of the country. Rationale See Objective 1 above for discussion of the theory of change. Outputs In addition to the outputs and DLI achievements reported above for Objective 1: Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721)  65 percent of medical officers (both Medical Officers of Health in the Public Health stream and Medical Officers at PMCIs) received training on essential service package on NCDs (target: 70 percent).  55 percent of PMCIs had transportation capabilities for dispatching the samples for investigations to apex laboratories (target: 55 percent).  55 percent of PMCIs had capabilities for primary detection and management of dengue (target: 55 percent).  60 percent of PMCIs were providing a standard primary oral health care package (target: 55 percent).  69 percent of PMCIs had basic emergency care facilities (target: 50 percent).  90 percent of annual procurement by the Medical Services Department is quicker than or equal to defined performance benchmarks (target: 50 percent).  34 out of 40 planned capacity-building programs and 8 out of 11 innovation grants were completed. The ICR did not report more specific information on these outputs, but noted that only US$2.99 million out of the US$5.22 million allocated for these activities was disbursed due to insufficient budgetary allocations from the National Budget Department.  Regarding the percentage of Medical Officer of Health areas offering mental health services in at least one PCMI, the ICR (Annex 1) reported this indicator was achieved but provided no figures (target: 35%).  Regarding the percentage of PMCIs with capability for tuberculosis screening and referral, the ICR (Annex 1) reported this indicator was achieved but provided no figures (target: 45%).  The indicator of the percentage of Medical Officer of Health areas with at least one PMCI providing routine nutrition counseling services by a qualified nutritionist was dropped.  The indicator of the percentage of food business operators inspected and classified as satisfactory according to the healthy food standards was dropped. Outcomes  550 PMCIs have the required capacities for providing comprehensive and quality care, achieving the target of 550. This was originally defined as the PCMI meeting a minimum of four of the following five elements: (i) having at least 25 percent of the adult population in its defined empanelment area screened and stratified for risk factors, (ii) having at least two trained medical officers and one nurse (public health) officer, (iii) having minimum set of operational diagnostic equipment and tools, (iv) having minimum availability of essential drugs and lab test capacity (on-site or through a networked/ contracted pharmacy or laboratory) and (v) undergoing quarterly supportive supervision facilitated by a checklist to ensure adherence to national quality and safety standards. Under the May 2022 restructuring, when project funds were 49 percent disbursed, two of the elements were relaxed as follows: (ii) at least one medical officer and one Public Health Nurse, and iv) availability of prioritized, though not necessarily all, essential drugs. Achievement is rated Modest. As reported in the ICR (page 6), although the project supported quality improvements in the 550 targeted PCMIs, as planned, Objective 2 was partially achieved. There were shortfalls in meeting the original (more rigorous standards). There were also persistent challenges in meeting the more relaxed standards, including drug supply issues and the lack of full coverage of the electronic medical information system. Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721) Rating Modest OBJECTIVE 3 Objective To provide immediate and effective response to an Eligible Crisis or Health Emergency Rationale In June 2020, the government requested activation of this component in response to the COVID-19 pandemic, reallocating funds from Component 2 to a common CERC pool that was created to support the COVID response. The priority interventions were identified in the government's Emergency Action Plan to prevent further spread and support economic recovery, including agricultural seed distribution to strengthen food security, tele-education and e-learning facilities for schools, and technology equipment for government institutions to ensure continuity of operations. As noted in the restructuring documents (Project Paper, para. 11), given that the CERC funds from the four contributing projects were pooled into a single account, direct contribution by this specific project cannot be determined. Outputs  Tele-education broadcasts in all nine provinces in line with the school curriculum (target: nine).  High-speed Wi-fi connections, computers, smart boards, and TVs for 651 rural schools. Formula- based school grants were also provided directly to all 10,165 government schools to strengthen their capacity to implement remote learning.  The ICR reported that the indicator on the number of direct project beneficiaries reached with seeds, planting material, compensation, equipment, works and services for COVID-19 recovery was "satisfactorily met", but no figures are reported (target: 100,000).  The ICR reported that the indicator on the number of institutions benefiting from improved digital tools for secure and resilient continuity of operations was "satisfactorily met", but no figures are reported (target: 8). Outcomes  The CERC was implemented according to the Emergency Action Plan. Achievement is rated Modest due to insufficient evidence about outputs delivered by the project. Rating Modest OVERALL EFF TBL OBJ_TBL Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721) OVERALL EFFICACY Rationale Achievement of the first objective is rated Substantial due to evidence of increased utilization of NCD services, including meeting of targets, although there are shortcomings noted in the robustness of the indicators and concerns about data quality due to reliance on manual records. Achievement of the second objective is rated Modest, as the project met its target of 550 PCMIs meeting minimum capacities but with significantly relaxed standards with some shortfalls in meeting originally retained standards, as well as persistent challenges, including drug supply issues and the lack of full coverage of the electronic medical information system. Achievement is rated Modest due to insufficient evidence about outputs delivered by the project, namely for the contingency emergency response activities. Therefore, overall Efficacy is rated Modest. Overall Efficacy Rating Primary Reason Modest Insufficient evidence 5. Efficiency At appraisal (PAD, page 42), the economic analysis primarily focused on the economic rationale for investing in the prevention, screening, and early management of NCDs, as an effective approach to mitigate national health expenditures and adverse effects on household welfare. NCD-related disability was considered to have a significant negative impact on labor force participation and productivity. The project was expected to increase efficiency of the health sector by promoting prevention (rather than costly treatment after complications) and encouraging utilization of primary care facilities (rather than specialist-level hospitals). The incremental costs to the government of restructuring the PHC system was also considered to comprise only a small fraction of the MOH budget. The economic soundness of the NCD approach was validated by the Disease Control Priorities research, which identified early detection of NCD risk factors followed by prevention therapies as highly cost- effective. At completion (ICR, Annex 4), the surpassing of targets for the key project indicator of the proportion of adults aged 35 and older screened and followed up for NCDs indicated efficiency in achieving this project outcome, with over 77 thousand DALYs averted over five years, equal to approximately US$ 288.0 million. This calculation reflected interventions for hypertension only, not other NCDs. A specific economic rate of return could not be calculated as the total projects costs included not only the Bank investment but also significant government investment. An updated analysis using the actual government spending on health over the five years of implementation suggested a slightly higher share of incremental costs in the government budget than was anticipated. However, the overall share was still considered low at 4.91% and 3.47% for 2022 and 2023. Implementation was significantly disrupted by political-economic crisis (Easter bombing 2019) and the COVID- 19 pandemic, both of which severely impacted the tourism industry. There was also an economic crisis in 2022 which led to high inflation and depleted foreign reserves. As reported in the ICR (page 10), the crisis affected the broader health sector, impacting medical supplies, mobility, and service delivery. Also, as project focus shifted to responding to the COVID-19 crisis, inadequate fund allocation (from the National Budget Department) and lower prioritization of the investment activities such as grants and operations research (Component 2) resulted in shortfall in disbursement (about 40 percent of the allocated funds remaining undisbursed). Nevertheless, Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721) outcomes particularly under Objective 1, supported by the PBF design, were still substantially achieved, while Objectives 2 and 3 were modestly achieved. Overall, implementation challenges encountered are considered minor shortcomings. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of objectives is rated High due to strong alignment with country conditions and Bank and country strategies. Efficacy is rated Modest due to substantial achievement of the first objective, modest achievement of the second objective, and modest achievement of the third objective. Efficiency is rated Substantial due to the strong economic rationale for investing in NCDs prevention and control, documented in the economic analysis, and to the implementation of the PBF scheme, despite significant exogenous challenges encountered during implementation. Overall outcome is rated Moderately Unsatisfactory due to shortcomings in achievement of two out of three project objectives (including insufficient evidence). a. Outcome Rating Moderately Unsatisfactory 7. Risk to Development Outcome The project helped to establish necessary institutional frameworks and policies to provide more comprehensive primary health care in Sri Lanka, particularly for NCDs. These included guidelines and protocols for primary care level institutions. Government commitment to these new reforms appears strong, with only incremental increases in operational costs to the government budget allocation expected. Supply chain management and personal health record systems are also in place, although the electronic health information system, critical to continuous and coordinated care, was yet to be implemented at the time of ICR Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721) completion. A follow-on Bank operation, the Sri Lanka Primary Healthcare System Enhancing Project (P181564) will contribute to sustaining outcomes by expanding the focus on NCDs from screening and treatment to comprehensive management. 8. Assessment of Bank Performance a. Quality-at-Entry The project, while representing a small proportion financially of the government's overall PHC system reorganization efforts, provided a critical contribution through technical support. This was enabled in large part by the various analytic work supported by the Bank, which informed the government's framework for addressing the emerging NCD burden. Much of this work was initiated during project preparation, with technical working groups established by the MOH. and helped to facilitate a smooth project start. This was critical given the necessary sequencing of activities. The PBF design added significant value by laying out and rewarding critical steps for PHC sector reform. The monitoring and evaluation design and arrangements were comprehensive, given the importance of verifying results throughout the project implementation period. The overall risk rating was appropriately assessed as substantial, given political and governance, stakeholder, and technical design risks. Mitigation measures identified at approval, such as stakeholder assessments, the use of international and national technical experts for capacity building, were partially effective. Unforeseen events of the COVID-19 pandemic and severe economic crises adversely impacted project implementation and outcomes. Quality-at-Entry Rating Satisfactory b. Quality of supervision The Bank team was responsive to changing country circumstances, most notably activating the contingency emergency response component and adjusting implementation accordingly (for example, transitioning from in-person to virtual training, offering telemedicine services). However, project implementation was overall marked by the negative impact of these external challenges (COVID-19 pandemic and economic crises), as well as systemic country issues including insufficient resource allocation and prioritization. The project was restructured, with M&E design revised accordingly, in response to the changing needs. The key activity of operationalization of a unique ID-based information management system for health facilities fell short of full implementation, and while project supervision documents flagged this issue, there was a lack of clear documentation of actionable solutions to address this problem. The ICR (page 14) also noted that discussions with provincial directors revealed other areas that required more attention during the supervision missions, such as reallocating human resources from secondary to primary care and insufficient focus on NCD screening for men and the working population. The documentation appeared Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721) to lack full candor, as implementation progress is consistently rated satisfactory while key aspects such as component activities and fiduciary performance remained moderately satisfactory. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The monitoring and evaluation design and arrangements were comprehensive, given the importance of verifying results throughout the project implementation period. The list of indicators was extensive and adequately measured project achievement. However, there were shortcoming in design that had implications upon monitoring and evaluation implementation. As discussed in the ICR (page 11), the key project indicator on the minimum capabilities of PCMIs was a composite measure that was challenging to capture with precision. The multiple elements comprising the indicator were primarily focused on assessing input or structural quality rather than process quality, making it inadequate for effectively tracking the quality of health service delivery. The indicator for screened NCD adults for follow-up did not specifically measure the nature of the active follow-ups (weak indicator). The M&E arrangements, while straightforward in design, relied on health information systems that did not materialize and therefore could not provide the data as planned. In addition, there were no evaluative activities planned such as baseline or follow-up surveys to measure impact. b. M&E Implementation The results framework, including definitions and targets for indicators, was revised during implementation to address shortcomings in the design and respond to significant impact of the COVID-19 pandemic. The key project indicator measuring minimum capabilities of PCMIs was revised, essentially relaxing the definition and the verification protocols. The original element that each PMCI have at least two medical officers and one nurse officer was revised to one medical officer with a backup arrangement from the nearest hospital. Another original element to have minimum availability of essential drugs and lab capacity was revised to a prioritized list. Also, the critical activity of the digitization of health records at the facilities was not completed (i.e. used manual records), which affected the follow-up of patients registered for NCDs, and resulted in the reliance on manual reporting instead, which raised concerns of reliability. Two of the four indicators for CERC did not report actual outputs achieved to compare against target, only noting that they were "satisfactorily met." Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721) c. M&E Utilization As reported in the ICR (page 12), M&E data was used to support implementation decisions and assess project progress. There was no information on whether data was shared with beneficiaries. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as an Environmental Category "C" project so therefore no environmental assessment was required. No safeguard policies were triggered. b. Fiduciary Compliance Financial management: Financial management performance was rated in the satisfactory range throughout the project period, with minor shortcomings noted due to delays in the submission of the audit report for Component 1. The ICR (page 13) did not report on results of audits but reported that there were no major problems in financial management. Procurement: Procurement performance was rated in the satisfactory range throughout the project period, with challenges in contract modifications and management noted. The ICR did not report on any major problems in procurement, aside from lengthy information technology-related procurement processes. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Overall outcome is rated Moderately Outcome Satisfactory Moderately Unsatisfactory due Unsatisfactory to shortcomings in achievement Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721) of two out of three project objectives (including insufficient evidence). Moderately Bank Performance Moderately Satisfactory Satisfactory Shortcomings in M&E implementation included the key activity of an electronic health information system (including unique digital ID system), Quality of M&E Substantial Modest concerns about reliability of manual records, robustness of a composite quality indicator, insufficient reporting on CERC activities. Quality of ICR --- Modest 12. Lessons Lessons drawn from the ICR (pages 15-16), adapted by IEG:  Harnessing strong government commitment along with significant technical assistance from development partners can help move forward a reform agenda, putting critical steps in place to ensure a smooth project start. In the case of this project, the Ministry of Health was strongly committed to reorganization of the PHC system to address the emerging NCD burden, and with early technical assistance on PHC policies and guidelines, helped to implement critical sequencing of project activities. The project design which laid out the critical steps through results areas and disbursement-linked indicators to incentivize the completion of those steps were important elements.  Community engagement in health service delivery can play a significant part in accountability and relevance. In the case of this project, the Friends of the Facility committees played an important liaison role between the PCMI and the community, by identifying main health needs of the population, facilitating active follow up services following screening, and stepping in to sustain service delivery during crises. 13. Assessment Recommended? No 14. Comments on Quality of ICR While acknowledging that this operation was a small part of a large government reform program, thus posing a challenge to enumerate outputs specific to this operation, the ICR could have provided stronger evidence on Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Sri Lanka: PSSP (P163721) the extent of the project's achievements. There was very limited information on the contingency emergency response activities, even if reported as part of the pooled operations. Also, it was not clear from the ICR how the essential drugs were reduced from a full list to a "prioritized" list but still met the minimum capacity. There is limited information on how the PCMIs were able to effectively carry out continuous care without the electronic medical information system in place. A clearer explanation of the reasons for the key implementation shortcomings - supply chain and digital ID system - would have helped to inform future efforts. Subsequent discussions with the project team addressed several of the above shortcomings. Lastly, the ICR would have been strengthened by a clearer explanation of the program logic of the PBF part of the project, namely by more fully explaining the logic of the results areas and disbursement-linked indicators in the narrative. Overall, the ICR quality is rated Modest due to insufficient evidence on some of the key outcomes reported. a. Quality of ICR Rating Modest Page 16 of 16