ICRR 10887 Report Number : ICRR10887 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 04/20/2001 PROJ ID : P010371 Appraisal Actual Project Name : Family Health Project Costs 62.93 39.28 US$M ) (US$M) Country : Pakistan Loan/ US$M ) 45 Loan /Credit (US$M) 28.1 Sector (s): Board: HE - Health (100%) Cofinancing 2.8 2.1 US$M ) 1.0 (US$M) 0.4 L/C Number : C2240 Board Approval 91 FY ) (FY) Partners involved : Closing Date 12/31/1999 04/30/2000 Prepared by : Reviewed by : Group Manager : Group : Roy Jacobstein Timothy A. Johnston Alain A. Barbu OEDST 2. Project Objectives and Components a. Objectives The project was the World Bank's first in the health sector in Pakistan. Project Objectives were: 1) To improve the health status of the population in Sindh and North-West Frontier (NWFP) provinces; 2) To increase the effectiveness of the existing health care network; 3) To build the institutional capacity to realize these objectives. b. Components There were three Project components: 1. Strengthening Health Services, focused on improved maternal health services including family planning (FP) and on integrating and expanding communicable disease control (CDC) activities. 2. Staff Development, consisting of continuing education, overseas training, and expanding the number of female paramedics via pre-service training. 3. Management and Organization Development, emphasizing enhanced management capacity including development of monitoring, information and financial systems and strengthening supervision. c. Comments on Project Cost, Financing and Dates Total project costs at appraisal were estimated at $62.9M, with a Bank loan of $45M, DFID contribution of 2.8M, Save the Children contribution of $1.0M, and GOP contribution of $14.1M. Latest estimates at closing were a total project cost of $39.3M, with a Bank loan of $29.1M, DFID contribution of $2.1M, Save the Children contribution of $.4M, and GOP contribution of $8.69M. In May 1997 the project was restructured, with cancellation of $7.9M. In October 1998 $1M was canceled because of misprocurement. In March 1999 $3.8M was canceled at the request of the Government of Sindh. Overall, $12.7M was canceled. Project objectives remained the same. 3. Achievement of Relevant Objectives: The project appears to have contributed to a number of improvements in health status and health services in both provinces. Contraceptive prevalence rates (CPR) and immunization rates rose in both provinces. At the health system level, the project led to a marked augmentation of staff, particularly female staff, and this contributed to noteworthy improvements in utilization of maternal health services. Institutional capacity was also enhanced, with striking increases in female enrollment in nursing schools, overseas training of provincial DOH personnel, and development of a fully functional Health MIS in both provinces. Additional significant staff training and development also took place. 4. Significant Outcomes/Impacts: Family planning service availability rose markedly, in both provinces, with the number of FP clients doubling and CPR rising, from 8.6% to 18% in NWFP and from 12.4% to 23.4% in Sindh. Full immunization rates rose dramatically as well, e.g., from 36% to 77% in urban NWFP and from 5% to 27% in rural Sindh. Antenatal care from qualified providers and institutional deliveries also rose, largely due to the greater availability of female paramedics, with an additional 769 posted to facilities, raising the total to 1800 overall. Two thousand additional staff were trained in management and supervision, almost twice the project’s target level. Pre-service training capacity was greatly enhanced, with establishment of two new schools, strengthening of existing ones, and preferential recruitment of female and rural students. A total of 25 in-service Health Development Centres were also established, at which over 15,000 staff were trained in various aspects of management, FP and/or PHC. The improvements in CPR and immunization rates cannot be entirely attributed to the project, but it likely contributed, particularly through the increased availability of female health care workers. 5. Significant Shortcomings (including non-compliance with safeguard policies): The First Health Project was implemented in a period of weak governance. This was reflected in staff absenteeism, political interference and four changes in government. The initial project design was overly ambitious and was not based on a realistic assessment of provincial DOH capabilities. In addition, at appraisal there was a lack of well-defined indicators, though this problem was substantially rectified at MTR. Integration of CDC programs at the service delivery level proved not to be possible due to administrative, institutional and legal constraints. Despite province-wide introduction of an HMIS, and significant reporting rates, routine use of the HMIS as a management tool has not yet taken hold. Financial monitoring and management was weak, particularly in Sindh, with an independent review of accounts indicating serious irregularities in 1998, leading to the refund of $1.23M of ineligible expenditures. 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTE: NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. 7. Lessons of Broad Applicability: • Baseline performance indicators and interim quantitative benchmarks, in terms of both outputs and health outcomes, should be identified at appraisal and regularly tracked in supervision as well as used in final evaluation. • Project emphasis on human resource development is important and can pay off in terms of improved management and service outcomes. • Gender priorities in health projects are important to articulate in project design, can be met in project implementation and can result in increased MCH/FP service use. • Financial management issues need to be integral components of the Bank’s continuous supervision. 8. Assessment Recommended? Yes No 9. Comments on Quality of ICR: The ICR was very well-written, concise and internally consistent. The Key Performance Indicators/Log Frame Index is particularly robust, with good indicators and benchmarks and a useful distinction between outcome/impact indicators and output indicators. The ICR could have done more, however, to assess the linkages between project inputs and changes in HNP indicators. Principal performance ratings and ratings for achievement of objectives were accurate and justified in the text by quantitative data and other qualitative findings.