ICRR 12254 Report Number : ICRR12254 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 09/22/2005 PROJ ID :P070294 Appraisal Actual Project Name :Health Sector Rehabilitation Project Costs 12.70 12.98 And Development Project US$M ) (US$M) Country :Timor-Leste Loan/ Loan US$M ) /Credit (US$M) 12.70 12.67 Sector (s):Central ): government US$M ) Cofinancing (US$M) NA 0.31 administration; Health L/C Number : FY ) Board Approval (FY) 00 Partners involved : Australia AID, European Closing Date 02/28/2002 12/31/2005 Commission, other donors under TFET were not named Evaluator : Panel Reviewer : Group Manager : Group : Elaine Wee-Ling Ooi Christopher D. Gerrard Alain A. Barbu OEDSG 2. Project Objectives and Components a. Objectives The Health Sector Rehabilitation and Development Project (HSRDP) was the first of two phases, of a multi -donor program, to restore health services to East Timor after its separation from Indonesia . This project, an emergency relief grant, was funded by the multi -donor Trust Fund for East Timor (TFET), for which IDA acted as the Trustee . The program objectives supported by this project, and as expressed in the PAD were to : (a) Restore access to a basic package of services, and (b) Lay the foundations for health policies and health system development . b. Components (or Key Conditions in the case of Adjustment Loans ): (i) Restoring access to basic health services ($9.5m planned ; $11.17m actual) -- provide selected, high priority primary care via contracted NGOs, improve supply and logistics of essential drugs, rehabilitate and equip health centers, and strengthen administrative /technical capacity at district and central levels . (ii) Health policies and health system development ($1.7m planned; $0.3m actual) -- develop health policy for the long term (via studies, evaluations, consultation and consensus building ), pilot options for designing Timor's new health system, develop health regulations and legislation, and develop human resources strategy . (iii) Program management unit ($1.5m planned: $1.51m actual) -- carry out project administration and M&E, build capacity, and support Timor's Interim Health Authority (IHA). c. Comments on Project Cost, Financing, Borrower Contribution, and Dates The United Nations Transitional Administration (UNTAET) was the provisional government of East Timor, and its Office of Health acted as the implementing agency for this project . While the IDA administered TFET supported all capital investments and most of the consultant fees and technical assistance (expenditures were $12.67 million), the project also received support for recurrent and operating costs ($0.31 million) from another UN Consolidated Fund for East Timor. While the ICR/PAD listed AUSAID and EC as the main cofinanciers (via TFET) of the project, others (UNICEF, WHO, UNFPA) also played a major role in meeting the needs of the country (see section 3, under Joint Assessment Mission (JAM)). The project was effective on 7/25/2000 and closed late on 12/31/2005 (ICR did not provide reasons). Designed as a 20 month project, it should have closed on 02/28/2002. 3. Relevance of Objectives & Design : The project objectives and design were relevant to East Timor then and now . Other donor comments (AUSAID) also endorsed the project focus, up -front, on policy development coupled with essential infrastructure rehabilitation . While no CAS, CEM or ESW was available at project preparation, the project based its design on the "Health and Education Background Paper " prepared in connection with the IDA -led multidonor JAM of 1999. Given the post-conflict situation in Timor (near breakdown of health infrastructure and the severe loss of management /technical professionals, including doctors, most of whom were Indonesians ), the paper had prioritized the urgent provision of health services, especially at subdistrict level, the development of a new health policy relevant to East Timor's needs and realities, and the re-establishment of a central health authority that would ultimately be the Ministry of Health (MOH). The project also took into account the poverty needs of the population . Approximately 63% of East Timorese are poor, live in rural areas and are engaged in agriculture . The project has targeted services at the rural poor . Additionally it has targeted the control of malaria and TB which especially afflict agricultural workers and heavily compromise their productivity and livelihood . 4. Achievement of Objectives (Efficacy) : Project objectives were largely achieved, and perhaps somewhat over -optimistic physical targets had been set given the short (20 months) implementation period planned. On the whole, given the emergency nature of the project and challenges from a post-conflict situation, project achievements were substantial . Objective (a) Restore access to a basic package of services -- Substantial . The project has laid the groundwork for strengthening the quality and quantity of basic primary care at district level . However due to data inconsistencies (DHS, Census, service records ), actual coverage and access figures are inconclusive . Further, as indicated in partner comments (section 9a), " ....the outcomes cannot be solely attributed to HSRDP I because HRSDP II came into effect a year later,... and the continuing contribution from ....other UN agencies and NGOs....". Vaccination targets (60%) were not likely to have been met (different sources citing 5% to 73%) there has been improvement in vaccination and health facility utilization rates . However outpatient utilization rates are very encouraging (0.75 visits per capita in 2000 to 2.13 visits in 2004; target was 2.5 visits) indicating a growing appreciation and trust by the population of government health services, and the greater availability of health centers . 28 health centers and a national medical supply center were constructed, and 36 health posts were renovated . Objective (b) Lay the foundations for health policies and health system development -- Substantial . The project contributed to developing a health policy framework (targeting health financing, human resources, role of external aid, pharmaceutical management, contracting /regulation of private health providers ) within which the new Timorese health system can be developed . Key stakeholders (local, including civil society, and international development partners) were consulted, engendering good ownership of the framework which was approved by the Timorese Council of Ministers in 2003. District level health plans have started to guide local level implementation . Service treatment protocols, a national formulary, and an essential drugs list have been developed . Legislation was passed (albeit delayed) granting autonomy and commercial management of the Central Medical Store (SAMES) - the latter requires substantial capacity building (staff training in procurement, management information system, transportation/logistics) before gains in productivity, and reduced shortages and prices of medical /pharmaceutical supplies, can be realized. Gains in capacity building and manpower planning for health were less than expected and development of a long-term human resources strategy has been deferred to be supported subsequently by AUSAID . The PMU was absorbed as planned into the new MOH, contributing further to its institutional development . 5. Efficiency : As this was an emergency operation and in the health sector, there was no requirement for calculating the ERR - neither the PAD nor ICR did so. The project was also laying the foundation for the "new" health system, entailing a certain amount of piloting of approaches, thus some allowance can be made for loss of efficiency . Nonetheless, the project was extended more than once and closed almost 4 years late. Gains in efficiency are expected to materialize only after program investments ( e.g. improved drug procurement/ distribution logistics, greater professionalism of workforce, linking budgets to workplans ) become fully operational. The earlier than expected replacement of international NGOs by Timorese health workers is a small but noted contribution to efficiency . 6. M&E Design, Implementation, & Utilization: Given the emergency nature of the project and the decision not to overburden the implementers, the project selectively identified health delivery (vaccination- and utilization- rates) and output/process ( district plans, hospital configuration plan, passage of pharmaceutical legislation ) indicators with which to gauge efficacy and track key project milestones. The M&E system was utilized as planned and inconsistencies in data sources are being attended to, as planned in phase 2, in strengthening health information systems . According to the ICR, the health sector is the only one with quantifiable indicators, supported by the Timorese Transition Support Program . 7. Other (Safeguards, Fiduciary, Unintended Impacts--Positive & Negative): At the start of the project international NGOs were recruited to serve as service providers immediately after the conflict. A significant number of these NGOs were replaced after a year with Timorese, resulting in cost savings, and demonstrating the higher than expected capacity of the Timorese health workers . 8. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: Substantial Substantial Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness . 9. Lessons: Even in post-conflict situations where reconstruction and basic services are urgently needed, the initiation of the longer term needs of a coherent policy framework should and can take place side by side . Similarly, in a post-conflict situation when many international players move in to provide assistance, it is necessary to map the location of health facilities relative to population needs and take inventory of needed repairs, as a basis for decisions on the future configuration of the health system . Documentation of the entire post -conflict process of the reconstruction of health systems is important . Currently there is little to guide future task managers / implementers involved in post-conflict situations. Documentation (through case studies as has been undertaken for East Timor ) and the dissemination of lessons are necessary . 10. Assessment Recommended? Yes No 11. Comments on Quality of ICR: The ICR is satisfactory and gives a balanced view of the achievements and difficulties of the emergency operation . However there are some gaps in the information provided . Firstly given the focus on TB and malaria by the PAD, the ICR should have discussed these interventions, if any . There are information gaps on key dates, and no reasons provided on project extensions (even though it closed almost 4 years late). Comments from AUSAID and in particular that from the Implementing Agency were very helpful and enlightening .