Document o f The World Bank FOR OFFICIAL USE ONLY Report No: 50 15 1-ID PROJECT APPRAISAL DOCUMENT ON A PROPOSED L O A N IN THE AMOUNT OF US$77.822 M I L L I O N EQUIVALENT TO THE REPUBLIC OF INDONESIA FOR A H E A L T H PROFESSIONAL EDUCATION QUALITY PROJECT August 28,2009 H u m a n Development Sector Unit East Asia & Pacific Region This document has a restricted distribution and may be used by recipients only in the performance o f their official duties. I t s contents ma y not otherwise be disclosed without W o r l d Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective June 16,2009) Currency Unit = Indonesian Rupiah IDR 1,000 = US$0.097 U S $ 1.00 = IDR 10,300.00 FISCAL YEAR January 1 - December 31 ABBREVIATIONS AND ACRONYMS AFDOKGI Asosiasi Fakultas Kedokteran CPCU Central Project Coordination Unit Gigi Indonesia (Association o f CPS Country Partnership Strategy Indonesian Dental Schools) DA Designated (Special) Account AIPKI Asosiasi I n s t i t u s i Pendidikan DEPKES Departemen Kesehatan (Ministry Kedokteran Indonesia o f Health) (Association o f Indonesian DGHE Directorate General o f Higher Medical Schools) Education AIPKIND Asosiasi InstitusiPendidikan DIPA Daftar Isian Pelaksanaan Kebidanan Indonesia Anggaran (Proposal for Budget (Association o f Indonesian Funding) Midwifery Schools) DUE Development o f Undergraduate AIPNI Asosiasi Institusi Pendidikan Ners Education Project Indonesia (Association o f EPSBED Evaluasi Program Studi Berbasis Indonesian Nursing Schools) Evaluasi D i r i (Self-evaluated APBN Anggaran Pendapatan dan Study Program) Belanja Negara (State Budget) FAP Financial Assistance Package ARSPI Asosiasi Rumah Sakit Pendidikan GFMRAP Government Financial Indonesia (Association o f Management and Revenue Indonesian Teaching Hospitals) Administration Project ASKESKIN Asuransi Kesehatan Masyarakat Go1 Govenpnent o f Indonesia (Health Insurance for the Poor) HE1 Higher Education Institutions BAN-PT Badan Akreditasi Nasional HPEQ Health Professional Education Perguruan Tinggi (National Quality Accreditation Body for Higher HWS Health Workforce and Services Education) Project BAPPENAS Badan Perencanaan IBI lkatan Bidan Indonesia Pembangunan Nasional (National (Indonesian Midwives' Development Planning Board) Association) BHE Board o f Higher Education ICB International Competitive BHMN Badan Hukum Milik Negara Bidding (State-owned Legal Entity) ID1 lkatan Dokter Indonesia BI Bank Indonesia (Indonesian Medical Association) BPK Badan Pemeriksa Keuangan IFLS Indonesia Family L i f e Survey (Supreme Audit Agency) IFR Interim (unaudited) Financial BPKP Badan Pengawasan Keuangan Report dan Pembangunan (Finance and IG Inspectorate General Development Supervisory Board - I-MHERE - Indonesia Managing Higher Ministry o f Finance) Education for Relevance and CBC Competency-based Curriculum Efficiency Project CBT Computer Based Testing JAMKESMAS Jaminan Kesehatan Masyarakat FOR OFFICIAL USE ONLY (Community Health Insurance (Finance Minister's Decree) Scheme) PHK Program Hibah Kompetisi KDI Kolegium Dokter Indonesia (Competitive Grants Program) (College o f Indonesian Doctors) PIP Proposal Implementation Plan Keppres Keputusan Presiden (Presidential PMM Project Management Manual Decree) POLTEKKES Politeknik Kesehatan (Health KKIiIMC Konsil Kedokteran Indonesia Polytechnic) (Indonesian Medical Council) Posyandu Pos Pelayanan Terpadu KPK Komisi Pemberantasan Korupsi (Integrated Health Services Post) (Corruption Eradication PP Peruturan Pemerintah Commission) (Government Regulation) JSF'PN Kantor Pelayanan PPNI Persatuan Perawat Nasional Perbendaharaan Negara (State Indonesia (Indonesia Nurses' Treasury Office) Association) MCQ Multiple Choice Questions PUSDIKNAKES Pusat Pendidikan Tenaga ME1 Medical Education Institution Kesehatan (Center for Health MMR Maternal Mortality Ratio Workforce Education - Ministry MoF Ministry o f Finance o f Health) MoH Ministry o f Health PUSKESMAS Pusat Kesehatan Masyarakat MoNE Ministry o f National Education (Community Health Center) MTKI Majelis Tenaga Kesehatan QCBS Quality-and Cost-Based Selection Indonesia (Indonesian Health QUE Quality o f Undergraduate Workforce Council) Education Project MTKP Majelis Tenaga Kesehatan SIB Surat I j i n Bidan (Midwife Provinsi (Provincial Health License) Workforce Council) SIL Specific Investment Loan NAA National Accreditation Agency SIP Surat I j i n Perawat (Nurse NACEHealthPro National Agency for Competency License) Examination o f Health SPICES Student-oriented, Problem-based, Professionals Integration o f Disciplines, NCBENCE National Competency-based Community Orientation, Early Examination and Systematic Clinical Exposure OSCE Objective Structured Clinical TOR Terms o f Reference Evaluation TPSDP Technological and Professional PDGI Persatuan Dokter Gigi Indonesia Skills Development Project (Indonesian Dentists' UNISBA Universitas Islam Bandung Association) (Bandung Islamic University) PDKI Perhimpunan Dokter Keluarga URGE University Research for Graduate Indonesia (Indonesian Education Project Association o f Family Physicians) usu Universitas Sumatera Utara PerMenKeu Peraturan Menteri Keuangan (University o f North Sumatra) V i c e President: James A d a m s Country Director: Joachim von Arnsberg Sector Manager: Juan Pablo U r i b e Task Team Leader: Puti M a r z o e k i This document has a restricted distribution and may be used by recipients only in the performance o f their official duties. I t s contents may not be otherwise disclosed without W o r l d Bank authorization. INDONESIA H e a l t h Professional E d u c a t i o n Quality P r o j e c t CONTENTS Page I. S T R A T E G I C CONTEXT AND RATIONALE ................................................................. 1 A . Country and Sector Issues ................................................................................................... 1 B. Rationale for Bank Involvement ......................................................................................... 2 C . Higher L e v e l Objectives to Which The Project Contributes .............................................. 3 . I1 PROJECT D E S C R I P T I O N ................................................................................................. 4 A . Lending Instrument ............................................................................................................. 4 B. Project Development Objective and Key Indicators ........................................................... 4 C . Project Components ............................................................................................................ 4 D. Lessons Learned Reflected in the Project Design............................................................... 6 E. Alternatives Considered and Reasons for Rejection........................................................... 7 . I11 IMPLEMENTATION .......................................................................................................... 8 A . Institutional and Implementation Arrangements ................................................................ 8 B. Monitoring and Evaluation o f Outcomes/Results ............................................................... 9 C . Sustainability..................................................................................................................... * . 10 D. Critical Risks and Possible Controversial Aspects ........................................................... 11 E. Loan Conditions and Covenants ....................................................................................... 12 . I V APPRAISAL SUMMARY ................................................................................................. 13 A . Economic and Financial Analyses .................................................................................... 13 B. Technical ........................................................................................................................... 14 C . Fiduciary ........................................................................................................................... 14 D. Social................................................................................................................................. 16 E. Environment ...................................................................................................................... 16 F. Safeguard Policies ............................................................................................................. 16 G. Policy Exceptions and Readiness...................................................................................... 16 A n n e x 1 C o u n t r y and Sector o r P r o g r a m B a c k g r o u n d : ......................................................... 17 A n n e x 2: M a j o r Related Projects F i n a n c e d by t h e Bank and/or o t h e r Agencies 20................. A n n e x 3: Results F r a m e w o r k and M o n i t o r i n g ........................................................................ 22 A n n e x 4: Detailed P r o j e c t Description ...................................................................................... 29 A n n e x 5: P r o j e c t Costs ............................................................................................................... 36 A n n e x 6: I m p l e m e n t a t i o n A r r a n g e m e n t s ................................................................................. 37 A n n e x 7: F i n a n c i a l M a n a g e m e n t and Disbursement A r r a n g e m e n t s ..................................... 39 A n n e x 8: P r o c u r e m e n t A r r a n g e m e n t s ...................................................................................... 49 A n n e x 9: B e t t e r Governance A c t i o n Plan ................................................................................. 55 Indonesia: H e a l t h Professional E d u c a t i o n Quality P r o j e c t ..................................................... 55 A n n e x 10: E c o n o m i c and F i n a n c i a l Analysis ........................................................................... 60 A n n e x 1 : Safeguard P o l i c y Issues ............................................................................................ 1 69 A n n e x 12: M e d i c a l E d u c a t i o n in Indonesia.............................................................................. 70 A n n e x 13: P r o j e c t P r e p a r a t i o n and Supervision ..................................................................... 73 A n n e x 14: Documents in t h e P r o j e c t F i l e ................................................................................. 74 A n n e x 15: Statement o f L o a n s and C r e d i t s .............................................................................. 76 A n n e x 17: Map IBRD 50151 ...................................................................................................... 81 INDONESIA HEALTH PROFESSIONAL EDUCATION Q U A L I T Y PROJECT PROJECT APPRAISAL DOCUMENT EAST ASIA AND PACIFIC EASHH Date: August 28, 2009 Team Leader: Puti Marzoeki Country Director: Joachim von Amsberg Sectors: Health (50%); Tertiary education Sector ManagedDirector: Juan Pablo Uribe (50%) Themes: Other human development (67%); Health system performance (33%) Project ID: P I 13341 Environmental category: N o t Required Lending Instrument: Specific Inirestment Loan .- ~ I - I ._ Project __ Finaricing_. t a Ih I [XI Loan [ ] Credit [ ] Grant [ ] Guarantee [ ] Other: For Loans/Credits/Others: Total Bank financing (US$m.): 77.822 Proposed terms: Variable Spread LIBOR based USD loan on standard terms repayable in 24.5 Borrower International Bank for Reconstruction and 74.972 2.85 77.822 Development Local Sources o f Borrowing Country 6.40 0.00 6.40 Total: 83.872 2.85 86.722 Borrower: Republic o f Indonesia Indonesia Responsible Agency: Ministry o f National Education Indonesia Annual 4.70 20.80 34.10 16.80 1.422 Cumulative 4.70 25.50 59.60 76.40 77.822 Project description ReJ PAD II.D., Technical Annex 4 Component 1: Strengthening Policies and Procedures for School Accreditation. The component would support the Government in developing a valid, transparent, credible and internationally competitive accreditation system for medical, dental, nursing, and midwifery education. Component 2: Certification o f Graduates Using a National Competency-based Examination The project would support the strengthening o f the conduct o f competence-based examination o f doctors, dentists, nurses, and midwives by: (i) establishing an independent national i) competence examination agency; ( iimproving the methodology and management o f the examination; and (iii) developing an item bank networking system. Component 3 : Results based Financial Assistance Package (FAP) for Medical Schools Building o n the agreements o n accreditation and certification, the project would allocate financial assistance packages (FAP) for selected medical schools to meet the accreditation standards. Component 4: Project Management A Central Project Coordination Unit (CPCU) would be established at the DGHE to manage and coordinate project implementation Which safeguard policies are triggered, if any? Re$ PAD I K F . , TechnicalAnnex 10 None Significant, non-standard conditions, if any, for: Re$ PAD III.F. Board presentation: NIA Loadcredit effectiveness: A Project Management Manual acceptable to the World Bank has been adopted by the Borrower. Covenants applicable to project implementation: A FAP Manual acceptable to the World Bank has been adopted by the Borrower prior to disbursement o f Component 3 I. STRATEGIC CONTEXT AND RATIONALE A. Country and Sector Issues 1. Indonesia has made impressive gains in expanding the reach o f health services over the past 25 years. During this period, the Government o f Indonesia (GoI) constructed more than 8,000 health centers (PUSKESMAS), about 22,200 subcenters (PUSTU) and 560 hospitals employing-in 2005-a total o f 415,000 staff (245,000 in health centers and 170,000 in hospitals). The private health sector grew even faster during the same period, partly as a result o f government policy that allows public sector staff to work part-time as private providers. 2. Increased access to health care contributed to significant improvements in certain health outcomes. L i f e expectancy at birth increased from 60 in 1986 to 69 years in 2007, while the Infant Mortality Rate (IMR) decreased from around 110 per 1,000 live births in the early 1980s to 34 per 1,000 live births in 2007. However, maternal mortality ratios (MMR) and malnutrition rates have fallen more slowly and Indonesia lags behind i t s peers for both indicators. The outpatient utilization rate at both public and private facilities i s l o w and declined sharply in 1997/8 with the financial and economic crisis and has not reverted to precrisis levels. The mixed picture on the performance o f Indonesia's health system indicates that Indonesia faces quality o f care challenges which go beyond improving access to health care. 3. The number o f medical schools-especially those that are privately managed-continues to increase, indicating a high demand to become a doctor, nurse o r midwife. In 2007, Indonesia employed more than 70,000 medical doctors-1 5,000 specialists and 55,000 general practitioners (KKI 2007), about 17,400 dentists (KKI 2007), about 200,000 nurses (PODES 2006) and almost 80,000 midwives (PODES 2006). Each year, around 5,000 doctors, 1,350 dentists, 34,000 nurses and 10,000 midwives graduate from Indonesia's 69 medical schools, 23 dental schools, almost 600 midwifery school and over 500 nursing schools; all categorized as higher education institutions under the jurisdiction o f the Ministry o f National Education (MoNE). 4. At the same time, the quality o f the schools-particularly new privately managed institutions-is very poor. This i s evidenced by the poor quality o f services delivered by a large proportion o f the doctors, midwives and nurses in Indonesia. The Indonesia Family L i f e Survey (IFLS) 1997 and 20071 vignette studies show that there was a significant improvement in quality during this period. Nevertheless, the average raw score-the percentage o f correct responses to the vignette questions- in 2007 for the three groups o f providers was poor; 45 percent for antenatal care; 62 percent for child curative care; and 57 percent for adult curative care. 5. Recent reforms have laid the foundation for improvement in health professional education but implementation o f these reforms i s only just starting. The enactment o f the National Education System Act (2003), Medical Practice Act (2004) and the Lecturers A c t (2006) provides the legal basis for improving the quality o f Indonesian doctors. The Medical Practice Act supported the establishment o f the Indonesia Medical Council (KKI) which, in 2006, produced the standard o f competencies for doctors and the standards for medical education. These formed the basis for the DGHE (MoNE) to introduce a major medical education reform by mandating all medical schools to implement a Competency-based Curriculum (CBC) along with the adoption o f problem-based learning and the integration o f various medical disciplines. In reality, given the different capacity and specificity o f the schools, the CBC i s The IFLS i s a panel survey conducted since 1993 covering 13 from the 33 provinces in Indonesia. The survey includes vignette questions to evaluate the knowledge and by association the skills o f doctors, nurses, and midwives in providing ANC, child and adult curative care. 1 being implemented in different ways in different schools, and there i s a consequently large gap in the quality o f education between the stronger and weaker schools. 6. Accreditation o f medical and dental schools as a measure o f education quality i s mandated in the 2004 Medical Practice Act. As with the accreditation o f other higher education institutions at academy level and above, this function i s the responsibility o f the National Accreditation Body for Higher Education (BAN-PT) under M o N E. In 2008, BAN-PT signed a memorandum o f understanding with the KKI to perform the accreditation o f medical and dental education institutions in an effort to make the instruments and processes more consistent with the characteristics o f medical and dental education. BAN- PT's toughest challenge t o establish i t s credibility as an accreditation body i s the lack o f autonomy in resource management, including i t s financial dependence o n M o N E . 7. Upon completion o f their studies, medical students receive their certification after passing a National Competency-based Examination (NCBE) which was introduced for the first time in June 2007. A I P K I (Association o f Indonesian Medical Schools) reported only 50 percent o f students passed the examination in 2007 with a passing score o f 45 out o f 100. In addition, the threshold for a passing score i s a matter o f continued debate. This result indicates most medical schools have n o t achieved the standards mandated by the KKI. 8. The development o f the regulatory framework for nurses and midwives lags far behind that o f doctors. The underlying problem i s the continuing debate over governance bodies and standards o f competencies for both o f these professions. The need t o establish the regulations t o ensure quality for midwifery and nursing professions becomes more pressing as the demand for this training grows and significant numbers o f n e w public and private schools are established without adequate oversight. This i s particularly important for private schools as public nursing and midwifery schools are organized under 33 Health Polytechnics (POLTEKKES) that are perceived as providing better quality education at a lower cost than the private ones. 9. There are overlapping processes in the accreditation o f nursing and midwifery schools. The Center for Health Workforce Education (PUSDIKNAKES) within the Ministry o f Health (MoH) accredits the public POLTEKKES, while BAN-PT accredits non-POLTEKKES and privately owned schools. While both P U S D IKN AKES and BAN-PT are working t o improve accreditation procedures, there i s not yet a common approach nor criterion. In addition, the accreditation process i s n o t aligned with international standards o f independence, credibility and transparency t o the public. 10. Although there i s an accreditation system for education institutions, there i s no legal entity to certify the competency o f graduating midwives and nurses. Nursing and midwifery students receive a graduation certificate from their schools without going through a nationally standardized competence testing process. Based o n the certificate, the Provincial Health Office issues the license as a midwife (Surat Ijin Bidan-SIB) or as a nurse (Surat Ijin Perawat-SIP). Registration with IBI (Ikatan Bidan Indonesia - Indonesian Midwives' Association) or PPNI (Persatuan Perawat Nasional Indonesia - Indonesian Nurses' Association) i s not mandatory. B. Rationale for Bank Involvement 11. The rationale for World Bank involvement in this area includes: Building quality assurance capacity through regulation involves a long-term institutional change process that i s well-suited to W o r l d Bank engagement and instruments, as laid out in the Country Partnership Strategy (CPS), and i s less appealing to bilateral agencies; 2 T h e Bank and Go1 have positive experience in i m p r o v i n g quality o f higher education through previous operations2 and the government has already taken initial steps to address the quality o f doctors, midwives and nurses under the Bank-financed Health Workforce and Services3 (HWS) project. A new operation would assure continued progress; T h e W o r l d Bank has access t o b r o a d international experience and can facilitate access to these resources to assist Go1 and other stakeholders to develop and implement accreditation and certification policies and procedures; and 0 T h e design, implementation and evaluation o f a national-scale p r o j e c t i s m u c h m o r e possible within th e context o f a Bank operation. C. H i g h e r L e v e l Objectives t o W h i c h T h e Project Contributes 12. T h e p ro j e c t will contribute t o better progress in achieving k e y health outcomes by ensuring higher value health care by i m p r o v i n g the q u a l i t y o f t h e health care providers: t h e doctors, dentists, nurses, and midwives. The project will do this by strengthening the accreditation and certification institutions which will lead t o improved quality o f the education o f these health professionals. 13. Assisting th e government in this area has strong l i n k s t o t h e W o r l d Bank's CPS. Firstly, strengthening the accreditation and certification system improves the accountability and effectiveness o f the schools as training institutions which i s an important investment in the production o f a high quality health workforce. Secondly, the education o f doctors, nurses and midwives f i t s w e l l with education as a core engagement area o f the CPS. Previous successful World Bank-funded operations include the Development o f Undergraduate Education (DUE) and the Quality o f Undergraduate Education (QUE) Projects. 'The Health Workforce and Services (HWS) project was a World Bank-financed project implemented by M o H and MoNE from 2003-2008. 3 11. PROJECT DESCRIPTION A. Lending Instrument 14. The lending instrument for the project i s a Specific Investment Loan (SIL). A S I L will meet the need for a specific investment to conduct policy reform in health professional education. I t provides the flexibility t o allocate the resources required to address various technical and institutional needs in improving the quality o f health professional education. B. Project Development Objective and Key Indicators 15. The specific Project Development Objective would be to strengthen quality assurance policies governing the education of health professionals in Indonesia. This will be achieved by: i) rationalizing and assuring competency-focused accreditation o f public and private health professional training institutions; ii)developing national competency standards and testing procedures for certification i) and licensing o f health professionals; and i i building institutional capacity to employ results-based grants for encouraging the use o f accreditation and certification standards in the development o f medical school quality. 16. The project's key performanceindicatorswill measure: the establishment o f an independent National Accreditation Agency (NAA); 0 the establishment o f an independent National Agency for Competency Examination o f Health Professionals (NACEHealthPro); 0 the percentage o f health professional schools (medical dentistry, nursing, and midwifery) that have gone through the accreditation process and have publicized the results; 0 the percentage o f graduates o f health professional schools (medical, dentistry, nursing, and midwifery) passing national competency testing at the f i r s t attempt; and the mean test score o f graduates from the Financial Assistance Package (FAP) recipient schools who have taken the National Competence Test. C. Project Components Component 1: Strengthening Policies and Proceduresfor School Accreditation (US$7.184 million). 17. This component will focus on improving the accreditation system o f medical, dental, nursing, and midwifery schools to make it comparable to internationally recognized systems. The objective i s t o have an independent body (National Accreditation A g e n c y N A A ) for the accreditation o f medical, dental, nursing, and midwifery schools. M o r e specifically, the project will assist in building consensus and develop accreditation policies, standards, processes and outcomes, and in piloting the new system prior to legalizing it for nationwide implementation. This includes developing an accreditation strategic framework, accreditation instruments, assessor recruitment system and training, and a policy mandating publication o f accreditation results for easy access by the general public. 18. An important area will be redefining the approach to accreditation from the former quantitative manner to a qualitative approach. Accreditation should have a formative aim. The project will revitalize medical education research and development initiated under the H W S project and, to ensure i t s independence, this function will be placed under the authority o f A I P K I (Asosiasi Institusi Pendidikan Kedokteran Indonesia - Association o f Indonesian Medical Schools). The project will support the Directorate General o f Higher Education (DGHE) in synchronizing data collection and analysis for 4 This accreditation and for EPSBEDS4 will help self-evaluation prior to the accreditation process and will inform decision making and planning processes in the institutions. Project resources would be used for the costs o f workshops and training, benchmarking, international and local technical assistance, legal assistance studies and surveys, office equipment, I T and audio visual equipment and furniture. Conzponent 2: Certijication o f Graduates Using a National Competency-based Examination (US$12.899 million). 19. The project will support the establishment o f an independent national evaluation center (National Agency for Competency Examination o f Health ProfessionaMNACEHealthPro) which will assure the standard o f education quality. NACEHealthPro will be responsible for developing, validating and implementing strategies, methodologies, and tools to evaluate the competence o f medical, dental, nursing and midwifery students prior to graduation from the respective education institutions. The challenge i s to develop an evaluation mechanism that measures not only knowledge but also clinical skills, attitude, ethics, and communication skills; and to develop policies introducing discipline in determining passing scores and transparent remedial examination for failing students. 20. The project will also assist the national evaluation center in establishing Computer-based Testing (CBT) and Objective Structured Clinical Evaluation (OSCE) facilities in about 12 medical schools that will function as regional centers. To improve the validity and reliability o f the examination, the project would assist with the establishment o f an item bank networking system. The system would also allow Higher Education Health Institutions to evaluate their teaching methods. Expenditures include I T and audiovisual equipment, computer software, skills laboratory equipment, office equipment, furniture, international and local technical assistance, international benchmarking, workshop and training. Component 3: Results-based Financial Assistance Package (FAP) for Medical Schools (US$61.4 million). 2 1. Building on the agreements on accreditation and certification, the project would allocate financial assistance packages (FAP) for selected medical schools to meet the accreditation standards. The project would adopt three key principles in allocating the FAPs: (i) results-based allocation o f resources; (ii) competition among medical schools according to their capacity; and (iii) fair partnership between leading and less advanced medical schools to build the capacity o f the latter schools according to their specific needs. The project would have three FAP schemes: 0 Scheme A : FAP for leading medical schools to build their international reputation and to strengthen Indonesia's global competitiveness; Scheme B: F A P to support weak capacity and new medical schools in achieving medical education standards mandated by the Indonesian Medical Council (IMC) through partnerships with a leading medical school; and Scheme C: FAP to support moderate capacity medical schools in achieving medical education standards mandated by the I M C . 22. The Board o f Higher Education @HE) i s responsible for establishing the guidelines for the FAP recipient selection and proposal approval process and for overseeing the FAP implementation. An FAP manual will be prepared by the BHE to guide the implementation o f the FAP. The BHE will also establish the criteria for grouping existing medical schools into four categories: leading, moderate, weak, ~ EPSBED is a Self-evaluated Study Program, a tool designed by the DGHE to monitor the statistics o f higher education institutions. 5 and new medical schools. There would only be one selection process during the lifetime o f the project which would be conducted during the first year o f the project. Each FAP recipient will implement the program for three year duration. Component 4: Project Management (USSS.239 million). 23. A C e n tra l Pro j e c t Coordination Unit (CPCU) w o u l d b e established at t h e DGHE. Project resources would finance incremental operating costs, project management consultant, office equipment, furniture and project monitoring and evaluation. D. Lessons L e a rn e d Reflected in the P ro j e c t Design 24. W o r l d Bank-funded higher education projects in Indonesia have used financial assistance as a t ool t o build th e capacity o f higher education institutions since t h e early 1990s. In general, the experience has been successful and financial assistance has been used not only in donor-supported higher education projects, but has also been institutionalized by the government to channel a part o f i t s o w n A P B N funds to higher education institutions. 25. Those projects have provided i m p o r t a n t lessons including: 0 the difficulties in linking financial assistance investments to outcomes; 0 limited use o f the financial assistance to support behavior change in improving teaching-learning quality; 0 challenges in maintaining transparency and avoiding conflicts o f interest; and 0 slow disbursement due to weaknesses in management arrangements o f the financial assistance. 26. T h e H e a l t h Professional Education Quality (HPEQ) P r o j e c t takes these lessons i n t o account and includes clear outcome indicators and a results f r a m e w o r k w h i c h allows investments t o be linked t o outcomes. HPEQ emphasizes improving teaching quality by furthering the implementation o f the CBC for health professional education. The CBC approach i s internationally recognized as a means for improved learning and teaching. During the preparation o f the HPEQ project various workshops were held in which existing and potential conflicts o f interest were openly discussed and agreements were reached o n ho w to address those in future. 27. All lessons p ro v i d e invaluable i n f o r m a t i o n t o ensure t h e FAP manual will avoid k n o w n pitfalls in th e management o f financial assistance. For example, given the high probability that standard goods such as laboratory equipment, text books, and I T equipment will be needed by all medical schools, HPEQ introduces the possibility o f collaboration among the FAP recipients to procure FAP financed packages with facilitation by a national procurement consultant. This arrangement would address fiduciary concerns while at the same time avoiding disbursement bottlenecks resulting from a lack o f familiarity o f medical schools with the W o r l d Bank procurement guidelines. 28. F r o m the health perspective, th e recently closed W o r l d Bank-financed H W S P r o je c t demonstrated th e i m p o r t a n t ro l e o f the professional associations in developing workforce-related policies. Taking this lesson into account, the professional associations and the association o f professional education institutions were actively involved in designing the HPEQ project and will continue to be actively involved in implementation. A second very important lesson from the earlier projects arises from the difficulty o f channeling resources t o nongovernment entities. HPEQ takes this lesson and provides clear fund channeling guidelines t o nongovernment implementing units agreed upfront with M o F . The details will be included in the Project Management Manual (PMM). 6 E. Alternatives Considered and Reasons for Rejection 29. Consideration was given to a project to improve the quality o f health care by addressing quality o f education and human resources @R)management policies. While it i s likely that improving the quality o f medical, dental, nursing and midwifery education by strengthening the education institutions will have an impact o n the quality o f service delivery, HR policies encouraging the employment o f these properly trained and certified providers will assure the impact. Developing and regulating HR-policies i s the stewardship function o f MoH. The Go1 decided not to include an HR management component in this project as involving multiple executing agencies and implementing units will increase project complexity. Learning from previous projects, a complex project negatively affects implementation performance and subsequently the achievement o f project objectives. For this reason, the Go1 preferred t o address HR management policies through other funding sources. 30. Allocating FAPs for nursing and midwifery schools a well as for medical schools was also s canvassed during the project appraisal process. There are far more nursing and midwifery schools than medical schools and the need to improve the quality o f these schools i s urgent. On the other hand, the development o f the accreditation system o f nursing and midwifery schools lags behind that o f medical schools. Un l i k e the case o f doctors, the governing body for certifying nurses and midwives does not yet exist to validate the competencies and the standard o f education. Without these standards n o accreditation system can be developed nor can accreditation standards be benchmarked for F A P proposals. Although FAPs for nursing and midwifery schools will not be provided under this project, the project will help in strengthening the regulatory framework o f both professions and support the establishment o f the governing body. Given existing circumstances, however, the desired outputs m a y occur late in the project implementation phase and would not allow sufficient time t o implement the FAPs. 31. Finally, consideration was given to allocating FAPs only to public medical schools. Indonesia's 69 medical schools together produce around 5,000 new doctors per year. This i s far below the demand for doctors in Indonesia. Some 57 percent o f existing medical schools are private institutions- making them very important suppliers o f medical graduates-however they also have lower accreditation status and quality than public school^.^ Improving the quality o f graduates from public schools by allocating FAPs only to those schools would continue to allow private school graduates to enter the market with poor qualifications, put patients at risk o f poor quality care and would widen the gap o f quality between public and private medical schools. Fo r this reason, the F A P will be made available to public and private medical schools. O f the 39 private medical schools in Indonesia, only 20 have gone through the accreditation process; 6 private medical schools have Level A accreditation, 11 have Level B accreditation, and 3 have Level C accreditation. 7 1. 1 1 IMPLEMENTATION A. Institutionaland Implementation Arrangements 32. The executing agency o f the project i s MoNE while the implementing unit i s the DGHE. M o H i s an important partner in this project and-although they are not an implementing agency-they will be closely involved in a l l policy discussions o n accreditation and certification. At the central level, project funds would be transferred into the budget document (DIPA) o f the DGHE. As the head o f the working unit (Satuan Kerja - SATKER), the Director General would be responsible f o r overall project implementation. The Authorized Budget Implementer (Kuasa Pengguna Anggaran - KPA) i s the Director o f Academic Affairs, DGHE. A Project Steering Committee would be established consisting o f representatives from Bappenas, MoNE, Mo H , and M o F. The steering committee w o u l d provide guidance and strategic directions for the project. 33. The Director General of Higher Education would be the Project Director and the Director o f Academic Affairs would be the Project Manager. The Central Project Coordination Unit (CPCU) would be established within the office o f the Directorate o f Academic Affairs. The Head o f the Sub- Directorate o f Curriculum and Study Programs would be the Vice-Project Manager, and would also function as the Commitment Maker (Pembuat Komitmen - PK) within the SATKER. The treasurer o f the SAT KER will also function as the treasurer o f the project. Payment verification will be conducted by the Finance Bureau o f M o N E . An Executive Secretary would be appointed t o lead the Project Secretariat within the CPCU. The project secretariat will be staffed by selected staff from DGHE and will be responsible for managing procurement, financial management and general administration for the project. The Project Director will appoint a person in charge (PIC) for each o f components 1, 2, and 3 and for project monitoring and evaluation. The persons in charge would be the relevant heads o f sub-directorate within the Directorate o f Academic Affairs. The project would employ the services o f consultants to provide technical as w e l l as managerial assistance as necessary to the Project Director. 34. A technical committee would be established by the Director General to ensure the project's annual plan i s consistent with intervention priorities identified during project preparation and to adapt the project as necessary to accommodate more recent developments. Membership o f the technical committee would consist o f representatives f r o m Bappenas, M o N E , M o H , MoF, KKI, BAN- PT, associations o f Health Higher Education Institutions (AIPKI, AFDOKGI, AIPNI, AIPKIND) and teaching hospitals (ARSPI) and professional associations (IDI, PDGI, PPNI, IBI).The technical team i s also responsible for monitoring project performance and for ensuring the achievement o f project objectives. The BHE would be responsible for establishing the guidelines for the FAP, organizing and overseeing the selection process o f the FAP, and overseeing FAP implementation. Implementing units o f the project will include BAN-PT, the NAA, the NACEHealthF'ro, associations o f education institutions (AIPKI, AFDOKGI, AIPNI, AIPKIND), and professional associations (IDI, PDGI, PPNI, IBI). Funds for these entities would be included in the PMM. 35. At the university level, the rector has the ultimate responsibility for project implementation. A Project Implementation Unit (PIU) will be established in the university to support implementation and administration o f the project. In autonomous (BHMN) and public universities, the P I U will consist o f university staff holding structural positions, including those working in the medical schools. Establishment o f the PIU should be completed before a contract i s signed between the university and the DGHE. B. Monitoring and Evaluation o f Outcomes/Results 36. The CPCU will be responsible for results monitoring o f the project. This includes reporting on project performance according t o agreed indicators for six-monthly reports to inform W o r l d Bank review missions; annual reports; and inputs to project baseline, mid term and the final evaluation. Project baseline, mid term, and final evaluation should report on the status o f the five k e y performance indicators o f the project. F o r the purpose o f monitoring and evaluation, the C P C U will develop required data collection instruments and a data recording and reporting system. The focal point for project monitoring will be the C P C U staff/consultant in charge o f project monitoring and evaluation. The person will collaborate closely with the P IC for each component to facilitate access to the various units implementing the project to get timely information on project outputs. 37. F o r Component 1, the primary source of information for the CPCU will be the information system o f the BAN-PT. The new independent accreditation agency (NAA) for health professionals will later provide this information once it i s established and functioning. The CPCU will monitor the progress o f component interventions and present the report during project review missions. 38. F o r Component 2, the primary source o f information for the CPCU will be the KKI/KDI and AIPKI consortium currently responsible for conducting the national competency based examination for medical doctors. Once the new independent certification agency for health professionals (NACEHealthPro) i s established and functioning, it will assume responsibility. Collaboration with KKIhTACEHealthPro will also provide the necessary information required for monitoring the performance o f the F A P recipients under Component 3 o f the project, particularly the performance o f participating schools in the National Competency Examination (NCE). The CPCU will also monitor the progress o f component interventions and present the report during project review missions. 39. F o r Component 3, the primary responsibility for monitoring the FAP implementationlies with the BHE. The BHE w o u l d do this through periodic meetings among the FAP recipients and through field visits. The BHE will conduct an annual performance evaluation o f the FAP recipients according to indicators listed in the Proposal Implementation Plan (PIP) as a prerequisite for allocating the yearly tranches. The focus o f the monitoring would be on the output indicators and the following aspects: (i) implementation o f the agreed PIP and overall disbursement o f the FAP resources; (ii) achievements in the five areas o f interventions (or six areas under Scheme A); (iii)performance in procurement o f resources and services; and (iv) overall management o f the FAP program. FAP monitoring will also assess the effectiveness o f interaction between the leading and the w e a k h e w medical schools. The BHE will develop a supervision plan t o ensure the required information i s available on time for the intended purposes. The C PC U will collaborate with the BHE t o access the required information for preparing agreed project reports. 40. The project will also support the strengthening o f the Sub-directorate o f Academic Evaluation under the Directorate of Academic Affairs in managing EPSBED. This includes developing software to allow online periodic data updating o f EPSBED by the health professional education institutions. An intervention under Component 1 is aligning EPSBED instruments with the accreditation instruments, and training the data providers in data entry and use o f the instruments. When there i s congruence between EPSBED and accreditation instruments, EPSBED can be another source o f information for monitoring progress o f health education institutions in achieving education standards. The project will also support the NAA in developing a NAA website which will provide the public with access to information o n the accreditation status o f health education institutions by province. The CPCU will periodically monitor the number o f "hits" by the public and include the information in the project monitoring reports. 9 C. Sustainability 41. Sustainability o f project initiatives will be attained t h r o u g h t h e following measures: T h e NAA and the N A C E H e a l t h P r o w o u l d include in t h e i r business plan a sustainability plan after project funding terminates. Possible ongoing funding for the NAA includes contributions from higher education health institutions based on the number o f students in the institution. The NAA would use the resources to finance the management and the conduct o f the accreditation process. Establishment o f the NACEHealthPro foresees that students would pay the NAA for the national examination at the CBT and OSCE centers through their schools. 0 Consistent implementation o f the policy t o p u b l i s h results o f school accreditation w o u l d influence school selection by f u t u r e students. This i s an incentive for the schools to maintain their education standards and would sustain the demand for accreditation and certification by the education institutions. 0 T h e DGHE i s committed t o reducing the gap between leading and w e a k e r h e w medical schools. The HPEQ-FAP manual calls for piloting the pairing o f leading medical schools with weakerhew medical schools. Learning from QUE and DUE, the D G H E i s likely to sustain successful financial assistance schemes. 10 D. Critical R i s k s and Possible Controversial Aspects Risk Rating M ith ill"" Mitigation- . - N e w government less supportive o f the The Higher Education L o n g Term M project. Strategy (2003-2010) places emphasis o n producing quality human resources to enhance national competitiveness. Project preparation has been accelerated to allow negotiations before the change o f government and DGHE and W o r l d Bank would conduct close monitoring o f political changes t o allow early inclusion o f potential new officials in project discussions. K e y stakeholders fa i l to reach agreement During preparation a stakeholder's M o n the standards, procedures and outcome analysis was conducted. Based o n the o f medical, nursing, and midwifery findings multi-stakeholders workshops school accreditation. and meetings were held t o create awareness about potentially sensitive decisions, develop roadmaps to address the issues and bring the stakeholders together. K e y stakeholders fa i l to reach agreement Participants o f the stakeholders' decision M on the independent body responsible for workshop during project preparation conducting accreditation and competence supported the idea o f having an testing o f graduates. independent body conduct accreditation and competence testing. The key stakeholders are active members o f the task force preparing project interventions, including establishment o f the independent body. There i s no system in place recording the The project allocates resources to M number and type o f health education develop a database t o track, record and institutions and their accreditation status. report the number o f schools and their sccreditation status. This would be the responsibility o f the DGHE under MONE. There i s no system in place recording the The project allocates resources to M number o f graduates taking competence ievelop a database to track, record, and testing for certification and recording the report the number o f graduates taking results. :ompetence testing and the result. This would be the responsibility o f the DGHE. More schools open without proper quality 4t pre-appraisal DHGE took the __ M :valuation during project implementation iecision to put a l l new requests for period -undermining the project efforts. 3pening o f new schools on h o l d until an iccreditation process i s in place. 11 T o Component Results Professional associations and associations The project applies recommendations N o f education institutions do not have and lessons learned from earlier projects, access to project resources and therefore and assures the fund channeling cannot participate actively in the project. possibilities for these nongovernment institutions are included in the legal agreement. Failure to sustain the operation o f the Close monitoring o f the business plans M NAA and the NACEHealthPro after the o f the NAA and NACEHealthPro for project ends. early corrective interventions as w e l l as bringing in international experiences and lessons learned. Weak and new medical schools lack The design o f the FAP requires leading M capacity t o prepare acceptable proposals medical schools t o provide assistance t o despite having highest need for the FAP. other schools in their cluster t o develop the F A P proposal through proposal preparation workshops and mentoring. Leading medical schools are reluctant to Leading schools are obliged t o include M be involved in the partnership to build the support to weaker schools (pairing capacity o f the weaker schools. mechanism) in their proposals. Proposals will not be approved without clear plans for capacity building f r o m leading schools to weaker schools and will be closelv monitored. FAP recipients are unfamiliar with the Manuals are prepared in a participatory H WB Procurement Guidelines, introducing manner to allow issues to be dealt with fiduciary risks and delays in the before finalization. Lessons learned f r o m completion o f procurement packages. earlier projects taken into account. Overall Risk Rating M E. Loan Conditions and Covenants 42. Condition o effectiveness: A Project Management Manual acceptable t o the W o r l d Bank must f be completed prior to project effectiveness. Condition o disbursement o Category 3: A FAP Manual acceptable to the W o r l d Bank must be f f completed prior to disbursement o f Component 3 Other Covenants: The Loan Agreement contains covenants that for the purposes o f carrying out FAPs and other activities for which the recipient o f Loan Proceeds i s not the government, each recipient will carry out such activities in accordance with the Loan Agreement, the relevant portions o f the Project Management Manual and FAP Manual, as applicable. 12 ` IV. APPRAISALSUMMARY A. Economic and Financial Analyses 43. I n general there i s a recognized need for quality enhancement in the provision o f health care throughout the developing world. Indonesia i s no exception and there i s a high implicit need for the proposed project given that there i s significant scope t o raise the level o f health knowledge among Indonesia's current practitioners. As the project i s oriented towards accreditation and certification, the economic analysis assumes that these activities lead to greater knowledge gains among entering cohorts o f practitioners. This in turn leads to elevated provider competency and hence better quality health care and improved health outcomes. There i s some evidence, albeit mostly f r o m the developed world, to support this causal linkage. 44. The Health Professional Education Quality project i s consistent with Indonesia's overall development plan expressed in the Country Partnership Strategy, 2009-2012 (World Bank 2008a). Public financing o f the project i s justified principally because o f the public goods nature o f accreditation and certification activities and because o f the positive externalities associated with improvements in health. Without clear revenue possibilities for project activities, at least at this moment for Indonesia, it i s highly unlikely that the private sector would invest in such services. 45. Due to the uncertainty surrounding the ultimate impacts o f accreditation and certification in the Indonesian context, this economic analysis takes a conservative approach. First, the analysis predicts modest gains in the quality o f care provided by a certified health practitioner trained in an accredited institution over and above the quality o f care available in Indonesia today. Second, the analysis only considers those health gains that are relatively easily calculated in monetary terms. These include reductions in the incidence o f l o w birth-weight infants, reductions in infant mortality, reductions in the number o f days o f w o rk lost to illness by prime age adults (aged 15-49), and reductions in prime age adult mortality. Clearly there are other dimensions o f population health that m a y gain from project activities, but these dimensions are not included in the analysis. Third, the provider competency gains from project activities are only likely to affect a finite number o f graduating cohorts in the near future given the inherent difficulties in forecasting outcomes further out in time. 46. Monte Carlo simulation methods generate a distribution o f anticipated total benefits, at the national level, to compare with costs. The total gross program cost i s US$76 m i l l i o n over five years, yielding a net present value o f US$68.87 m i l l i o n given the anticipated disbursement schedule. The median present value o f total costs averted i s estimated at US$380.80 million, yielding a gross benefit- cost ratio o f 5.53. Every possible outcome determined through the analysis i s associated with a substantially higher present value o f total costs averted in comparison with actual project costs. The benefit-cost ratio ranges over the full interval (1.57, 10.56). The vast majority - 80 percent, o f all simulations - range over the truncated interval (3.40, 7.63). 47. Savings to the health care system are likely to accrue from reduced public expenditures resulting from more accurate diagnosis, more appropriate treatment, and changes in utilization rates as a result o f improved health. Nevertheless, modeling these costs i s difficult without specialized information and forecasts. Hence the analysis does not estimate a net benefit-cost ratio f o r health system savings through improved provider quality. Even with the very conservative assumptions o n program impact- made in response to an environment o f great parameter uncertainty and only a partial accounting o f all possible benefits-the anticipated gross benefits are substantial. The net benefits would be even greater. 13 48. On completion o f the project, modest recurrent costs will be incurred by the Go1 budget, consisting mainly o f the costs o f continuing to administer the accreditation and certification bodies- NAA and NACEHealth Pro. These administrative costs are estimated to total U S $ l .14 m i l l i o n (in 2009 dollars) per year after project completion. This amount i s small (0.006 percent) in comparison with total government spending in the education sector and within the same small order o f magnitude (0.011 percent) when education expenditures are calculated against central government commitments. B. Technical 49. Strengthening the education institutions by improving accreditation policies, standards, procedures and outcomes i s the first o f two key aspects to improving the quality o f graduates of medical, dental, nursing, and midwifery education institutions. The project would assist in establishing an accreditation system for health professional education institutions comparable to international standards, at the same time accommodating the specific needs o f Indonesia. Development o f the new system would be participatory, involving a wide range o f stakeholders to build support and commitment. Stakeholders include BAN-PT, KKI, associations o f education institutions, professional associations and government authorities within M o N E and M o H . A key output i s the establishment o f an independent NAA for health professional education. 50. Secondly, the project will strengthen the certification o f graduates by improving the validity, reliability and transparency of the system and the methodology for student evaluation. The introduction o f C B C for health professional education in 2005 has changed the education process in health higher education institutions. As a result, there i s an urgent need to develop a student evaluation system appropriate for measuring these competencies. In this regard, the project would support the government in establishing an independent agency (the NACEHealthPro) responsible for designing and conducting a standardized national final evaluation o f students o f medicine, dentistry, nursing and midwifery. This includes building this institution's capacity in managing the national examination for the four professions, the establishment o f the evaluation centers and capacity building o f the centers and health higher education institutions, the development o f evaluation methodology, and the establishment o f an item bank networking system t o support reliable and valid evaluation o f students. 51. The largest component o f the project in financial terms i s the FAF' to medical schools. The design o f the FAP aims at assisting the medical schools in meeting the national standards o f education. Effective use o f the F A P by the medical schools would be reflected in improved performance measured by the performance o f the students in the NCE. The FAP would be provided competitively and has three schemes t o accommodate the need o f the wide range o f capacity o f existing medical schools. The F A P design aims at reducing the gap between leading medical schools and weaker and new medical schools through a partnership in developing and implementing the FAP proposal. It introduces a more systematic approach and more efficient use o f resources f o r building the capacity o f weaker and n e w medical schools, compared to the current ad hoc partnership among the medical schools. C. Fiduciary 52. The project i s a Specific Investment Loan (SIL) of about US$77.822 million with overall risk o f the project assessed as `substantial' before mitigation and `moderate' after mitigation. Identified risks include: (i) significant amount o f FAP allocation for selected universities with medical the schools (68 percent o f the loan) while the type o f expenditures could not be defined during the FM capacity assessment, exposing the project to higher risk; (ii) considerable amount o f soft expenditures the (training, workshop and incremental operating cost) vulnerable to misuse; (iii) limited supervision and monitoring under similar ongoing projects such as I-MERE (Indonesia - Managing Higher Education 14 for Relevance and Efficiency) within the DGHE; and (iv) n o complaint handling mechanism i s yet in place. An action plan has been designed and included in this document to help mitigate those risks. 53. Accounting policies and procedures for this project will follow the government accounting standards (PP No. 24/2005) and government accounting systems (PerMenKeu No. 59/PMK.06/2005). All project activities and expenditures will be included in the central government budget (DIPA) and administrative and accountability procedures for approving expenditures and disbursing funds t o beneficiaries will follow government procedures. Project financial transactions will be recorded in the government accounting system and included in government accountability reports. The Financial Unit will prepare a separate set o f project financial reports that are suitable for project- monitoring purposes. The C P C U will be responsible for preparing an aggregate Interim (unaudited) Financial Report (IFR) and submitting the report to the Bank o n a quarterly basis in a format agreed with the Bank. 54. The Inspectorate General (IG) o f MoNE will undertake regular internal audits at the central and university level based on Terms o f Reference (TOR)acceptable to the Bank. A copy o f the audit reports would be provided to the Bank. The annual financial statements f o r the project will be audited by an external auditor. The annual audit reports and audited financial statements would be furnished to the Bank not later than six months after the end o f the government fiscal year. The audit will be conducted in accordance with the TORacceptable to the Bank and agreed at negotiations. 55. The applicable disbursement methods for funds are by: (i)advance; (ii)reimbursement; and (iii) direct payment. A Designated (Special) Account (DA) denominated in U S dollars will be opened by DG Treasury in the Bank o f Indonesia (BI) or a commercial bank acceptable to the Bank in the name o f the M o F . The DA will be used solely to finance eligible project expenditures. The ceiling o f advance t o the DA i s US$7 million. The CPCU and PIUs will prepare and submit quarterly IFRs. W h e n an IFR has been produced, the applications for reporting o n use o f DA funds will be based o n the IFRs and an advance to the DA will be made for six-monthly projected expenditures. The Bank will notify MoF about this change as appropriate. All documentation for expenditures will be retained at the implementing unit and will be made available to the auditors for the annual audit and t o the Bank and i t s representatives if requested. 56. Procurement for the project would be carried out in accordance with the World Bank's Guidelines: "Procurement under IBRD Loans and I D A Credits" dated M a y 2004 and revised in October 2006; and Guidelines: "Selection and Employment o f Consultants by World Bank Borrowers" dated May 2004 and revised in October, 2006, and the provisions stipulated in the Loan Agreement. Overall procurement risk o f the project i s assessed as "high". Considering the weak capacity o f participating schools in undertaking public procurement, and t o ensure competitiveness, efficiency, and a smooth process, procurement for goods with similar characteristics (for example laboratory equipment, textbooks, and computers) under the FAP will be coordinated through a j o i n t procurement process -taking into account geographical location o f the schools. The Higher Education Institutions will coordinate a j o i n t procurement process. A staff member from DGHE experienced in W orld Bank procurement guidelines will be part o f the committee. In addition, the procurement process will be assisted by an individual procurement consultant or a consulting firm attached t o the CPCU. Nonsimilar goods will be procured individually by the respective school, but these purchases are expected to be the exception. The procurement plan o f each FAP recipient and i t s revisions-if any-will be reviewed and approved by the Wo rl d Bank prior to implementation. 15 D. Social 57. This project addresses the need to strengthen the regulatory framework of key health providers including nurses and midwives who often have the greatest involvement in, and impact on, key public health indicators. Indonesia has a mixed record regarding health outcomes, with child mortality and l i f e expectancy having made significant progress, while maternal mortality and malnutrition among young children have not. This can, in part, be explained by the l o w quality o f service providers and the poor regulatory framework guiding the quality o f service providers. The lower income quintiles, in particular, often use the services o f nurses and midwives, while these two groups o f providers have the weakest regulatory framework. The project does not affect communities o f indigenous peoples and thus the Indigenous Peoples (LP) Policies are not triggered. 58. Under Scheme A of the FAP, leading medical schools would get FAP allocations without competition to build their international reputation and to strengthen Indonesia's global competitiveness. In return, each leading medical school would have an obligation to assist one weak and one new medical school under Scheme B. As a role model for the other medical schools, the leading medical schools would demonstrate their social responsibility by mentoring weaker schools. E. Environment 59. This project will have no significant impact on the environment and thus the requirement to conduct an environment assessment i s not triggered. T h e safeguards r e v i e w team endorsed the assessment of t h e task t e a m a n d assigned Category C for t h e project. Any c i v i l work undertaken through the FAP program in the project i s limited to the rehabilitation o f existing lecture rooms and laboratories. The FAP manual includes this provision and the manual would be used as reference not only by the medical schools preparing the proposal but also b y the reviewer o f the F A P proposals. F. Safeguard Policies Safeguard Policies Triggered by the Project Yes No Environmental Assessment (OP/BP 4.01) [I [ XI Natural Habitats (OP/BP 4.04) [I [ XI Pest Management (OP 4.09) [I [ XI Physical Cultural Resources (OP/BP 4.1 1) [I [ XI Involuntary Resettlement (OP/BP 4.12) [I [ XI Indigenous Peoples (OP/BP 4.10) [I [ XI Forests (OP/BP 4.36) [I [ XI Safety o f Dams (OP/BP 4.37) [I [ XI Projects in DisputedAreas (OP/BP 7.60) [I [ XI Projects on International Waterways (OP/BP 7.50) [I [ XI A'ote: By supporting the proposed project, the Bank does not intend to prejudice the final determination o f the parties' claims on the disputed areas. G. Policy Exceptions and Readiness 60. Government and World Bank readiness criteria were fulfilled by negotiations: 0 Signed decrees for the CPCU, Steering Committee and Technical Committee and staffing; Final Draft o f the Project Management Manual; 0 Budget allocated for first year implementation 0 Procurement plan for the first 18 months. 16 Annex 1: Country and Sector o r Program Background INDONESIA: Health Professional Education Quality Project 1. Indonesia has made impressive gains in expanding the reach o f health services over the last 25 years. The government succeeded in significantly increasing the number and improving the distribution o f public health hospitals and PUSKESMAS (World Bank 2008b) and establishing an extensive outreach effort through the POSYANDU.6 There has also been impressive growth in the availability o f private health providers and hospitals. These efforts have successfully expanded the size o f the total health workforce which no w encompasses more than 70,000 doctors, more than 200,000 nurses and about 80,000 midwives. In addition, efforts to balance availability and workforce incentives to ensure accessibility through innovative contractual schemes, such as the PTT, and legislation seeking to rationalize and regulate dual practice, have had a positive impact. 2. Law 40/2004 and the implementation o f ASKESKIN' helped remove financial barriers to utilization o f public as well as private sources o f care. Prospects for the success and sustainability o f these efforts are more likely thanks to ongoing efforts to take advantage o f the opportunities created through decentralization and the redefinition o f the roles o f central, provincial and district level health authorities. Efforts t o continue these improvements in the quantity o f service delivery will need to continue, particularly to overcome disparities in accessibility across regions and to assure equity in access to both preventive and curative services. 3. Much more attention and effort i s needed to improve the quality of the health care provision and doctors, midwives and nurses as the prime providers o f these services. Fo r example, trends in the M MX - a good indicator o f performance o f the health system and the quality o f service provision-are illustrative o f the need for more attention to quality. While MMR i s notoriously difficult to measure, and estimates in Indonesia are highly variable, there i s overall agreement that the MMR for Indonesia i s the highest in the region and high for Indonesia's income level. The most accurate and recent estimates put Indonesia's MMR between 250 and 400 per 100,000 live births, close to the level in 1990. This lack o f progress i s likely t o be highly correlated with the l o w quality o f service provision by the main providers o f health services. 4. Improving the quality o f health human resources, namely the s k i l l s and competencies of the key health professionals-doctors, nurses and midwives-is key to improving the effectiveness o f both public and private channels o f service delivery. In particular, attention to the quality o f service provision in the private sector i s important as this i s a major source o f health care; almost h a l f o f those seeking care do so at a private facility (World Bank 2008b). Quality o f care i s l o w in general in Indonesia (IFLS 2007) and not necessarily better when provided by private providers (World Bank forthcoming). 5. Indonesia in 2007 has more than 70,000 medical doctors-15,000 specialists and 55,000 general practitioners (KKI 2007), about 17,400 dentists (KKI 2007), about 200,000 nurses (PODES 2006) and almost 80,000 midwives (PODES 2006). A large and growing number o f schools are producing significant numbers o f n ew graduates each year. Fo r example, there are 69 medical schools, o f which 39 are privately owned. Existing medical schools produce around 5,000 n e w doctors per year. There are 595 and 5 18 registered midwifery and nursing schools respectively. O f these, around 83 percent Posyundu or Integrated Health Services Post i s a community-based health service providing growth monitoring and nutrition promotion, immunization, diarrhea control, antenatal care and family planning services. ' ASKESKIN i s the health insuranceihealth card program for the poor, started in 2005, now succeeded by JAMKESMAS, the health insurance program for the poor and near poor. . 17 o f midwifery schools and 52 percent o f nursing schools are privately managed. The midwifery and nursing schools produce around 10,000 midwives and 34,000 nurses per year. 6. There i s no comprehensive and reliable data on the quality o f education and very little understanding o f the quality o f the graduates who finish their education and enter the labor market. The IFLS 1997 and 2007 vignette studies show a significant improvement in the quality o f health care delivered by doctors, nurses and midwives in both the public and the private sectors, and in urban and rural areas across Indonesia. Despite the improvement, however, the average raw score-the percentage o f correct responses to the vignette questions in 2007 for the three groups o f providers was poor-around 45 percent for ante natal care, 62 percent for child curative care, and 57 percent for adult curative care, suggesting continuing l o w quality o f care by those providers (World Bank forthcoming). 7. The enactment o f the National Education System Act (2003), Medical Practice Act (2004) and the Lecturers Act (2006) has been used a the legal basis for improving the quality o f s Indonesian doctors. The Medical Practice Act supported the establishment o f the Indonesian Medical Council (KKI), which produced the competencies for doctors and the standards o f medical education in 2006. These standards were the basis for the DGHE (MoNE) to introduce a major medical education reform by mandating all medical schools to implement a CBC along with the adoption o f problem-based learning and the integration o f various medical disciplines. Given the varying capacity and specificity o f the medical schools, however, the CBC has not been implemented equally across schools and, consequently, there i s a large and growing gap in quality between the stronger and weaker schools. New medical schools need a recommendation from ID1and the relevant provincial government before M o N E will issue an operating license. The accreditation process for the new medical school will take place after two years. 8. BAN-PT i s tasked with the accreditationof medical schools a mandated under the National s Education System Act. Although BAN-PT makes `independent accreditation decisions, i t s lack o f autonomy in MoNE's hierarchy and i t s financial dependence jeopardize BAN-PT's credibility. In addition, as the only legal accreditation body for all higher education institutions including health, BAN- PT's task i s daunting. In 2007, BAN-PT had only completed the accreditation o f 45.6 percent o f 17,844 study programs. BAN-PT also faces the challenge to adapt the accreditation instruments and procedures to address the specific characteristics o f each o f the professional education institutions. As part o f the continuing effort to strengthen the accreditation o f medical schools, B A N - P T recently entered a memorandum o f understanding with the KKI to make the accreditation instruments and procedures more medical school specific. 9. Before graduating from medical school, students undertake the NCBE which was introduced for the first time in June 2007. The KKI has commissioned a consortium comprising KDI, A I P K I and the Indonesian Association o f Family Physicians (PDKI) to develop the NCBE. The current examination consists o f 300 multiple-choice questions which measure knowledge, but not clinical skills, attitudes and ethics. Moreover, the consortium i s under pressure to set a l o w passing grade for the examination in order to avoid high failure rates. The threshold for a passing score i s a matter o f continued debate and i s exacerbated by the growing number o f new schools. In 2007, only 50 percent o f students passed the examination with a passing score o f 45 out o f 100 (AIPKI 2007) an indication that a large number o f medical schools are providing a poor quality o f education. 18 10. MoNE i s responsible for licensing the operation o f new midwiferyhursing schools8, both public and private, on the recommendation o f M o H and the relevant professional associations (IBI for midwifery and PPNI for nursing). The current regulatory framework for the establishment o f nursing and midwifery schools i s suboptimal at best-especially for private schools-which explains the large number o f schools o f doubtful quality. At the same time, the demand for training as a midwife or nurse i s extremely high in both public and private schools. Those seeking to become midwives or nurses, however, have little guidance as t o where to get the best training and the best price. 11. There are overlapping processes in the accreditation o f nursing and midwifery schools as the PUSDIKNAKES accredits the public POLTEKKES, while BAN-PT accredits non- POLTEKKES and privately owned schools. While both P U S D I K N A K E S and BAN-PT are working to improve accreditation procedures, there i s not yet a common approach or criteria nor i s the accreditation process aligned with international standards o f independence, credibility and transparency to the public. 12. Unlike the case with doctors, Indonesia has no authorized body to certify midwives and nurses. The IBI and P P N I have each drafted a l a w that includes the establishment o f a council as the governing body for each profession, but debate o n the draft l a w i s s t i l l ongoing. Given the urgency o f strengthening the governance o f both professions, the M o H proposed a temporary measure o f establishing an Indonesian Health Workforce Council (MTKI) under the M o H with a branch in each province (MTKP). This i s seen as an alternative solution while waiting for a legally binding decision. 13. I n the absence o f a governing body, the current practice i s that students receive a graduation certificate from their schools without going through a nationally standardized competence test. Based o n the certificate, the Provincial Health Office releases the license as midwifehurse (Surat Ijin Bidan-SIB and Surat Ijin Perawat-SIP). Registration with IBI (for midwives) or PPNI (for nurses) i s not mandatory. However, when a midwife/nurse requests a license t o open a private practice from the local government, she has t o obtain a letter o f recommendation from IBI or PPNI respectively. There are currently n o standards for assuring continuing education or for license renewal. There is a wide range o f levels in nursing and midwifery qualifications in Indonesia. The majority o f schools offer D 3 3 (academy) level education while D4 qualifications are mainly used for the training o f D lecturers. Nurses can also train to Sl(Bachelor), S2 (Masters) and S3 (Doctorate) level while midwives can attain a D4 or S1 qualification. Public D3 schools are organized as Health Polytechnics (POLTEKKES). There are 33 POLTEKKES spread more or less evenly across the country. 19 A n n e x 2: M a j o r Related Projects Financed by the Bank and/or other Agencies INDONESIA: H e a l t h Professional Education Quality Project 1. World Bank-funded higher education projects in Indonesia have used block grants as a tool to build the capacity o f higher education institutions since the early 1990s; these projects included University Research for Graduate Education (1994-2001), Development o f Undergraduate Education (1996-2002), and Quality o f Undergraduate Education (1997-2004). The Implementation Completion Reports (ICR) provided satisfactory ratings for a l l three projects. Since then, block grants were used not only in other donor-supported higher education projects such as the ADB-funded Technological and Professional Skills Development Project, but were institutionalized by the government for channeling a part o f i t s o w n A P B N funds t o higher education institutions. 2. Managing Higher Education for Relevance and Efficiency Project (ZOOS-). I-MERE i s an ongoing W o r l d Bank financed lending program implemented by the DGHE, M o N E . The project development objective i s t o create an enabling environment f o r the evolution o f autonomous and accountable public higher education institutions and to develop effective support mechanisms for the improvement o f quality, relevance, efficiency, and equity in higher education. Project activities focus on constructing an enabling environment at the central level while strengthening institutional management and supporting innovation to improve performance and results. The project has two components: (i) reform and oversight o f the higher education system; and (ii) grants t o improve academic quality and institutional performance. Component one supports, inter alia, a transition in the quality assurance system t o emphasize institutional accreditation and licensing o f professional fields through the strengthening o f BAN-PT. The project rating at mid-term review was satisfactory for the development objective and moderately satisfactory for implementation progress. The moderately satisfactory rating for implementation progress i s primarily the result o f difficulties in conducting procurement under the project leading to slow disbursement. 3. Health Workforce and Services Project (2003-2008). The project was implemented by M oH , M o N E and the Indonesian Medical Association. I t s objective i s to: (i) redefine their roles and responsibilities vis a vis health workforce policy, planning and management; and (ii) strengthen their institutional capacity for effective stewardship in fulfilling the functions o f policy making, legislation, regulation, quality assurance/control and technical assistance t o provinces and districts. A subcomponent o f the project supported the enhancement o f the quality o f medical education by increasing institutional capacity t o organize and manage medical education, improving the quality o f formal medical education, and enhancing the learning and teaching environment for both undergraduate and postgraduate medical education and training. Although the project's final Implementation Status Report (ISR) rated the overall achievement o f the project development objective as moderately unsatisfactory, the rating f o r the medical education subcomponent was satisfactory. 4. Technological and Professional Skills Development Project (2000-2006). The overall objective o f the ADB-funded Technological and Professional Skills Development Project (TPSDP, 2000- 2006) i s t o improve the country's human capital t o help achieve sustainable economic growth and to reduce poverty by reducing inequities. The project has two inter-related parts: (i) strengthening the governance and management capacity o f the public and private higher education system, improving equity for disadvantaged students, and supporting a skills retooling program; and (ii) strengthening existing study programs and supporting new study programs in priority disciplines, upgrading study centers for women, and strengthening community and industrial relations. TPSDP was a sector loan and covered six years o f the government's Long-term Higher Education Development Plan. The project supported 75 grants based o n proposals that were selected competitively according to agreed guidelines and procedures. 20 5. Program Hibah Kompetisi (PHK). The GoI-funded Program Hibah Kompetisi (PHK) initiative consists o f four distinct kinds o f competitive grants (A-1, A-2, A-3, and B) each o f which i s targeted at a different tier o f higher education institution. Program A-1 i s aimed at building institutional capacity in study programs that have scored an accreditation ranking o f C (satisfactory) from the BAN-PT. Program A-2 i s aimed at increasing the internal efficiency o f study programs with an accreditation rank o f B (good). Program A-3 i s aimed at increasing external efficiency in study programs that have received an accreditation rank o f A (very good). program B i s aimed at promoting excellence in undergraduate programs that not only have an accreditation rank o f A but that have also been in operation for five years. 21 Annex 3: Results F r a m e w o r k and M o n i t o r i n g INDONESIA: H e a l t h Professional Education Quality Project Results F r a m e w o r k To strengthen quality Establishment o f an independent MoH, M o N E assurance policies governing the National Accreditation Agency decision makers at education o f health ( N W various levels and professionals in Indonesia Establishment o f an independent public are able to use through: National Agency for data on accreditation i) rationalizing and assuring Competency Examination o f status to reach competency-focused Health Professionals decisions on higher accreditation o f public and NACEHealthPro; education in the private health professional Percentage o f health health sector; training institutions; professional schools (medical, Health policy and ii) developing national dentistry, nursing, and program managers competency standards and midwifery) that have gone will be able to assess testing procedures for through accreditation process quality o f recruits at certification and licensing o f and publicize results; entry on basis o f health professionals; and Percentage o f graduates o f national standards; i) i i building institutional health professional schools and capacity to employ results-based (medical, dentistry, nursing, and Current and potential grants for encouraging use o f midwifery) passing national students will be able accreditation and certification competency testing at f i r s t to assess quality o f standards in the development o f attempt; and schools based on medical school quality. the mean test score o f graduates examination from the FAP recipient schools performance. who have taken the National Competence Test. Iiiterriiediate Outcome Ilit1icittors ...."" Outcomu MonitorinP~ Responsibility for policies on (See for each profession below.). Developing specific professional accreditation, national accreditation certification and school and certification licensure are clear and accepted standards and policies by key stakeholders. will enable evidence- based improvement policy and program decision making at medical, dental, nursing and midwifery schools. Medical Education: KKI provides oversight on standards Medical schools will for accreditation o f medical schools, have clear guidance for the content and conduct o f NCBE. developing and improving the quality o f the study programs. 22 ~. Interniecliate Outcoinrs Use o f Interrnctliate Outcome Blonitoriiirr e7 Dental Education: KKI provides oversight on standards Dental schools will have for accreditation o f dental schools, clear guidance for the content and conduct o f NCBE. developing and improving the quality o f the study programs. Nursing Education: Go1 has established a formal entity Nursing schools will with responsibility for oversight o f have clear guidance for accreditation standards, competency- developing and based examination o f nursing improving the quality o f profession. the study programs. Midwifery Education: Go1 has established a formal entity Midwifery schools will with responsibility for oversight o f have clear guidance for accreditation standards, competency- developing and based examination o f nursing improving the quality o f profession. the study programs. Public and private medical, 0 Education and competency Accreditation and dental, nursing and midwifery standards are agreed by all dissemination o f schools are aware of, and using, stakeholders and published. accreditation status will: national standards to develop 0 Help M o H and/or and implement quality DGHE target improvement programs. resources to schools on basis o f program rather than randomly; and Publication o f accreditation status will inform students o f school quality. The NAA develops annual 0 The NAA has independent and Transparency in planning and budgeting, applies adequate budget to conduct accreditation regular external audits and accreditation; and management will publishes the results. 0 The NAA has access to adequate encourage schools to numbers o f suitably trained contribute resources assessors. actively to the NAA and w i l l ultimately sustain the NAA. Professional associations, A charter supporting the Consensus and charter sssociations o f education establishment o f the w i l l strengthen the institutions, M o N E and M o H NACEHealthPro including i t s tasks mandate and the sgree on the framework for and responsibilities, and working credibility o f the :onducting an N C B E for the arrangements. independent agency and four health professions. will accelerate school participation in the NCBE. 23 1 Intermediate Outcomes Intermediate Outcome fndicators I Use a f Ititerniediatu Outcome Monitoring MoNE (BHE) uses the standards Numbedpercentage o f medical Using the standards o f o f basic medical education and a schools receiving financial support basic-medical education transparent proposal review t o strengthen the program. in reviewing proposals process to identify schools will ensure that the F A P requiring additional resources program i s consistent in within the weak, new, and helping the medical moderate medical school schools achieve better clusters. performance in 1 accreditation. Arrangements for Results Monitoring 1. The C P C U will be responsible for results monitoring o f the project. This includes preparing the status o f project performance according to agreed indicators for six monthly reports t o inform W o r l d Bank review missions; annual reports; and inputs to project baseline, mid term and the final evaluation. Project baseline, mid term, and final evaluation should report o n the status o f the f i v e k e y performance indicators o f the project. F o r the purpose o f monitoring and evaluation, the C P C U will develop required data collection instruments and a data recording and reporting system. The focal point for project monitoring will be the C P C U staffkonsultant in charge o f project monitoring and evaluation. The person will collaborate closely with the person in charge for each component to facilitate access to the various units implementing the project to get timely information o n project outputs. 2. For Component 1, the primary source o f information for the C P C U will be the information system o f the BAN-PT until the new independent accreditation agency (NAA) f o r health professionals i s established and functioning. The C P C U will monitor the progress o f component interventions and present the report during project review missions. 3. For Component 2, the primary source o f information for the C P C U will be the KKI/Consortium o f KDI and AIPKI currently responsible f or conducting the national competency based examination for medical doctors until the new independent certification agency for health professionals (NACEHealthPro) i s established and functioning. Collaboration with KKI/NACEHealthPro will also provide the necessary information required for monitoring the performance o f the FAP recipients under Component 3 o f the project, particularly the performance o f participating schools in the National Competency Examination. The CPCU will also monitor the progress o f component interventions and present the report during project review missions. 4. For Component 3, the primary responsibility for monitoring the FAP implementation lies with the BHE. The BHE would do this through periodic meetings among the FAP recipients and through field visits. The BHE will conduct an annual performance evaluation o f the FAP recipients according to indicators listed in the PIP as a prerequisite for allocating the yearly tranches. The focus o f the monitoring would be on the output indicators and following aspects: (i) implementation o f the agreed PIP and overall disbursement o f the FAP resources, (ii) achievements in the five areas o f interventions (or 6 areas under Scheme A), (iii) performance in procurement o f resources and services, and (iv) overall management o f the F A P program. F A P monitoring will also assess the effectiveness o f interaction between the leading and the weakhew medical schools. The BHE will develop a supervision plan t o ensure the required information i s available o n time for the intended purposes. The C P C U will collaborate with the BHE to access the required information for preparing agreed project reports. 24 5. The project will also support the strengthening o f the Sub-directorate o f Academic Evaluation under the Directorate o f Academic Affairs in managing EPSBED. This includes developing software to allow online periodic data updating o f EPSBED by the health professional education institutions. An intervention under component 1 i s aligning EPSBED instruments with the accreditation instruments, and training the data providers in data entry and use o f the instruments. When there i s congruence between EPSBED and accreditation instruments, EPSBED can be another source o f information for monitoring progress o f health education institutions in meeting the fulfillment o f some standards o f education. 6. A The project will support the N A in developing a NAA website. The website will provide the public with an access to information o n the accreditation status o f health education institutions by province. The C P C U will monitor the number o f "hits" by the public periodically and include the information in the project monitoring reports. 25 T tr: P? 3 2w u a 2 8 .- 3 * + f c3 3 m rr: u 00 N n A n n e x 4: D e t a i l e d P r o j e c t D e s c r i p t i o n I N D O N E S I A : H e a l t h P r o f e s s i o n a l E d u c a t i o n Quality P r o j e c t 1. T h e p ro j e c t will contribute t o better progress in achieving k e y health outcomes by ensuring higher value health care th ro u g h i m p r o v i n g the quality o f the health care providers; the doctors, dentists, nurses, and midwives. The project would do this through strengthening quality assurance policies governing the education o f health professionals in Indonesia through: i)rationalizing and assuring competency-focused accreditation o f public and private health professional training institutions; ii)developing national competency standards and testing procedures for certification and licensing o f health professionals; and iii) building institutional capacity to employ results-based grants for encouraging use o f accreditation and certification standards in the development o f medical school quality. Component 1: Strengthening Policies and Proceduresfor School Accreditation (US$7.184million). 2. This component would support the government in developing a valid, transparent, credible and internationally competitive accreditation system for medical, dental, nursing, and midwifery education. Subcomponent 1.1:Development o Strategic Framework, Policies and Proceduresfor Accreditation. f 3. Building o n c u rre n t initiatives t o strengthen the accreditation system o f health higher education institutions, the subcomponent w o u l d support the establishment o f a comprehensive accreditation approach consistent with an international standard o f independence, credibility and transparency. T o this purpose, the project will recruit the necessary international and local technical assistance including legal assistance. An important area will be the redefinition o f the approach to accreditation from the former quantitative manner t o a qualitative approach. Accreditation should have a formative aim, and therefore, the accreditation report will indicate the standards the school i s not complying with. This allows the school to improve the quality o f i t s program and t o reach a better accreditation status. 4. A possible system o f accreditation ratings will be: (i) - full accreditation; ( iB - partial A i) accreditation with a need f o r a follow-up r e p o r t a f t e r a certain time; ( i ) - partial accreditation i iC with a need f o r a follow-up field visit after a certain time; (iv) D - probation; and (v) E - n o accreditation. Specific interventions include: (i) building consensus among k e y stakeholders o n the general principles for the accreditation o f health higher education institutions; (ii) developing an accreditation master plan for health higher education institutions in alignment with international standards; (iii)developing the concept, and establishing an independent accreditation body, f o r health higher education institutions (NAA), and providing capacity building through international training, management and operational support t o the institution for a maximum o f three years after i t s establishment; and (iv) harmonizing regulations and policies. Expenditures include workshops, international and local technical assistance, legal assistance, benchmarking and short courses, and some office equipment and furniture for the independent accreditation body. Subcomponent 1.2: Development o Standards o Health Education Programs and Standards o f f f Competencies. 5. T h e p ro j e c t w o u l d assist th e stakeholders in developing and strengthening standards and guidelines as a prerequisite f o r establishing a valid accreditation system. Priority areas are: (i) 29 i) developing competence and education standards for nursing and midwifery; ( i harmonizing the standards o f competence o f doctors, dentists, nurses, and midwives; ( i ) i i establishing a certification system for clinical instructors as w e l l as building their career pathways; (iv) developing guidelines for clinical instructor training; (v) establishing standards for teaching hospitals and other clinical training sites; and (vi) revitalizing the function o f medical education development and research initiated under the H W S Project, but expand the scope t o also include dentistry, nursing and midwifery. 6. T h e subcomponent w o u l d also accommodate various activities t o strengthen the associations o f education institutions (AIPKI, AF'DOKGI, AIPNI, and AIPKIND) as the frontline stakeholders in preserving education quality. These include technical assistance, mentoring o f A I P N I and AIPIUND by AIPKI, publication o f scientific journals, and information technology to support the work o f the associations. Expenditures include workshops, training, local and international technical assistance, legal assistance, I T services and hardware, printed materials, surveys and studies. Subcomponent 1.3: Development o Accreditation Instruments. f 7. BAN-PT, in collaboration with AIPKI, has commenced t h e process o f i m p r o v i n g the instruments t o accredit medical schools. M o r e work i s needed, however, t o develop accreditation instruments for specific health professions, and to employ an international peer review process for the instruments. The work under this subcomponent would concentrate o n supporting BAN-PTNAA in developing and piloting accreditation instruments for medical, dental, nursing and midwifery education through a participatory process, and disseminating the instruments to the relevant education institutions. Expenditures include technical assistance, workshops and travel. Subcomponent 1.4: Development o a Pool o Assessors. f f 8. Credible assessors are a prerequisite f o r a credible accreditation system as t h e assessors have the i m p o r t a n t task o f establishing whether a school complies with every standard. With the adoption o f the competency-based curriculum, the challenge i s t o recruit assessors with expertise in clinical training. The project would assist BAN-PTNAA in improving the quality and credibility o f assessors for the accreditation o f each education program by improving: (i) the identification and recruitment system o f assessors; (ii) establishing a basic training program for selected assessors; and (iii) conducting regular refresher training for the assessors. Expenditures include technical assistance, workshops and training. Subcomponent 1.5: Establishment o an Accountability System for Accreditation o Health Higher f f Education Institutions. 9. There i s l i m i te d awareness a m o ng prospective students o r t h e i r parents and t h e p u b l ic in general about accreditation and t h e accreditation status of education institutions as an i n d i c a t o r o f education quality. The project would support efforts t o develop a directory o f the accreditation status o f health higher education institutions and publicize the directory and upload accreditation results into the website o f the accreditation agency. Expenditures include I T services, hardware, and printed materials. Subcomponent 1.6: Data Management to Support the Accreditation System. 10. Interventions u n d e r this component w o u l d include developing a strategic alliance between the f ut ure NAA and various units within M o N E t o harmonize and streamline t h e collection o f data f r o m health higher education institutions f o r accreditation, E P S B E D a n d o t h e r purposes. This includes reaching agreement on the data and the instruments used to collect the data. Moreover, project resources would assist the health higher education institutions in building capacity for information 30 management and conducting self-evaluation for continuous quality improvement. Expenditures include software development, workshops, travel and training. Component 2: Certijication o f Graduates Using a National Competency-based Examination (US$12.899 million). 11. The project would support the strengthening o f the conduct o f competency-based examination o f doctors, dentists, nurses, and midwives. Strategies include: (i)establishing an independent national competence examination agency; (ii) improving the methodology and management i ideveloping an item bank networking system. The latter i s aimed at improving o f the examination; and ( i ) the validity and reliability o f the examination while, at the same time, providing an opportunity for medical students to conduct self-assessment and for the medical schools to evaluate their teaching methods. Subcomponent 2.1: Establishing an Independent NACE Healthpro. 12. The main stakeholders o f medical, dental, nursing and midwifery education have reached consensus about the need to have an NCBE to ensure the achievement o f competency standards by each health profession. Medical and dental education i s leading in this area, as a national examination committee started implementing an N C E in 2007, although for the time being this i s limited to paper- based examination using multiple-choice questions. Given that hundreds o f nursing and midwifery schools are producing thousands o f graduates each year, the need to have a similar approach for nursing and midwifery i s urgent. The challenge i s t o establish an independent national examination agency with the task o f developing, validating and implementing tools and strategies t o evaluate the competence o f the four health professions. 13. T o this end, the project would support the establishment o f NACEHealthPro with a mandate to ensure the efficient use o f resources and to cover medical, dental, nursing, and midwifery examination. This i s because the competencies o f the four professions are interlinked. Interventions under this subcomponent would be: (i) development o f the national competency-based the examination framework; (ii) the establishment o f NACEHealthPro and i t s business plan; (iii) benchmarking o f Agency managers t o well-known similar international organizations; (iv) capacity assessment o f the examination centers; (v) an external expert review on the establishment o f the Agency; (vi) management and operation support o f the Agency for a maximum o f three years after i t s establishment; (vii) standardization o f the Agency management quality; and (viii) policy studies and a research program on evaluation. For the purpose o f the research program, the NACEHealthPro will establish guidelines for research selection and awards. Such guidelines are subject to review and approval by the W orl d Bank prior to execution. Eligible expenditures for the subcomponent include workshops, nondegree training, benchmarking, international and local technical assistance, studies, office equipment, I T and audiovisual equipment and furniture. Subcomponent 2.2: Improving the Methodology and Management o the National Competency-based f Examination. 14. Many education institutions have already taken the initiative to improve the quality of written examinations; some have even started to apply an OSCE. However, nonstandardization o f the methodology, assessment tools and arbitrary passing scores has resulted in variable examination quality among education institutions. The project would assist the NACEHealthPro in developing a computer- based examination system (CBT) to improve validity, accuracy, and efficiency o f the examination, Assessing the competency o f health professionals requires the evaluation o f professional skills, in 31 addition to knowledge. In this context, the project would assist in the establishment o f a standardized OSCE system. 15. Through the project, the NACEHealthPro would establish approximately 12 CBT and OSCE centers throughout Indonesia. Although these centers would be located in leading medical schools, they would be used not only for medical examination but also for dental, nursing and midwifery examination. Interventions under the subcomponent would include: (i) developing C B T and OSCE guidelines; (ii)establishing C B T and OSCE centers; (iii) item blueprinting through a series o f workshops to ensure validity and reliability; (iv) workshops o f a panel o f experts to conduct item selection for final assessment tools; (v) establishing standardized patients for OSCE implementation; (vi) workshops for standard setting including the establishment o f passing scores; and (vii) activities t o develop international recognition, including benchmarking, international technical assistance, and participation in international scientific forums and conferences. Expenditures include I T and audiovisual equipment, computer software, skills laboratory equipment, office equipment, furniture, international and local technical assistance, workshops and training. Subcomponent 2.3: Developing an Item Bank Networking System to Support the National Competence Examination. 16. A valid examination requires the availability o f a secured pool o f high quality items and accountable management o f the items and the project would support the NACEHealthPro to establish an IT-based item bank networking system. Having the system w o u l d assist the Agency in developing standardized tools in items management, t o keep them current and representative o f all learning domains, and in improving the sustainability o f the test. Interventions under this subcomponent would include: (i) the development o f student assessment procedures and guidelines; (ii) overseas benchmarking fo r learning and networking with overseas item bank systems; (iii) development o f sofhvare for student assessment; (iv) national and regional training o n the use o f the software; (v) workshops on item development and item review; (vi) technical assistance; and (vii) monitoring and evaluation. Expenditures include I T and audiovisual equipment, computer software, furniture, international and national technical assistance, international benchmarking, workshops and training. Component 3: Results-based FAPfor Medical Schools (US$61.4 million). 17. Many o f Indonesia's health higher education institutions need to strengthen the performance o f their institutions to reach the current accreditation requirements. Investment to improve medical education quality and, more importantly, capacity-building o f the medical schools i s needed for future performance. The project therefore includes a competitive FAP t o create the opportunity for medical schools to obtain the needed resources to scale up their performance and achieve the national accreditation standards. The project would adopt three key principles in allocating these FAPs: (i) results- based allocation o f resources based o n evaluation o f annual performance; (ii) competition among fair medical schools according to their capacity; and (iii) partnership between leading and less-strong medical schools to build the capacity o f the less-strong schools according to their specific needs. 18. The project would have 3 FAP schemes: 0 Scheme A provides FAPs for 10 leading medical schools to build their international reputation and to strengthen Indonesia's global .competitiveness. Leading medical schools would get the FAP without competition but, in turn, each leading medical school has the obligation to assist one weak and one new medical school under Scheme B. The size o f the FAP i s US$l.O m i l l i o n for each leading medical school covering three years. 32 Scheme B provides FAPs to support weak capacity and new medical schools in achieving medical education standards mandated by the KKI through partnering with a leading medical school. The BHE would award the F A P to about 13 weak and seven new medical schools through a competitive selection process among medical schools within the group with similar capacity to ensure fair competition. The selected medical schools would implement the FAP in collaboration with a leading medical school. Each leading medical school would work with one weak and one new medical school. The F A P i s a maximum o f US$1.5 m i l l i o n for each recipient o f Scheme B funding covering a three year implementation period. Under Scheme C FAPs will support moderate-capacity medical schools in achieving medical education standards mandated by the KKI. The FAP will be awarded t o 10 medical schools through a competitive selection process. Moderate-capacity medical schools would implement the FAP individually without a partnership arrangement with a leading medical school. The F A P allocation i s a maximum o f US$1.5 m i l l i o n for each medical school over a three year implementation period. 19. The BHE i s responsible for establishing the guidelines for the FAP recipient selection, proposal approval process and overseeing FAP implementation. A FAP manual will be prepared by the BHE t o guide the implementation o f the FAP, including the criteria and grouping o f the 69 medical schools into leading, moderate, weak and new medical schools. There would only be one selection process during the lifetime o f the project to be conducted during the first year o f the project. 20. Under the three schemes the FAP could be used to finance interventions in the following areas: 0 Improving the implementation o f the CBC according to the SPICES model. Activities could include fulfilling the needs for establishing student-centered learning, early significant clinical exposure, adjusting student evaluation t o be consistent with the CBC, and periodic review o f the curriculum t o ensure achievement o f competencies; 0 Strengthening teaching, training and learning facilities. Activities could include modernizing and strengthening libraries, computer centers t o allow e-learning through e-libraries, and establishing electronic connectivity and high-speed internet facilities t o allow networking among the medical schools. The system will also assist communication between the leading and partner medical schools; 0 Development o f the medical faculty. Activities could include support for recruitment systems and the training o f clinical instructors, the training o f examination item writers, the training o f Problem Based Learning (PBL) problem writers, and the training o f PBL tutors; 0 Strengthening the Medical Education Unit. This could include staff recruitment system and physical/office facilities improvement, and staff capacity building through short- and long-term in-country and overseas training; and 0 Establishing a data management capacity. This could include building capacity to manage a database o n medical education, data analysis and reporting for education planning and development, institution decision making and accreditation purposes. 21. A range o f eligible expenditures have been determined under the FAP. These include workshops, teaching and laboratory equipment, degree and nondegree training, scholarships for poor students f ro m underserved areas (maximum 10 students per school), information technology, technical assistance, mi n o r building renovation and enhancing library collections. Under Scheme A, the allocation o f resources for goods should not exceed 20 percent o f the total package, while under Schemes B and C, the maximum allocation for goods i s 60 percent. Each F A P recipient would provide matching funds to the FAP received. The size o f the FAP under each scheme and the procedures for allocating the matching fund will be explained in the FAP Manual. 22. A s a result o f this project, the leading medical schools are expected t o b e better resourced and developed in the five areas described above. A large percentage o f the FAP funds for Scheme A should, therefore, be allocated t o build the schools' research and development capacity and their international reputation. Additional eligible expenditures for leading medical schools are a peer-reviewed research fund, overseas seminars and internships, publication fees, patent registration fees, staff and student exchanges, and overseas technical assistance. Selection o Leading Medical Schools Under Scheme A: f 23. An accreditation status o f `A' i s t h e basic c r i t e r i a f o r categorizing medical schools i n t o leading medical schools, and 16 medical schools can be categorized as leading medical schools. F r o m this group, 10 medical schools will be selected to become leading medical schools eligible for participation in the F A P program under Scheme A. The criteria for selecting the 10 schools will be included in the FAP Manual. Selection o f leading medical schools will entail a desk review o f existing data, and a field visit to check the level o f commitment o f the leaders in the leading medical schools. The six nonselected schools will be grouped with moderate medical schools, and will be eligible to participate in the F A P competition under Scheme C. Selection o Scltools under Scheme B: f 24. T h e aim o f the selection process is t o pair weak and new medical schools with a leading medical school t o establish a long-term collaboration in capacity building t h r o u g h structured and comprehensive partnership arrangements. The selection process would be as follows: W e a k and n ew medical schools in each cluster w o u l d prepare individual proposals guided by the FAP Manual. The BHE would also organize proposal preparation workshops to help the medical schools in developing their initial proposals; T h e proposal f r o m weak and n e w medical schools w o u l d include rigorous self-evaluation. This process would analyze the institution's strengths, weaknesses, opportunities and threats (a S WOT analysis), identify problems, gaps, and strategic issues and develop a strategic proposal addressing the needs; T he w ea k and n ew medical schools w o u l d s u b m i t t h e i r proposals t o t h e BHE. The BHE would organize a peer-review evaluation process o f incoming proposals to select about 13 proposals from weak (Scheme B l ) and about seven proposals f r o m n e w (Scheme B2) medical schools; T h e BHE w o u l d th en pair each w e a k and each n e w medical school with a leading medical school. Consideration would be given to the existing regional distribution o f medical schools and any previously established partnership to ease interaction between the medical schools. The BHE would establish the partnership in consultation with both parties t o minimize the risk o f an incompatible collaboration that could hamper implementation; and E a c h selected medical school w o u l d then prepare a j o i n t proposal with t h e p a r t n e r leading medical school. The j o i n t proposal will then be submitted for the second peer-review process to assess quality, feasibility and compatibility. The review process m a y include site visits as required. U p o n completion o f the review, the review team w o u l d provide feedback to the medical schools on the proposals and, depending o n the result o f the review, the schools would have an opportunity t o revise the proposal as appropriate before the FAP award process. 34 f Selection o Schools Under Scheme C: 25. T he selection process u n d e r Scheme C will b e similar t o that f o r Scheme B but w o u l d n o t include steps (iv) and (v). Program Management, Monitoring and Supervision: . 26. At the university level, th e rector has t h e ultimate responsibility f o r p r o j e c t implementation. A P I U will be established in the university receiving the FAP to support implementation and administration o f the project. In autonomous (BHMN) and public universities, the PIU will consist o f university staff holding structural positions, including those working in the medical school. Institutions receiving the FAP would be responsible for allocating their o w n resources t o manage the F A P which would be considered as the matching fund. These could include resources f o r staffing, office operational cost, office equipment and furniture, and other management expenses. 27. T he primary responsibility f o r m o n i t o r i n g the FAP implementation lies with the BHE. The BHE would do this through periodic meetings among the F A P recipients and through field visits. The BHE will conduct annual performance evaluation o f the FAP recipient according t o indicators listed in the PIP as a prerequisite for allocating the yearly tranches. There will also be monitoring o f the following aspects: (i) implementation o f the agreed PIP and overall disbursement o f the FAP resources; (ii) achievements in the five areas o f interventions (or six areas under Scheme A); (iii) performance in procurement o f resources and services; and (iv) overall management o f the FAP program. FAP monitoring will also assess the effectiveness o f interaction between the partner medical schools. The BHE will develop a supervision plan t o ensure the required information i s available o n time f o r the intended purposes. The C P C U will collaborate with the BHE to access the required information for preparation o f agreed project reports. World Bank Supervision 28. W o r l d Bank a p p ro v a l is required f o r t h e following stages in t h e FAP process: (i) FAP i) Manual; ( iselection a n d award process; and ( i ) annual p r o c u r e m e n t plan f r o m each FAP i ithe recipient and any revision o f th e procurement plan. Performance Indicators 29. There will be an evaluation o f the t r e n d in the mean test scores o f graduates f r o m the FAP recipient schools w h o have ta k e n the N a t i o n a l Competence Test. FAP recipient and non-FAP recipient schools would be compared, while the percentage o f graduates passing the national competency testing at their first attempt would be evaluated during project mid-term review and final evaluation. Component 4: Project Management (US$5.239 million). 30. Project implementation w o u l d be coordinated and administered by a CPCU established within the DGHE. Project resources would finance incremental operating costs, with the exception o f c i v i l servant salaries and honoraria, project management consultants, office equipment, furniture and project monitoring and evaluation, including project baseline, monitoring, and final evaluation. 35 Annex 5: Project Costs INDONESIA: Health ProfessionalEducation Quality Project Component 1 - Str for school accreditation I I Component 2 - Certification o f graduates using 1 10,440 1,120 1 11,560 national competency-based examination i Component 3 - Results-based FAP for medical 59,950 550 60,500 schools Component 4 - Project Management 4,250 410 4,660 Total Baseline Cost 80,320 ~ 2,650 82,970 Price Contingencies 3,530 i 200 3,730 Total Project Costs 83,872 j 2,850 86,722 I31<1) HE1 Total Project Cost By Financier ..-. .- __ US$,OOO usszoOO . g s , o o o 1. Training and workshops, incremental 6,584 600 01 7,184 operating costs, research expenditures, consultant services and goods under Component 1. 2. Training and workshops, incremental 12,099 800 01 12,899 operating costs, research expenditures, consultant services and goods under Component 2. 3. Financial Assistance Package under 55,000 0 Component 3. 4. Training and workshops, incremental 4,139 1,100 5,239 operating costs, consultant services and goods under Component 4. T o t a l Project Costs 1 77,822 36 Annex 6: Implementation Arrangements INDONESIA: Health. Professional Education Quality Project 1. The executing agency o f the project i s MoNE while the implementing unit i s the DGHE. M o H i s an important partner in this project and-although they are not an implementing agency-they will be closely involved in a l l policy discussions o n accreditation and certification. At the central level, project funds would be transferred into the budget document (DIPA) o f the DGHE. As the head o f the working unit (Satuan Kerja - SATKER), the Director General would be responsible for overall project implementation. The Authorized Budget Implementer (Kuasa Pengguna Anggaran - KPA) i s the Director o f Academic Affairs, DGHE. A Project Steering Committee would be established consisting o f representatives from MoNE, M o H , BAPPENAS, and Mo F . The steering committee would provide guidance and strategic directions f o r the project. 2. The Director General o f Higher Education would be the Project Director and the Director of Academic Affairs would be the Project Manager. The Central Project Coordination Unit (CPCU) would be established within the office o f the Directorate o f Academic Affairs. The Head o f the Sub-Directorate o f Curriculum and Study Programs w o u l d be the Vice- Project Manager, and would also function as the Commitment Maker (Pembuat Konzitmen - PK) within the SATKER. The treasurer o f the S A TK E R will also function as the treasurer o f the project. Payment verification will be conducted by the Finance Bureau o f M o N E . An Executive Secretary would be appointed to lead the Project Secretariat within the CPCU. The project secretariat will be staffed by selected staff from DGHE and will be responsible for managing procurement, financial management and general administration for the project. The Project Director will appoint a person in charge (PIC) for each o f components 1, 2, and 3 and for project monitoring and evaluation. The persons in charge would be the relevant heads o f subdirectorate within the Directorate o f Academic Affairs. The project would employ the services o f consultants to provide technical as w e l l as managerial assistance as necessary t o the Project Director. 3. A technical committee would be established by the Director General to ensure the project's annual plan i s consistent with intervention priorities identified during project preparation and to adapt the project as necessary to accommodate more recent developments. Membership o f the technical committee would consist o f representatives from Bappenas, MoNE, MoH, MoF, KKI, BAN-PT, associations o f Health Higher Education Institutions (AIPKI, AFDOKGI, AIPNI, AIPKIND) and teaching hospitals (ARSPI) and professional associations (IDI, PDGI, PPNI, IBI).The technical team i s also responsible for monitoring project performance and for ensuring the achievement o f project objectives. The BHE would be responsible for establishing the guidelines for the FAP, organizing and overseeing the selection process o f the FAP, and overseeing FAP implementation. Implementing units o f the project will include BAN-PT, the NAA, the NACEHealthPro, associations o f education institutions (AIPKI, AFDOKGI, AIPNI, AIPKIND), and professional associations (IDI, PDGI, PPNI, IBI). Funds for these entities would be included in the PMM. 4. At the university level, the rector has the ultimate responsibility for project implementation. A Project Implementation Unit (PIU) will be established in the university to support implementation and administration o f the project. In autonomous (BHMN) and public universities, the PIU will consist o f university staff holding structural positions, including those working in the medical schools. Establishment o f the PIU should be completed before a contract i s signed between the university and the DGHE. 37 Organization Structure at the Central Level for Project Implementation STEERING COMMITTEE PROJECT DIRECTOR Echelon 1 *BAPPENAS *MONE : 1 Director General of Higher Education PROJECT MANAGER TECHNICAL Board of Higher COMMITTEE Director of Academic Affairs Education *Echelon 2: I * BAPPENAS MONE VICE PROJECT MANAGER . Head of su b-Oirectorate of Curriculum & S t u d y P r o g r a m * ID1 AIPNI * PPNI * AIPKIND * IBI * ARSPI * AFDOKSI * PDGI Component 2 Component 3 Head of Section of Head of Section of Head of Sub-directorate Head of Sub-directorate Academic Achievements Academic Quality of Academic Evaluation of Academic Quality 38 Annex 7: Financial Management and DisbursementArrangements INDONESIA: Health Professional Education Quality Project A. Executive Summary and Conclusion 1. The purpose o f the project's financial management assessment i s to determine whether the financial management system o f the implementing agency, the DGHE under M o N E has the capacity to produce timely, relevant and reliable financial information o n the project activities and if the accounting system for the project expenditures and underlying internal controls are adequate t o meet fiduciary objectives and allow the Bank to monitor compliance with agreed implementation procedures and appraise progress towards i t s objectives. 2. The HPEQ Project i s designed as a S I L with total project expenditure o f US$77.822 million. The financial management assessment for the project was carried out in accordance with the guidance issued by the Financial Management Sector Board o n November 3, 2005." For this assessment, the team reviewed the financial management capacity o f DGHE (MoNE) including visits t o five possible F A P recipients representing different categories: leading medical schools (Airlangga University, University o f Nort h Sumatra - USU), a moderate capacity medical school (Hang Tuah University), a weak capacity medical school (Methodist University) and a new medical school (Bandung Islamic University/ UNISBA). 3. The overall risk o f the project i s assessed as `substantial' before mitigation and `moderate' after mitigation. This risk assignment i s based on (i) significant amount o f FAP allocation for selected the medical schools within universities (68 percent o f the loan) while the type o f expenditures could not be defined at this stage, exposing the project to higher risk; (ii) considerable amount o f soft expenditures the (training, workshop and incremental operating cost) vulnerable t o misuse; (iii) limited supervision and monitoring under similar ongoing projects within the DGHE (I-MHERE);and (iv) n o complaint handling mechanism i s yet in place. An action plan has been designed and included in this assessment report t o help mitigate those risks. 4. This assessment has concluded that, after the implementation o f the actions stated in the Proposed FM Action Plan below, the project will satisfy the Bank's financial management requirements as stipulated in OP/BP 10.02. B. Country Issues 5. The recently concluded PEFA assessment indicates that in recent years Indonesia has made significant changes in the way public fmances are managed and in increasing transparency and independent oversight. In almost a l l areas o f public financial management, a sound regulatory framework i s in place. Advances have been made in budget preparation, and in instituting a state budget that combines the previously separated recurrent and development budgets. However, internal control in the execution o f budget by spending agencies has not scored w e l l overall. Internal audit exist in a l l ministries but generally lack the capacity t o carry out risk based auditing. 6. The Bank and several other development partners are engaged with Go1 in providing assistance in several areas o f weakness. The Government Financial Management and Revenue Administration Project (GFMRAP) i s the Bank's primary channel for assisting the Go1 on these issues. The Bank i s also working lo"Financial Management Practices in World Bank-financed Investment Operations" 39 with internal auditors o f four line ministries-including MoNE and an external auditor-to strengthen their capacity. 7. The risk-mitigating measures proposed for the program have been designed taking the assessment o f the country-level P F M issues into account. Strengths and Weaknesses Strengths Prior experience o f the DGHE in implementing World Bank-funded projects provides familiarity with W orld Bank financial management and disbursement procedures and requirements. Providing grants to universities i s an ongoing program in DGHE. The DGHE would be able to use the manual for the implementation o f the I-MHERE program for HPEQ with some adjustments to accommodate special features o f the project. The visit t o universities found that, in general, universities have sound financial management practices in their operations, and with sufficient guidance from the DGHE/CPCU, participating universities would have the capacity to produce timely, relevant and reliable financial information on the FAP-financed activities. Weaknesses The project will involve competitive funding allocations to medical schools, amounting to US$50 m i l l i o n (68 percent o f the loan). Up to 40 percent o f the fund will be used for incremental operating costs, workshops and training considered as high risk expenditures. In addition, there are risks related to the transparency o f the FAP selection process, the adequacy o f complaint handling mechanisms and the adequacy o f monitoring mechanisms. Other than the FAP, about 20 percent o f the loan allocation i s intended t o finance expenditures namely training, workshops and incremental operating costs, increasing the risk exposure o f the project. Learning f ro m I-MHERE implementation, the supervision and monitoring o f financial assistance at the university level by the C P C U was weak, and there i s a risk that similar situations will recur in the project. Risk Assessment Summary A detailed analysis o f the financial management risks arising from the country situation, the proposed project entities, and specific project features and related internal controls was completed during the assessment, and i s summarized below. These risks have been rated o n a scale o f high, substantial, moderate and Zow. The overall fmancial management risk i s assessed as `substantial' before mitigation and `moderate' after mitigation provided that a l l mitigating measures are effectively implemented and the project i s effectively supervised. 40 _- I1 I Sumntary Comments Itcsidual Condition of And Risk Mitigarioa Risks h'cgotiationdo f' f.:ftectitcncss I____ j (Y/K'!)- Country L e v e l I 1. Public Financial Government recognizes existing weaknesses Management in public financial management and has several programs t o improve it, including through the Bank-financed project (GFMRAP). However, making substantial progress on the country issues has been s I" -- Entity Level 1. Implementing S The project will be implemented by the M Y, TOR o f Entity/Organization DGHE (MoNE). DGHE i s currently consultant by Status o the entity f implementing I-MHERE (IBRD 4789-IND negotiations and IDA 4077-IND). The supervision o f I - MHERE project implementation at the university level by the C P C U has been weak, and this i s related t o the reluctance o f M o F at that time t o use the loadcredit fund t o f i a n c e consultants t o assist the CPCU i n supervising the project. Risk Mitigation: The DGHE has agreed t o recruit consultants t o assist with the supervision o f project implementation M H more limited. The FAP will only be given to selected universities with a medical faculty. Selection criteria and guidelines are currently under preparation. Risks may arise due t o poor supervision and monitoring mechanism o f the program implementation resulting from unavailability o f DGHE staff and a limited budget for supervision as found in I-MHEREimplementation. Risk Mitigation: The guidelines should include but n o t be limited t o the following: 1. Dissemination o f information about the FAP program; 2. Publication o f the F A P proposal and the use o f the fund in the university/medical faculty announcement board; 3. Multiple transfer o f the FAP fund based o n satisfactory progress; 41 4. Development o f a clear monitoring mechanism from the central level and ensuring resources are available t o conduct monitoring; 5. Establishment o f a complaint handling mechanism. Details o f the mechanism n are discussed i the project's Anti- Corruption Action Plan (ACAP); and Consultants will be hired t o assist the CPCU in conducting project supervision Overaft Project Risk - B. Control Risk 1. Budgeting M Budget preparation for FAP t o universities in M Delay in issuing and the DGHE i s w e l l defmed, and while there effectiveness o budget f have been delays in execution, performance documents is improving. Decrees o n budget execution for the program should be issued immediately after budget documents are issued. 2. Accounting M The DGHE will follow government M Reliability of accounting standards. accounting system 3 . Internal Control S Payment verification will rely o n the M Y, Final Draft Inadequate payment government system. Internal control related PMM by verification risk mainly occurs at the university level. Negotiations A t the DGHE level, payment o n project expenditures i s verified by the finance unit at the M o N E Secretary General level. A t the university level, each faculty has i t s own budget, and payment o n expenditures i s verified by the f i a n c e unit at the university level. Learning from past experience, the risk was related to difficulties i n conducting verification o f payment for soft expenditures. Risk Mitigation: 1. Publication o f the F A P proposal and the use o f the F A P fund o n the university/medical faculty announcement board. l . There will b e a separate bank account for the FAP. ?. There will be a specific reporting format acceptable t o the W o r l d Bank o n the use o f the FAP. 42 4. All contracts and receipts should have a note stating all vendors as-e subject to review by MoNE's IG/BPKP/BPWKPK. 4. Flow o f Funds M Fund transfer to the university account i s in tranches and will be based on satisfactory performance o f the FAP recipients. ~ M 5 . Internal Audit M There i s extensive involvement o f the I G in conducting internal audit reviews o f grants DGHE to I G and program to universities but the report i s TORfor internal mainly on administrative matters. audit by negotiations Risk Mitigation: Internal audits w i l l be systematically undertaken at regular intervals o f the university receiving the FAP from the DGHE based on a TOR acceptable to the World Bank. The DGHE will issue a letter to the IC, MoNE requesting the I G to conduct internal audits o f the project. Copies o f the audit reports are to be provided to the World Bank. 6. External Audit I S The program will be audited by BPKP-based M Y, letter from on a TORacceptable to the World Bank. The DGHE to BPKP project audit report should be submitted to and TORfor the World Bank within six months after the external audit by end o f fiscal year. Annual audit TORfor the negotiations Y project and appointment o f auditor has to be acceptable to the World Rank. -- __ O t c r d l Cvntrol R i d :i G. Implementing Arrangements 11. The executing agency o f the project i s MoNE while the implementing unit i s the DGHE. M o H i s an important partner in this project and-although they are not an implementing agency-they will be closely involved in a l l policy discussions o n accreditation and certification. At the central level, project funds would be transferred into the DIPA o f the DGHE. As the head o f the SATKER, the Director General would be responsible for overall project implementation. The KPA i s the Director o f Academic Affairs, DGHE. A Project Steering Committee w o u l d be established consisting o f representatives from MoNE, M o H , BAPPENAS, and M o F . The steering committee would provide guidance and strategic directions for the project. 12. The Director General o f Higher Education would be the Project Director and the Director of Academic Affairs would be the Project Manager. The Central Project Coordination Unit (CPCU) would be established within the office o f the Directorate o f Academic Affairs. The Head o f the Sub- Directorate o f Curriculum and Study Programs w o u l d be the Vice-Project Manager, and w o u l d also function as the Commitment Maker (Pembuat Komitmen - PK) within the S A T K E R . The treasurer o f the S A T K E R will also function as the treasurer o f the project. Payment verification will be conducted by the 43 Finance Bureau o f M o N E . An Executive Secretary would be appointed to lead the Project Secretariat within the CPCU. The project secretariat will be staffed by selected staff f r o m DGHE and will be responsible for managing procurement, financial management and general administration for the project. The Project Director will appoint a person in charge (PIC) for each o f components 1, 2, and 3 and for project monitoring and evaluation. The persons in charge would be the relevant heads o f sub-directorate within the Directorate o f Academic Affairs. The project would employ the services o f consultants to provide technical as w e l l as managerial assistance as necessary to the Project Director. 13. At the university level, the rector has the ultimate responsibility for project implementation. A P I U will be established in the university to support implementation and administration o f the project. In autonomous (BHMN) and public universities, the P I U will consist o f university staff holding structural positions, including those working in the medical schools. Establishment o f the PIU should be completed before a contract i s signed between the university and the DGHE. 14. The' DGHE i s currently implementing I-MHERE (IBRD 4789-IND and IDA 4077-IND). In i t s FY 2007 annual audit, the project received an unqualified audit opinion with follow-up action o n weaknesses in the internal controls related to a requirement to improve the control system in general and in procurement in particular as w e l l as in project monitoring and payment verification. A letter requesting follow-up action o n audit findings was sent on September 5, 2008 and a response received during supervision o n February 2009 indicating follow-up action o n findings at the central level has been completed while the status o f the findings at the university level has not been updated. 15. As part o f the project's financial management assessment, F M S visited five universities representing each category o f FAP recipients, namely the following: (i)Airlangga University and the University o f N o r t h Sumatra (USU)-leading medical school category; (ii)Hang Tuang University-moderate capacity medical school category; (iii)Methodist University-weak capacity medical school category; and (iv) Universitas Islam B a n d u n m I S B A - n e w medical school category. 16. The visit found in general that universities have sound financial management practices in their operations, and with sufficient guidance from the DGHE/CPCU, participating universities would have the capacity to produce timely, relevant and reliable financial information o n the FAP-financed activities. H. Financial Management Arrangements 17. The project accounting policies and procedures for this project will f o l l o w the government accounting standards (PP N o . 2412005) and government accounting systems (PerMenKeu No . 59/PMK.o6/2005) issued in accordance with Finance and Treasury Laws. The following are the summary procedures from budgeting to reporting. All project activities and expenditures will be included in the central government budgets and administrative and accountability procedures for approving expenditures and disbursing funds to beneficiaries will follow government procedures. Budgeting 18. The project budgeting system will follow the existing government budgeting system. The project budget will be included in the annual government budget and M o N E budget document (DIPA). There i s a risk that the late issuance o f DIPA m a y result in project implementation delays. Delay in DIPA issuance and effectiveness ma y be minimized by the early preparation o f the decree for the working unit 44 (SATKER). The team would be able t o develop the implementation plan early and make the required adjustments when the approved DIPA differs from the proposal. Accounting and Reporting 19. All financial transactions will be recorded in the government accounting system and included in government accountability reports. The CPCU will prepare an IFR suitable f o r project-monitoring purposes and submit the report to the World Bank o n a quarterly basis in a format agreed with the World Bank. The IFR will include inputs received f i o m the participating universities. Fund Flow Expenditures at the DGHE Level 20. The DGHE shall be responsible for procurement and activities under the project components in accordance with the agreed budget. When expenditures are due for payment, the DGHE will submit an SPP (payment request) to the finance unit within M o N E f o r eligibility review (for eligibility o f expenditures and documentation completeness). The finance unit will then issue an S P M (payment order). The SPM and the supporting documentation will then be submitted to the KPPN (government treasury office), followed by the issuance o f the SP2D (remittance instruction) to B I or other government bank to credit the payee's accounts at their respective banks and to debit the project's Special Account for the W orld Bank portion. Expenditures at the University Level 2 1. A contract agreement will be entered into by each participating university with DGHE (MoNE). Each university will prepare the first IFR (university IFR) to request the first quarterly payment o f the contract. Subsequent payments can only be made based o n the submission o f the university's subsequent IFRs. The IFRs will also acquit the use o f funds received f r o m the previous IFR. Based o n the university's IFR (first and subsequent), the DGHE will submit a request for payment (SPP) t o the finance unit within M o N E for eligibility review (for eligibility o f expenditures and documentation completeness). The finance unit will then issue the SPM. The S P M and the supporting documentation will then be submitted t o the KPPN followed by the issuance o f the SP2D (remittance instruction) to B I or other government bank t o credit the payee's accounts at their respective banks and t o debit the project's Special Account for the Wo rl d Bank portion. I. Action Plan to Mitigate the R i s k s 22. The overall risk o f the project i s assessed as `substantial' before mitigation and `moderate' after mitigation. This risk assignment i s based o n (i) significant amount o f FAP allocation f o r selected the medical schools within universities (68 percent o f the loan) while the type o f expenditures could not be defined at this stage, exposing the project to higher risk; (ii) considerable amount o f soft expenditures the (training, workshop and incremental operating cost) vulnerable t o misuse; (iii) limited supervision and monitoring under similar ongoing projects within the DGHE (I-MHERE); and (iv) n o complaint handling mechanism i s yet in place. An action plan has been designed and included in this assessment report to help mitigate those risks. 45 1. A Central Project Coordination Unit (CPCU) i s established and suitably staffed. The SATKER which sets Satker arrangements arrangements should allow for adequate segregation o f and CPCU establishment and duties between project financial verification functions staffing acceptable to the World and project implementation. Bank. 2. Appointment o f consultants to assist the supervision TORo f consultant acceptable to Completed and monitoring o f project implementation, including the Bank. that at the university level. - P M M to include procedures to be followed by all Final draft P M M acceptable to Completed implementing units o f the project. This should include: the World Bank. (i) financial management and disbursement all procedures for the project; (ii) annual budgeting and Final P M M acceptable to the By Effectiveness i) work plan; (i ienhanced control on payment; (iv) World Bank and formally adopted segregation o f duties among payment authorization and by DGHE. "commitment maker" functions; (v) financial reporting formats; (vi) supervision; (vii) internal audit arrangements; and (viii) anti-corruption plan. FAP Manual to also include the following: Final FAP Manual acceptable to Before , (i) i) socialization process; ( iproposal preparation and the World Bank and formally disbursement o f i iflow o f funds; (iv) guidelines for review process; ( i ) adopted by DGHE. Category 3 withdrawal and accountability mechanisms; (v) supervision and monitoring mechanisms; and (vi) complaint handling mechanisms. C. Training Training o f all project staff on the necessary skills to P M M and FAP Manual training Ph4M Training as carry out respective duties as described in the P M M the f i s t activity and FAP Manual. after loan effectiveness, F A P Manual training prior to selection process. D. Internal Audit Arrangement with the I G to conduct internal audit Letter o f confirmation to the Completed based on TORacceptable to the World Bank. World Bank informing the internal audit arrangements o f the project. Copies o f these audit reports to be provided to the World Bank. Copies o f internal audit reports to During be sent to the Bank and the implementation external auditor. E. Annual audit Arrangement o f annual audit in accordance with D G H E letter .to the auditor Completed specific TORacceptable to the World Bank. (acceptable to the World Bank) confirming the audit arrangements (on special purpose financial statement o f the project) with audit TOR attached. 46 J. Disbursement Arrangements 23. The applicable disbursement methods are by: (i) i) advance; ( i reimbursement; and (iii) direct payment. In order to facilitate disbursements, a Designated (Special) Account (DA) denominated in U S dollars will be opened by DG Treasury (MoF) at BI or a commercial bank acceptable to the Bank in the name o f the M o F . The DA will be used solely t o finance eligible project expenditures. The ceiling o f advance to the DA i s US$7 million. 24. DG Treasury will authorize i t s relevant Treasury Offices (KPPNs) located near the implementation units t o authorize payments o f eligible project expenditures by issuance o f SP2D (remittance orders) charging the DA. F o r this purpose, DG Treasury shall issue a circular letter t o the relevant K P P N Offices providing guidelines and criteria for eligible project expenditures in accordance with the grant agreement. Although management o f the DA will be under the responsibility o f DG Treasury, the CPCU will be responsible for reconciling the DA and preparing applications for withdrawal o f advances and reporting the use o f the DA, duly approved by DG Treasury before their submissions t o the Bank. Copies o f DA bank statements will be provided to the C P C U by DG Treasury. 25. Applications for reporting o f the use o f DA funds will be supported by: (i)s t o f payments together li with records evidencing such expenditures, against contracts that are subject to the Bank's prior-review; or (ii) statement o f expenditures (SoEs) fo r a l l other expenses; and (iii) reconciliation statement. Reporting DA o f use o f DA funds and application for an advance to DA m a y be submitted in a single application. Applications for reimbursement will be supported by the same documentation referred to earlier as w e l l as evidence that payments were made (for example a bank statement). Applications for direct payment will be supported by records evidencing eligible expenditures (for example, copies o f receipts or supplier invoices). Frequency o f application reporting expenditures paid from the DA will be o n monthly basis. The minimum size o f application for direct payment i s US$lOO,OOO. 26. The CPCU and PIUs will prepare and submit quarterly IFRs. When an IFR has been produced o n time, the applications for reporting o n use o f DA funds will be based o n the IFRs and an advance to the DA will be made for six-monthly projected expenditures. The Bank will n o t i f y M o F o f this change as appropriate. 27. All documentation for expenditures will be retained at the implementing unit and shall be made available to the auditors for the annual audit and to the Bank and i t s representatives if requested. FAPs will be disbursed based satisfactory performance by schools-but eligible expenditures are as actually identified. To be. clarified in minutes o f negotiations. 47 K. Internal Control: Internal Audit 29. Internal audits are to be systematically undertaken at regular intervals at the central and local level based on TORacceptable to the World Bank. The DGHE should issue a letter to MoNE's I G requesting the I G t o conduct internal audits o f the project. Copies o f the audit reports should be provided t o the Bank. L. External Audit Arrangements 30. The project's financial statements will be audited annually. The annual audit reports and audited financial statements shall be furnished t o the W o r l d Bank not later than six months after the end o f the government fiscal year (June 30 o f the following year). The audit will be conducted in accordance with the audit TORacceptable to the Wo rl d Bank and agreed at negotiations. TYPE OF AUDIT REPORT EXEC~TING 2 DUEDATE AGENCY . . Project Financial Statement (a single audit DGHE (hloNE) Six months after the end o f the opinion comprising the project account and government's fiscal year. DAiSoE account) 48 Annex 8: Procurement Arrangements INDONESIA: Health ProfessionalEducation Quality Project A. General 1. Procurement for the proposed project would be carried out in accordance with the ,World Bank's Guidelines: "Procurement under IBRD Loans and IDA Credits" dated M a y 2004 and revised in October 2006; and Guidelines: "Selection and Employment o f Consultants by World Bank Borrowers" dated M a y 2004 and revised in October, 2006, and the provisions stipulated in the Loan Agreement. The various items under different expenditure categories are described below: 2. For each contract to be financed by the Loan, the different procurement or consultant selection methods, estimated costs, prior review requirements, and time frame have been agreed between the Government o f Indonesia and the Bank in the Procurement Plan. The Procurement Plan shall be updated at least annually or as required to reflect the actual project implementation needs and improvements in institutional capacity. 3. For National Competitive Bidding (NCB) and Shopping, standard documents, acceptable to the Bank shall be used. The N C B standard documents shall incorporate the Bank's N C B conditions. For selection o f consulting firms, the Bank's Standard Request for Proposals shall be used. For NCB, the Borrower shall follow the procedures outlined in Keppres No. 80/2003, as supplemented with additional conditions which have been agreed with the Go1 and included as an attachment to this Annex. 4. Procurement o f Goods. Contracts estimated to cost US$200,000 or more shall be procured through ICB. For contracts which are estimated to cost less than US$200,000 equivalent, procurement may be carried out through NCB. While those estimated to cost less than US$50,000 equivalent, procurement may be carried out through Shopping. 5. The procurement o f goods under this project are likely to include: skills laboratory equipment, text books, computers and laptop, I T equipment, printing, and furniture. The procurement will be done using the Bank's Standard Bidding Documents (SBD) for all ICB, the National SBD (for NCB), and standard requests for quotations (for shopping) agreed with or satisfactory to the Bank. 6. Procurement o f Works. All procurement for contracts above US$3 million or equivalent per package will follow I C B procedures. Procurement for contracts o f less than US$3 million or equivalent will follow N C B procedures. Procurement for contracts less than US$50,000 or equivalent should follow shopping procedures. 7. The procurement o f works under this project will mostly be small rehabilitation, most likely for the skills laboratory o f the participating higher education institutions. The procurement will be done using the Bank's SBD for all ICB, the National SBD (for NCB), and standard requests for quotations (for shopping) agreed with or satisfactory to the Bank. " 8. Goods to be procured by participating higher education institutions will probably be o f similar characteristics and nature, such as skills laboratory equipment, textbooks, and computers. Considering the weak capacity o f participating institutions in undertaking public procurement, and to ensure competitiveness, efficiency, and smooth procurement process, it was agreed with the Borrower that procurement for goods with similar characteristics and nature from the participating Higher Education Institutions will be coordinated through a joint procurement process, taking into account the institution's geographical location. The Higher Education Institutions concerned will coordinate a joint procurement 49 process. A staff member fro m DGHE with experience in W o r l d Bank guidelines f o r procurement will be a part o f the committee. In addition, the C P C U will recruit a project management consulting firm to assist the CPCU in managing procurement under the FAP program, particularly by providing procurement assistance to the Higher Education Institutions and conducting capacity building as appropriate. Goods which are not similar will be procured individually by the respective Higher Education Institutions. These packages are expected to be small (less than US$50,000), and therefore, the procurement should follow the shopping procedures. These Higher Education Institutions will also be monitored and supported by the CPCU. 9. Consultant Selection. Except for contracts estimated to cost less than US$200,000, selection o f consultants (firms) will follow the Quality-and Cost-Based Selection (QCBS) method. There will be one contract o f Project Management Consultant (procurement specialist, financial management specialist, and monitoring and evaluation specialist) with a total cost estimate o f about US$500,000 equivalent selected in accordance with the QCBS method. In cases o f contracts estimated to cost less than US$200,000 equivalent, the Selection Based o n Consultants' Qualifications (CQS) method may be used. The participating higher education institutions will mostly select individual consultants; n o selection o f consultant f i r m s i s expected. 10. Shortlists o f consultants for services estimated to cost less than US$400,000 equivalent per contract ma y be composed entirely o f qualified national consultants in accordance with the provisions o f paragraph 2.7 o f the Consultants Guidelines. B. Assessment o f the Capacity o f ImplementingAgency's to Implement Procurement 11. The DGHE as the central project coordination unit (CPCU) will do the procurement o f goods and services under components 1, 2, and 4 o f the project, while the public and private higher education institutions will do the procurement o f works, goods, and services under component 3. 12. The assessment was conducted o n April 1 t o 8, 2009 which included j o i n t field visits with the task team's FM Specialist, and also with members o f the project preparation team o f the Borrower. Discussions were conducted with five candidate H E I s to represent the various capacity levels o f the implementing units. These were two public universities (University o f Airlangga and University o f N o r t h Sumatra), and three private universities (Hang Tuah University, Methodist University o f Indonesia, and Bandung Islamic University). The sample universities were based o n the three schemes, leading medical schools, weak capacity and new medical schools, and moderate capacity medical schools. During the assessment it was found that the private universities visited have limited capacity t o undertake public procurement, and the system in place to administer procurement i s inadequate. Private higher education institutions procurement guidelines and procedures are adopted f r o m Keppres N o . 80/2003. The largest procurement conducted was around US$40,000 per package using the shopping procedures. These institutions have, o n average, two procurement staff who have attended the public procurement certification training conducted by BAPPENAS. However, this does not ensure adequate procurement capacity. The bidding committee i s an ad hoc team consisting o f members from various departments. 13. The Bank carried out an assessment o f the capacity o f DGHE as the implementing agency to implement procurement actions for the project. The DGHE (MoNE) has experience in implementing Bank-financed projects through the implementation o f IBRD 4789-IND and IDA 4077-IND (Indonesia Managing Higher Education for Relevance and Efficiency Project), and ADB L 1 7 9 2 - I N 0 (Technical and Professional Skills Development Sector Project). The public universities have experience in Bank's procurement under IBRD 4789-IND and IDA 4077-IND (Indonesia Managing Higher Education for Relevance and Efficiency Project). 50 14. M o s t findings and recommendations below are based on an assessment o f the DGHE (MoNE), and some participating higher education institutions, which was undertaken by the Bank in April 2009. The Bank identified the following key issues and risks concerning procurement for implementation: Corruption and Collusive Practices: This has been a nationwide issue, which has normally involved both bidders/consultants and government officials. The general procurement environment i s weak; Capacity o f the procuring units: Some o f MoNE's staff are familiar with the Bank's procurement procedures, however there i s no guarantee that they will be assigned as procurement staff for this project since the decree establishing the team i s yet to be issued; Independency o f procurement decisions: Although M o N E and the members o f the bidding/evaluation committee are authorized t o make their o w n independent decisions in accordance with Keppres N o . 80/2003, they can sometimes be influenced by other parties such as higher-level officials (for example, their superiors) or by external parties(for example, bidders or consultants). This can cause significant delays in the procurement processing, and can result in contracts being given to low-quality consultants/contractors and to the delivery o f low-quality services/works. Procurement delays: L o w capacity in knowledge o f Bank Procurement Guidelines and procedures, late initiation o f the procurement process and long evaluation process (especially in hiring consultants) normally contribute to significant delays in the award o f contracts. Unavailability o f detailed implementation/procurement plans at initial stage o f implementation: The risk i s particularly associated with the grant component to H E I s . The names o f participating H E I s along with their procurement packages which will be included in the grant proposal m a y not be identified at the time o f negotiation or even at Loan Effectiveness. Table 1 below presents a l i s t o f measures agreed between the Go1 and the Bank to mitigate the abovementioned risks. The overall procurement risk o f the project i s considered `high'. Table 1: Agreed Corrective Measures Decree to establish the project's organizational Negotiations MoNE structure and staffing, including procurement officer. * A t least one procurement workshop conducted prior to Agreed at Appraisal MoNE the project launch, attended by all procurement committees, including MoNE-DGHE staff who will be involved in the project's procurement. Draft P M M developed to streamline all agreed Negotiations MoNE procurement procedures and reporting under this project. Standard FWP and SBDs for use in the project. Negotiations lote: *The CPCU will recruit consultantk to assist the procurement ol :er in conducting procurement under the project. The consultar can be an individual consultantis or consulting firm depending on the procurement workload o f the project. The services o f a consulting firm will most likely be required whenthe FAP progam under Component 3 has started. 51 C. Procurement Plan 15. The Borrower has developed a procurement plan under component 1, 2, and 4 o f the project. This plan has been agreed between the Borrower and the Project Team o n April 30, 2009 and i s available at the DG o f Higher Education (DGHE). I t will also be available in the project's database and in the Bank's external website. The Procurement Plan will be updated in agreement with the Project Team annually or as required to reflect the actual project implementation needs and improvements in institutional capacity. D. Frequency of Procurement Supervision 16. In addition to the Bank's prior review and supervision missions, post review o f procurement actions will be conducted by the Bank on an annual basis. E. Details o f the Procurement Arrangements Involving International Competition Goods, Works, and Nonconsulting Services The following table lists contract packages to be procured following international competitive bidding: r I Descriptiol-- ~~ 1 -- - Est. Cost 1 (C!SCj Procurement Method P-v l)&tcstic 1 Prcfcrcricc .. , ' Review By Expected Bid Opening I .- I- iiiiltioii) -i __ .I I ~ i-!k!l\ .- 7 - l l _ l _, 1 Desktop Computer and 0.49 IC B No No Prior June 2010 Laptop Desktop and Stabilizer 0.25 ICB No No Prior September 20 10 Skills Lab Instrument 1.24 ICB No No Prior June 20 10 The first N C B for works and goods conducted by each HEI, a l l ICBs, and a l l direct contracting will be subject t o prior review by the Bank. Consulting Services The following table lists the consulting assignments: 17. Consultancy services (firms) estimated t o cost US$200,000 or above per contract, US$SO,OOO (individuals) or above per contract and any single-source selection o f consultants (firms) will be subject to prior review by the Bank. In addition, the TORfor a l l consulting services will be reviewed by the Bank prior to entering a contract. 18. Shortlists o f consultants for services estimated t o cost less than US$400,000 equivalent per contract ma y be composed entirely o f qualified national consultants in accordance with the provisions o f paragraph 2.7 o f the Consultants Guidelines. 52 Attachment A to Annex 8 ClarificationsRelating to National Competitive Bidding Procedures I. General The procedures to be followed for N C B shall be those set forth in Keppres No. 80/2003 with the clarifications and modifications described in the following paragraphs required for compliance with the provisions o f the "Guidelines for Procurement under IBRD Loans and IDA Credits" dated M a y 2004 (the Guidelines). 2. Registration (a) Bidding shall not be restricted to preregistered f i r m s and shall not be a condition for participation in the bidding process; and (b) Where registration i s required prior t o award o f contract, bidders: (i)shall be allowed a reasonable time t o complete the registration process; and (ii) shall not be denied registration for reasons unrelated t o their capability and resources to successfully perform the contract, which shall be verified through post qualification. 3. Prequalification 0 Prequalification shall not used for simple goods and works. Normally, post qualification shall be used. Prequalification shall be required only for large o r complex works with the prior `no objection' o f the Bank. W h e n pre- qualification shall be required: 0 o (b) Eligible bidders (both national and foreign) shall not be denied pre- qualification, and o (b) Invitations to pre-qualify for bidding shall be advertised in at least one (1) widely circulated national daily newspaper a minimum o f thirty (30) days prior to the deadline for the submission o f pre-qualification applications. 4. Joint Ventures A bidder declared the lowest evaluated responsive bidder shall not be required to form a joint venture or to subcontract part o f the work or part o f the supply o f goods as a condition o f award o f the contract. 5. Preferences (a) N o preference o f any kind shall be given to national bidders. (b) Regulations issued by a sectoral ministry, provincial regulations and local regulations, which restrict national competitive bidding procedures to a class o f contractors or a class o f suppliers shall not be applicable to procurement procedures under the Credit or the Loan. 53 6. Advertising (a) Invitations to bid shall be advertised in at least one (1) widely circulated national daily newspaper allowing a minimum o f thirty (30) days for the preparation and submission o f bids and allowing potential bidders to purchase bidding documents up t o twenty-four (24) hours prior the deadline for the submission o f bids. (b) Bid documents shall be made available, by m a i l or in person, to a l l who are willing t o pay the required fee. (c) . Bidders domiciled outside the area/district/province o f the unit responsible for procurement shall be allowed to participate regardless o f the estimated value o f the contract. (d) Foreign bidders shall not be precluded from bidding. If a registration process i s required, a foreign firm declared the lowest evaluated bidder shall be given a reasonable opportunity for registering. 7. Bid Security Bid security, at the bidder's option, shall be in the f o r m o f a letter o f credit or bank guarantee f i o m a reputable bank. 8. Bid Opening and Bid Evaluation (a) Bids shall be opened in public, immediately after the deadline for submission o f bids, and if bids are invited in two (2) envelopes, both envelopes (technical and price) shall be opened at the same time. (b) Evaluation o f bids shall be made in strict adherence to the criteria declared in the bidding documents and contracts shall be awarded t o the lowest evaluated bidder. (c) Bidders shall not be eliminated f r o m detailed evaluation o n the basis o f minor, non-substantial deviations. (d) N o bidder shall be rejected merely o n the basis o f a comparison with the owner's estimate and budget ceiling without the AssociatiodBank's prior concurrence. 9. Rejection o f Bids (a) All bids shall not be rejected and new bids solicited without the Bank's prior concurrence. (b) When the number o f responsive bids i s less than three (3), re-bidding shall not be carried out without the Bank's prior concurrence. 54 Annex 9: Better Governance Action Plan Indonesia: Health ProfessionalEducation Quality Project 1. The HPEQ project aims at contributing t o the improvement o f community health status by strengthening the quality o f the health workforce. This will be achieved through the following components: (i)Strengthening policies and procedures for school accreditation; (ii)Assuring the standard o f education quality through certification o f graduates using national competency-based examination; and (iii) Improving school quality through results-based FAP funding. 2. Learning from the experience o f similar projects such as I-MHERE, issues m a y arise- particularly in the provision o f financial incentives and the implementation o f subprojects. There are r i s k s o f kickbacks paid associated with provision o f grants and procurement. The risks m a y be aggravated by lack o f transparency-in the selection and award o f grantees and in the use o f funds-and lack o f capacity, especially in procurement. Mitigation o f R i s k s 3. A substantial proportion o f the IBRD loan for this project will be disbursed as grants to medical schools and professional associations. It is, therefore, critical for the project t o be transparent on the procedures for selection criteria and evaluation method, and in announcing the process o f selection and award o f grants, at least in the project website. 4. In order to ensure fairness o f the process, a system should be developed t o resolve complaints. Various channels should be provided to report complaints, and the channels should be w e l l communicated and made available at the website. 5. During project implementation, problems m a y occur due t o lack o f capacity or experience in Bank-funded projects. The Project Management Manual will specify measures t o mitigate risks in procurement, including provisions on wider advertisement, disclosure and reporting. 6. The DGHE will disseminate information and w o r k with professional associations, school associations, national and local media and other c i v i l society organizations in monitoring the progress and resolving issues in the course o f the project. 7. Reasonable sanctions should be imposed on project implementers w h o abuse the power entrusted t o them, such as by suspending disbursement or making them ineligible for future grants. In a l l procurement contracts, evidence o f corruption, collusion and fraud are grounds for termination o f contracts, with possible additional penalties imposed. Lessons learned will be widely disseminated through the project website and newsletter. S., Specific terms o f reference provided to the monitoring and evaluation officer will include, inter alia, implementation o f the plan and resolution o f complaints. A matrix outlining the risks to good project governance and the proposed measures to mitigate them i s provided below. 55 R i s k s Matrix: Corruption Opportunities for Corruption Mitigation Measures Mapping Area DISCLOSURE OF. E'ORhl.4TlON Dissemination o f - Lack o f transparency; and - DGHI? \\fill establish a project-specific \\ ebpagc;o information - N o third party monitoring. display critical information about the project; - Information such as manuals, selection announcements, criteria and procedures, awards, procurement-related information, and numbers to register complaints should be made available; - Civil society organizations should be invited at least to the project launching; and - The P M U w i l l disseminate information and w i l l work with civil society organizations interested in monitoring the project. __ OF F ~ t ~ ~ P I ~ ~ ~ ?X I 4SCIAL IYCENTIVES ._ I Selection o f - Lack o f transparency: - Announce the selection on website and grantees - Unfair process: and newspapers; - Conflict o f interest. - Publication o f criteria and procedures for selection process; - Announce award on the website; and - Establish and communicate complaints handling mechanism. Implementation - Misuse o f funds. - Tighter supervision by both the government and the Bank team; - Annual audit performed by BPKP and I G to include the use o f funds by individual schoolslassociations; and - Enforce sanctions. Flow o f funds - K i c k backs to government officials. - Define transparent criteria for payment o f grant tranches in the Project Operational Manual. Reporting and Filing . Project documents (such as - Establish clear guidelines in the Project procurement, financial, contract, audit, Operational Manual on reporting and filing; implementation reports) are - Tighter supervision by both the government and intentionally available or not well the Bank team; and documented. - Annual audit performed by BPKP and I G to include the use o f funds by individual schoolslassociations. Accreditation and . Nontransparent and nonobjective - Activities inherent in project components are certification process process o f evaluation; and designed to strengthen accreditation and Bribes paid to obtain better evaluation. certification process to address such risks. P Capacity o f n Lack o f independent judgment i - Build capacity o f all actors involved in Procurement evaluation process. procurement; Committees - Procurement w i l l be under the coordination o f 56 Clorruption Opportunities for Corruption Mitigation Mcasures Slapping Area CPCU and supported by the Project Management Consultant Firm; - Representatives o f relevant universities will be invited to conduct a joint bidding process for procurement o f goods with similar technical specifications with facilitation from a procurement consultant at the central level. Advertising - Improper advertising (for instance, - Project Operational Manual to establish standard using newspapers with limited advertisement for the project, and minimum circulation); and requirements for newspapers to be used. - Fictitious advertising. Quality of delivered - Delivered products/services are o f lower - Quality evaluated as part o f monitoring and productslservices n quality than t h e ones specified i the evaluation; and technical specification; and - Enforce sanctions as defined in Keppres No. - Kickbacks from contractors, suppliers or 80/2003 and other relevant Indonesian laws. consultants. " - ~ _ _ _ T I ;SHOPS - I Reporting - False reporting. Project Operational Manual w i l l define supporting documents to be presented for payments. Regular audit w i l l be performed to verify payments. _ " __ __ .. - -- . ~- )LING . I____ __ . - - Complaints - Issues not recognized and addressed. - DGHE will establish a project-specific page to the Resolution existing website. The website will displa). information on the project, selection and procurement-related information, manuals. and channel to register complaints: - Other channels o f complaints should be available (mail, text messages, phone hotline). The website address, phone numbers and mail address will be stated in project documents and publications: - DGHE w i l l be staffed with personnel responsible for managing complaints; - Serious complaints will be investigated by IG. BPKP or law enforcement agencies: and - Resolution o f complaints will be monitored by the Monitoring and Evaluation Officer. -. - _ "_. _. :ME DIES I - --I__ Enforcement . Sanctions nia!, be inconsistent and not In all procurement contracts. evidence o f enforced. corruption, collusion or fraud will be grounds for terminating relevant contract, possibl!. with additional penalties; - In cases ofcorruption. disbursement can be suspended or stopped completely i f cases are not dealt with effectively; - Failure to report cases and take appropriate action w i l l be investigated and may be grounds for disciulinan action: and 57 Corruption Opportunities for Corruption Mitigation Measures Mapping Area -. .. .- I - Guidelines on different leve-isof sanctions to government officials, consultants, and medical schools or professional associations to be defined in the Project Operational Manual. I Transparency - Lessons learned related to past - Lessons learned related to past corruption cases corruption cases and the subsequent and the subsequent remedial actions w i l l be remedial actions are not well documented and made available through website. communicated and thus repeated. I Selection o f grantees newspapers; - Publication o f criteria and procedures for selection process; - Announce award in the website; and - Establish and communicate complaints handling mechanism. Implementation Flow o f funds Reporting and Filing _I__- documented. ACCRE DITATlOh __II___ - _.- . "_- . . Accreditation and - Nontransparent and nonobjective - Activities inherent in project components are certification process process o f evaluation; and designed to strengthen accreditation and - __ Bribes paid to obtain better evaluation. -11" I certification Drocess to address such risks. ___ Capacity o f - Lack o f independentjudgment in - Build capacity o f all actors involved in Procurement evaluation process. procurement; Committees - Procurement w i l l be under the coordination o f CPCU and supported by the project management consultant fim; and - Representatives o f relevant universities w i l l be invited to conduct a joint bidding process for procurement o f goods with similar technical specifications with facilitation from a procurement consultant at the central level. Advertising - Improper advertising (for instance, - Project Operational Manual to establish standard using newspapers with limited advertisement for the project, and minimum circulation); and requirements for newspapers to be used. . Fictitious advertising. 58 Mitigation Measures M a pping Area - Quality evaluated as part o f monitoring and productslservices quality than the ones specified in the evaluation; and technical specification; and - Enforce sanctions as defined in Keppres No. - Kickbacks from contractors, suppliers or 80/2003 and other relevant Indonesian laws. consultants. I __ - __ _. - ___ -- I . . - .- _ _ - False reporting. - Project Operational Manual w i l l define supporting documents to be presented for payments. Regular audit will be performed to verify payments Resolution available (website, email, mail, text messages, phone hotline) and well communicated; and - DGHE w i l l be staffed with personnel responsible for managing complaints. SANCTIONS AND 1tERIEI)iES - Sanctions may be inconsistent and not Enforcement I enforced. corruption, collusion or fraud w i l l be grounds for terminating relevant contract, possibly with additional penalties; - In cases o f corruption, disbursement can be suspended or stopped completely if cases are not dealt with effectively; - Failure to report cases and take appropriate action w i l l be investigated and may be grounds for disciplinary action; and - Guidelines on different levels o f sanctions to government officials, consultants, and medical schools or professional associations to be defined in the Project Operational Manual. Transparency - Lessons learned related to past - Lessons learned related to past corruption cases corruption cases and the subsequent and the subsequent remedial actions w i l l be remedial actions are not well documented and made available through website. communicated and thus reDeated. 59 Annex 10: Economic and Financial Analysis12 INDONESIA: Health ProfessionalEducation Quality Project Project Relation to Indonesia's Development Context 1. The Health Professional Education Quality (HPEQ) project i s w e l l situated within the overall development context o f Indonesia as expressed in the Country Partnership Strategy, 2009-20 12 (World Bank 2008a). The overarching theme o f this strategy i s "Investing in Indonesia's Institutions" and highlights the need for effective and accountable institutions that can translate available resources into better development outcomes. 2. In order to enhance institutional capacity, WB-Go1 partnership activities hope to build or strengthen three domains o f activity: (i) legal and regulatory framework overall and in each sector; (ii) the operational, technical, and administrative policies, procedures, and standards; and (iii) organizational structures, including clarity in definitions o f roles and responsibilities. The project's establishment o f independent oversight bodies empowered by public and transparent criteria o f accreditation and certification will achieve progress in each domain. The project will strengthen the regulatory framework o f the medical education and the health provider sectors. I t will establish standards o f medical education and practitioner knowledge expected o f future cohorts o f graduates. Finally HPEQ clearly delineates the roles and responsibilities o f each quality assurance body as w e l l as the participant stakeholders. 3. One o f the core engagements in the country partnership strategy i s support t o the education sector which aims to continue the W o r l d Bank's close partnership with M o N E . The HPEQ project f i t s naturally into the engagement framework that i s designed to support k e y components o f the government's education reform agenda including building social accountability mechanisms, and promoting external transparency and access t o information. 4. Finally, Indonesia's progress towards achieving i t s health-related MDGs t o date has been uneven. While currently on-track to achieve MDG 4-reductions in child mortality-Indonesia will need rapid progress to achieve MDG 5-improved maternal health. Through the anticipated improvements in practitioner competency and quality, this project will assist the attainment o f both MDG targets as will MDG 6-continued progress against HIV, malaria, and other diseases. Justification for Government Financing 5. Public financing o f the project i s justified principally because o f the public goods nature of accreditation and certification activities and because o f the positive externalities associated with improvements in health-most notably through reducing the burden o f communicable disease as a result o f quality improvements. 6. The services provided to society by quality-related accreditation and certification are either in the nature o f a public good (in the case o f the provisions o f accreditation and certification) or the nature o f a private good with significant positive externalities (in the case o f disease transmission reduction through improved health care). In regards to public goods, the consumption o f these services by an individual or household does not diminish the value o f consumption for other individuals or households. That is, public goods are nonrivaled-the accreditation and certification o f one institution or individual does not reduce the availability o f such services for others. In regards to a private good with significant positive externalities, a successfully treated sick individual as a result o f improved quality o f care n o longer presents a danger to others (and hence infections averted). Without clear revenue possibilities for project l2See Annex 14 for references f o r t h i s section. 60 activities, it i s highly unlikely that the private sector would invest in such services t o a degree that i s socially optimal. Indeed, given the current lack o f consumer familiarity with health sector accreditation, as well as generally l o w levels o f health knowledge in the population, the private sector equilibrium o f many o f these activities ma y w e l l be zero provision. 7. Given the public health nature o f the interventions supported through the project and their large externalities, Go1 should be supporting activities in this area even if that means reducing spending o n alternative health activities for which public financing i s less justified. Examples o f these alternative activities include the production o f private goods with fewer positive externalities such as direct provision o f care for n o n communicable diseases, or public spending o n secondary health care which positively favors well-off households (World Bank 2008b). Economic Analysis 8. The project will fund quality enhancements in health sector higher education through three main activities: (i) rationalizing and strengthening the accreditation o f public and private medical, dental, nursing, and midwifery schools; (ii) implementing competency-based certification; and (iii) providing results-oriented resources t o assist health education institutions meet these challenges. The project activities will focus o n the establishment o f an independent body-the N A A - f o r the accreditation o f health education 'institutions as w e l l as an independent national evaluation center-the NACEHealthPro. Project funds will also be allocated to provide financial assistance for selected medical schools to meet the accreditation standards. 9. Together the 69 medical schools in Indonesia produce around 5,000 new doctors per year. This i s far below the demand for doctors in Indonesia. Around 57 percent o f existing medical schools are private institutions-making them very important suppliers o f medical graduates. However, they also have lower accreditation status and quality than public schools. Simply improving the quality o f graduates from public schools would not address the quality o f care issues for patients visiting graduates o f private schools. Currently the vast majority o f patients do not distinguish the educational pedigree o f the medical provider and hence this provides l i t t l e incentive for private schools to invest in quality upgrades o n their own-students at these schools will not be willing to pay more for certification if this does not affect future demand for their services. Hopefully this will change in the future and the activities under the HPEQ project should help prompt such change. Nevertheless, given the current situation, as w e l l as the positive health externalities from improved quality o f care, the FAP will be made available to public and private medical schools. 10. In general there i s a recognized need for quality enhancement in the provision o f health care throughout the developing world. Research has demonstrated variations in adherence t o evidence-based procedures fo r malaria in Kenya (Zurovac et a1 2004) and between public and private clinics in Vietnam (Tran et a1 2005). Mexican women receive better prenatal care at social security institutions than elsewhere (Barber et a1 2007), and insured South Africans receive higher-quality treatment than their uninsured peers do (Chabikuli et a1 2002). 11. The implicit need in Indonesia for the proposed project activities i s high since there i s significant scope to raise the level o f health knowledge among Indonesia's current practitioners. Preliminary results from the 2007 Indonesia Family L i f e Survey (IFLS) found that the percentage o f health practitioners reporting correct responses t o vignette questions i s only 45 percent for antenatal care, 62 percent for child curative care, and 57 percent for adult curative care. Additionally, in 2007, only 50 percent o f graduating medical students passed the qualifying examination with a passing score o f 45 out o f 100 (AIPKI 2007). 61 12. I t i s difficult, however, t o understand exactly h o w practitioner-related quality o f care affects ultimate population health given the numerous n o n competency related constraints (such as l o w staffing or inadequate infrastructure) in the health system. Credible estimates from the developing w o r l d o f the impacts o f improved care o n health outcomes are rare. One example i s f r o m Bjorkman and Svensson (2006) who find that a Ugandan community monitoring program increased health sector quality o f care. One year into the program, the investigators estimated a significant difference in the weight o f infants (a 0.17 z-score increase) and a markedly lower number o f deaths among children under 5 (a 33 percent reduction in child deaths) although sole attribution o f health gains to improvements in health care quality i s not possible as utilization of treatment had also increased. 13. In developed country settings, there i s a stronger empirical link between provider quality and health outcomes. McClellan and Staiger (2000) found quality o f care to be strongly related to heart attack mortality. A comparison o f high- and low-quality hospitals in the United States delineates a difference o f 10 percentage points in the 30 day Acute Myocardial Infarction mortality rate. Another study in the US, an evaluation o f the National Surgical Quality Improvement Program (NSQIP), found that since the inception o f the NSQIP data collection process, 30-day postoperative mortality after major surgery in the quality-enhanced hospitals had decreased by 27 percent and the 30-day morbidity by 45 percent (Khuri et a1 2002). 14. Since the project specifically focuses o n accreditation and certification, the economic analysis assumes that accreditation and certification leads to greater knowledge gains among entering cohorts o f practitioners. This, in turn, leads t o elevated provider competency and hence better quality health care and improved health outcomes. There i s some evidence, again mostly from the developed world, t o support this causal linkage. 15. For the United States, Silber et al. (2002) find that certification o f anesthesiologists i s significantly linked to a 13 percent reduction in mortality, after controlling for various observable factors. Clark et al. (1998) directly link provider education t o better child health outcomes in an intervention focused on child asthma. In this study, children treated by physicians who had received a supplementary education intervention had significantly fewer symptoms and fewer follow-up office visits, nonemergency physician office visits, emergency department visits, and hospitalizations. In a related vein, Ross et al. (1999) find that a failure o f education or training i s deemed responsible for 20 percent o f a l l adverse events observed in the Quality in Australian Health Care Study. 16. M o v i n g to the developing country context, Quimbo et al. (2008) find that national level accreditation influences quality o f care in the Philippines. Barber and Gertler (2009) determine that a one standard deviation increase in quality (as measured through knowledge vignettes) o f Indonesian practitioners reduces prevalence o f child stunting by six percentage points (increasing length by 0.5 cm). Barber et al. (2007) argue that lower-quality care in Indonesia is-to a large extent-a manifestation o f educational quality and conclude that training i s an important strategy t o use when quality deficiencies result from lack o f skills. 17. Due t o the uncertainty around the ultimate impacts o f accreditation and certification in the Indonesian context, the economic analysis necessarily takes a conservative approach. The analysis links improved quality o f care over a practitioner's early career, and hence population health outcomes, to the quality o f received training and education. The analysis then monetizes the gains in a select variety o f health dimensions and compares them to project costs. 18. The approach adopted i s highly conservative in at least t w o dimensions: (i) analysis posits the especially modest gains in the quality o f care provided by a certified health practitioner trained in an accredited institution over and above the quality o f care available today; and (ii) the analysis only 62 considers dimensions o f health gains that are relatively easily translated into monetary terms. These dimensions include reductions in the incidence o f l o w birth-weight infants, reductions in infant mortality, reductions in the days o f w o rk o f prime age adults (aged 15-49) lost t o illness, and reductions in prime age adult mortality. N o attempt i s made to assess gains to non-infant children or older age adults although assuredly these populations will also benefit from improvements in quality o f care. Additionally any externalities from improved health, such as infections averted mentioned above, are not monetized and hence unaddressed in this conceptual framework. In this regard the analysis establishes a lower limit o n all possible monetized benefits resulting from an improvement in practitioner competency and quality o f care. 19. An additional benefit o f the project not formally addressed in the analysis concerns the equity implications o f project activities. These activities should benefit the poor and vulnerable since, as noted in the Health Sector Public Expenditure Review (World Bank 2008b), out-of-pocket payments are high given that public spending o n the health system i s low. Furthermore utilization o f nurses and midwives i s proportionally higher among the poor. Improved knowledge-based competency among these providers will result in fewer repeat health facility visits as diagnoses become more accurate and curative services more effective. Finally, catastrophic health spending-an unfortunately common cause o f entry into poverty (Xu et al2007)-wiIl occur less frequently as the quality o f care improves. 20. The analysis assumes that project activities ultimately result in improved knowledge among accredited health institution graduates only by the fifth year o f the project in order to allow for project scale-up and a sufficient proportion o f nursing and midwifery schools to be fully accredited. The knowledge gains from the program are then extended to the subsequent nine graduating cohorts o f health practitioners. After the tenth cohort graduates and enters the labor force, fifteen years will have elapsed since project initiation. Forecasting the quality o f care o f practitioners graduating more than 10 years in the future, and the gains fro m such quality, i s not addressed by this analysis due t o the challenges in understanding the disease environment, the structure o f the health system, the pattern o f utilization, and the overall level o f national development at that point in time and afterwards. For the same reasons, the analysis assumes that quality gains in health care from improved provider competency extend only for the first ten years o f practice. At that point, future developments in medicine m a y necessitate further professional training in order to maintain diagnostic and curative skills. 21. Table 1 depicts the k e y parameter values and the sources o f information from which they are derived. Every year 5,000 doctors and 44,000 nurses and midwives graduate (this information comes from the respective professional associations o f doctors, nurses, and midwives). This analysis assumes that all graduates j o i n the ranks o f practicing health providers. 22. Each newly graduated doctor will be expected t o attend 9.07 births per year o f practice while a nurse or midwife will attend 12.89 births (estimated through the combination o f information from the 2007 D H S and the U S Census Bureau International Database). Infant and c h i l d health appear particularly susceptible t o practitioner quality. In low-income settings, the quality o f prenatal care i s very likely to affect child survival and growth. In these settings there tends to be a high prevalence o f maternal conditions that promote l o w birth weight and/or increased mortality, including low energy intake, l o w prepregnancy body mass index, and hypertensive disorders o f pregnancy. Persistent untreated illnesses during pregnancy can result in l o w birth weight infants that are less l i k e l y to catch up in growth. Currently in Indonesia the Infant Mortality Rate stands at 34 deaths per 1,000 births, while the proportion o f infants born with l o w birth weight i s 0.146. 23. In terms o f valuing the gains from reduced infant mortality, there i s n o consensus in the research literature on the ethically fraught task o f h ow to value a l i f e saved. One conservative possibility i s to use 63 the resource costs of alternative means o f saving a life. Alderman et al. (2004) suggest that the resource cost o f saving an infant's life through a measles campaign i s US$1,250. This i s the value adopted here. 24. The same authors adopt a conservative approach to valuing the gains from averting a l o w birth weight infant by estimating the costs o f various dimensions o f morbidity and mortality o f l o w birth weight, including lowered mortality, reduced additional costs o f neonatal medical attention and subsequent illnesses, reduced lifetime productivity due to stunting, and so on. Taken together the authors arrive at a total cost o f US$5SO per l o w birth weight delivery averted. 25. Turning to the indicators for prime-age adults, the analysis combines information o n disease burden and health-seeking behavior as recorded in Choi et al. (2007) with the U S Census Bureau International Database. Each practitioner i s estimated to encounter an average o f 374 unique health events affecting prime-age adults per year. Seldom do these events result in mortality-the age-adjusted prime- age adult mortality rate i s 2.32 deaths per 10,000 population. M u c h more likely i s for this adverse health event to result in days o f w o rk or other key activities lost due to elevated morbidity. The 2007 I F L S estimates the average days o f w o rk lost to an adverse health event to be 6.45 days. 26. This cost o f foregone earnings i s typically the greatest microeconomic cost o f premature adult mortality or morbidity. The number o f productive work years lost to premature adult mortality i s assumed to average 20 years across individuals. Real wages are set to grow an average o f 2 percent a year (a conservative assumption compared with economic projections), with a standard deviation o f 0.5 percent in order to account for uncertainty over the future wage path. 27. The impact o f the project can be modeled as a series o f interventions that affect the following four health indicators applicable t o the patient population treated by the affected cohorts o f practitioners: (i) infant mortality rate; (ii) proportion o f infants born with l o w birth weight; (iii) number o f the the the prime-age adult w o rk days lost to illness; and (iv) the prime-age adult mortality rate. In expectation, each indicator i s reduced o n average by 1 percent from the population norms depicted in Table 1 as a result o f enhanced provider competence. F o r example, if the infant mortality rate among patients o f practitioners not benefiting fro m the project i s 34 deaths per 1,000 births, the equivalent mortality rate among practitioners trained under the HPEQproject i s 33.66 deaths per 1,000 births. 28. These relative health gains due to enhanced quality are maintained even as health outcomes such as IMR experience secular gains in the overall population. This expected 1 percent gain in overall health outcomes for the patients o f accredited and certified practitioners i s consistent with a universal improvement in knowledge-based competency o f 1 percent applied to all entering cohorts o f practitioners. I t i s also consistent with a larger gain in quality for a subset o f new graduates with n o change in competency for others, for example a 5 percent gain in patient health outcomes accruing t o 20 percent o f entering practitioners. This latter possibility m a y be more realistic given the heterogeneity in current school quality as recognized by the project design. 29. The l i k e l y impacts o f the proposed intervention are then treated as random variables with hypothesized distributions, also listed in Table 1. Typically the standard deviation o f a posited impact i s h a l f the expected gain, so that 95 percent o f a l l potential impacts vary between zero and twice the expected gain. The analysis adopts Monte Carlo simulation methods to generate a distribution o f anticipated total benefits, at the national level, to compare with costs. A total 500 simulations are run. 30. The analysis also adopts a discount rate o f 5 percent, a typical value for the evaluation o f health projects. Table 2 presents the range o f benefit-cost figures as determined in the Monte-Carlo analysis. The total gross program cost i s US$76 million over five years, yielding a net present value o f US$68.87 million given the anticipated disbursement schedule. The median present value o f total costs averted i s 64 estimated at US$380.80 million, yielding a gross benefit-cost ratio o f 5.53. Indeed every point in the range o f possible outcomes, depicted in the kernel density plot in Figure 1, i s associated with a substantially higher present value o f total costs averted. In n o simulation i s the estimated gross benefit less than cost and the benefit-cost ratio ranges over the full interval (1.57, 10.56). The vast majority, 80 percent, o f a l l simulations range over the truncated interval (3.40, 7.63). 3 1. In terms o f the relative contributions o f the constituent interventions t o overall gains, 13.2 percent o f mean gains derives from the reduction in the incidence o f l o w birth weight babies, 6.2 percent from lowered infant mortality, 41.9 percent fr om reduced time costs o f prime-age adult morbidity, and 38.8 percent from lowered prime-age adult mortality. O f course in any particular simulation the allocation o f overall gains across constituencies will vary. 32. Although savings to the health care system i s likely due to reduced public expenditures originating from more accurate diagnosis, more appropriate treatment, and changes in utilization rates as a result o f improved health, modeling these costs is difficult without specialized information and forecasts. Hence the analysis does not estimate a net benefit-cost ratio that accounts for health system savings from improved quality. Even with the very conservative assumptions o n program impact, made in response to an environment o f great parameter uncertainty and only a partial accounting o f a l l possible benefits, the anticipated gross benefits are substantial. The net benefits would be even greater. Fiscal Sustainability 33. The completion o f the project will bequeath modest recurrent costs to GoI, as presented in Table 3. The uses o f recurrent expenditures falling to the central government are constituted by the costs o f continuing t o administer the NAA and NACEHealthPro. These administrative costs are estimated to total U S $ l . 14 mi l l i o n (in 2009 dollars) per year after project completion. 34. This amount i s strikingly small (0.006%) in comparison with the total government spending in the education sector, which i s currently US$17.9 billion per year and m a y increase as a share o f total government expenditure in coming years. The amount i s within the same small order o f magnitude (0.01 1%) when education expenditures are restricted solely t o those committed by the central government. 35. Furthermore, the budgetary impact o f the project's recurrent expenditures o n total government expenditures may very w e l l be overestimated. The economic analysis for this project discussed the possibility o f direct costs averted within the public health sector as a result o f fewer adverse health events and lower utilization rates. If such were to occur, this would also lessen the budgetary burden by a significant proportion due to savings in the public health sector. 36. One project risk concerns the fungible nature o f donor-provided development spending since, in principle, funds are transferable across possible projects. This project might w e l l have been undertaken without external financing and so it i s unclear what a project's estimated net benefits convey about the grant's effect o n development. If this i s the case, the donor's funds are actually financing some other project that would not have been carried out otherwise. in this specific context o f Indonesian health professional education, however, the project funds will be devoted to establishing two important national health accreditation and certification bodies with the technical assistance o f the W o r l d B a n k Group. i t i s not clear that these two bodies would have been established without such engagement. As such, fungibility concerns are mitigated. 65 Figure 1. Distribution of costs averted as a result of improvements in practitioner quality I I I 0 200 400 600 800 Value of health improvements (millions USD) Dashed line depicts total discounted cost of project 66 v1 2 8 u d 0 0 B 0 N N m t- 2 0 0 8 m x N x 00 i 2 00 2 a 0 vi N 7 2 i m 0 00 'ci 00 a r- hl rcI 09 N x hl vi P r- r- 3 r'r ? CJ hl I n a d B Y a L, 2 VI VI Y 0 Y .3 8 0 Y .3 E! id Y G E! Y VI Y VI Y rr 8 0 8 g s cd g 8 D > Y 8 0 I= VI VI % 8 2 pi 8 Annex 11: Safeguard Policy Issues I N D O N E S I A : Health Professional Education Quality Project 1. The safeguards review team endorsed the assessment o f the task team and assigned Category C for the project. Therefore, safeguard clearance o f the project has been delegated t o E A S H D . 2. The project does not affect communities o f indigenous peoples and thus the IP Policies are not triggered. The project will provide F A P funding to selected medical schools, and an eligible spending under the FAP i s providing access for the poor to medical education through a scholarship program. The medical schools will not actively seek scholarship candidates from the poor community. This i s consistent with the government policy o f promoting access o f underprivileged students to higher education. 3. This project will have n o significant impact on the environment and thus the requirement to conduct an environment assessment i s not triggered. There i s a possibility that some FAP resources will be used for c i v i l works, but this will be limited to the rehabilitation o f existing classrooms or laboratories. Under the Indonesian environmental impact assessment regulations, they are categorized below the standard requirements for a formal environmental impacts assessment. The FAP Manual will state the provisions o f c i v i l work and the manual would be used as reference not only by the medical schools preparing the proposal but also by the reviewer o f the FAP proposals. Whether such activity will be included will only be known when the grant requests are submitted. 69 Annex 12: Medical Education in Indonesia INDONESIA: Health Professional Education Quality Project Background 1. The current ratio o f doctors to population shows that Indonesia i s lagging behind other countries with a similar level o f income. The ratio o f doctors to population in 2007 i s 25 per 100,000, lower compared t o Malaysia and the Philippines with 70 and 58 per 100,000 respectively. This aggregate number however masks unequal distributions o f doctors with more concentration in Java/Bali provinces and urban areas. Unavailability o f doctors-especially in remote rural areas-and high absenteeism in public health facilities forces health personnel without proper training nor legal support to deliver health care, exposing the health personnel to legal risks and patients t o medical risks. 2. As a response t o the need to deploy more doctors t o rural remote areas, the government i s encouraging the production o f more health professionals. The number o f medical schools has mushroomed in the last decade, and the role o f the private sector has become more prominent for there i s a high demand for doctors that can not be fulfilled only by public sector. Currently, there are 69 medical schools, an increase o f more than 30 percent from the previous year. Existing medical schools produce around 5,000 doctors per year. In terms o f geographical distribution, the schools remain concentrated, with 80 percent located in Java and Sumatra. 3. The quality o f health professionals has been identified as one o f the contributing factors t o the overall l o w quality o f health services in Indonesia. Quantitative measurement o f the quality o f health personnel from the I F L S shows l o w quality in general. On average, practicing health professionals- including doctors-only knew about one-half o f the standard treatment regime for cough in adult care, fever and diarrhea in child care, and prenatal care. 4. The rapid growth o f health professional education institutions creates challenges in ensuring the quality o f education and the outputs. In the absence o f a clear framework for establishing new health professional schools and monitoring the education inputs, process, and outputs, the quality o f health providers and, ultimately, the services provided i s questionable, and therefore putting people's health in danger. The quality o f the schools depends largely o n the licensing, accreditation and certification process. The licensing o f medical education institutions i s conducted by the DGHE based o n the reference from the IDI. The exact procedures for licensing n e w medical schools i s unclear and need to be reviewed to make sure that n e w schools have adequate learning resources and process to produce quality health professionals. 5. With the rapid growth o f health professional schools, it i s crucial for Indonesia to have strong standards and mechanisms for school accreditation and certification o f the graduates. At the present time, the accreditation o f medical schools i s conducted by BAN-PT using generic instruments for a l l study programs, not necessarily accommodating the specific characteristics o f medical education. The development o f a specific accreditation instrument for medical schools i s currently ongoing, but it i s well- known that a proper accreditation instrument for medical schools alone i s not sufficient. M o s t importantly, there needs t o be an independent and accountable accreditation system. Current Strategies to Improve the Quality o f Medical Education in Indonesia A. Implementation o f the CBC 6. In order to improve the quality o f medical education, the DGHE started the implementation o f a new curriculum for medical education-the C B C in 2005. The KKI validated the new curriculum by 70 releasing the `Standards for Basic Medical Education', and `Competency Standards for Primary Care Physicians' in November 2006. These standards define seven areas o f competencies to be achieved by students using the new curriculum. The features o f the C B C distinguishing it from the previous one are the shorter length o f study-by one year-and the horizontal and vertical integration o f various disciplines. A major shift from the previous curriculum i s the use o f the SPICES model13, the underlying paradigm o f a competence based curriculum. In consequence, the M E I s have t o make substantial adjustments in their study programs including the use o f PBL methodology and ensure the availability o f learning resources. A feature specific to Indonesia i s the flexibility extended to the medical schools t o incorporate locally specific health issues as 20 percent o f the n e w curriculum content. The development and the implementation o f the C B C were facilitated by the HWS-a project financed by the W o r l d Bank. 7. The DGHE mandated the implementation o f the C B C immediately after the dissemination o f the final draft o f the new curriculum in 2004. Supported with resources from the H W S Project, DGHE and the association o f medical schools (AIPKI) developed a schedule for the implementation o f the CBC, starting with five M E I s in 2005 and covering 52 medical schools in 2008. The schedule took into account the level o f capacity o f each medical school based on an assessment conducted at the beginning o f the H W S Project, showing a wide range o f capacity among the medical schools in the availability and quality o f learning resources such as lecturers, lecture rooms and library. 8. The PBL was a new teaching-learning method for most medical schools. Prior to 2005, only a few medical schools had introduced the PBL, even fewer used the method consistently. Those schools had the advantage o f the QUE Project, another W o r l d Bank-financed project, t o introduce the P B L . The new method required integration o f individual disciplines into human body systems or an integration within basic sciences (horizontal) and between basic and clinical sciences (vertical). Applying PBL, and in a wider sense the SPICES Model, was a departure from the deep rooted `traditional' teaching-learning process, and therefore required a complete change o f the teacher's mind set. Such fundamental changes were made possible by the strong commitment o f the school leaders, ample resources and external technical assistance. A rapid assessment in 14 medical schools conducted by the WB in 2008 found medical schools in Indonesia implemented a "Hybrid PBL", marked by a large share o f lecture sessions compared t o self-learning and tutorial sessions, repetition due to lack o f integration, and separation o f basic and clinical sciences. 9. The nature o f the C BC i s very complex and the majority o f medical schools have difficulties in understanding the essence, not to mention in implementing the curriculum. The H W S Project has funded workshops, training, fellowships, and technical assistance to support the implementation o f the new curriculum, particularly to provide information t o academic staff o n C B C implementation strategies and to strengthen the capacity o f Medical Education Units established in each medical school. A long-term effort t o provide technical support in developing and implementing C B C was the establishment o f a national center for medical education called the National Medical Education Development and Research Center (NMEDRC). Unfortunately, the center existed only during the life-span o f the project, and the task force foreseen t o replace the center was never established. However, the need for a technically capable body with similar functions to N M E D R C was widely acknowledged. 10. Students from medical schools implementing the C B C since 2005 will enter the f i r s t rotation o f the clinical training in mid-2009. During the three semesters o f the clinical period, the students will master the required competencies as primary physicians, and therefore the clinical training i s a crucial phase in CBC. The medical schools urgently need to address the issue o f the monitoring o f clinical l3SPICES: Student-oriented, Problem-based, Integration o f disciplines, Community Orientation, Early and Systematic clinical exposure. Originally, the `E' stands for `Elective' but for Indonesia it has been changed to `Early exposure to clinical experience'. 71 teaching, an important aspect o f clinical training. Currently, clinical teaching i s conducted in some central and local government hospitals, military and privately owned hospitals, meaning the medical schools do not have control over the hospitals to emphasize the quality o f clinical teaching. In many cases, one hospital i s used by more than one medical school reducing student exposure t o patients. Other new approaches to clinical teaching should be developed, such as the strategy f o r helping students t o learn the process o f clinical reasoning. Given the current status and the large number o f weak and new medical schools, required investment to properly implement the C B C i s s t i l l quite significant. B. Certification o f Medical Doctors - The National Competency-Based Examination 11. Under the Medical Practice Law, the responsibility to undertake a national competency-based examination (NCBE) lies with the KKI, but the KKI s t i l l relies o n the consortium consists o f AIPKI and the KDI to prepare and conduct the examination. The N C B E was administered for the first time in mid- 2007. The test consists o f 3 00 multiple-choice questions, a methodology appropriate for measuring knowledge but not competence in clinical skills and attitudes. M o s t recent estimates by A I P K I indicate only 50 percent o f students passed the examination with a passing score o f 45 out o f 100 in 2007. In addition, the threshold for the passing score was decided arbitrarily by the consortium. I t i s clear there i s an urgent need for support t o develop a more valid evaluation system o f student competencies. This may include a longer-term goal o f establishing an independent national examination center. 12. In the n e w system, to be certified as doctors, graduates have t o undertake a one-year internship program upon completion o f their training in the medical schools. The preparation o f the internship program i s under the leadership o f the M o H and the first wave o f interns will enter the program in 201 1. C. Accreditation o f Medical Schools 13. The accreditation o f universities i s conducted by the BAN-PT w h i c h i s a part o f the National Accreditation Board (BAN). The accreditation process for medical schools i s similar to that o f the other study programs and the instrument used does not yet take into account the specificity o f medical schools. BAN-PT also suffers from understaffing and has difficulties in recruiting the proper experts in medical education. Realizing the limitations, the KKI and the BAN-PT, the two bodies with the authority to accredit the medical schools, forged a collaboration to strengthen the accreditation of medical schools. This includes the development o f a new instrument for the accreditation o f medical schools. The new instrument reflects the components o f medical education according to the standards published by the KKI. The next steps will be to test the instrument and to recruit and train the assessors. 14. The existing accreditation process o f medical schools i s not yet aligned with international standards o f independence, credibility, and transparency to the public. Although BAN-PT i s an independent institution as stated in the government r e g ~ l a t i o n ' ~ , does n o t have direct control over i t s it operational budget, raising some doubts over i t s independent status. In addition, the accreditation system i s punitive and summative rather than formative. I t lacks incentive f o r the schools t o improve themselves as the results rank medical schools into three levels (A, B, C) and the status lasts for five years. Improvement o f the accreditation system including the publication o f accreditation results will create motivation among the medical schools to improve the performance o f the schools and, ultimately, the quality o f medical education. Government Regulation (PP) l4 N o 191200.5 72 Annex 13: Project Preparation and Supervision INDONESIA: Health ProfessionalEducation Quality Project ~~ Planned Actual P C N review November 25,2008 Initial P I D to PIC November 11,2008 Initial I S D S to PIC December 14,2008 Appraisal June 1,2009 Negotiations June 25,2009 August 2 1,2009 BoardRVP approval September 24,2009 Planned date o f effectiveness November 15,2009 Planned date o f mid-tenn review April 1, 2012 Planned closing date December 3 1,20 14 Key institutions responsible for preparation o f the project: Directorate General o f Higher Education, Ministry o f National Education Board o f Higher Education, Ministry o f National Education Bank staff and consultants who worked on the project included: Name Title Unit Puti Marzoeki Task Team Leader EASHD Claudia Rokx Lead Health Specialist EASHD Pandu Harimurti Health Specialist EASHD Jed Friedman Senior Economist DECRG Yogana Prasta Operations Adviser EACIF Susiana Iskandar Senior Education Specialist EASHD Ratna Kesuma Operations Officer EASHD Novira Asra FM specialist EAPCO Imad Saleh Senior Procurement Specialist EAPCO Paulus Bagus Tjahjanto Procurement Specialist EAPCO Dayu Nirma Amunvanti Anti Corruption Specialist EACIF Susan Stout Operations and M and E, Consultant EASHD Pierre Jean Accreditation Expert, Consultant EASHD Gordon Page Certification Expert, Consultant EASHD Rosalia Sciortino Sociologist, Consultant EASHD Christopher Smith Higher Education Consultant EASHD Ratih Hardjono Communication Expert, Consultant EASHD Tetty Rachmawati Communication, Consultant EASHD Khadrian Adrima Costab, Consultant EASHD Estie W. Suryatna Team Assistant EASHD Bank funds expended to date on project preparation: 1. Bank resources: US$ 160,000 2. Trust funds: - 3. Total: US$ 160,000 Estimated Approval and Supervision costs: 1. Remaining costs to approval: US$ 5,000 2. Estimated annual supervision cost: US$ 65,000 73 Annex 14: Documents in the Project F i l e INDONESIA: Health Professional Education Quality Project 1. Standar Kompetensi Dokter, KKI, 2006. 2. Standar Pendidikan Profesi Dokter, KKI, 2006. 3. K e y Issues Related to Medical Education in Indonesia, Consultant Report, 2008. 4. Indonesia-Health Professional Education Quality Project, Stakeholders Analysis, Consultant Report, 2008. 5. Indonesia's Doctors, Midwives and Nurses: Current Stock, Increasing Needs, Future Challenges and Options, World Bank, 2009. 6. Handbook on Competency-based Assessment in Indonesia Medical Schools, Gordon Page, 2008. 7. Instrumen Akreditasi Pendidikan Kedokteran, BAN-PT/KKI, 2009. 8. Penjaminan Mutu dalam Pendidikan Dokter, Dikti-Depdiknas, 2008. 9. Rancangan Undang-Undang Praktik Keperawatan, PPNI, 2006. 10. Laporan Hasil Pengkujian Pengembungan Ketenagaan Perawat dan Bidan, Depkes, 2002. 11. Pedoman Sertijikusi Tenuga Kesehatan, Depkes, 2008. 12. Higher Education Long-Term Strategy 2003-20 10, Dikti-Depdiknas, 2003. 13. Kajian Terhadap Sumberdaya Pembelajaran Pendidikan Dokter Pada Institusi Pendidikan Dokter di Indonesia, Dikti-Depdiknas, 2008. 14. Laporan Panja Pendidikan Kedokterun, Komisi X DPR-RI, 2008. 15. Implementation Completion Report Development o f Undergraduate Education, World Bank, 2003. 16. Implementation Completion Report Quality o f Undergraduate Education, World Bank, 2004. Bibliography AIPKI. 2007. "Report on competence testing o f Indonesian MDs 2007". Commission o f Indonesian Doctors Competence Testing. Alderman, H., J. Berhman and J. Hoddinott. 2004. "Hunger and malnutrition". Copenhagen Consensus Challenge Paper. Barber S.L., S.M. Bertozzi and P.J. Gertler. 2007. "Variations in prenatal care quality for the rural poor in Mexico". Health Affairs, 26(3): w3 10-w323. Barber, S.L. and P.J. Gertler. 2009. "Health workers, quality o f care, and child health: Simulating the relationships between increases in health staffing and child length." Health Policy, forthcoming. Barber, S.L., P.J. Gertler and P. Harimurti. 2007. "Difference in access to high-quality outpatient care in Indonesia." Health Affairs, 26(3): w352-w366. Bjorkman, M., and J. Svensson. 2006. "Power to the people: Evidence from a randomized field experiment o f a community-based monitoring project in Uganda". World Bank Policy Research Working Paper # 4268. Chabikuli, N., H. Schneider, D. Blaauw, A.B. Zwi and R. Bruaha. 2002. "Quality and equity o f private sector care for sexually transmitted diseases in South Africa". Health Policy and Planning, 2002 Dec; 17 S~ppl:40-6. 74 Choi, Y.J., J. Friedman, P. Heywood and S. Kosen. 2007. "Forecasting health care demand in a middle income country: Disease transitions in East and Central Java, Indonesia". World Bank mimeo. Clark, N.M., M. Gong, M.A. Schork, D. Evans, D. Roloff, M. Hurwitz, L. Maiman, R.B. Mellins. 1998. "Impact o f Education for Physicians on Patient Outcomes". Pediatrics, lOl(5): 83 1-836. Khuri, S.F., J. Daley and W.G. Henderson. 2002. "The comparative assessment and improvement o f quality o f surgical care in the Department o f Veterans Affairs". Arch Surg, 137:20-27 McClellan, M. and D. Staiger. 2000. "Comparing the quality o f health care providers". Forum for Health Economics h Policy: Vol. 3: (Frontiers in Health Policy Research), Article 6. htt~://www.bepress.com/fhep/3/6. Quimbo, S.A., J.W. Peabodv, R. Shimkhada, K. Woo and 0. Solon. 2008. "Should we have confidence if a physician i s accredited? A study o f the relative impacts o f accreditation and insurance payments on quality o f care in the Philippines." Social Science h Medicine, 67(4):505-10. Ross, M.W., B.T. Harrison, R.W. Gibberd and J.D. Hamilton. 1999. "An analysis o f the causes o f adverse events from the Quality in Australian Health Care Study". Medical Journal o Australia, 170: f 41 1-415. Silber, J.H., S.K. Kennedy, 0. Even-Shoshan, W. Chen, R. Mosher, A. Showan and D. Longnecker. 2002. "Anesthesiologist board certification and patient outcomes". Anesthesiology, 96(5): 1044-1052. Tran, T., V.T.M. Dung, INeu and M.J. Dibley. 2005. "Comparative quality o f private and public health . services in rural Vietnam". Health Policy and Planning, 20(5):3 19-327. World Bank. 2008a. Country Partnership Strategy for Indonesia FY2009-20 12: Investing in Indonesia's Institutions for Inclusive and Sustainable Development. World Bank, Jakarta. World Bank. 2008b. Investing in Indonesia's Health: Challenges and Opportunities for Future Public Spending. Health Public Expenditure Review 2008. World Bank, Jakarta. Xu, W., D.B. Evans, G. Carrin, A . M . Aguilar-Rivera, P. Musgrove and T. Evans. 2007. "Protecting households from catastrophic health spending". Health Affairs, 26(4): 972-983. Zurovac, D., A.K. Rowe, S.A. Ochola, A.M. Noor, B. Midia, M. English and R.W. Snow. 2004. "Predictors o f the quality o f health worker treatment practices for uncomplicated malaria at government health facilities in Kenya". International Journal o Epidemiology, 33(5): 1080-109 1. f 75 Annex 15: Statement o f Loans and Credits INDONESIA: Health ProfessionalEducation Quality Project Difference between expected and actual Original Amount in US$ Millions disbursements ProjectID FY Purpose IBRD IDA SF GEF Cancel. Undisb. Orig. F m . Rev'd PO90991 2010 ID-URBANWATER SUPPLY 23.56 0.00 0.00 0.00 0.00 23.56 0.00 0.00 P115199 2009 Public Expend. Supp. Facility (DPL- 2,000.00 0.00 0.00 0.00 0.00 1,995.00 0.00 0.00 DDO) P107661 2009 ID-BOS K I T A Project 600.00 0.00 0.00 0.00 0.00 416.89 -181.61 0.00 PO92218 2009 ID- Indo Infrastructure Finance Facility 100.00 0.00 0.00 0.00 0.00 100.00 0.00 0.00 PO96532 2009 I D : Dam Operational Improvement 50.00 0.00 0.00 0.00 0.00 50.00 0.00 0.00 (DOISP) P100740 2009 PINTAR 110.QO 0.00 0.00 0.00 0.00 109.73 0.00 0.00 PO96921 2008 ID -National UPP (PNPM UPP) 167.68 125.00 0.00 0.00 0.00 165.68 -121.68 0.00 PO97104 2008 ID-BERMUTU 24.50 61.50 0.00 0.00 0.00 74.32 23.23 0.00 P105002 2008 National Program for Community 341.19 190.00 0.00 0.00 0.00 373.21 50.00 0.00 Empower PO79906 2007 ID-Strategic Roads Infrastructure 414.46 0.00 0.00 0.00 174.98 206.38 95.17 21.91 PO83742 2007 ID-Farmer Empower. Agric.Tech.&Info 32.80 60.00 0.00 0.00 0.00 71.04 19.63 0.00 PO89479 2006 ID-Early Childhood Education and Dev 0.00 67.50 0.00 0.00 0.00 52.17 6.23 0.00 PO85375 2006 ID-WSSLIC Il(PAMSIMAS) l 0.00 137.50 0.00 0.00 0.00 92.89 5.31 0.00 PO77175 2006 ID-Domestic Gas MarketDevelopment 80.00 0.00 0.00 0.00 0.00 38.29 24.12 0.00 Proj , PO71296 2005 ID-USDRP 45.00 0.00 0.00 0.00 0.00 31.97 21.17 0.00 PO76174 2005 ID-Initiatives for Local Govern Reform 14.50 15.00 0.00 0.00 0.00 15.68 14.27 0.00 PO78070 2005 ID-Support for Poor and Disadvant Areas 138.00 35.00 0.00 0.00 0.00 34.19 24.99 0.00 PO84583 2005 ID-WP3 67.30 7 1.40 0.00 0.00 0.00 17.53 -33.76 0.00 PO85133 2005 Govt Fin1 M g t & Revenue AdminProject 55.00 5.00 0.00 0.00 0.00 51.95 50.15 42.80 PO92019 2005 ID Kecamatan Development Project3B 80.00 203.00 0.00 0.00 0.00 1.41 -126.97 8.39 PO85374 2005 ID-HIGHER EDUCATION 50.00 30.00 0.00 0.00 0.00 53.78 39.22 0.00 PO74290 2004 ID-Eastern IndonesiaRegion Transp. 2 200.00 0.00 0.00 0.00 1.00 89.14 90.14 0.00 PO71316 2004 ID - Coral Reef Rehab and Mgmt Prog I1 33.20 23.00 0.00 0.00 0.17 28.46 24.62 0.00 PO64728 2004 ID-LAND MANAGEMENT &POLICY 32.80 32.80 0.00 0.00 0.16 27.92 23.85 0.00 DEVT PO79156 2003 ID-KECAMATAN DEV. 3 45 50 45.50 0.00 0.00 0.00 2.26 0.97 0.00 PO63913 2003 ID-Java-Bali Pwr Sector & Strength 141.00 0.00 0.00 0.00 0.00 43.62 43.62 43.62 PO59931 2003 ID-Water Resources & 1rr.Sector M g t 45.00 25.00 0.00 0.00 0.00 34.58 32.10 1.99 Prog PO72852 2002 ID-UPP2 29.50 206.00 0.00 0.00 0.00 3.94 -146.62 -11.12 PO59477 2000 ID-WSSLIC I1 0.00 77.40 0.00 0.00 3.95 3.34 -0.39 0.00 Total 4,920.99 1,410.60 0.00 0.00 180.26 4,208.93 - 22.24 107.59 76 INDONESIA STATEMENT OF IFC's Held and Disbursed Portfolio In Millions o f U S Dollars Committed Disbursed IFC IFC FY Approval Company Loan Equity Quasi Partic. Loan Equity Quasi Partic. 2006 Bank Danamon 155.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2004 BonaVista School 1.00 0.00 0.00 0.00 1.00 0.00 0.00 0.00 2006 Buana Bank 5.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2006 Centralpertiwi 45.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2004 Medan NP School 1.75 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2002 P.T. Gawi 11.05 0.00 0.00 3.49 4.90 0.00 0.00 3.49 1989 PT Agro Muko 0.00 2.20 0.00 0.00 0.00 2.20 0.00 0.00 1997 PT Alumindo 2.73 0.00 0.00 0.00 2.13 0.00 0.00 0.00 1989 PT Astra 0.00 0.20 0.00 0.00 0.00 0.20 0.00 0.00 1994 PT Astra 0.00 0.19 0.00 0.00 0.00 0.19 0.00 0.00 2003 PT Astra 0.00 0.12 0.00 0.00 0.00 0.12 0.00 0.00 PT Astra Otopart 0.00 0.70 0.00 0.00 0.00 0.70 0.00 0.00 2005 PT Astra Otopart 24.00 0.00 0.00 0.00 24.00 0.00 0.00 0.00 2000 PT Bank NISP 0.00 2.85 2.86 0.00 0.00 2.85 2.83 0.00 2002 PT Bank NISP 0.00 2.04 0.00 0.00 0.00 2.04 0.00 0.00 2004 PT BankNISP 35.00 0.00 0.00 0.00 35.00 0.00 0.00 0.00 1997 PT Berlian 0.00 3.35 0.00 0.00 0.00 0.00 0.00 0.00 1993 PT Bina Danatama 0.05 0.00 0.00 0.00 0.05 0.00 0.00 0.00 1996 PT Bina Danatama 0.00 0.00 2.58 4.81 0.00 0.00 2.58 4.81 2004 PT Ecogreen 30.00 0.00 0.00 0.00 30.00 0.00 0.00 0.00 2005 PT Ecogreen 25.00 0.00 0.00 0.00 20.00 0.00 0.00 0.00 PT Grahawita 0.00 0.00 3.75 0.00 0.00 0.00 3.75 0.00 1991 PT Indo-Rama 0.00 3.82 0.00 0.00 0.00 3.82 0.00 0.00 1995 PT Indo-Rama 0.00 1.57 0.00 0.00 0.00 1.57 0.00 0.00 1999 PT Indo-Rarna 0.00 0.81 0.00 0.00 0.00 0.81 0.00 0.00 2001 PT Indo-Rama 20.00 0.00 0.00 0.00 0.33 0.00 0.00 0.00 2004 PT Indo-Rama 48.00 0.00 0.00 0.00 41.00 0.00 0.00 0.00 1992 PT K I A Keramik 0.23 0.00 0.00 2.00 0.23 0.00 0.00 2.00 1996 PT K I A Keramik 1.65 0.00 0.00 53.49 1.65 0.00 0.00 53.49 1995 PT K I A Serpih 4.50 0.00 0.00 49.50 4.50 0.00 0.00 49.50 1997 PT Kalimantan 9.38 0.00 0.00 0.00 9.38 0.00 0.00 0.00 PT Karunia ( U S ) 16.45 0.00 0.00 3.56 16.45 0.00 0.00 3.56 2006 PT Karunia ( U S ) 20.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 PT Makro 0.00 2.34 0.00 0.00 0.00 2.34 0.00 0.00 2000 PT Makro 0.00 1.21 0.00 0.00 0.00 0.71 0.00 0.00 2006 PT Makro 0.00 0.66 0.00 0.00 0.00 0.66 0.00 0.00 1998 PT Megaplast 0.00 2.50 0.00 0.00 0.00 2.50 0.00 0.00 1993 PT Nusantara 0.00 0.00 10.16 7.90 0.00 0.00 10.16 7.90 2004 PT Prakars (PAS) 15.36 0.00 0.00 3.20 15.36 0.00 0.00 3.20 1997 PT Sayap 0.83 0.00 0.00 0.00 0.83 0.00 0.00 0.00 77 2001 PT Sigma 0.00 1.03 0.00 0.00 0.00 1.03 0.00 0.00 2006 PT TAS 7.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 1995 PT.Viscose 7.81 0.00 0.00 0.00 7.81 0.00 0.00 0.00 2004 PT Viscose 8.31 0.00 0.00 0.00 8.31 0.00 0.00 0.00 1997 PT Wings 0.72 0.00 0.00 0.00 0.72 0.00 0.00 0.00 2001 Sunson 11.62 0.00 0.00 7.35 11.62 0.00 0.00 7.35 2005 WOM 0.00 15.82 0.00 0.00 0.00 15.74 0.00 0.00 2006 WOM 20.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 2004 Wilmar 33.33 0.00 0.00 0.00 33.33 0.00 0.00 0.00 Total portfolio: 560.77 41.41 19.35 135.30 269.20 37.48 19.32 135.30 Approvals Pending Commitment FY Approval Company Loan Equity Quasi Partic. 2005 BankNISP SELF 0.03 0.00 0.00 0.00 2006 Bank NISP Swap 0.00 0.00 0.00 0.00 2006 Orix Indonesia 0.08 0.00 0.00 0.00 Total pending commitment: 0.11 0.00 0.00 0.00 78 Annex 16: Country at a Glance INDONESIA: Health Professional Education Quality Project East tower. Asia B middle. -_ - _ _ -- -- D e v e l o p m e n t d i a m o nd' Indonesia Pacific income 225 6 15% 3 437 Life expectancy 1650 2 80 1867 GNI (Atlas method US$ billions) 373 1 4 74 6 465 I A v e r a g e a n n u a l growth, 2001-07 Popuiation (%j 13 08 11 GN I Gross Laborforce (%) 19 12 15 per primary M o s t r e c e n t e s t i m a t e ( l a t e s t year available, 2001-07) capita enrollment R Infant mortality (pef lOOOlive births) 50 68 26 43 71 24 42 65 41 I Chiid malnutntion (%ofchildien under5) 14 53 25 Access to improved water source Access to an improved water source (%ofpopulationj 80 37 88 Literacy (%ofpopo/afionage 1 3 5 90 91 89 Gross primaryenro llment (%of school-age population) 14 1 13 1 111 - --Indonesia Male 16 in ll? Lo wer-middle-income group , ~~ Female m 13s M9 KEY E C O N O M I C R A T I O S a n d L O N G - T E R M T R E N D S 1987 1997 2006 2007 , GDP (US$ billions) 759 2157 I 364 5 432 8 i 302 318 25 4 24 9 Trade 310 25 4 30 a 28 9 279 26 1 -28 4 6 27 25 34 24 08 692 632 38 9 370 300 86 35 4 M4 7 Indebtedness 1987-97 1997-07 2006 2007 2007-11 37 63 64 Indonesia 23 51 52 ods andservices 51 80 76 STRUCTURE o f the ECONOMY lgg7 Growth o f c a p i t a l a n d G D P (Oh) (%or GDPj Agriculture Industry M anufactunng 8 9 268 280 270 Services 404 396 401 394 Household finai consumption expenditure General gov t finai consumption eqenditure 94 68 86 83 Imports of goods and services 224 281 256 253 GCF -GDP 1987-97 '997-07 l G r o w t h o f e x p o r t s a n d i m p o r t s (%) (average annual grordh) Agriculture 31 28 34 Industry 98 31 45 Manufacturing TI9 42 46 Services 82 46 74 89 Household finai consumption expenditure 87 36 57 General gov't final consumption expenditure 36 62 96 39 - -10 Gross capital formation Imports of goods and services 130 P O 15 41 12 86 20 89 ~ -Expods -Inpods - Note: 2007 data are preliminaryestimates. This tablewas producedfrom the Development Economics LDB database. , 'Thediamonds showfour keyindicators inthecountry(inboid) comparedwith its income-groupaverage If dataaremissing,thediamondwill be incomplete 79 Indonesia , PRICES andGOVERNMENT FINANCE 1987 1997 2006 2007 Domestic prices (%change) 88 61 D.1 65 I n f l a t i o n (Oh) 20 - I Consumer prices Implicit GDP deflator 154 't26 13.6 115 Government finance (%of GDP, includes correnf grants) Current revenue 15.3 7.2 18.1 15.3 Current budget balance -40 5.2 6.6 5.2 Overall SurDluSldefiCit -08 -12 -0.9 -22 TRADE 1987 1997 2006 2007 (US$ millions) l E x p o r t a n d i m p o r t l e v e l s ( U S $ mill.) I Total exports (fob) 53,444 130,799 I%,13l 125 000 Fuel t3.154 27,619 29,221 000 Estatecrop .. 3,785 5,483 6,329 Manufactures .. 21,268 42,764 46,MO 75 000 Total imports (cif) 41,680 79,777 91724 50 000 I Food 2.983 4,709 6,884 Fuel and energy .. 4,047 19,028 21994 25 000 Capital goods .. ii.573 15.411 19,038 0 ' , 01 02 03 04 05 06 07 Export price index (20OO=r)O) 88 68 65 Import price index(2000=r)O) 71 83 68 sEYports EInpOrtS Terms o f trade (2000=WO) .. a4 82 95 BALANCE of PAYMENTS 1987 1997 2006 2007 I C u r r e n t a c c o u n t b a l a n c e t o G D P (X) (US% millions) Exports of goods and services 18,271 63,239 115,032 t30,439 115 Imports of goods and services 15,972 62,830 95,493 08.458 Resource balance 2,598 818 19.539 21.981 Net income -7,308 -'12,664 -14,465 -15,875 Net current transfers 257 1.034 4,863 4,904 Current account balance -2,098 -9,890 9,937 11021 Financing items (net) 3,357 -6.385 -3,035 1533 Changes in net reserves -1,259 B,275 -6,902 -a,543 01 02 03 04 05 06 07 Memo: Reserves including gold (US$ millions) 4.8% 7,396 43,083 55,626 Conversion rate (DEC. local/US$) 1643 8 2,909.4 9,159.3 9,%3 4 E X T E R N A L D E B T and RESOURCE FLOWS (US$ millionsj 1987 1997 2006 2007 C o m p o s i t i o n o f 2 0 0 6 d e b t ( U S $ mill.) I Total debt outstanding and disbursed 52,535 t36,273 00,956 IBRD 7,391 9.991 7,423 6.821 IDA 865 715 1,318 1550 Totaldebt service 7,001 19,737 20,434 IBRD 875 1,848 1.827 1803 IDA a 26 37 38 Composition of net resourceflows Official grants 195 213 1,071 Official creditors 2,523 534 54 Private creditors 301 5,992 4,776 Foreign direct investment (net inflows) 385 4,677 5,580 Portfolio equity(net inflows) World Bank program 0 -4 987 1898 F 41,023 I Commitments 1,418 813 749 1,383 A-IBRD E - Bilatwal Disbursements 1,374 899 1.01L 986 B - IDA D - Other nultilaterai F - Private Principal repayments 36 1 1,155 1,430 1396 C-IMF G - Short-ter Net flows 1,013 -266 -418 -409 Interest payments 525 709 434 445 Net transfers 488 -975 -852 -855 Note This tablewas producedfrom the Development Economics LDB database 9/24/06 80 95° 100° 105° 110° 115° 120° 125° This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and INDONESIA any other information shown on this map do not imply, on SELECTED CITIES AND TOWNS the part of The World Bank Group, any judgment on the PROVINCE CAPITALS legal status of any territory, 15° or any endorsement or NATIONAL CAPITAL acceptance of such INDONESIA boundaries. RIVERS MAIN ROADS MYANMAR VIETNAM RAILROADS PHILIPPINES PROVINCE BOUNDARIES 10° 10° INTERNATIONAL BOUNDARIES THAILAND Sulu Sea 135° 140° Banda Aceh L A Y 5° A S BRUNEI 5° 1 Medan M Natuna I Celebes Talaud Is. Pematangsiantar Besar Tarakan Sea PACI F I C OCE AN 24 A Simeulue 19 Morotai 2 Manado SINGAPORE 23 Nias Tanjungpinang Ternate Halmahera Pekanbaru 25 0° 3 Pontianak K AL IM ANTAN 26 Gorontalo Waigeo 0° Lingga 20 Samarinda Manokwari Me Padang 30 Biak Balikpapan Palu Sorong 4 5 Peleng Obi nt Siberut Jambi Bangka 21 aw Pangkalpinang SULAWESI Misool 32 Yapen Jayapura SUMATERA Palangkaraya Mamuju Sula Is. Ceram ai 9 22 6 Palembang Belitung 27 Amahai Fakfak 28 Is Buru 7 Bandjarmasin 29 Kendari 33 PAPUA . NEW GUINEA Bengkulu Parepare Ambon Timika Puncak Jaya 8 Muna (5030 m) 5° Bandar Java Sea Makassar Kai PAPUA Enggano Lampung 11 Baubau Banda 31 Is. JAKARTA 0 200 400 Kilometers Serang Sea Aru Is. 12Bandung Semarang Madura 10 13 Wetar Surabaya Babar Tanimbar 0 100 200 300 400 Miles JAWA Yogyakarta 16 Sumbawa Alor Moa Is. 15 Bali Lombok Raba Flores 14 Merauke 95° 100° 105° Denpasar Mataram Ende Arafura Sea 18 TIMOR-LESTE 17 PROVINCES: 10° Waingapu Sumba Timor 10° Kupang 1 NANGGROE ACEH DARUSSALAM 12 JAWA BARAT 23 KALIMANTAN TIMUR 2 SUMATERA UTARA 13 JAWA TENGAH 24 SULAWESI UTARA 3 RIAU 14 D.I. YOGYAKARTA 25 GORONTALO 4 5 SUMATERA BARAT JAMBI 15 16 JAWA TIMUR BALI 26 27 SULAWESI TENGAH SULAWESI BARAT INDIAN OCEAN 6 BENGKULU 17 NUSA TENGGARA BARAT 28 SULAWESI SELATAN IBRD 33420R2 7 SUMATERA SELATAN 18 NUSA TENGGARA TIMUR 29 SULAWESI TENGGARA AUGUST 2008 8 LAMPUNG 19 RIAU KEPULAUAN 30 MALUKU UTARA 15° 15° 9 BANGKA-BELITUNG 20 KALIMANTAN BARAT 31 MALUKU 10 BANTEN 21 KALIMANTAN TENGAH 32 PAPUA BARAT AUSTRALIA 11 D.K.I. JAKARTA 22 KALIMANTAN SELATAN 33 PAPUA 115° 120° 125° 130° 135° 140°