ICRR 11487 Report Number : ICRR11487 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 06/24/2003 PROJ ID : P008793 Appraisal Actual Project Name : Reform Of Higher Project Costs 69.8 84 Education And Research US$M ) (US$M) Country : Romania Loan /Credit (US$M) Loan/ US$M ) 50 49.6 Sector (s): Board: ED - Tertiary Cofinancing 10 10 education (90%), Adult US$M ) (US$M) literacy/non-formal education (10%) L/C Number : L4096 Board Approval 96 FY) (FY) Partners involved : European Union Closing Date 06/30/2002 09/30/2002 Prepared by : Reviewed by : Group Manager : Group : Helen Abadzi Ridley Nelson Alain A. Barbu OEDST 2. Project Objectives and Components a. Objectives The project was a sector investment loan meant to improve: (a) the responsiveness of the system to the demands for new professional and managerial skills required by a market economy; (b) resource allocation to generate incentives for more efficient performance; and (c) access to higher education with increased equity for talented but needy students. Specific objectives were to: (i) improve the capacity of the recently formed higher education councils and the individual institutions to fulfill their responsibilities in the reform strategy; (ii) develop the new undergraduate and continuing education programs demanded by the transition to a market economy; and (iii) develop the new postgraduate education and research training needed to supply the next generation of academic staff in high demand fields of study, and the professionals with advanced training in the new fields demanded by the transition to a market economy b. Components (a) Management capacity improvement (US$14.3 million, funded by the European Union); (b) Undergraduate and continuing education (US$27.6 million); (c) postgraduate education and research (US$42.1 million). The latter two components were funded by the Bank. c. Comments on Project Cost, Financing and Dates The project closed after an extension of three months. An undisbursed balance of US$400,000 was canceled. 3. Achievement of Relevant Objectives: Specific objectives were substantially achieved. (i) Research councils received substantial training through study visits and preparation for grants approvals; (ii) a grants approval program became operational and reviewed about 4000 proposals; grants were distributed to 43 institutions (18 of these got 98.9 percent of the funds). (iii) As a result of the program, market-oriented programs became available and are attended by students. Overall, research management capacity was greatly strengthened, and the ability to carry out research improved likelihood of future funding by other donors. However, it is uncertain whether the larger goal of reforming the management of the entire system was met. Grant submissions were voluntary and many professors were too busy earning a living, so only 4% applied. So, the best organized and most competent universities got most funds. Many of the smaller institutions did not benefit to the extent expected at appraisal, and have not reaped the benefits of the reform. Poor students’ access to stipends did not increase to the extent expected, as 70% of the stipend budget continued to be given for small merit-based scholarships. 4. Significant Outcomes/Impacts: A total of 937 grants were awarded for research and for development of new specialties in degree and non-degree programs, including the masters doctorate degrees. Through the grants, research in various fields advanced, and the number of teaching staff with doctorates increased from 7,829 in 1996 to 11,939 in 2001. Masters degree students increased by almost 300%, from 4,521 in 1996 to 17,884 in 2001, and doctoral students increased by 77%, from 14,472 in 1996 to 25,616 in 2002. Government ownership was strong, and preparation benefited from a Japanese grant facility for preparation and needs assessment. Councils for research and finance developed formulas for a fair distribution of resources, forms for application and financial audits, and monitored the implementation process. A policy matrix was developed, which constituted a set of expected changes that would happen in the system as a result of implementing this project. The newly established National Council for Evaluation and Accreditation has facilitated the accreditation of 10 universities, which account for 28% of the higher education enrollment. 5. Significant Shortcomings (including non-compliance with safeguard policies): There were significant procurement and disbursement problems. The special account was too small for the expenditures, and delays in audit reports or in accounting by grant recipients blocked its replenishment, making it impossible for other researchers to receive money. Thirty percent of grant amounts consisted of counterpart funds, which were disbursed irregularly, so professors who had received awards did not know when they would receive the funds. When notified that the funds were available they had 60 days to disburse them. Thus they could not carry out many planned activities. Bureaucracy became time-consuming. Unless the university had money to lend professors, they could not carry out the funded work. (Some universities did, but others could not.) The PCU had underestimated the ability of universities to overcome these complexities. There were separate project management units for the European Union funding, an issue which created complexities and inefficiencies. 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Satisfactory Institutional Dev .: High High Sustainability : Likely Likely Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTE: NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. 7. Lessons of Broad Applicability: - Rigid procurement and disbursement requirements imposed on smaller institutions (such as schools) or individuals who receive grant funds may adversely impact their ability to implement their projects. Special efforts may be required to limit the paperwork and deadline obligations that grant recipients face. — -Small grants (such as stipends or research) often have a portion contributed by the government. Government delays in paying counterpart funds disrupt important educational functions of individuals. Arrangements should be made during appraisal to shift counterpart funds to components that will not disrupt individual activities. - Donor collaboration should translate into effective and efficient arrangements for the project coordination, monitoring, and evaluation. - Ethical dimensions must be considered in research program as well as the need to monitor compliance with them. Countries that begin donor-funded research programs must create criteria and guidelines for the ethical treatment of experimental subjects. 8. Assessment Recommended? Yes No Why? Part of a cluster audit (already completed) 9. Comments on Quality of ICR: The ICR was clear and presented quantitative inputs and procedural issues in detail . However, it did not delve in greater depth on professors' perceptions regarding grant awards or the extent to which the greater goal of the project was met through the processes initiated by the project .