ICRR 11244 Report Number : ICRR11244 ICR Review Operations Evaluation Department 1. Project Data: Date Posted : 08/19/2002 PROJ ID : P003988 Appraisal Actual Project Name : Id-phrd Ii Project Costs 95.00 52.20 US$M ) (US$M) Country : Indonesia Loan/ US$M ) 69.00 Loan /Credit (US$M) 48.52 Sector (s): Board: ED - General Cofinancing industry and trade sector US$M ) (US$M) (47%), Tertiary education (22%), Central government administration (16%), Sub-national government administration (15%) L/C Number : L3825 Board Approval 95 FY ) (FY) Partners involved : Closing Date 09/30/2000 10/31/2001 Prepared by : Reviewed by : Group Manager : Group : Gillian M. Perkins Helen Abadzi Alain A. Barbu OEDST 2. Project Objectives and Components a. Objectives The project's objectives were to foster economic development and industrialization and facilitate policies of deregulation and decentralization by upgrading the quality of professional, managerial, and science and technology staff in key government ministries and agencies both at local and national levels and, on a limited scale, in the private sector. The private sector objective was dropped one year after effectiveness, following a government directive in 2/96 that fellowship holders would be required to find employment in the public sector on completion of their studies . Since the project was not formally restructured, evaluation is based on the original project objectives . b. Components [A] Fellowships and training (US$94.3 m.): The three subcomponents would be implemented by (i) the National Development Planning Agency, (ii) the Ministry of Finance, and (iii) the Agency for Assessment and Application of Technology. These agencies would undertake similar activities in their respective subject areas to : Provide domestic and overseas degree and non -degree fellowships to staff of key government ministries and agencies; Provide overseas fellowships in subjects not yet available in Indonesia to fresh bachelor's degree graduates, on a pilot basis; Improve re-entry procedures for utilization of fellowship holders, through tracer studies and strengthening of relevant personnel policies; Facilitate overseas degree and non -degree programs for private and public firms by making available such services as testing, placement and monitoring . [B] National Steering Team for Overseas Training (NASTOT) (US$0.7 m.): Strengthen the capacity of NASTOT in formulating policy, coordinating and monitoring civil service fellowships and pilot programs for fresh graduates; undertake studies and other preparatory activities for setting up a national merit -based fellowship program. c. Comments on Project Cost, Financing and Dates Total project costs amounted to about 55% of estimates, due primarily to overestimation of unit costs at appraisal and to depreciation of the rupiah which lowered the US$ cost of domestic training . Loan cancellations amounted to US$12 m. in 1998-99 and a further US$6.5 m. in mid-2001. Government costs of US$3.7 m. were estimated from the loan expenditures on the basis of the agreed disbursement percentages for each category (and excluded fellows' salaries, estimated at US$9m.). The project was extended twice, closing 13 months behind the original schedule . 3. Achievement of Relevant Objectives: Performance on specific objectives was mixed : The project met or exceeded its quantitative targets for fellowships to civil servants, providing a total of 2,513 fellowships. Overseas training accounted for 25% of the fellowships and 85% of project costs. Sound selection procedures were successful in identifying strong fellowship candidates . Virtually all the beneficiaries of training maintained high grade point averages and the great majority completed their studies on time . The project was not successful in its objective of piloting overseas training for the private sector . The subcomponent for provision of fellowships to new graduates was dropped one year into implementation . The 46 graduates (of a planned 90) who had already been assigned to overseas programs completed their studies and returned to take up civil service positions . Although efforts were made, there is as yet no clear evidence of improvement in re -entry procedures and utilization of returning fellows. Provision of testing, overseas placement and monitoring services for private and public firms was partially successful. One of the implementing agencies has been providing testing and information services but not placement or monitoring. The small component for strengthening NASTOT became largely redundant when the pilot for new graduates was dropped and plans for developing a national merit -based fellowship scheme were abandoned . 4. Significant Outcomes/Impacts: Data are available so far only for the earlier cohorts but indicate a potentially important impact on public sector capacity: virtually all beneficiaries of overseas training returned to the country and at least three quarters of all fellows were working in appropriate posts following completion of their studies . In line with recent development of the tertiary sector in Indonesia, the share of domestic training was greatly increased under PHRD II, improving cost effectiveness . The project was well-positioned to support local capacity -building efforts when the government sought to accelerate decentralization . Data are not provided for the shares of beneficiaries from different levels of government, but domestic training targets were increased at mid -term to allow expansion of urgently needed training of regional staff. Capacity of the implementing agencies was enhanced in management of fellowship services : transparent, merit-based selection procedures, established under PHRD I, were reinforced; and dependence on consultants' services of education exchange organizations for placement, coordination and monitoring was largely eliminated. 5. Significant Shortcomings (including non-compliance with safeguard policies): Doubtful relevance of objectives : It is doubtful whether further, large -scale, in-service training of permanent civil servants continued to warrant high priority in the Bank's program . A 1997 Bank study questioned the continuing scale and impact of civil service training in the country . It found that, as a result of tertiary education expansion, the civil service could start recruiting applicants already possessing the necessary level of education and confine in-service training to well-defined, specific needs. It recommended giving priority to establishing the necessary incentive regime and institutional framework to support broad -based investment in human resources . Ad hoc project design : The decision to broaden the scope of the project to address needs in the private sector was taken at the appraisal stage -- too late to carry out the necessary preparatory work and ensure government commitment. The SAR made a strong case for extending assistance beyond the public sector but did not address the issue of cost recovery, or of consistency with national policy for financing tertiary education in general . The project's potential impact was also limited by lack of integration with broader civil service reform : efforts to improve re-entry procedures and utilization of returning fellows were constrained by personnel policies that were beyond the scope of the project and over which the implementing agencies had no control . The proposed training was not linked to specific, monitorable improvements in public sector performance with regard to the project's stated objectives (of fostering economic growth, industrialization, deregulation and decentralization ). Weak financial management : At times during project implementation, one or more of the agencies had difficulties demonstrating precisely the financial status of its subcomponent . For the ICR, government expenditures had to be deduced from the Bank's figures for loan expenditures by category . 6. Ratings : ICR OED Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately Satisfactory The shortcomings noted above warrant qualification of the outcome rating . Institutional Dev .: Substantial Substantial Sustainability : Highly Unlikely Likely The benefits of the project -- primarily upgraded skills of public sector staff -- are likely to be sustainable in so far as beneficiaries are returning to (or remaining in) the country and are continuing to work in appropriate posts . Bank Performance : Satisfactory Satisfactory Borrower Perf .: Satisfactory Satisfactory Quality of ICR : Satisfactory NOTE: NOTE ICR rating values flagged with ' * ' don't comply with OP/BP 13.55, but are listed for completeness. 7. Lessons of Broad Applicability: Adding to project objectives at a late stage of preparation can jeopardize both technical soundness and national ownership of the project. "Link programs," combining initial studies at a domestic university with completion of a program at an overseas university, can be an effective way of meeting higher level training needs : link programs are more cost effective than purely overseas training, and they can allow the implementing agency to tailor the study programs closely to specific needs. Continuity of staff on both the Bank and borrower's sides is important for successful implementation . 8. Assessment Recommended? Yes No 9. Comments on Quality of ICR: The ICR is satisfactory. The rating for sustainability should be based on the likelihood of sustaining the benefits of the project itself, not on whether the program is likely to be (or should be) continued.