Document of The World Bank FOR OFFICIAL USE ONLY Report No.: 20677 IMPLEMENTATION COMPLETION REPORT TURKEY NATIONAL EDUCATION DEVELOPMENT PROJECT (Loan TR-31920, 3192S, 3192A) June 5, 2000 Human Development Unit Europe and Central Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENC'Y EQUIVALENTS (as of 30 June 1999) Currency Unit Turkish Lira (TL) US$1 TL 418,189.00 AVi:RAGE EXCHANGE RATES 1990 1991 1992 1993 1994 1995 1996 1997 1998 2,606.36 4,168.11 6,864.36 10,965.42 29,669.76 45,678.23 80,797.53 151,238.90 259,626.66 1999 416,685.50 WEIGHTS AND MEASURES Metric System ABBREVIATIONS AND ACRONYMS APK Council for Research, Planning and Coordination BEP Basic Education Project BEPP Basic Education Pilot Project BOE Board of Education CES - Computer Experimental Schools CLS - Curriculum Laboratory Schools ERC Education Resource Center ERDC - Education Research and Development Center ERDD - Education Research and Development Department FRTVC Film, Radio and Television Center ICR - Implementation Completion Report MADU - Management Assessment and Development Unit METARGEM Vocatonal/Technical Education Research and Development Center MIS Management Information System MOF - Ministry of Finance MONE - Minist:ry of National Education NEDP - National Education Development Project NFVT Non-Formal Vocational Training PCU - Project Coordination Unit SAAT Student Achievement Assessment Test YOK - Council for Higher Education TURKEY'S FISCAL YEAR January 1 - December 31 Vice President: Johannes Linn, ECAVP Country Director: Ajay Chhibber, ECCO6 Team Leader: Michael Mertaugh, ECSHD Sector Leader: Annette Dixon, ECSHD FOR OFFICIAL USE ONLY IMPLEMENTATION COMPLETION REPORT TURKEY NATIONAL EDUCATION DEVELOPMENT PROJECT (Loan TR-31920, 3192S, 3192A) Table of Contents Preface Evaluation Summary Part I. Project Implementation Assessment A. PROJECT OBJECTIVES Background and Project Objectives Evaluation of Project Objectives B. ACHIEVEMENT OF PROJECT OBJECTIVES C. IMPLEMENTATION RECORD AND MAJOR FACTORS AFFECTING THE PROJECT D. BANK PERFORMANCE E. BORROWER PERFORMANCE F. ASSESSMENT OF OUTCOME G. PROJECT SUSTAINABILITY H. FUTURE OPERATION I. KEY LESSONS LEARNED Part II. Statistical Tables Table 1: Summary of Assessment Table 2: Related Bank Loans/Credits Table 3: Project Timetable Table 4: Loan/Credit Disbursements: Cumulative Estimated and Actual Table 5: Key Indicators for Project Implementation Table 6: Key Indicators for Project Operation Table 7: Studies Included in Project Table 8A: Project Costs Table 8B: Project Financing Table 9: Financial and Economic Costs and Benefits Table 10: Status of Loan Covenants Table 11: Status of Compliance with Operational Manual Statements Table 12: Bank Resources: Staff Inputs (actual) Table 13: Bank Resources: Missions Appendices: A. Implementation Completion Mission's Aide-Memoire B. MONE's Contribution to ICR C. YOK's Contribution to ICR D. Map This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not be otherwise disclosed without World Bank authorization. IMPLEMENTATION COMPLETION REPORT TURKEY NATIONAL EDUCATION DEVELOPMENT PROJECT (Loan TR-31920, 3192S, 3192A) Preface This is the Implementation Completion Report (ICR) for the National Education Development Project (NEDP) in Turkey for which Loan TR-31920, TR-3192S and TR-3192A in the amount of US$90.2 million equivalent was approved on May 18, 1990, and made effective on July 31, 1990. The loan was closed on June 30, 1999, instead of the original closing date of June 30, 1997, after three extensions (June 30, 1998, December 31, 1998 and June 30, 1999) totalling two years. As of 1 November 1999 (the date of last transaction), the amount of US$ 700,683.65 equivalent from the Loan 3192S-TU was canceled and also as of 20 April 2000 (the date of last transaction), the amount of US$ 20,429,958.35 equivalent from the Loan 3192A-TU was canceled, totalling the canceled amount to US$ 21,130,642.-. In summary, US$ 15.0 million, US$ 36.9 million and US$ 17.2 million have been disbursed from the 31920-TU, 3192A-TU and 3192S-TU, respectively. The ICR was prepared by Ferda Sahmali, Task Team Leader, ECCTR and Robert Schware, Senior Information Technology Specialist, EMTII. Helpful comments and assistance were received from Michael Mertaugh, Education Program Team Leader, ECSHD. The ICR was reviewed by Marlaine E. Lockheed (MNSHD) and Peter Russell Moock (EASHD). Preparation of this ICR began during the Bank's final supervision/completion missions in June and October 1999. It is based on material in the project files as well as information received during the course of supervision. The two implementing agencies (Ministry of National Education and Council for Higher Education) contributed to the preparation of this document by providing written input, which is attached in the appendix. Comments were also received from the implementing agencies. TURKEY NATIONAL EDUCATION DEVELOPMENT PROJECT (Loan TR-31920, 3192S, 3192S) Evaluation Summary Introduction 1. There has been a long history of Bank lending to Turkey's education sector focused primarily on improving vocational/technical education and manpower training with a view to relieving scarcities of technically qualified and skilled labor. The National Education Development Project (NEDP) was the first project which was designed to focus on quality enhancement and management improvement of the Government's broader sectoral development program. Project Objectives 2. The Project's three broad objectives were to: (i) improve the quality of primary and secondary education for increasing student achievement to a level approaching the average OECD countries; (ii) improve the quality and relevance of teacher education to a level consistent with norms for such training in OECD countries; and (iii) improve management and administrative skills and practices in the Ministry of National Education (MONE) for more efficient and effective use of resources. Achievement of Project Objectives 3. The second objective of the Project was largely attained. Progress under the first and third objectives was less clear cut. Although a number of concrete actions were carried out under the project to improve inputs to learning and education management, the educational benefits in many cases will take time to spread these inputs throughout the system. In several key areas, the project's contribution to improved educational quality was developmental. Achievement of the full quality gains throughout the system requires replication of the activities developed under the Project. This is necessarily a long-term undertaking which is in progress in most of the key project areas: pre-service and in-service teacher training, student assessment, curriculum laboratory schools, computer laboratory schools, basiceducation pilot project and mother and child development program. In a few cases, quality gains were frustrated by inertia within the Ministry, rivalry among units of the Ministry, and the difficulty of attracting staff and managers with the requisite qualifications (notably curriculum development and management and development). For example, the Project supported the establishment of a new curriculum development unit which was intended to consolidate the curriculum development functions of several bodies within the MONE and the Board of Education (BOE). The new unit was created and its staff were trained, but the consolidation of curriculum development functions of other units ii did not occur. As a result, some aspects of curriculum development have benefitted from the project, while others have not. 4. There were also serious problems of ownership at the start of implementation, both on the part of senior Ministry management and the line units which were expected to implement the various components of the Project. This problem resulted partly from the departure at Project initiation of the Undersecretary under whom the Project was designed, and partly from the fact that many of the key line units were not sufficiently involved in project preparation. Project implementation took two years to get under way, and moved very slowly for the next three years. Project implementation improved thereafter, due largely to the efforts of a new Undersecretary who made a personal commitment to retrofit ownership among some of the key line units involved in the Project. 5. The Project resulted in setting up two new units, namely, the Educational Research and Development Department (ERDD) and Management Assessment and Development Unit (MADU) at the MONE. ERDD is responsible for curriculum development and succeeded in developing 21 new curricula. ERDD is also responsible for establishing 208 Curriculum Laboratory Schools (CLS) in 23 provinces, responsible for establishing for the first time testing (student achievement assessment) and evaluation system, and for conducting six educational studies. MADU is responsible for establishment and implementation of management assessment procedures and development of functional job descriptions. Management assessment studies conducted by MADU resulted in the publication of two regulations on the "Appointment, Evaluation, Promotion, and Transfer of Ministry of National Education Administrator"and on the "Norm Cadres of the Ministry of National Education Schools' Administrators and Teachers". MADU also played a leading role in training and implementing "Total Quality Management and Process Reengineering" at the MONE. 6. The Project funded the development of 53 Computer Experimental Schools (CES) as an experimental platform to explore how far computer education can facilitate instruction and provide tools to improve teaching and learning. The success of the CES led to the expansion of number of computer experimental schools from 53 to 235 schools, financed with project savings. The schools were equipped with a computer laboratory and teacher rooms were also equipped with a PC each, and 2,395 teachers were trained on computer literacy. 7. The Project funded a modern Computerized Textbook Distribution System in 21 regional offices of the MONE. The offices were fully equipped with computers and inventory control software. The users of the system provided specialized training to operate the system. However, only in one of the 21 offices, the Computerized Textbook Distribution System is fully operational. The System in the remaining 20 offices is not operational due to ineffective management and lack of qualified personnel. 8. The Project assisted in improving the quality of pre-service teacher education by providing laboratory equipment to 42 faculties of education managed by the Council for Higher Education (YOK). Curriculum and material development were completed and 23 textbooks were published and distributed to the Faculties of iii Education. Masters and doctoral fellows were sent abroad and 21 masters fellows and 52 PhD fellows obtained their degrees, in addition to 18 fellows who were sent for post-doctoral research in their field. 9. The Project supported expansion and upgrading of the in-service teacher training, mainly on computer literacy and specialized subject areas. In-service training was decentralized. MONE provided in-service training for 577,756 trainees via 12,529 courses. This number far exceeded the figure foreseen in the Staff Appraisal Report (SAR). 10. Six research studies on educational subjects funded by the Project were completed and published. A management information system was designed and implemented for the first time at MONE in order to decentralize operations to local offices and to provide a network of information flows between schools and local offices and between local offices and MONE's central organization. 11. The NEDP was restructured on 18 January 1996 to include three additional components financed by Loan savings: (a) a Basic Education Pilot Project; (b) Secondary Education Studies; and (c) a Mother and Child Development Program. Under the Basic Education Pilot Project, various educational materials, teachers' guides and student activity books were distributed to 276 village schools; In-service training for inspectors and teachers was provided; a school funds program was initiated in 30 village schools; upgrading, furnishing and extensions of 14 rural schools were completed; pre and post evaluation studies were conducted to measure the achievements of the rural education activities. Secondary Education Studies provided assistance to the MONE to help develop a strategy to reform secondary vocational/technical education to make it more affordable and more responsive to the market needs. The Mother and Child Development Program provided training to 35,000 mothers and pre-school children in 59 provinces of Turkey. Twenty five short films on mother and child education were broadcasted on national and local TV channels. Implementation Record. Bank Performance and Borrower Performance 12. The NEDP was implemented by MONE and YOK. The two implementating agencies maintained two separate Project Coordination Units (PCUs). Both PCUs were slow in launching the Project implementation. This was due to the delay in signing Technical assistance (TA) contracts by both implementing agencies. The MONE signed the TA contract two years after the effective date of the Loan; it took YOK four years to sign the TA contract. As there was one project special account, the two PCUs had difficulty in managing the payments from the special account and on many occasions there would not be sufficient funds to make a payment due to delay in replenishment requests. Procurement of equipment and materials under International Competitive Bidding took more time than was envisaged; this caused considerable delays in the implementation. 13. During implementation, the Project's two agencies found the Bank performance to be satisfactory, but changes in task managership delayed timely action to deal with administrative and technical problems in implementation of the Project. iv 14. Borrower performance in implementation was adversly affected by high turnover of staff at senior and middle management levels, and inadequate availability of counterpart funding and delay in releasing the funds. This created an uncertainity concerning project planning and implementation. Procurement delays and cancellations adversly affected the completion of the Project. Project Sustainability 15. Overall prospects of Project sustainability are assessed as uneven due to individual characteristics of each component of the Project. Sustainability of most MONE components depends upon sufficient allocation of funds and assignment as well as retention of qualified staff, whereas YOK components are more likely to be sustainable due to commitment of the faculties of education. Restructuring of the faculties of education also increased the sustainability of the YOK components. 16. Sustainability of Project interventions in the Ministry of Education suffers from a problem which is present, to some extent, in all ministries: the difficulty of attracting qualified staff with civil-service salaries. This was a particular problem in areas - such as film, radio, and television production - where there are better professional opportunities outside the Ministry of National Education. While this problem could be addressed through more liberal and more imaginative use of consultants, the Ministry is reluctant to do so - largely because of the resentment which results from the high salaries paid to consultants in some specializations. The large institutional consultant contracts for educational support to the Ministry of National Education may also not have left behind the permanent benefits which were foreseen because of ownership problems. Throughout implementation of the Project, the Ministry felt that the consultants were forced upon it. The Ministry did not make full use of the consultants, and in many cases did not provide counterpart staff who could internalize and sustain the potential benefits of the consultants. Future Operations 17. The MONE intends to sustain the achievements of the NEDP through the Basic Education Program. The Program is supporting in-service training for teachers, inspectors and school principles, is providing textbooks and educational materials, is implementing an IT program that will phase in different instructional objectives starting with computer literacy and moving towards more skill-based and subject matter related training, is supporting renovation and extension of school buildings etc. The MONE also intends to initiate secondary general and vocational education project as a complementary project to the Basic Education Program. Lessons Learned 18. The main lesson learned through implementation of the Project is that project design needs to be based on a realistic assessment of constraints, and a recognition of which constraints can be mitigated by project design and which cannot be. The design of the project assumed that the improved inputs would bring about the expected improvements in educational outcomes, and assumed that the Ministry would continue to be committed to the Project even after the departure of the Undersecretary who conceived the Project. The Project had been designed principally v by this Undersecretary and a group of like-minded collaborators - including the Bank appraisal team and a number of hand-picked consultants. This group was excited by the Project design and committed to making it work. But there was limited involvement in Project design among the line units which were expected to implement the Project, and, consequently, little of this commitment or excitement. There was also no prior agreement on the part of those units which were expected to relinquish some of their responsibilities under the Project, and in the event failed to do so. 19. In retrospect, it would have been better to recognize the structural constraints at the start, and to design interventions into the Project to work around them, rather than to assume them away. This would have required the full involvement of the line units from the start of project design. Doing so would require more time for project preparation and would perhaps require more modest project objectives and a narrowing of project scope, but would produce better implementation results. vi TURKEY NATIONAL EDUCATION DEVELOPMENT PROJECT (Loan TR - 31920, 3192S, 3192A) Part I. Project Implementation Assessment Project Identity Project Name National Education Development Project Loan No. 31920, 3192A, 3192S Country Turkey Sector Human Development A. PROJECT OBJECTIVES Background and Project Objectives 1. Prior to the implementation of the National Education Development Project (NEDP), the Bank's support to Turkey's education sector focused primarily on expansion and improvement of vocational/technical education and manpower training. This reflected the high priority which the Government attached to the provision of trained manpower to the productive sector, and its previous reluctance to borrow for general education. In line with the Government's priorities, the Bank has provided five educational loans to Turkey, all of which were designed to improve the Country's vocational education and manpower training capabilities. The first loan (Ln. 748-TU) signed in 1971, and completed in 1981, supported the development of secondary vocational education and trade schools and vocational teacher education colleges. The subsequent loans (Ln. 2536-TU and Ln. 2776-TU) signed in 1985 and 1987, respectively, continued to support the upgrading and expansion of secondary vocational schools and trade programs. The other two loans (Ln. 2399-TU and Ln. 2922-TU), signed in 1984 and 1988, respectively, supported the development of post- secondary technician training centers, and Faculties of Technical/Vocational Education under Council for Higher Education (YOK), and special programs for tourism training under the Ministry of Tourism. 2. Following the Bank's 1986 Education Sector Survey Report, which made a wide range of recommendations for sector reform and improvement, the Government's own evaluation of the system, as stated in the SAR, confirmed the Report's major conclusions and presented a plan for reform, based on its recommendations. Subsequent to Bank identification missions in February and May 1988 and preparation assistance missions in September 1988 and February 1989, the NEDP was prepared by the Government. Pre-appraisal discussions were held in June 1988 and the Project was appraised in October/November 1989. 3. The NEDP was designed to focus on quality enhancement and management improvement of the Government's broader sectoral development program. The Project addressed, through the objectives listed below, three of the major sectoral issues, i.e. student achievement, educational efficiency and organizational management. The Project's three broad objectives were to: (i) improve the quality of primary and secondary education for increasing student achievement to a level approaching the average in OECD countries; (ii) improve the quality and relevance of teacher education to a level consistent with norms for such training in OECD countries; and (iii) improve management and administrative skills and practices in the Ministry of National Education (MONE) for more efficient and effective use of resources. 4. The Project was designed to be jointly implemented by the MONE and YOK over a seven year period. The Project focused on high priority medium-term sectoral development actions, along with a number of longer-term measures designed to strengthen various aspects of Turkey's education infrastructure. The detailed components of the Project were as follows: Improving Quality of Primary and Secondary Education 5. US$39.0 million was allocated by the World Bank, including physical and price contingencies (total cost of US$73.9 million), to improve quality of primary and secondary education for increasing student achievement to a level approaching the average in OECD countries. 6. This component had eight sub-components: (a) Curriculum Management and Policy Development, (b) Curriculum Development, (c) Curriculum Laboratory Schools, (d) Computer Experimental/Laboratory Schools, (e) Testing and Evaluation, (f) Education Studies, (g) Quality of Textbook and Materials, and (h) Distribution of Textbooks and Materials. Improving Quality and Relevance of Teacher Education 7. US$32.3 million was allocated by the World Bank, including physical and price contingencies (total cost of US$82.7 million), to improve the quality and relevance of teacher education to a level consistent with norms for such training in OECD countries. 8. This component had two sub-components: (a) Pre-service Teacher Education, and (b) In-service Teacher Training. Improving Management Skills and Practices 9. US$18.9 million was allocated by the World Bank, including physical and price contingencies (total cost of US$20.6 million), to improve management and administrative skills and practices in the MONE for more efficient and effective use of resources. 10. This component had four sub-components: (a) Management Assessment and Development, (b) Research and Planning, (c) Management Information, and (d) Research in Vocational/Technical Education. 2 11. Total loan financing provided under the project was US$90.2 million including physical and price contingencies. Of this amount, US$67.1 million was allocated to activities to be implemented by the MONE, and US$23.1 million to activities to be implemented by YOK. The Government's contribution to the NEDP was planned to be US$87.0 million. The total estimated cost of the NEDP was US$177.2 million. 12. The NEDP was restructured on 18 January 1996 to include three additional components financed by Loan savings totalling US$15.33 million: (a) a Basic Education Pilot Project (US$13.1 million); (b) Secondary Education Studies (US$0.5 million); and (c) a Mother and Child Development Program (US$1.73 million). 13. The loan was closed on June 30, 1999 (the original closing date was of June 30, 1997). The Project closing date was extended three times to June 30, 1998, December 31, 1998 and June 30, 1999, upon the request of the Borrower, in order to complete project implementation including the three activities added under the restructuring of the Project. They extended the total implementation period from seven to nine years. Evaluation of Project Objectives 14. The objectives of the NEDP were clearly defined. However, in retrospect, the expectations of the appraisal team for the first and third objectives were unrealistically ambitious. The first objective of the NEDP (improving the quality of education) through reform of curriculum management and policy development was unrealistic. Many of the necessary measures to attaining such an objective were outside the responsibility and influence of a Bank-financed investment program. The SAR stated that "all curriculum development units/groups dispersed throught the MONE would be consolidated into a unified Education Research and Development Center (ERDC) ... and the Center would have the responsibility for all development activities in the education sector, ... and Board of Education (BOE) would continue its leadership role of advising the Minister on approval of curriculum and related development outputs". Achievement of this objective would have required new legislation, as well as redefinition of the roles and responsibilities of the BOE and other units of the Ministry. This did not take place as envisaged in the Project due to internal rivalries in the Ministry and inertia of the existing structure, which proved highly resistant to attempts to change roles and responsibilities of individual units. The first objective aimed to restructure the MONE's core activities like curriculum development, textbook quality improvement, student achievement assessment, and educational research and development. Under this objective it was assumed that structural changes and technical improvements would trigger improvement in quality of education. However, the Project paid too little attention to how these changes were to be effected and -in particular- to establishing incentives for cooperation (rather than overt resistance) among the various units of the Ministry. In retrospect, the lack of involvement of many of the line units of the Ministry in the design of the Project created a bias against successful implementation. 15. The second objective of the NEDP (improving the quality and relevance of teacher education) was relevant and realistic. Pre-service teacher education was 3 transferred to YOK under the Project. This change was a positive step which aided the implementation of the second objective of the Project. 16. The third objective of the NEDP (improving management skills and practices) was unrealistic. Measures which were supposed to be taken to improve administrative efficiency and general management of the education system were ambitious and not realistic considering the lack of qualified personnel and the lack of legislative framework. This was not adequately foreseen by the Bank at the time of appraisal. B. ACHIEVEMENT OF PROJECT OBJECTIVES 17. Overall, the project was substantially successful in achieving its objectives. In meeting its first objective, Improving the Quality of Education, the Project was partially successful. Curriculum Management and Policy Development: The Education Organization Law (Act number 3797 dated 30 April 1992), to restructure the MONE failed to create and establish an Education Research and Development Center (ERDC) which would unify the curriculum, textbook and materials development activities dispersed throughout the Ministry. What the new law did in fact was to add another unit under the name of Education Research and Development Department (ERDD) for curriculum development to other educational units which develop curricula (i.e., General Directorate for Primary Education, General Directorate for Secondary Education etc.). Moreover, the BOE was left intact and continued developing and approving the c urricula, textbooks and educational materials. This was clearly not the intention of the NEDP as stipulated in the SAR. On the contrary, the Project envisaged the elevation of the BOE's image and stature within the MONE and the Country by promoting the BOE to a high level executive body which would be a true policy decision making and approval unit. The BOE was not intended to be responsible for managing an operation, whose own work it would have to approve. This is a conflict of interest which need to be addressed by the MONE. Or the other hand, the ERDD struggled to establish itself in an environment which was not conducive to its well being. In addition, initially the head of the ERDD was a political appointee who was not capable of leading such a new unit. However, towards the end of the Project a new head of the ERDD was appointed who was dedicated and committed to the success of the unit. In short, the legal and regulatory basis for the ERDD was not sufficient to realize the achievements stated in the SAR, and all it was done was the establishment of an additional unit for the MONE. * Curriculum Development: The ERDD was slow in developing curriculum for various administrative and legal reasons. Later the ERDD succeeded in developing 21 curricula since its establishment. The ERDD developed the curricula by using a new "Curriculum Development Model". Although this Model was recognized and recommended by the BOE to be used by other curriculum development units at the Ministry, the BOE managed to approve only 3 curricula in three years which are being used in schools since last school year. The remaining 18 curricula were awaiting the BOE's approval during the preparation 4 of this ICR. In short, establishment of a curriculum development model, increased the awareness of the decision makers on various issues that are linked with professional, comprehensive and systematic curriculum development process. * Curriculum Laboratory Schools (CLS): In 23 provinces, 208 CLS including 147 basic education schools, 37 general secondary schools, 16 Anatolian secondary schools and 8 Anatolian teacher training schools were provided with educational equipment. The equipment consisted of PCs, lap-tops, printers, data show machines, fax machines, modems, scanners, TV sets, video recorders and video cameras, overhead projectors and screens, tape recorders, projectors, cameras, flip charts and physics, chemistry, biology and life sciences laboratory sets were funded from the Loan. The majority of these schools have access to the internet and have developed their own WEB pages. School furnitures (individual student desks, student lockers, laboratory tables, library shelves, etc.) were procured through Government contribution. Local funds were used to renovate and maintain CLS in order to meet established building standards. Music and drawing rooms, computer and science laboratories, libraries, teacher study rooms, teacher common rooms and multi-functional halls were established. 6630 CLS principals, teachers, counselors and inspectors received in-service training to use the improved facilities of the CLS in more effective teaching. Educational technology formators were provided training on effective use of the equipment, and appointed to 23 provinces, where CLS exist. The formators in turn provided training to the CLS teachers and administrators in use of the equipment. The quantum improvement in teaching and lecturing equipment in the CLS was immediatelly recognized by the intended beneficiaries. There was and remains very strong demand for these schools on the part of parents, teachers and students. In spite of this strong demand the administrators resisted the pressure to increase enrollments beyond the levels foreseen at appraisal (40 students per class). As a result, the high quality of these schools was preserved and the CLS component fully achieved its objective of improving teaching and learning conditions and establishing a model for the rest of the system to follow. Although the evaluation of the CLS was not completed yet, the MONE issued a new regulation on 20 October 1999, to expand the implementation of CLS throughout the Country, however, leaving the question of funding the CLS open. In short, CLS activity, under the NEDP, achieved its objective in establishing those schools, providing them with the necessary equipment, and providing training to the staff of the schools. * Computer Experimental/Laboratory Schools (CES): The CES aimed to develop 53 schools as an experimental platform to explore how far computer education can facilitate instruction and provide tools to improve both teaching and learning. The schools were equipped with a computer laboratory, and teacher rooms were also equipped with a PC each. 2395 teachers were trained through in-service training programs. The pilot was evaluated by an international team of consultants in October 1996. The independent review of the CES found that administrators, teachers, and students were all enthusiastically and creatively using the equipment and software. In the schools visited by the evaluators it was clear that the students were utilizing the facilities offered by their laboratories on a daily basis and the schools and the educational communities they serve welcomed the technology in 5 the 53 CES. This justified a request from the Ministry to expand the educational opportunities offered by the new tools, and an additional 182 schools were included in an unplanned expansion of the pilot. In short, the project realized its quantitative objectives in terms of training outputs and establishing facilities and equipment. At the same time, unexpected benefits were also realized from the experiment. In some cases, support from the Parent Teachers Associations (PTA) had resulted in upgraded equipment, additional hardware, and even hired technicians. Parents increasingly wanted their children enrolled in a CES/CLS school, leading to overcrowding of some classrooms, and continue to want to send their children to these schools. The size of classes, the interests of students and parents, and the need to provide greater access to the laboratories resulted in additional weekend classes, mostly in computer literacy. Based on the evaluation of CES, and its popularity with parents, teachers and students, it appears that CES was very successful. However, a more definitive account must await the results of more formal assessment of actual learning outcomes. Overall, CES was substantially successful in achieving its objectives. Testing and Evaluation: Testing and evaluation was applied in MONE schools for the first time under the Project. This was an important first step in the use of student assessment as a diagnostic tool for improving the effectiveness of teaching throughout the Country. ERI)D prepared and regularly conducted on a yearly basis Student Achievement Assessment Tests (SAAT) in Science, Mathemathics, Social Sciences and Turkish. The SAAT were carried out on 347,222 CLS students and 38,275 non-CLS students, totalling 385,497 students. The SAAT paved the way for ERDD to research, produce, validate and standardize tests for establishing national norms on student cognitive achievement. A Test Item Data Bank with 6,396 test items was developed. Out of this number, 3,137 test items were piloted and validated. The ERDD successfully administered the mathematics and science achievement tests under the Third International Mathematics and Science Study Repeat (TIMS-R) in 204 selected schools to 8000 13 year-old students. Despite these achievements, very significantly ERDD suffered continuously from inadequate infrastructure and budgetary constraints. For instance, ERDD was denied adequate space, equipment such as computers and related software and most Importantly qualified staff and funds. * Education Studies: Six educational research studies were carried out to help define a strategy for future education development in key areas. Studies were conducted on: (i) Development of a teacher testing program to establish base-line data against which to measure improvements in teacher quality; (ii) A school mapping study to determine projected student growth patterns and trends in population shifts; (iii) In-service teacher training as a factor for consideration in the promotion of education personnel; (iv) Analysis of actual teacher loads throughout the system; (v) Inventory of instructional equipment and supplies in schools to determine availability and extend of needs; (vi) Assessment of the use of science laboratories. In addition, EARGED organized a research study on "Education Studies Program for Turkey" and based on this research a "Study Model" was developed so as to define the methods of future studies. Although the subjects of the research studies were crucially important for the education system, the studies concerned were not well done by the Turkish academicians, and related units of MONE failed to respond to the outcome of the studies. The units to which the 6 research studies were sent did not show a particular interest to these studies and as a result they failed to make benefit out of these studies. * Quality of Textbook and Materials: The Textbook Technical Quality Control Unit which was established in the ERDD under the Project was later transferred to the Publications Department. This was because the ERDD could not accommodate the Unit as the ERDD was not established as a center. The transfer of the Unit however did not help solve the Unit's legal status as the BOE prevented the Publication Department from letting the Unit be established in this department by arguing that anything to do with publication was under its jurisdiction. The BOE was quite difficult to work with as it could not develop ways to sharing any responsibility which may be related to textbook content. This rift between the two departments of the Ministry hindered the improvement of the quality of textbooks and materials and prevented the establishment of pre-press publication capabilities at the MONE. Although the SAR envisaged that 80% of the textbook printing would be transferred from MONE to the private sector, only 50% of the textbook were printed by the private sector. * Film, Radio and Television Center (FRTVC): The Project funded a major local training program on non-textbook teaching and learning materials design and development. Seminars, workshops and peer-assisted training were also conducted for the FRTVC staff for main areas such as script writing, radio and television sound recording, lighting for television, camera work, staff organization for effective radio and television production, pre-production training etc. Facilities renovation to install the equipment to be procured under the Project was realized, but the amount of time spent to develop specifications for equipment procurement and the repeated re-biddings for equipment procurement reduced the effectiveness of the training carried out. The first two biddings of the MONE to procure the FRTVC equipment were cancelled for various reasons which stemmed from the Bank, MONE and the bidders. At the third attempt to procure the equipment, only five packages out of 12 packages could be procured. However, the main packages, i.e. color TV studios, which would have updated the production of non-textbook teaching and learning materials capacity of the Center was not realized. Failure to procure the main technical equipment and failure to maintain the existing studios and equipment reduced the effectiveness of the Center. Therefore, the attempt to improve the production capacity and capability of the FRTVC through the provision of modem, updated equipment for better quality materials was unsuccessful. * Distribution of Textbook and Materials: A modem computerized textbook distribution system was established in the Publication Department and its 21 regional offices. The offices were fully equipped with computers and inventory control software. The users of the system received specialized training to operate the system. Two comprehensive reports were prepared. The recommendations were provided to improve management of textbook storage and distribution. During implementation of the Project, the system was regularly upgraded and the staff were re-trained. However, inspite of the provided facilities and repeated training, the lack of commitment of the Publications Department prevented efficient operation of the system. Out of 21 regional offices, only Konya office 7 was run efficiently and fully operational. In short, lack of commnitment by the MONE was a continuous problem during the implementation of the Project. 18. In meeting its second objective, Improving Ouality and Relevance of Teacher Education, the Project was substantially successful. * Pre-service Teacher Education: Pre-service teacher education was implemented by YOK. Despite the late start of technical assistance (4 years after loan effectiveness) YOK made a concentrated effort to make up for the delay, and almost all the activities foreseen under this component were accomplished with professional skill and dedication. The component was also expanded from 34 Faculties of Education to 42 Faculties, in order to include several new Faculties of Education. Equipment and educational materials (computer laboratories, education resource room, overhead projectors, video equipment, physics laboratories, chemistry laboratories, biology laboratories, music room and social science equipment) were provided to the 42 Faculties under the component. The curriculum and materials development were completed and 23 textbooks were published and distributed to the related departments of the Faculties of Education. The books were widely used by faculty members and students. In order to meet the high demand for these books YOK made the books available on the internet for the use of all concerned. The master and doctoral programs were completed, and 21 master fellows (against the planned 50 fellows) and 52 PhD fellows (against the planned 62 fellows) obtained their degrees. There were also 18 fellows (against the planned 20 fellows) who were sent for post-doctoral research in their respective fields. Some fellows upon their return went to their respective faculties. Some master fellows were given scholarship by the MONE to further their studies abroad. The fellowship program faced several problems. Timing was a major factor in implementing the program. Due to the late start of the technical assistance, the timing required for PhD training (33 months) extended beyond the original closing da:e. Therefore, without the extension of the Project, the PhD training would not have been completed. Language proficiency was another problem, as YOK could not find well-qualified fellows to be sent abroad. But, overall, YOK with the assistance of the technical assistance contractor, managed to accomplish the program satisfactorily. The Project's extension allowed the completion of two major activities under NEDP. The Faculty-School Partnership Program, and the Teacher Education Standards and Accreditation System. YOK and MONE made a considerable effort to improve faculty-school partnership. A national system of faculty-school partnership was set up after nationwide training courses run by the Partnership Working Party, in which 2,003 received training. This led to each faculty of education entering into partnership arrangements with the schools it uses for student-teacher work in schools. The proposed approach to accreditation was successfully piloted in six faculties in February 1999, and a comprehensive book on standards and accreditation was published as the basic reference for future accreditation activity. In this regard, national and international conferences were held on standards and accreditation in Edirne and Ankara. YOK also conducted dissemination courses to inform program heads of the standards and the accreditation process, and 366 people attended those courses. In add.ition, 36 future accreditation assessors were trained. 8 * In-service Teacher Training: In-service teacher training was implemented by MONE's In-service Training Department. The Project supported expansion and upgrading of the In-service Training Department. It also strenghtened its capabilities for planning and conducting training needs assessment, and designing in-service training to meet identified needs. The project also supported the preparation and development of in-service teacher training materials and programs. Due to the extremely large number of staff, the in-service teacher training was decentralized. MONE provided in-service training for a total of 577,756 trainees in 12,529 courses. This number far exceded the figure of 200,000 foreseen in the SAR. Of this figure, 251,476 were trained centrally in 4,061 courses; 326,280 trainees were trained locally through 8,468 courses. The Project gave special emphasis to the establishment of four Education Resource Centers (ERC) to assist in curriculum and institutional materials dissemination, to serve as regional depositories of teacher resource materials which teachers could use independently of organized training programs. The Project had foreseen the establishment of two ERC in Hasanoglan (near Ankara), one in Isparta and one in Istanbul. Later on, ERC were established in Ankara, Aksaray, Icel and Izmir. Although the ERC were equipped with computers, science and language laboratories, the first three Centers were used only for computer and language training, whereas the Center in Izmir was not established due to lack of suitable space. The ERC were not successful in achieving the goals and objectives as defined in the Project, mainly due to the lack of commitment by MONE. MONE failed to provide qualified administrative and technical staff to those centers. MONE also failed to explain the purpose of establishing these centers and did not provide incentives to make use of these places. As a result, administrators showed little interest in running the centers, and teachers were not keen in making use of these centers. 19. In meeting its third objective, Improving Management Skills and Practices, the Project was partially successful. * Management Assessment and Development: The two units (Management Development Unit and Management Assessment Unit) as mentioned in the SAR, were unified under the Management Assessment and Development Unit (MADU) by MONE at beginning of the implementation of the Project. Several organizational structures were attempted for the establishment of the unit. An attempt to integrate the unit into the General Directorate of Personnel or to the Office of a Deputy Undersectary was not materialized. Eventually MADU was given a separate status through Ministerial approval with its own regulation. MADU The committee of managers as stipulated in the SAR was not appointed. Procedures for management assessment were established and implemented. Procedures for developing functional job descriptions were established and implemented. MADU developed job descriptions for central units of MONE, Ankara Provincial Directorate of Education and primary and secondary school principles. MADU also initiated and completed management assessment studies, which resulted in the publication of a regulation as the Council of Ministers decision due to the importance of the regulation. The regulation concerned was on the "Appointment, Evaluation, Promotion, and Transfer of Ministry of National Education Administrator". Another important regulation on the "Norm Cadres of the Ministry of National Education Schools' Administrators and Teachers" was 9 also published in the Official Gazette. MADU played a leading role in training and implementing Total Quality Management and Process Reengieering. Inspectors were trained in the process of management assessment, but the effort could not be sustained due to opposition shown by the Board of Inspectors. Despite the active role played by MADU in management development and assessment activities, the uncertainity of the legal status of this unit was a cause for concern with regard to the future activites of this unit. Research and Planning: Six research studies stipulated in the SAR were completed and published, after substantial delay, by ERDD. The studies concerned were on: (i) Teacher assessment; (ii) Enrollment estimation/school mapping; (iii) Inventory of instructional materials and equipment; (iv) In-service training for promotion; (v) Teaching loads; and (vi) Science laboratories for demonstration. In order to strengthen and improve the planning and coordination abilities of Planning and Coordination (APK), staff was trained on computer usage. Strenghtening the establishment of a process concerning budgetary requests and working with provincial directorates and State planning Organization was realized. Finalization of the Education Master Plan covering the schooll years up to 2011 was completed. However, due to the introduction of the Eight-Year Continuous Compulsory Education Law, the APK began reviewing the Education Master Plan so as to reflect the changes brought by the Law. The Management Information System intended for APK was not yet established, therefore, APK continued to collect, analyse and disseminate infornation through traditional channels. * Management Information: The Management Information System (MIS) was designed to decentralize operations to local offices and to provide a network of information flows between schools and local offices and between local offices and MONE's central organization. The MIS pilot began in 1996 with provincial offices in Ankara and Cankiri, and two sub-provincial offices in Polatli and Cankaya. Nationwide expansion of the pilot to all provincial and sub-provincial centers was a large and new effort for the MONE. The system managers had not only to plan and execute a national roll-out, but do so within the context of changes to software functionality due to changes in regulations including new eight-year basic education schools and a strategic shift by Government to utilize iTTnet, the nationwide information technology infrastructure being developed by Turkish Telecom because of the more expensive costs of deploying and sustaining a stand alone network. The pace of change affected by introduction of new information technologies is virtually unparalled in other technical contexts. As a result, the project equipped the provinces and sub-provinces, developed MIS application software (delivered significantly behind schedule, however), provided training to both network and data administrators, but did not deliver the dedicated computer network that would link to the provincial directorates of education, thereby enabling efficiency gains in work loads and flows of information to be demonstrated from the effort. This sub-component was partially successful in achieving its objectives. * Research in Vocational/Technical Education: Although the Project did not directly refer to Vocational/Technical Education Research and Development Center (METARGEM), initially, METARGEM acted as a host to the establishment of Project Management Unit by providing administrative support. The Center 10 benefited from the NEDP through equipment, technical assistance and fellowships abroad. However the Center has no legal status. This prevents it from recruiting qualified staff to carry out its main research on vocational and technical education. 20. The second objective of the Project was largely attained. Progress under the first and third objectives was less clear cut. Although a number of concrete actions were carried out under the project to improve inputs to learning and education management, the educational benefits in many cases were less than expected. The project design had focussed on improved inputs to learning and education management, and assumed that the improved inputs would bring about the expected improvements in educational outcomes. In a number of cases, this assumption proved false. Inertia within the Ministry, rivalry among units of the Ministry, and the difficulty of attracting staff and managers with the requisite qualifications meant that the Project fell short of achieving the full potential benefit of the improved inputs. For example, the Project supported the establishment of a new curriculum development unit which was intended to consolidate the curriculum development functions of several bodies within the Ministry of National Education and the Board of Education. The new unit was created and its staff were trained, but the consolidation of curriculum development functions of other units did not occur. As a result, some aspects of curriculum development have benefitted from the project, while others have not. 21. There were also serious problems of ownership at the start of implementation, both on the part of senior Ministry management and the line units which were expected to implement the various components of the Project. This problem resulted partly from the departure at Project initiation of the Undersecretary under whom the Project was designed, and partly from the fact that many of the key line units were not sufficiently involved in project preparation. Project implementation took two years to get under way, and moved very slowly for the next three years. Project implementation improved thereafter, due largely to the efforts of a new Undersecretary who made a personal commitment to retrofit ownership among some of the key line units involved in the Project. 22. The Additional Components of the Project was substantially successful. * Basic Education Pilot Project (BEPP): In order to assist the Government in promoting its new goal of universal basic education, seminars, orientation courses, follow-up and evaluation meetings were held to improve the knowledge and skills of the central and provincial project implementation teams. With regard to rural education activities; teachers' guides and student activity books were reproduced and distributed to teachers and students. Various educational materials were also distributed to 276 village schools. In-service training for inspectors and teachers were provided. A Schools Funds program was initiated in 30 village schools to enhance school community collaboration, and to increase enrollment and retention (especially girls). In addition, a basic school buildings handbook was developed by under the rural education activities of the Project. Upgrading, furnishing and extensions of 14 rural schools in three provinces were completed. Pre and post evaluation were contracted out to measure the achievements of the rural education activities. Three new model schools were designed but due to the introduction of the Basic Education Program, the construction of the schools was cancelled. The 11 BEPP was substantially successful, and provided relevant information and experience for the implementation of the BEP. * Secondary Education Studies: As basic education expanded, the Government anticipated increased demand for secondary education (Grades 9 - 11). The potential expansion of secondary education raised questions about the affordability and cost effectiveness of education programs at this level. Secondary Education Studies provided assistance to the MONE to help develop a strategy to reform secondary vocational/technical education to make it more affordable and more responsive to the market needs. Accordingly, five research studies were conducted with the help of the technical assistance contractor. Results of those studies were used to prepare a concept paper and a proposal for a new project which was submitted to the Bank for review and comments. The Bank reviewed the concept paper and recently helped obtain a US$500,000 grant in order to help prepare secondary education and. higher education projects. * Mother and Child Development Program: The program was an on-going activity first initiated under the Industrial Schools Project, continued under the Non- formal Vocational Training (NFVT) Project. Upon the completion of the NFVT project, the program was transferred to the NEDP. The objective of this program was to encourage school attendance and reduce early drop-outs among children from poor households. The MONE recognized the need for special efforts to encourage parents to send their children to school and to provide early educational inputs to improve children's performance once they start school. The Program is a 25-week program, targeting mothers of five-year old children, which aims to empower the mother to fullfil her role as a potential educator of her children. The mothers were trained in a wide range of child rearing topics, cognitive development of the child, family planning and reproductive health. The Program is jointly implemented by the MONE and the Mother and Child Education Foundation. Over the past four years of Program implementation 35,000 mothers and children in 59 provinces were provided with training. In addition, 300 teachers participated in refreshment seminars. Nine orientation and evaluation meetings were held with the participation of 574 MONE officials. Twenty-five short films on mother and child education were broadcasted on national and local TV channels. The program was evaluated and it was concluded that the program was successful in achieving its aforementioned objectives. It is planned that the program will be resumed under the second Adjustable program Loan (APL) for Basic Education Project. C. IMPLEMENTATION RECORD AND MAJOR FACTORS AFFECTING THE PROJECT 23. Implementation Record. The NEDP was implemented by MONE and YOK. The two implementating agencies maintained two separate Project Coordination Units (PCU). Both PCU were slow in launching the Project implementation. This was due to the delay in signing TA contracts by both implementing agencies. MONE signed the TA contract two years after the effective date of the Loan; it took YOK four years to sign the TA contract. Lack of communication and cooperation between the two PCU affected project implementation, especially procurement and disbursement. For 12 instance, as there was one project special account, the two PCU had difficulty in managing the payments from the special account. On many occasions there would not be sufficient fiunds to make a payment due to delay in replenishment requests. 24. The two implementing agencies (MONE and YOK) had only the SAR which was written in a short and concise style. For example, the absence of a Project Operational Manual (POM) concerning the Project caused considerable delays and difficulties during project implementation. Especially during procurement of TA services at the initial implementation period of the Project, the lack of TOR caused confusions and delay not only by the implementing agencies per se, but also by the Ministry of Finance (MOF) which was responsible for providing the required "visa" for TA contracts. An important and comprehensive project such as the NEDP, whose aim was to reform the education sector, should have had a much more detailed documents which would have assisted and guided the implementing agencies. 25. As the largest allocation of funds under NEDP was provided for the procurement equipment and materials (US$63.6 million) and procurement of TA services (US$26.6 million), the Project implementation revolved around those activities. As the Project suffered considerable delays in procurement of TA and, to a lesser degree in procurement of equipment and materials, the PCU's of MONE and YOK played a critical role in achieving the goals of the Project. Initially, the MONE PCU suffered from lack of legal status and a high turnover of staff who were seconded to the unit. But later on the MONE PCU managed to establish itself and accumulate invaluable experience with regard to coordinating the project activities. In fact, the PCU became the "nerve center" of the NEDP within the MONE and showed considerable willingness with regard to the ownership of the Project compared with the major implementing agencies within the Ministry. The experience and the will to own the Project led to accumulation of strength at the PCU which made it a decision maker on behalf of the implementing units. Sometimes this had a negative effect on the Project concerning especially procurement of equipment and materials. For instance, the PCU, despite the availability of funds, would not procure the necessary equipment and materilas neede by the implementing unit(s) of the MONE. 26. TA contractor for MONE provided 55.5 month period of technical services. The major activity of the TA contractor was to organize training programs, fellowships and study tours and provide consultancy services as stipulated in the SAR. During the implementation of the training programs, fellowships and study tours, MONE adopted the approach of reducing training duration and increasing the number of trainees. Also, the selection of trainees was not rigorous enough, as MONE should have made available at least three applicants for each training position. The continuity of training is also another issue which need to be addressed by the MONE. TA consultants (national and international) worked in the implementation units at different levels of the MONE. However, the consultants could not work effectively due to lack of qualified counterpart staff and language problem. Although the loan funds were used to provide English language training for selected MONE staff, this was not taken seriously by the recipient of the training. These deficiencies at MONE in making use of the consultants services caused a significant technical and financial loss to the implementing agency. In short, MONE should have made better 13 use of the opportunity of the Project facilities in order to strenghten its project management, coordination and implementation capacity. 27. The YOK's PCU, was staffed mainly by part-time academicians who did not have sufficient time to devote themselves to the Project activities. This led to almost devolving the responsibility of the implementing agency to the TA contractor. Fortunately for YOK, the TA contractor turned out to be an efficient, honest and a professional contractor which made every effort for successful implementation of the Project. However, the arrangement of fellowships by the TA contractor faced with problems of timing, location and language training of suitable fellows. Only 42% of master quota and 854% of the doctoral quota were achieved. The unspent funds from the fellowship budget were redirected to the provision of short-term fellowships and study tours. Difficulties encountered in the implementation of the curriculum work included the lack of counterpart staff to work with the foreign consultants. In short, the YOK component of the Project was completed by the closure of the Project, inspite of four-year delay in implementation at the beginning of the Project. 28. Major Factors Affecting the Project. Major factors affecting the Project were examined under two headings. Factors Subject to Government Conmrol. 29. Counterpart funding. Inadecuate availability of counterpart funding and the delay in releasing the theoratically available funds towards the end of the fiscal year, so that the funds could not be used efficiently in the remaining period, was a major handicap in implementing project activities on time. For instance, every fiscal year, the MOF allocated funds to the MONE in the national budget. Although the budgetary funds were supposed to be used by the MONE, the MOF released the funds in tranches on a quarterly basis. In addition, MOF imposed a limit on the amount of the tranches to be spent. The MONE was not allowed to spent more than 30% in the first quarter. This approach by MOF, clearly created an uncertainity concerning project planning and implementation by MONE. Counterpart funding was not a major problem for the YOK. 30. MOF Visa. The visa requirement of the MOF concerning the procurement of services and goods was another serious cause of delay in implementing the Project. For instance, the MOF, instead of looking into the budgetary consequences of a given contract, often interfered with the content of the contract, especially concerning fellowships and study tours abroad. 31. Audit. Both MONE and YOK kept detailed project accounts. Unlike the case for many other Europe and Central Asia countries, Bank funded projects expenditures in Turkey is audited by Treasury Comptrollers. However, the audits which were carried out by the Undersecretariat of Treasury Comptrollers were not consistent with internationally accepted auditing and accounting standards. This need is being addressed as a country issue, because it affects all projects and will require legislative change. 32. Political and Economic Instability. The design of the NEDP was extremely complex and was difficult to implement. Political instability and weak coalition 14 governments hampered decisive management of the Project. In addition, macroeconomic instability, compounded with austerity measures during the life of the NEDP, slowed project implementation. Turkey suffered from chronic inflation and a foreign currency crisis for the last decade. The unexpected devaluation of the Turkish Lira in 1994 worsened the fiscal deficit, which affected the Project negatively. Shortage of counterpart funds often slowed implementation. Factors Subject to the Implementing Agencies' Control. 33. The Education Organization Law. The Education Organization Law, Act Number 3797, of April 30, 1992, was enacted in order to re-structure the MONE and create the unit (ERDC) which was envisaged in the SAR. However, the law failed to create the ERDC as it was envisaged to be the center-piece of the sector reform foreseen under the NEDP. The Ministry was aware that the ERDC was designed to be one of the key elements of MONE's system for strengthening the quality of education for the Country's children and youth. The ERDC was expected to develop and recommend educational standards and to provide special leadership and guidance for educational programming and development. As such, ERDC needed to be a major source of knowledge, a leader in educational research, curriculum and materials development, and educational evaluation, etc. It was very important, therefore, that the most experienced and highly respected educators, with the best of minds were assigned to lead and work in that important Center. With that in mind, the Project foresaw that the ERDC would be administratively attached to the MONE's Undersecretariat which was responsible for managing the other aspects of general education. In this context, the ERDC's unique role and responsibilities vis-a-vis other units were a pre-requisite to forging a true partnership between the ERDC and those units. Forging a partnership between the BOE and the ERDC was particularly crucial and the Bank as well as the MONE officials were fully aware that the ERDC would never be able to function as it was agreed, unless the BOE was fully committed to make it work. 34. As the MONE did not establish the ERDC, the creation of ERDD, to be administratively affiliated with the Deputy Undersecretary, was simply another additional unit which was authorized to conduct curriculum development, thereby making the responsibility for curriculum development in the MONE even more confused and unwieldly than when the Project began. Moreover, generally either the BOE or the other curricula development units ignored the existence of ERDD and refused to cooperate with it. In addition, the first head of the ERDD whose appointment lasted for seven years was not committed to make the ERDD an operational unit. His appointment was political. This made it difficult for the ERDD to collaborate with other units of the MONE. Especially, the BOE, determinedly kept its reservations about the ERDD's role for developing curricula. For example, the BOE procrastinated in approving the curricula developed by ERDD based on the curriculum development model endorsed by the Minister's administrative circular. Accordingly, the objectives which led to the creation of the ERDC -- namely, to consolidate and improve curriculum development -- were not met due to the lack of clear concensus within the MONE on the process of curriculum development. This resulted in a veiled competition and rivalry between the ERDD and BOE. The ERDD was at a loss in this competition in that the MONE did not provide adequate resources 15 and support to allow the ERDD to develop a permanent and professional institutional capacity. 35. Staffing. The lack of qualified staff during the implementation of the NEDP was a recurrent problem which was not solved by the MONE, despite the MONE's renewed efforts to appoint qualified staff. This affected the Project's institutional development negativelly, and reduced the impact of the TA consultants due to the lack of qualified counterpart staff and the lack of cooperation with other units within the MONE. The staff who were assigned to work with the consultants or those who were selected for fellowship training suffered from insufficient knowledge of the English language. This prevented the staff concerned from communicating and working with the consultants effectivelly, and prevented them from getting the benefit of fellowship training. The staff turnover at senior and technical level was another impediment which affected the Project implementation. During the nine-year implementation period of the NEDP eight Ministers, four Undersecretaries, three Deputy Undersecretaries and three PCU Directors changed, in addition to numerous changes in the administrative and technical personnel of the implementing units. D. BANK PERFORMANCE 36. Preparation. It is not clear, in the light of subsequent experience that the Bank team which prepared and appraised the Project fully appreciated the implications of the institutional environment and political pressure within which the NEDP would be implemented. Now, ex post, it appears that the team did not adequately take into consideration the large number of activities required to realize the Project objectives, the complexity of involving fourteen different departments. The lack of baseline data for significantly new initiatives limited the extent to which monitorable performance indicators could be used to monitor project progress and evaluate achievement of the developed objectives. The appraisal team lacked expertise in management information systems resulting in a less than realistic assessment of the institutional environment within which the MIS would be implemented. 37. An important and comprehensive project such as the NEDP, should have had a much more detailed documents which would assist and guide the implementing agencies. Whereas, the two implementing agencies (MONE and YOK) had to contend themselves with the SAR only, which was written in a short and concise style. For instance, the absence of a POM concerning the Project caused considerable delays and difficulties during project implementation. 38. Supervision. During the implementation of the NEDP the Bank paid a special attention to monitoring the implementation progress, procurement and disbursement activities. In this regard 25 supervision missions including the completion mission were arranged between June 18, 1990 and October 14, 1999. The staff turnover was high concerning the task management, this created deficiencies in detecting administrative and technical problems timely which were directly involved in implementation of the Project. For instance, with regard to the establishment of the ERDC, administratively belonging to the Undersecretariat of MONE, the Bank failed to realize that the The Education Organization Law, Act Number 3797, dated April 30, 1992, did not allow the establishment of the Center as it was agreed during loan 16 negotiation and as stipulated in the SAR. It took the Bank almost a year to notice that the MON'E had not lived up to its commitment. However, the Bank after a year delay did remind the Minister by writing a letter and explaining to him that the Ministry's committment for making the ERDD function as agreed between the Bank and the Government of Turkey. As judged from retrospect, Bank supervision in the early stages of the Project was coupled with supervision of other on-going projects resulting in a lack of focus on the Project and not enough attention to establishing milestones with the implementing agencies and to subsequently track these in relation to Project progress. E. BORROWER PERFORMANCE 39. Preparation. The Borrower took the initiative in Project preparation and deserve credit for this, as well as for addition of three new components included at a later stage as logical follow-on from other related projects (Mother and Child Development Program, Secondary Education Studies), or as pilots for new planned initiatives, such as the Basic Education Pilot to lead to the Basic Education Program. 40. Implementation. The Borrower's implementation performance was adversely affected by two factors. The first related to high turnover of staff, which manifested itself in various ways in all parts of the Project, slowing implementation and delaying procurements. For instance, although the senior officials of the MONE reconfirmed their commitments in making the ERDD function, the MONE failed to live up to those commitments because of changes in the high level management. Also, the high staff turnover in medium administrative and technical staff was particularly detrimental to the Project's implementation. For example, the head of Computer Experimental Schools component left to return for full-time position at the university, and her successor headed the CES for only a short period and then retired. Second, there were delays in releasing counterpart funds for the construction of four ERC, expansion and refurbishment of CLS. Overall, borrower performance should be rated as satisfactory, but this should be qualified for each component. F. ASSESSMENT OF OUTCOME 41. The overall rating of the project is partially satisfactory vis-a-vis implementation performance. However, judged by individual components, the performance has been uneven. Conversely, the inclusion of three components to the Project late in its life, which were intended to lay the ground-work for a related future projects, can be regarded as a further interest to implement new projects jointly with the Bank. G. PROJECT SUSTAINABILITY 42. MONE Component. There is evidence that the MONE has assumed ownership of the CLSs, however, allocation of adequate funds in the national budget is important for maintaining the current standards of the 208 CLS. It is also important that sufficient funds be allocated to provide the opportunity for non-CLS to reach the 17 standards achieved by the CLS under the NIEDP. However, during preparation and appraisal, the question of cornstraints, such as programming, phasing, and prioritization of the new initiatives like CLSs and CESs do not seem to be comprehensively addressed. Also, not withstanding the cost of expanding the CLSs to non-CLSs including rural schools, the problem of attracting the qualified staff to run the system has been a persistent challenge at the MONE. In line with this, the sustainability of CLS would be liZely. For the part of the Project that attempted to provide computer literacy training to teachers, sustainability is likely, since MONE has incorporated formal courses for teachers into its system and has supported operating costs. For CES, sustainability is likely because of the enactment of Basic Education Law No. 4306, which stipulates that the Government's objective is "that all basic education age students have access to computers in the learning process" in order to attain computer literacy, support and enhance the existing curricula, and open the computer laboratories to the community as technology-intensive learning environments. For testing and evaluation, the MONE has shown that it is capable of conducting tests and questionnaires to assess student achievements and has conducted TIMS-R for the first time;hence the sustainability of SAATs is likely provided that the Government allocate adequate funds for the continuation of this important work. As the textbook technical quality cont-ol unit was not established as originally envisaged in the SAR, and because the software for distribution and inventory control of textbooks could not become fully operational, the sustainability of improving the quality and distribution of textbooks is unlikely. With regard to in-service training, MONE has shown its ability to provide the training by exceeding the targeted number of teachers who are trained at central and local levels. Therefore, the sustainability is likely provided that enough resources are allocated to the department. The ERC were unsuccessful in achieving the goals and objectives as defined in the Project, mainly due to the lack of commitment by MONE, and hence the sustainability of the activity is unlikely. With regard to management assessment and development, the MONE has assumed responsibility for improving management skills and practices and accordingly, conducted assessment studies for each MONE units, enacted new rules and regulations for appointment, evaluation, promotion of MONE administrators and developed norm cadres for schools, trained MONE staff in related subjects. However, despite these achievements and MONE's commitment, and the likely sustainability of the unit, the Bank has concerns about the lack of legal status of MADU. There is evidence that MONE has assumect ownership of the ILSIS and shows readiness to ensure continued maintenance and operation. The MONE has established a help-desk facility. The Project's technical consultants and software developer have trained counterpart staff. ILSIS application software has a three year warranty. But as the technical support levels are not comnpetitive with other career opportunities in the IT industry, the technical support staff turnover will continue. If compensation cannot be changed, MIS sustainability remains uncertain. The execution of the research and planning actvities of the MONE are contingent upon the appointment of qualified staff and appropriate infrastructure. Therefore, without having the qualified staff, i.e. education economists, education planners, and fully functioning MIS, the sustainability of research and planning at the MONE is unlikely. With regard to the sustainability of the components added at a later stage of the NEDP; the signing of the loan agreement of Basic Education Project provided the MONE with the opportunity to make a smooth transition in the implementation of the new project; the MONE, based on the outcome of the Secondary Education Studies, would like to launch a new project in the field of secondary vocational and technical education, and based on the 18 succesful implementation of the Mother and Child Development Program, is considering extension of the Program activities under Basic Education Project (BEP). 43. Sustainability of Project interventions in the Ministry of National Education suffers from a problem which is present, to some extent, in all ministries: the difficulty of attracting qualified staff with civil-service salaries. This was a particular problem in areas - such as film, radio, and television production - where there are better professional opportunities outside the Ministry of Education. While this problem could be addressed through more liberal and more imaginative use of consultants, the Ministry is reluctant to do so - largely because of the resentment which results from the high salaries paid to consultants in some specializations. The large institutional consultant contracts for educational support to the MONE may also not have left behind the permanent benefits which were foreseen because of ownership problems. Throughout implementation of the Project, the Ministry felt that the consultants were forced upon it. The MONE did not make full use of the consultants, and in many cases did not provide counterpart staff who could internalize and sustain the potential benefits of the consultants. 44. YOK Component. The Project has significantly increased the capacity of the Faculties of Education to provide pre-service teacher education through the improvement in training facilities and equipment. Institutional strengthening of faculties through fellowships, course content and training has made the sustainability of quality of pre-service teacher education likely. In addition, YOK has restructured the Faculties of Education to cater to the new requirements of the pre-service teacher training, and also established the national committee for teacher education. This is an evidence that the implementation of the goals and objectives of the Project have led YOK to a stronger commitment to continue the Project activities. H. FUTURE OPERATION 45. The MONE intends to sustain the achievements of the Project through an improved basic education quality program, which will support in-service training for teachers, inspectors and school principles, provide textbooks and educational materials, and implement an IT program that will phase in different instructional objectives starting with computer literacy and moving toward more skill-based and subject matter related training. Classrom teachers and students in grades 4 - 8 will for the first time in their school have access to computers for their use, administrative and teaching tasks, and will begin their own exploration on how best to use computers in learning and teaching. A number of other initiatives designed to improve the quality of basic education, such as renovation and extension of school buildings around the Country, are also planned. The MONE also intends to initiate secondary general and vocational education project as a complementary project to the Basic Education Program. YOK is also in the process of developing a new higher education reform project. He bank has already provided a grant money to MONE and YOK in order to assist the two agencies develop project proposals for the new initiatives. 19 I. KEY LESSONS LEARNED 46. Project Design. The main lesson learned through implementation of the Project is that project design needs to be based on a realistic assessment of constraints, and a recognition of wvhich constraints can be mitigated by project design and which cannot be. The design of the project assumed that the improved inputs would bring about the expected irnprovements in educational outcomes, and assumed that the Ministry would continue to be committed to the Project even after the departure of the Undersecretary who conceived the Project. The Project had been designed principally by this Undersecretary and a group of like-minded collaborators - including the Bank appraisal team and a number of hand-picked consultants. This group was excited by the Project design and committed to making it work. But there was limited involvement in Project design among the line units which were expected to implement the Project, and, consequently, little of this commitment or excitement. There was also no prior agreement on the part of those units which were expected to relinquish some of their responsibilities under the Project, and in the event failed to do so. In retrospect, it would have been better to recognize the structural constraints at the start, and to design interventions into the Project to work around them, rather than to assume them away. This would have required the full involvement of the line units from the start of project design. Doing so would require more time for project preparation and would perhaps require more modest project objectives and a narrowing of project scope, but would produce better implementation results. 47. Legal and Institutional Environment. A project such as the NEDP, which aimed to reform and restructure basic and secondary education system, should have been launched after the legal base was obtained to implement the Project. If this was not possible the Bank should have taken serious measures in order to rectify the defect concemed in line with the agreed Project goals and objectives. With the NEDP, this was not the case as the law which was enacted at the beginning of the Project, contrary to the agreed objectives created another unit in the MONE, instead of a unified curriculum development center, which worsened the institutional environment at the MONE. Future project preparation teams should be aware of the institutional, as well as legal constraints of the implementing agencies prior to launching a project. For instance, the Basic Education Project had the advantage of being operational within a legal framework compatible with the Project's goal and objectives. 48. Human Factor. Leadershlip is important in launching new programs. Visionary and effective leadership is a necessary, but not sufficient condition for succesful project implementation. As change permiates all levels of an organization, the assignment of qualified staff become involved in implementation of the Project on a daily basis is a very important factor. The NEDP suffered a chronic shortage of qualified full-time staff to work at the Project. This affected the institutional strengthening of the implementirng agencies, especially in the case of MONE the utilization of international and national consultants, as well as fellowships and study tours. 49. Project Coordination. In a project which had multi-implementing agencies/units such as the NEDP, the PCUs played a key role in administering and 20 coordinating the Project. Keeping the PCUs adequatelly staffed so that they could work efficiently with both consultants and the implementing units become a critical success factor of the Project. In addition, ownership of the Project by the implementing units was also crucially important and helped make collaboration between the PCU, the consultants and implementing units to be beneficial. 21 Part II. Statistical Tables Table 1: Summary of Assessments A. Achievement of Objectives Substantial Partial Nelgible Not alicable Macro Policies [] a ol Sector Policies [ El E Financial Objectives El 0 0l Institutional Development 17 I a Physical Objectives El [E3l El Poverty Reduction [ l E [ Gender Issues [7 El E ox Other Social Objectives n E E I] Environmental Objectives E E E [E3 Public Sector Management El E Private Sector Development El E E ixi Other (specify) [7 E E E (Continued) 22 B. Project Sustainability Likely Unlikely Uncertain (V) (1) (vi) C C~~~~~~~~~~~LL Highly C. Bank Performance satisfactory Satisfactory Deficient (1) ~~~~~~(1) (/) Identification 0 3 i Preparation Assistance I IX 0 Appraisal l E l Supervision i EJ Highlv D. Borrower Performance satisfactory Satisfactory Deficient (V) (V () Preparation [2 [ Implementation [E l Covenant Compliance [ m2 Operation (if applicable) J i:i Highly Highly E. Assessment of Outcome Satisfactory Satisfactorv Unsatisfactory unsatisfactory (/) (V) (V) (V) 23 Table 2: Related Bank Loans/Credits Loan Credit Title Purpose Year of Approval Status First Education To improve the quality 1971 Closed (Ln. 748-TU) of educational programs and to assist: in meeting well-defined manpower needs in growing industrial areas. Industrial Schools To improve the quality 1985 Closed (Ln. 2536-TU) of programs, to increase the quantity of trained industrial manpower and to improve ihe planning and management capacity of t:he Directorate of Technical and Vocational Education in the Ministry of national Education. Non-formal To increase and improve 1987 Closed Vocational the supply of well- Training trained labo:- for (Ln. 2776-TU) manufacturing industries, and to improve employment and income generating opportunities for disadvantaged groups with little education and training. Industrial Training To support the 1988 Closed II (Ln. 2922-TU) Governmeni's efforts to improve the quality of trained technicians for the industrial sector, to meet urgent needs for well-trained technical and vocational teachers in secondary, and non- formal vocational/technical training. 24 Employment and To improve allocative Active Training efficiency through the (Ln. 3541-TU) diversification of employment services; to improve the absorption of unemployed, unskilled workers in productive employment; to improve the efficiency of labor market decisions by providing better information on employment prospects; and to promote women's productive employment. Basic Education I Achieve universal 1998 Active coverage in an expanded, 8-year basic education cycle (formerly 5-year); to improve the quality and relevannce of basic education; and make the basic education schools a learning resource for the community. Table 3: Project Timetable Steps in Project Cycle Date Planned Date Actual Identification May, 1988 Preparation February, 1989 Appraisal November, 1989 Negotiations Letter of Development Policy (if applicable) Board presentation > April 26, 1990 Sianing May 18, 1990 Effectiveness July 31, 1990 First tranche release (if applicable) Midterm review (if applicable) Second and third tranche release (if applicable) Project completion December 31, 1996 June 30, 1999 Loan closing June 30, 1997 October 31, 1999 25 Table 4: Loan/Credit Disbursements: Estimated and Actual (US$ millions) FY91 FY92 FY93 FY94 FY95 FY96 FY97 FY98 FY99 FY00 Appraisal 4.6 24.8 28.0 21.6 9.4 1.2 0.7 Estimate _____ Actual 0.1 1.4 1.8 2.6 8.2 14.3 7.4 19.8 12.5 1.0 Actual as % 2.2 5.6 6.4 12.0 87.2 1191.6 1057.1 of Estimate I I _I I I Date of final disbursement: 1 November 1999 (for Loan 3192S-TU) and 20 April 2000 for (for Loan 3192A-TU) Table 5: Key Indicators for project Implementation Indicators for project implementatiorn not included in the SAR. Table 6: Key Indicators for Project Operation Indicators for project operation not included in the SAR. 26 Table 7: Studies Included in Project Study Purpose as defined at Status Impact of Study appraisal/redefined 1. Teacher Development of a teacher Completed Measurement of assessment testing program to improvements in establish base-line data teacher quality against which to measure againts the base- improvements in teacher line data has not quality over the next been conducted yet. several years as a result of Therefore the effect the proposed project of the project activities. activities on teacher quality has not been determined. 2. School Implement a school Completed Based on this mapping mapping study to study, MADU determine projected prepared a draft student growth patterns regulation on and trends in population "ducation Districts shifts and resulting unique and Education education needs of the Councils". The respective regions and draft regulation provinces. was submitted to the approval of MONE administration. 3. In-service Feasibility of including in- Completed Benefitting from training for service teacher training as the study, the promotion a factor for consideration regulation on the in the promotion of "Appointment, educational personnel. Evaluation, Promotion and Transfer of Ministrty of National Education Administrator" was prepared. The regulation was published in the Official Gazette as the decision of the Council of Ministers. 27 4. Teaching Analysis of actual teaching Completed Based on the loads loads throughout the outcome of the system and implications study, MONE for increased efficiency provided incentives and cost savings. to teachers who were assigned extra workload. Workloads could not be reduced due to insufficient number of teachers. 5. Science An assessment of the Completed Based on the laboratories for extent to which science recommendations demonstration laboratories are used and of the study, the advisability of a science laboratories equipping schools with of 208 Curriculum science equipmenl and Laboratory Schools materials appropriate for in 23 provinces teacher demonstrations were equipped with instead of small groups or science laboratory individual laboratory equipment and work. guidebook for science laboratories was published by ERDD and distributed to 208 schools. 6. Educational This study was not Completed The conclusion of studies program envisaged in the SAR, the program was for Turkey however, the purpose of used in research the study was to provide a studies planned and program to the MC)NE conducted by with regard to the ERDD, APK and educational studies. METARGEM. 28 Table 8: Project Costs Implementing Agency Appraisal Estimate Actual (US$M) (US$M) Local Foreign Total Local Foreign Total MONE ConstructionlRenovation 41.5 41.5 27.7 27.7 Classroom Furniture 8.1 - 8.1 8.1 - 8.1 Instructional Equipment 4.2 37.6 41.8 19.4 33.0 52.4 Instructional Materials 3.9 3.9 7.8 0.5 2.5 3.0 Technical Assistance Consultant Services - 4.1 4.1 0.5 8.0 8.5 Fellowships 0.3 3.2 3.5 - 5.1 5.1 Local Training 0.4 - 0.4 2.2 - 2.2 Development Support - 2.3 2.3 5.0 - 5.0 MONE Base Cost 60.7 48.8 109.5 63.4 48.6 112.0 Physical Contingencies 6.1 4.9* 11.0 - - - Price Contingencies 12.0 9.4* 21.4 - - MONE Sub-total 78.8 63.1 141.9 63.4 48.6 112.0 YOK ____ ____ Construction/Renovation 1.6 - 1.6 10.2 - 10.2 Classroom Furniture 0.9 - 0.9 2.9 - 2.9 Instructional Equipment 0.8 7.2 8.0 0.4 5.7 6.1 Instructional Materials 0.1 0.4 0.5 - 0.2 0.2 Technical Assistance Consultant Services 0.1 2.5 2.6 - 4.6 4.6 Fellowships 2.4 10.9 13.3 1.1 9.1 10.2 Local Training 0.4 - 0.4 0.1 - 0.1 YOK Base Cost 6.3 21.0 27.3 14.7 19.6 34.3 Physical Contingencies 0.6 2.1 2.7 - - Price Contingencies 1.3 4.0 5.3 1.9 0.9 2.8 YOK Sub-total 8.2 27.1 35.3 16.6 20.5 37.1 Total: Baseline Costs 67.0 69.8 136.8 78.1 68.2 146.3 Physical Contingencies 6.7 7.0 13.7 - - Price Contingencies 13.3 13.4 26.7 1.9 0.9 2.8 TOTAL: PROJECT COSTS 87.0 90.2 177.2 80.0 69.1 149.1 * Allocated for TA and Basic Education Pilot Project (BEPP) 29 Table 9: Project Financing Appraisal estimate (US$M) Actual (US$M) Source Local Foreign Total Local Foreign Total Costs Costs Costs Costs IBRD - 90.2 90.2 - 69.1 69.1 Government 87.0 - 87.2 80.0 - 80.0 Total 87.0 90.2 177.2 80.0 69.1 149.1 Table 10: Financial and Econonmic Costs and Benefits No formal financial or economic cost benefit analysis was undertaken for this Project. Table 11: Status of Legal Covenants Agreement Section Covenant Present Original Revised Description Comments type status Fulfillment Fulfillment of covenant Date Date Loan Schedule Impleme Compil 09/01/90 Primary None Agreement 5, para. ntation ed with Teacher B.2 (a) after Education delay Steering Committee to be established by YOK. Loan Schedule Impleme Compil 09/01/90 Secondary None Agreement 5, para. ntation ed with Teacher B.2 (b) after Education delay Steering Committee to be established _ _ _ _ __ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ by Y O K . 30 Loan Schedule Impleme Compil 09/01/90 Board of Agreement 5, para. ntation ed with Education C.2 and after to undertake para. delay review of A.1 (a) arrangement s for planning, developing & updating curricula, textbooks & instructional materials. Loan Schedule Impleme Compil 09/01/95 MONE None Agreement 5, para. ntation ed with shall C.2 and complete an para. assessment A.3 of facilities required for project's curriculum laboratory schools & project's computer education schools. Loan Schedule Impleme Not 06/10/96 MONE to Agreement 5, para. ntation compile ensure C.3 and d with employment Schedule of not less 2, para. than 3 A.3 qualified specialists in tests & measureme nts to assist in ERDC's testing and evaluation center. 31 Loan Schedule Monitori Compil 12/31/90 PCUs of None Agreement 5, para. ng, ed with YOK and C.4 review MONE to and prepare and reporting furnish to the Bank a plan for annual assessment of progress in project implementat _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ io n . _ _ _ _ _ Table 12: Status of Compliance with Operational Manual Statements Project compliant with applicable Bank Operational Statements (OD or OP/BP) Table 13: Bank Resources: Staff Inputs (actuals) Stage of project cycle Weeks US$(000) Preparation to appraisal Not available Not available Appraisal Not available Not available Negotiations through Board Not available Not available Approval Supervision 46.6 183.1 Completion 5.2 7.0 Total 51.8 190.1 32 Table 14: Bank Resources: Missions Stage of Month/ No. of Days Specialized Performance Rating Types of Project Cycle Year Persons in staff skills Problems Field represented Status Developms Through Appraisal Appraisal Trough Board Approval Supervision 1 06/1990 1 SES Lack of qualified staff at .________PCU. 2 12/1990 1 SES Behind schedule, slow implement ation. 3 09/1991 2 SES, PROC Personel changes in the PCU. Slow TA selection process. 4 02/1992 1 3.8 SES Behind schedule slow implement ation. 5 09/1992 2 2.0 SES, PROC Considera ble delay (2 years) in project implement ______ ~~~~~~~~~~~~ation. 6 01/1993 2 5.5 SES, 00 7 07/1993 3 SES, 00, Lack of SA cooperatio n between the implement ing units of the MONE. 33 8 10/1993 1 00 ERDD implement ation _________ difficulties 9 05/1994 3 TM, SA, Implement MS ation _______ problems 10 11/1994 2 TM, MS Lack of achieveme nt of developme nt _ ~~~~~~~~~~~~~objectives. 11 02/1995 1 PO Long delays in disbursem ent and implement ation. 12 05/1995 2 6.3 TM, PO The institution al changes the Project was to designed to facilitate could not be achieved and loan funds could not be used for the purposes originally intended. 13 12/1995 5 4.6 DC, PPM, Initiation TM, PO, of the 00 Loan Amendme nt to add three new componen _________________ ts. 14 05/1996 1 _ TM 15 07/1996 2 TM, 00 34 16 12/1996 1 TM, Procureme nt delay of FRTVC equipment 17 05/1997 3 TM, PEE, Delay in SITS curriculum developme nt 18 11/1997 2 TM, SITS Slow disbursem ent 19 05/1998 2 TM, SITS Slow disbursem _______ ___________ ent. 20 12/1998 5 TM, PEE, Delay in PROC, PO, procureme SITS nt of FRTVC equipment 21 05/1999 3 TM, PEE, Seven SITS packages of FRTVC equipment procureme nt cancelled. Completion . 09/1999 3 TM, PEE, SITS Specialists: SES : Senior Education Specialist 00 Operations Officer TM Task Manager MS : Management Specialist PO : Project Officer DC : Division Chief PPM : Project Portfolio Manager PEE : Principal Education Economist SITS : Senior Information Technology Specialist SA : Systems Analyst PROC : Procurement Officer 35 Appendix: A AIDE - MEMOIRE REPUBLIC OF TURKEY NATIONAL EDUCATION DEVELOPMENT PROJECT (LN. 3192-TU) World Bank Intplementation Completion Mission 13 September - 14 October 1999 Final INTRODUCTION 1. Based on the Terms of Reference dated 10 September 1999, the World Bank conducted an Implementation Completion mission of the above Project between 13 September - 14 October 1999. The mission comprised of Ms. Ferda Sahmali (Mission Leader and Human Development Specialist), Mr. Michael Mertaugh (Education Program Team Leader) and Mr. Robert Schware (Senior Information Technology Specialist). The team met with officials from the Ministry of National Education (MONE), the Council for Higher Education (YOK) and che Treasury to discuss the achievements, outstanding problems and likely sustainability of the NEDP, and to provide inputs to the lmplementation Completion Report (ICR). The mission also made field trips to Istanbul, Canakkale, Siirt, Aksaray, Adana, Mersin and Konya to visit Curriculum Laboratory Schools, Computer Experimental and Laboratory Schools, Education Resource Centers, MONE Printing House and Faculties of Education. The members of the mission wish to express their sincere thanks to their counterparts for their assistance to the work of the mission, and to the staff of the schools and institutions visited for their kind reception and efforts made to contribute to the mission's work. 2. Since this aide-memoire is a step in the preparation of the project ICR, it does not follow the usual format previously used in supervision mission aide-memoires. The ICR will follow the standard format, but this aide-memoire concentrates on the achievements, outstanding problems, and likely sustainability of the Project's goals and objectives implemented by different units of NIONE and YOK. IMPROVING QUALITY OF EDUCATION Curriculum Management and Policy Development: 3. Achievements: The SAR oF the NEDP states that "All curriculum development units/groups dispersed throughout the MONE would be consolidated into a unified Education Research and Development Center (ERDC). The Center would have the responsibility for all development activities in the education sector, including curricula, textbooks, materials, educational research, testing and evaluation etc. In addition, the ERDC would be the operational research and development arm of the MONE for curriculum and instructional materials development, educational research, and evaluation. Whereas the BOE would continue its leadership role of advising the Minister on approval of curriculum and related development outputs". The Education Organization Law, Act Number 3797, of April 30, 1992, failed to create the ERDC as a Center which would assume the responsibilities mentioned above and would operate directly through the leadership of the Undersecretary of the MONE. The ERDC was created as a department which could not unify the curricula activity which is still dispersed throughout the MONE. Accordingly, the BOE still continue to develop and approve curricula, textbooks and educational materials, whereas the intend of the NEDP was to elevate the BOE's image and stature within the Ministry and the Country as a whole by enabling the Board to be a high level executive body which it should be a true policy, decision making, and approval agency for the Ministry, without having the lower level charge/burden of managing an operation whose own work it would have to approve. 4. Problems: (i) The legislation instead of creating a Center unifying all development activities in the education sector, and directly functioning under the leadership of the Undersecretary, the legislation, added another unit developing curricula to those already in existence. This led to a confusion and increased the lack of cooperation between those units that are engaged in development activities. (ii) The legislation left BOE intact, and accordingly, BOE continues to develop and approve curricula, textbooks and other related educational materials. 5. Sustainability: As ERDC could not be established the "center piece" unit foreseen in the Project, and the BOE is functioning in the same old way, comments could not be made about the sustainability of the curriculum management and policy development. Curriculum Development: 6. Achievements: The Education Research and Development Department (ERDD) has developed a "Curriculum Development Model" under the Project to be used for developing curriculum. This model has enabled the ERDD to increase the number of curriculum to be developed by the unit. The Board of Education (BOE) recognized this model and recommended it for curriculum development. ERDD, by using this model has developed 21 curricula since the beginning of the project. Out of the 21 curricula, 7 curricula belong to basic education schools programs (Grades 1-8), and the remaining 15 curricula belong to secondary school programs (Grades 9-1 1). Primary Life Sciences (Grades 1,2,3), Biology (Grades 9,10,11) and Mathematics (Grades 1-8) curricula have been approved by the Board of Education (BOE) and are being implemented in schools. 7. Problems: (i) Although initially ERDD was slow in developing curriculum for various administrative reasons, later on the ERDD speeded up the development of curricula and therefore increased the number of curricula to 21 to be presented to the BOE, but the ERDD did not receive the same response and enthusiasm by the BOE which has continuously impeded the work of the ERDD. The BOE has always been reluctant to cooperate with ERDD and declined to send its expert personnel to work with ERDD in the process of developing a curriculum. In fact, BOE all along, since the beginning of the Project perceived ERDD as its rival and did all it can to slow down the work of ERDD; for instance, there are still 18 curricula developed by ERDD awaiting the approval of the BOE. (ii) Lack of adequate space, poorly furnished offices and lack of qualified experts to assist ERDD in curriculum develepment commissions, have impeded the efficiency of the ERDD. 8. Sustainability: As rivalry and lack of cooperation continue to persist between BOE, ERDD and other curriculum developing units of the MONE (e.g. Directorate General for Basic Education, Directorate General for Secondary Education, etc.) sustainability of curriculum development activity will remain unstable. Curriculum Laboratory Schools (CLSs): 9. Achievements: In 23 provinces, 208 Curriculum Laboratory Schools (CLSs) have been selected covering the seven regions of Turkey. The selected CLSs include 147 Basic Education Schools, 37 General Secondary Schools, 16 Anatolian Secondary Schools and 8 Anatolian Teacher Training Schools. Education Research and Development Department (ERDD) which was established under the NEDP, developed the "Model of the Curriculum Laboratory Schools". Based on the Model, the ERDD developed a "CLSs Standards" concerning equipment, furnishing, facilities and building standards. In line with the CLSs Standards, 208 schools have been provided with educational equipment using the Loan funds. The educational equipment include computers (PCs and lap-tops), printers, data show machine, fax machines, modems, scanners, TV sets, video recorders and video cameras, overhead projectors and screens, tape recorders, projectors, cameras, flip charts, and physics, chemistry and biology and life sciences laboratory sets. School furniture (individual student desks, students lockers, laboratory tables, library shelves etc:.) of the CLSs have been funded under the local budget. Local funds have also been used to renovate and maintain CLSs so as to meet the building standards concerned. In this context, music and drawing rooms, computer and science laboratories, libraries, teacher study rooms and teacher common rooms have been established. ERDD has also developed human resources standards and provided training for administrators, teachers, counselors and inspectors based on the standards concerned. Educational Technology Formators have been trained and appointed to every province. The formators provided training to the teachers and administrators concerning the use of equipment at CLSs. As part of the implementation of the CLSs a "School Development Model" has been initiated so as to piovide flexibility to CLSs in adapting themselves to new developments in the education sector. Within the framework of this model, annual school development plans have been prepared and implemented. This model has resulted in a close cooperation between the school administration, teachers, parent and students. 10. Problems: (i) The popularity of the CLSs has increased the demand to these schools. Parents who want to register their children to the CLSs at the beginning of every school year caused an increase in the number of students planned in each classroom from 30 students to 40 per classroom. The CLSs are under immense pressure even to increase the intake to each classroom above 40. (ii) The procurement of photocopy machines could not be realized due to a disagreement between the supplier firm and the MONE. The matter is being dealt with at the court. (iii) The high turnover of CLSs administrators and teachers is also affecting the performance of these schools negatively. (iv) Although CLSs have been selected in provinces where Faculties of Education existed, cooperation between the two educational establishments has not been realized as envisaged in the NEDP. Faculties of Education used CLSs as places for internship for their students, but failed to provide support for the development of CLSs. 11. Sustainability: There is evidence that the MONE has assumed ownership of CLSs. In this regard, MONE has approved a regulation for the dissemination of the CLS Model to be implemented in other non-CLS MONE schools so as to increase the standards of non-CLSs through out the Country. This is a correct step in the right direction. However, allocation of sufficient funds is crucially important to maintain the current standards of CLSs and also to enable non-CLSs to reach the standards achieved by the CLSs under the NEDP. Computer Experimental Schools (CESs): 12. Achievements: At the time of project preparation it was envisaged that there would be a pilot project designed to fully equip 53 selected secondary schools with Computer Laboratories. The schools were equipped with a computer laboratory which contained 20 monitors, with one server, a printer, a modem, office suite software and educational games. Teacher rooms were also equipped with a PC each. Assistance also included instructor training, establishing criteria for instructional software selection, and the development/adaptation of instructional software. A committed and enthusiastic director of the CES Department advanced the pilot activities. An additional 182 schools were included in an unplanned expansion of the pilot. Of the total 235 computer laboratories, 118 are in general high schools, 72 in basic education schools, 32 in Anatolian high schools, 13 in Anatolian Teacher Training high schools, and 1 is in a Science high school. Nearly half the number of schools are also participating in the Curriculum Laboratory Schools (CLS) component of the project, which provides them with additional audio-visual equipment. Thus the component achieved a quantum leap from what was envisaged at the beginning of the project in terms of building computer laboratories. The project has also successfully provided the appropriate staff training to laboratory coordinators ("Formators"), administrators and teachers. The Ministry creatively adopted a cascade model whereby teachers from the 53 pilot school project who passed an examination would become Formators, and would in turn receive specialized training and then train both the students and teachers in their respective schools. School directors and other administrators, as well as parents, were inforrned about the objectives of the program. To a growing extent, parents want their children enrolled in a well-equipped school, and teachers see opportunities for students and themselves in being introduced to IT. The size of classes, the interests of students and parents, and the need to provide greater access to the laboratories have resulted in additional weekend classes, mostly in computer literacy. In most cases, the facilities are well-used. An evaluation instrument to assess attitudes of students, teachers, administrators, and parents toward the introduction and use of computer laboratories in schools was administered. Teachers, principals, and parents in general considered the use of computer laboratories improved the quality and choice of educational options. PCU staff have become very knowledgeable and proficient at handling equipment procurement to the Bank's satisfaction, while at the same time meeting requirements of the Turkish administrative procedures with respect to Ministry of Finance and customs clearances, and VAT payments. Finally, the project has greatly increased the capacity of the Ministry to articulate a vision of how computers should be used in schools, which has led to an IT component of the Basic Education Program and the EU's Basic Education Supporting Project. 13. Problems: (i) A difficulty which manifested itself throughout the project was the non-availability in the market of Turkish software, and also software closely related to the curricula that could complement and reinforce the teaching provided in the schools. Due to a lack of Turkish software, an estimated 10% of the original 53 schools were not making effective use of the computer laboratories. A specific parallel problem was the lengthy approval process for software once it was decided which software to procure--the Board of Education took over a year to approve the software. (ii) The assessment instrument did not evaluate how the program affects what and how students learn, and whether the software has been useful in complementing classroom instruction. (iii) Teachers have not been trained in use of subject-specific instructional software, and how to integrate these tools into daily teaching practices. (iv) Training has been of uneven quality, with some very good training provided by the contracted universities, and others unsatisfactory. (v) Availability of teachers and/or formators t keep the laboratories open and operational was less than fully e ffective, some schools without ministerial-appointed teachers have had to hire computer trainers from the private sector. (vi) There is so far little evidence that the PCs procured for principles are used for anything but object of prestige. 14. Sustainability: (i) The ability to upgrade equipment in existing schools in response to technological progress will be a factor determining whether the schools maintain their capacity to deliver IT-supported learning. (ii) Similarly, refresher training courses will have to be provided to Formators., and pre- and post tests should be undertaken to evaluate the quality of this training. Up until now, the Ministry has used universities throughout the country to supply the training during the summer. To a growing extent, future training should be undertaken with participation from qualified private sector training firms in the IT sector, which generally have more experience in IT training than universities, have the equipment at their disposal, and may be able to provide better pupil- computer ratios. (iii) The role of teachers is also very important, and a considerably larger number must be trained to use and operate laboratories. (iv) Pilot projects should be planned that will lead to implementation of online learning, integration of IT in the teaching role, and extension of the Formator program to teachers and students, all of which will support further integration of IT into schools. (v) Careful selection in the future of schools to be provided laboratories is required, since widespread diffusion of the technology among those with financial resources who can afford it may diminish use of the laboratories. (vi) The availability of formators or teachers to operate the laboratories must be guaranteed before the school year, and action taken to obtain the services of private instructors if they are unavailable. (vii) Due to language difficulties and the inappropriateness of most overseas software to the existing curricula, the increasing availability of Turkish instructional software, quality reference and educational game software, will be a factor in determining whether future initiatives to computerize schools can continue to operate viably. Testing and Evaluation: 15. Achievements: The ERDD has initiated Student Achievement Assessment Tests (SAAT) as envisaged under NEDP on a national basis. SAATs have been regularly conducted in Science (at Grades 5,8,9), Mathematics (at Grades 5,8), Social Sciences (at Grades 5,8,11) and Turkish (at Grades 5,8). SAATs have been carried out on 347,222 CLSs students and 38,275 non-CLSs students totaling 385,497 students. By conducting SAATs, ERDD has achieved a capability for researching, producing, validating and standardizing tests for establishing national norms on student cognitive achievement, for comparative analysis and interpretation of achievement test data, and for periodically gauging the quality of the national education system - both in terms of achievement gains in-country and as measured against comparators of selected other countries. SAATs have been conducted in 208 CLSs, and ERDD has already started conducting SAATs in non- CLSs since 1998. Within the context of SAATs, a Test Item Data Bank has been developed by the ERDD. There are 6396 test items which have been stored at the Bank of which 3137 test items have been piloted tested and validated. In addition, the ERDD has conducted questionnaires on students in order to identify the elements which influence the success of the students by comparing the test results with the outcome of the questionnaires. The ERDD has successfully completed the mathematics and science achievement tests under the Third International Mathematics and Science Study Repeat (TIMSS-R) in 204 selected schools to 8000 13-year old students (Grade 8) in May 1999. There are 41 countries which are participating in this study. TIMS-R data entry, totaling 10 million, have been completed and forwarded to Hamburg Data Processing Center and to the National Research Center at University of Boston a branch of International Association for the Evaluation of Student Achievement (IEA) in Holland. The result of the assessment will be completed and forwarded to the ERDD by September 2000. The ERDD is also in preparation of applying Program of International Student Assessment (PISA) project tests to 15-year old students. PISA covers 32 countries 28 of which are OECD member countries. The study will be conducted in Turkish reading skill, mathematics and science. Schools selection and implementation will begin in the year 2000. 16. Problems: (i) ERDD has been chronically suffering from inadequate infrastructure to carry out its duties professionally. For instance, ERDD still does not have an access to e-mail so as to communicate with its international counterparts. In addition, lack of PCs has been impeding the work of ERDD. (ii) ERDD has been lacking, due to budgetary constraints, qualified consultants to assist its personnel in carrying out their duties. (iii) The lack of cooperation and friction between ERDD and related departments at MONE has been a continuous problem. As a result, ERDD has been suffering from lack of recognition by other departments of MONE. 17. Sustainability: The SAATs study has been initiated for the first time under NEDP. ERDD has shown that it is capable of conducting tests and questionnaires to assess student achievements. In order to continue developing the scope of the SAATs the ERDD should be allocated adequate funds to conduct these important work. The existing experience and knowledge accumulated through SAATs should be made use of with regard to the implementation of the Basic Education Project. Education Studies: 18. Achievements: Six research studies that are outlined in the NEDP have been contracted out by the MONE for the purpose of assisting the Ministry conceming its sectoral development program. The studies carried out on (i) Development of a teacher testing program to establish base-line data against which to measure improvements in teacher quality; (ii) A school mapping study to determine projected student growth patterns and trends in population shifts; (iii) In-service teacher training as a factor for consideration in the promotion of education personnel; (iv) Analysis of actual teacher loads throughout the system; (v) Inventory of instructional equipment and supplies in schools to determine availability and extend of needs; (vi) Assessment of the use of science laboratories. Upon the completion of the studies, ERDD continued to conduct studies by carrying out research studies on its own, and providing in-kind support to researchers from universities. ERDD due to budgetary constraint could not contract out research studies to research organizations. 19. Problems: Although the six research studies have been distributed to the related departments of MONE, the recommendations of the studies have not been fully realized due to financial, personnel, and administrative reasons. 20. Sustainability: Research studies should be conducted continuously, as the outcome of these studies will provide the MONE with a guide in pursuing the MONE's decision making with regard to sectoral development program. The sustainability of these studies can only be maintained so long as the decision takers make use of the outcome of the studies concemed. Quality of Textbooks and Materials: 21. Achievements: The two studies to (i) investigate the range of specifications for up-graded physical production quality textbooks, determine by volume and quality level the cost of up-grading, and make recommendations for appropriate quality levels to adopt, and to (ii) investigate textbook reni:al possibilities and make recommendations for a textbook rental policy initiative have been carried out. Special emphasis on improving the quality of textbooks with regard to content and physical production specifications and with regard to the quality of book covers and papers as well as the type of binding, some progress have been achieved albeit under different units of the Ministry and not under ERDC as stipulated in the SAR. For instance, a Textbook Technical Quality Control Unit to upgrade the physical production quality of textbooks, has been established initially under ERDD, but transferred to the Publications Department as the establishment of the ERDD under The Education Organization Law, Act Number 3797, of April 30, 1992, did not allow the Unit function through the ERDD. The outcome of the second study on textbook rental possibilities and textbook rental policy initiative implemented in a limited way which eventually let to the suspension of the textbook rental activity. With regard to the transfer of textbooks printing load from MONE to the private sector, although the SAR envisaged 80% of the textbooks to be printed by the private sector, presently, 50% are being printed by the private sector. 22. Problems: (i) Presently, the Textbook Technical Quality Control Unit operates within the Publications Department bereft of its legal status. (ii) Lack of cooperation and coordination between the BOE and Publications Department resulted in a slow down in making available the books at the beginning of every school year. (iii) With regard to the production of non-textbook teaching and learning materials, the MONE's Film, Radio and Television Center (FRTVC) could not produce the teaching and learning materials due to delays in the procurement of the equipment concerned. However, five of the twelve procurement packages have been realized towards the end of the Project. The main procurement packages concerning the renovation of the FRTVC color TV studios did not take place. The MONE is expected to procure the FRTVC color studio package under Basic Education Project. (iv) Repeated cancellation of the procurement of FRTVC equipment packages caused the resignation of several qualified personnel who went to work at private sector. (v) High turnover in the management is also another significant factor which has affected the implementation of the component. 23. Sustainability: Several of the forgoing problems raise questions as to the sustainability of this component of the Project. Undoubtedly, some achievements has been realized, but the rift between the BOE and the Publications Department prevented the Ministry from a potential gain with regard to the improvement in quality of textbooks and materials. Distribution of Textbooks and Materials: 24. Achievements: In order to establish a modem textbook storage and distribution system, the Publication Department and its 21 regional offices have been fully equipped with computers and inventory control software and users of this system have received specialized training to operate the system. During the implementation of the Project, the system has been upgraded and the personnel have been re-trained. 25. Problems: (i) Although the system have been installed in the Publication Department of the MONE and its 21 regional offices at the beginning of the Project, numerous obstacles have been encountered with regard to making the system fully operational. For instance, despite the training provided by the supplier firn and the MONE's General Directorate of Educational Technology, due to the lack of qualified technical personnel, the system is far from being fully operational. (ii) Lack of commitment by the Publication Department with regard to the appointment of qualified technical and administrative personnel to run the system has been a continuous problem during the life of the Project. 26. Sustainability: As the provision of improved warehousing and materials handling equipment and facilities, and implementation of a modem computerized inventory, ordering and distribution accounting capability and improved delivery and retailing arrangements could not be fully attained, the sustainability of this component has been poor. IMPROVING QUALITY AND RELEVANCE OF TEACHER EDUCATION Pre-Services Teacher Training: 27. Achievements: Pre-Service Teacher Training component of the NEDP was implemented by the Council for Higher Education (YOK). Although there has been considerable delays in the start of implementation, the goals and objectives of have been attained. Initially, the component covered 34 Faculties of Education and later on this figure was increased to 42 Faculties. Equipment and educational materials including books have been provided to all Faculties of Education. The curriculum and materials development activities have been completed with the help of the subject panels, expatriate and Turkish consultants. Urtder the component, 23 books have been published including 19 books on the main subjects of the teacher education. Master program in accordance with the project plan has been completed and 21 master fellows have obtained their master degrees, in addition to 52 PhD fellows who have been awarded their doctorate degrees. With regard to post-doctoral studies, 18 candidates have been sent abroad to conduct research in their respective fields. Study and monitoring tours took place to the USA and the UK to monitor the progress of the fellows. As an additional activity of the extension of the YOK component of the NEDP, a faculty-school partnerships program has been initiated with the schools of the MONE to set up an internship system for student- teacher work in schools. Another additional activity in this component was the agreement on standards for teacher education courses in the faculties of education and to set up an accreclitation system by which the achievement of the standards will be judged and teacher e6ucation programs accredited. 28. Problems: Acquiring Technical Assistance faces considerable delay at the beginning of the Project. This let to delay in procurement as well sending the fellows abroad for graduate and post-graduate studies. The two year extension enabled the fellows to complete their studies abroad. 29. Sustainability: The project has significantly increased the capacity of the faculties of education to provide teacher education. This has been due to the improvements in training facilities, procurement of relevant equipment and the realization of fellowships abroad, course content and training, which resulted in institutional strengthening of the faculties of the education. YOK accordingly has restructured the Faculties of Education to cater for the new requirement of the pre-service teacher training and also established the National Committee for Teacher Education. This is an evidence that the implementation of the goals and objectives of the Project have led YOK to a strong commitment for the continuation of the Project activities after the completion of the Project. In-Service Teacher Education: 30. Achievements: The NEDP assisted the In-service Training Department in expanding and strengthening its capabilities for planning and conducting training needs assessment and designing the implementing the in-service programs to meet the identified needs. Training has been decentralized and accordingly in-service training has been provided on a central and local level. MONE achieved in providing in-service training 577,766 trainees in 12529 courses. Centrally 251,476 trainees have been provided with training and locally 326,290 trainees have been trained. Four Educational Resource Centers (ERCs) in Aksaray, Ankara, Icel and Izmir have been established under the NEDP. The ERCs have been furnished and equipped so as to carry out the special purpose training on a selected basis for various groups of education personnel on topics not normally covered in the other programs. ERCs provided in-service and special purpose training for in-service teacher trainers, school inspectors, and provincial directorate staff etc. 31. Problems: (i) ERCs were supposed to assist in curriculum and institutional materials dissemination, to serve as regional depositories of teacher resource materials which teachers could use independently of organized training programs and to assist the ERDD and universities in school based research. The lack of qualified technical and administrative personnel prevented these ERCs from fulfilling their duties. 32. Sustainability: Except for the in-service training carried out at central and local level in four ERCs, the ability of the ERCs to sustain the remaining activities is doubtful due to the lack of commitment by the MONE. IMPROVING MANAGEMENT SKILLS AND PRACTICES Management Assessment and Development: 33. Achievements: Management Assessment and Development Unit (MADU) was established under the Project, in May 1994, by the approval of the Minister. With regard to the assessment activities, MADU has developed a Model to be used in assessing the units of the Ministry and applied this Model on the assessment of CLSs administrators' level of independence. With regard to the development activities, MADU has completed job descriptions of all positions covering total ministerial units and accordingly, an important Regulation with regard to "Appointment, Evaluation, Promotion and Transfer of Ministry of National Education Administrators" completed and published at the Official Gazette on 30 April 1999. The regulation was published as a decision of the Council of Ministers due to its importance. The aim of the regulation is to define the qualifications of the administrators at every level of the MONE administration and to regulate appointment, evaluation, transfer, recruiting and dismissal of the administrators by taking into consideration the criteria of career, suitability, record, seniority, examination grades etc. This achievement was accomplished through close cooperation between MADU and General Directorate of Personnel of MONE. Another important Regulation on the "Norm Cadres of the MONE Schools' Administrators and Teachers" also published in the Official Gazette on 10 August 1999 to establish and identify the rules of appointment of personnel to MONE schools. MADU has also been engaged in promoting Total Quality Consciousness in Education Admninistration System and drafted a "Regulation on Total Quality Control in Education". Another regulation on "Education Districts and Education Councils" has also been drafted by MADU and presented to the Undersecretary for approval. 34. Problems: As MADU has been established by the approval of the Minister, it has no legal status. The staff working in this department are secondments from the other units of the MONE. The uncertainty of MADU's legal status is a cause of concern for the future of this important unit. 35. Sustainability: There is ample evidence that the MONE has assumed ownership of MADU and the Ministry has shown its commitment concerning the activities of MADU despite the uncertain legal status of the unit. Research and Planning: 36. Achievements: Research, Planning and Coordination Council (APK) of the MONE improved the qualifications of its staff through external and locally organized training so that it can upgrade its planning, research and coordination functions. The cooperation with the State Planning Organization in preparation of the Education Master Plan has been a major activity of the Council, however, upon the enactment of the eight year continuous basic education law, the Council had to concentrate its time on the Basic Education Program implementation activities. 37. Problems: As the establishment of the Management Information System (MIS) has significant implications for APK's functions, delays in the JLSIS applications directly affected the collection and processing of statistical and budgetary data. The activities outlined in the SAR was partially fulfilled as a result of this delay. 38. Sustainability: The execution of the activities of the APK as outlined in the SAR are contingent upon the appointment of qualified personnel. For instance, education economists and researchers and education planners should be made available to assist APK in carrying out its duties. Management Information: 39. Achievements: As part of a first stage devolution of planning and decision- making responsibility from the central Ministry to the provincial directorates, the project implemented a pilot MIS project called ILSIS (MIS for provincial and sub-provincial directorates.) This pilot began in 1996 with provincial offices in Ankara and Cankiri, and two sub-provincial offices in Polatli and Cankaya. The pilot was designed to support the operations of the local offices, increase the speed and quality of services, streamline work-flow within the local offices, support the transfer and authority of information analysis and decision making to local offices, and train local and central personnel in the use of information technology (IT) for improved decision making. Government decided in 1997 to expand the pilot project to 80 provinces, 24 sub-provinces, and 34 secondary schools, which were to be connected through a dedicated wide-area data communications network (WAN). This would allow for integration of local and central level databases. The project has fully equipped the provinces and sub-provinces, developed application software modules (personnel, statistics, budget and accounting, facilities and others), provided training to both network and data administrators, and, following recent delivery of the application software, started to train MONE and provincial and sub-provincial staff on ILSIS applications. Installation of the hardware and systems software was completed in May 1999. There has been a progressive need for provincial and sub-provincial IT staff to receive updating training, and the Ministry has taken action in this direction. However, the project did not deliver the dedicated computer network that would link to the provincial directorates of education. 40. Problems: (i) There were continued delays in the ILSIS application software development, many of which were due to factors beyond the control of the software developer, such as modifications required in software modules due to changes in regulations including new eight-year basic education schools. As a result, the final application software was delivered 15 months later than scheduled. (ii) In the preparation and procurement processes, the Ministry was planning to build its own computer network for the WAN connectivity. However, Government decided instead to utilize TTnet, the nationwide information technology infrastructure being developed by Turkish Telecom because of the more expensive costs of deploying and sustaining a stand alone network. This strategic shift required new technical specification requirements (broadband switches and leased lines were no longer necessary), and subsequent procurement delays. Educational access to this network was not available at the close of the project. However, as part of the project completion process, MONE has provided the Bank with a request to finance under the Basic Education Project the WAN connection with Turkish Telecom, modems, and fiber-optic termination hardware, and has given assurances that application has been made to Turkish Telecom for related approvals. (iii) The loss of technical support staff to the private sector is a reality given the fact that with sufficient training in use and management of computer networks qualified persons can eam up to three times a monthly salary in the IT sector. High personnel turnover is particularly apparent in the provinces. 41. Sustainability: There is evidence that MONE has assumed ownership of the ILSIS, and shows readiness to ensure continued maintenance and operation. The Ministry has established a help-desk facility at EGITEK. The project's technical consultants and software developer have trained 20 counterpart technical staff who will provide network support and maintenance assistance to all directorates. There is a 3-year warranty on the ILSIS application software. An evaluation process is in place to ascertain costs and benefits of ILSIS, as measured by efficiency gains in work loads and flows of information resulting from the fully implementec. ILSIS. Ongoing training for technical personnel is planned in system maintenance and use of ILSIS application software modules, which will meet the need for updating training. But the reality is that technical support salary levels are not competitive with other career opportunities in the IT industry, and thus technical support staff turnover will continue. If compensation cannot be changed, non- monetary incentives need to be present (e.g., variety of work, size and variety of projects, job security, opportunities to obtain experience/additional experience). The extent of MONE's commitment to project sustainability remains open without offering sufficient employment condition and salaries to attract qualified technicians. Center for Research in Vocational-Technical Education: 42. Achievements: The Vocational and Technical Education Research and Development Center (METARGEMi, initially, provided administrative support for the establishment of Project Management Unit to manage all of the MONE's education development projects including those financed by the Bank. METARGEM has conducted research activities in vocational and technical education. The unit has developed a model for re-structuring vocational and technical education in Turkey. The Center has also been engaged in the program development activities with regard to vocational and technical education. 43. Problems: METARGEM has been established with Ministerial approval, but it has not attain legal status yet. 44. Sustainability: Despite the uncertain legal status of the unit, the Ministry has committed itself to the activities of METARGEM. ADDITIONAL ACTIVITIES 45. The NEDP was re-structured on 18 January 1996 so as to allow the inclusion of three additional activities to be funded under the Project. Basic Education Pilot Project: 46. Achievements: Basic Education Pilot Project (BEPP) was a preliminary project providing the grounds for the presently implemented Basic Education Project. With regard to the rural education component of BEPP seminars, orientation, follow-up and evaluation meetings were held in order to improve the knowledge and skills of the central and provincial project implementation teams. Teachers guides and student activity books were reproduced and distributed to teachers and students. Various educational materials have been distributed to 276 village schools. In-service training for inspectors and teachers to improve their understanding the philosophy and basic principles of active learning have been provided. School funds program was initiated in thirty village schools to enhance school-community cooperation, to increase enrolment and retention (especially girls), to encourage the parents and the community to participate in school related activities and get involved in decision making process. In addition, basic school buildings handbook was developed. Upgrading and extension of 14 rural schools in three provinces have been completed and furnished. Pre and post evaluation have been conducted in order to measure the achievements of the rural education component of the Project. 47. Problems: During the course of the implementation of BEPP, architectural and engineering projects for three new schools have been prepared and tendered. Due to the enactment of the new Basic Education Law, the MONE cancelled the tenders and did not go ahead with the construction. 48. Sustainability: The signing of the Loan Agreement of the Basic Education Project, provided the MONE with the opportunity to make a smooth passage with regard to implement the new Project. Secondary Education Studies: 49. Achievements: The "Feasibility Studies on Secondary General and Vocational Education" have been completed and based on the outcome of the Study, the MONE has prepared a Concept Paper for Secondary Vocational and Technical Education Project and submitted it to the consideration of the Bank 50. Problems: Problems have not been encountered with regard to the preparation of the feasibility studies. 51. Sustainability: The MONE, in addition to the Basic Education Project, would like to launch project preparation with the aim of signing a loan agreement to implement a new project in the field of secondary vocational and technical education. Mother and Child Development Program: 52. Achievements: The Mother and Child Development Program has implemented with the collaboration of the MONE and the Mother and Child Education Foundation since 1994. The program is a 25-week program targeting mothers of five year old children and to empower the mother to fulfil her role as a potential educator of her children. The participants of the Program have trained in a wide range of child-rearing topics, cognitive development of the child, family planning and reproductive health. Initially, the Program has been funded through the Non-Formal Training Project and later was extended under the NEDP as of 1996. Over the past four years of its implementation, the Program has covered 35,000 mothers and children in 59 provinces. In terms of training, 92 coordinators and 400 teachers have been trained. In addition 300 teachers have participated in refreshment seminars. Nine orientation and evaluation meetings have been conducted with the participation 574 Ministry officials. Twenty five short films on mother and child education have been broadcasted on national and local TV channels. An extensive evaluation of the Program has been conducted and the results of the evaluation has shown that the Program was successful in fulfilling its main target. 53. Problems: No significant problems have been encountered during the implementation of the Program. 54. Sustainability: Based on the successful implementation of the Program, the MONE is considering the extension of the Program's activities under the Basic Education Project. World Link Program: 55. Achievements: This activity was not envisaged under the NEDP, but MONE decided a pilot should be launched. because of its new IT program which phases in different instructional objectives starting with computer literacy and moving toward more skill-based and subject matter related training, including use of the Intemet. WorLD is establishing educational on-line communities for secondary school students and teachers around the world, in order to expand distance learning opportunities, improve educational outcomes, facilitate cultural understanding across nations, and promote broad-based support for economic development. The pilot project in 19 secondary schools from 16 provinces started in September 1998. With extensive inputs from an EGITEK team that is evolving to play the role of a "center of excellence" in relation to use of the Intemet in schools, in-service training for directors, teachers and students has been provided to the pilot schools, along with technical assistance. Because of the general enthusiasm about the Internet, 12 new schools have requested Ministry approval to join the program. Extra budgetary income for the pilot has come from WorLD, which has also provided technical assistance to carry out a monitoring and evaluation study of the pilot. The monitoring and evaluation protocol is likely to be used in the formative and summative evaluation of the IT component of the Basic Education Program. EGITEK has demonstrated commitment to provide pedagogical support to the schools in terms of local training sessions, and implementing monitoring and evaluation instruments designed to gather information about the achievements and changes to teachers, students and parents of the courses worked on in the project. 56. Problems: (i) The most serious immediate problem facing the program is the lack of time teachers and students reportedly have for computer-related activities within their school schedule in general, and to participate in the design of collaborative learning projects in particular. When participation in a pilot is during "spare time", the scope for realizing benefits may well be limited. (ii) A specific problem parallel to that is the lack of familiarity with online learning settings, and the difficulties of using both the Internet and Internet-based educational materials to improve curriculum usage. There remains a question mark as to the effect of introducing an educational on-line community program when the framework for IT use and collaborative learning has not been firmly established. 57. Sustainability: An important aspect of the program is that schools need to have Internet access, as well as visits by program coordinators to discuss with teachers and students problems in use of the Internet in teaching, and ways to share information and collaborate on projects. The first priority of the Ministry and WorLD should be to ensure that existing schools and identified teachers and students continue to participate in the program, and to provide training and learning events oriented towards the design and implementation of collaborative interdisciplinary curriculum-based projects. Evidence that the schools have assumed ownership of this activity and plan to continue it should be demonstrated. CONCLUSION 58. Following the completion of this mission and receipt of final information from the implementing agencies, an Implementation Completion Report (ICR) will be completed. Implementation Completion Report will cover, inter alia, the achievement of project objectives, the implementation record and major factors affecting the project, project sustainability, an assessment of outcome, future operations and the key lessons learned. Appendix: B MONE IMPLEMENTATION COMPLETION REPORT REPUBLIC OF TURKEY NATIONAL EDUCATION DEVELOPMENT PRJECT MONE COMPONENT EVALUATION SUMMARY Like in many other developing countries, Curriculum Development, teacher training, equipment supply and using technology in education seem to be the most important issues in Turkey However, because of insufficiencies and deficiencies in pre-service teacher training, lack of equipment in schools, not using the Educational Technology Means well enough, not upgrading the teachers continuously through in-service training and other methods have come out as elements hindering expected developments in education process. It has always been in the Governments Plans so far to support the education of young population and help them in becoming productive, decision maker, problem solver and strong willed individuals by increasing their quality and skills parallel to the development plans of the country. Delays occurred in upgrading curricula programs and not being able to provide a rapid development because of the lack of sources to be used in order to increase the quality in pre-service and in-service training have continuously affected the quality of the education in a negative way. According to the data given in "National Education 1999 with Numerical Data" published by APAK of the MONE, there are 7,936 pre-schools, (7,389 are of state and 587 are of private) in the whole country; and there are 44,525 primary schools, (43,874 are of state and 651 are of private); there are 2,611 general high schools, (2,170 are of state and 441 are of private). Total number of the pre-schools, primary and high schools is 55,112. When we subtract the number of private schools, which is 1,679, from this number, it becomes 53,433. Institutions giving vocational and technical training, Apprenticeship Training Centers and Public Training Centers are not included in this number. On looking at the number of the schools giving general education and the size of the education sector, the difficulty to man- age them from the center can be understood easily. National Education Development Project (NEDP) has been prepared in order to elevate the primary and secondary education up to the averages of OECD countries by increasing the quality and accreditation, to elevate pre-service and in-service teacher training up to the standards of OECD countries by increasing the quality and accreditation, to become more effective and economical in resource use by improving administrative and operational skills and implementations of the Ministry of Na- tional Education. It is envisaged that for establishing a new curriculum policy, increasing the quality of text-books and educa- tion materials, establishing a new testing and evaluating approach in order to observe the quality of education continuously, for researches to increase the quality of pre and in-service teacher training by means of establishing a new managerial ap- proach to the education system and 177.2 Million USD source has been allocated. 15 Ministry Units, having a relationship with the Formal General Education directly or indirectly, have been included in the NEDP and these units have been given service through "Technical Consultancy Services" and furnished and equipped through national and international tenders realized within the scope of the project. A- BACKGROUND AND PROJECT OBJECTIVES According to the census done on 30 November 1997, the population of Turkey is 62,865,574 and the increasing rate of the population is 1.62 % annually. 21,983,217 of the population live in rural regions while 40,882,357 live in cities. When the annual increase rate taken into consideration, it is expected that the population in 2005 will be 69,825,000 and in 2010 it will be 74,119,000. Republic of Turkey gives priority to the education of the youth ( who make the 70% of the population) by increasing the edu- cational level of its students. Though many improvements about this issue have been recorded when years are taken as the basis, schooling rates in primary and secondary education are not yet as required. It should be increased. The government has been trying to elevate the level of schooling rates up to the objectives defined in the Development Plans by signing agree- ments with international financing institutions as well as making use of State Budget and Public Contribution. As of 1999, the schooling rate in primary education has been 88.9 % and in secondary education it has been 54.7%, 23.9% of which belongs to the vocational and technical education. 1 Schooling Rates And The Number Of Schools, Teachers And Students According To The Grades Of Educational Institutions EDUCATION GRADE NUJMBER NUMBER NUMBER SCHOOLING OF OF STU- OF TEACH- RATE (%) SCHOOLS DENTS ERS Pre-School 7,976 207,319 11,825 8.9 Primary Education 44,525 9,512,044 316,991 88.9 Secondary Education 5,708 2,013,152 39,644 54.7 General Secondary Education 2,611 1,094,610 70,936 30.8 Vocational Technical Secondary Edu- 3,097 918,542 68,728 23.9 cation Sub Total 58,209 11,732,515 468,460 Non-Formal Education 6,280 2,935,929 44,042 Higher Education" 6,567 1,322,345 63,223 22.4 GRAND TOTAL 64,556 15,980,789 575,725 * Rates are figures of the years 1996-1997 acadernic year. Source: APAK Numerical Data National Education 1999 (Ankara 1999) ** Figures about Higher Education belongs to the 1997-1998 academic year. The governments have been aiming to decrease the general education share of the secondary education to 35 % but to in- crease the share of vocational and technical education to 65 % in their Development Programs. According to the data of 1996-1997 academic year, the existing schooling mate of primary education throughout the country was 88.9 % and the objec- tive for the year 2000 is 100%. Schooling rate of secondary education was 54.7%, 30.8% of which belongs to the general high schools and 23.9 % belongs to the vocational and technical high schools. Development objective envisages the school- ing rates for the secondary education as 75% and for the general high schools as 40.5 %, and for the vocational and technical education as 34.5% by the year 2000. When we take the figures aimed for the year 2000 as the basis, we can see that the schooling rate of the general high schools is 54% and of vocational and technical high schools to be 46%. Final objective of the government is to decrease the schooling rate in secondary education to 35 % in general high schools and to increase it to 65% in vocational and technical high schools. In order to reach planned objectives, new projects should be designed in addition to the present ones. Besides available man- power (directors, teachers and other personnel) resources like buildings, institutions, time, money and every kind of equip- ment, production and education tools should be used in all programs based on the cooperative working principles with the organizations and institutions concerned. Besides these, with respect to the Development Plans, Government Programs and the needs of business market, it is necessary to reclesign the process of vocational and technical education so that the students can develop the problem solving and decision making skills and therefore to prevent huge crowds from entering universities and to elevate the quality in education and training up to the averages of OECD countries. That is why, a number of meas- urements should be taken in order to re-examine and re-structure both general education and vocational education with re- spect to the pre-service and in-service teacher training, curricula programs, documents, physical facilities, financing and man- agement issues so that the capacity of the education system can be upgraded to meet the needs of Turkey in the years of 2000s at the international level. 1. National Education Development Project (Ln-3192-TU) has been proposed to the World Bank (WB) by the Turkish Government in order (a) to elevate the primary and secondary education up to the averages of OECD countries by increasing accreditation and quality. (b) to elevate teacher training up to the averages of OECD countries by increasing accreditation and quality. (c) to increase the productivity and effectiveness in resource use by increasing the management and administrative skills of the MONE. An "Appraisal Mission" formed by the WB visited Turkey in October and November 1989, and re- viewed sectors dealing with the project and prepared Staff Appraisal Report (SAR) #8328-TU dated 18 May 1990 after the proposal Republic of Turkey. According to the Loan Agreement, which was signed with respect to the objectives defined above between the Republic of Turkey and the WB, the WB agreed to give US $ 90.2 Million of the total cost of the project, which was US $ 177.2 Million. Remaining US $ 87 million was the commitment of the Republic of Turkey. 2. It has been envisaged that necessary studies should be done to review, renew and upgrade education programs, text-books and education materials of each field, to establish of physical capacity of the project schools and equip them, automation of stock control and delivering systems of text-books and to establish of a testing and evaluation unit within the ERDC to con- nect extensively with the program and education material producers, to make necessary arrangements for each teacher to be taken into the in-service training at least once in every five years, and to prepare new programs for this and build new In- service Training Centers and widespread in-service training at the local level throughout the country; in order to widespread authority and responsibility gradually into the provincial and sub-provincial level as well as the (Management Information System) computer network to be established to increase the administrative and operational skills of the MONE. The following 2 objectives have been included in the project with the Loan Amendment dated January 29,1996: (i) spreading basic education that was increased from 5 years to 8 years, (ii) a feasibility study in order to improve vocational technical education, (iii) spreading pre-school education, through the Technical Consultancy Services and overseas fellowship training and the units under the project improving the quality and accreditation in the primary and secondary education have all been envisaged. B- ACHIEVEMENT OF PROJECT OBJECTIVES FIRST OBJECTIVE 3. Curriculum Development: The MONE has established Education Research and Development Directorate (ERDD) as it was foreseen by the project. Studies of testing and evaluation, research and curriculum development within the ERDD, which was established in 1992, have been continuing. It has been foreseen in the project that the curriculum developed by ERDD shall be experimented in CLS and that successful ones shall be made widespread across the country. Primarily " Curriculum De- velopment Model " was prepared by ERDD. The model in question was suggested by Board of Education to be used in the activities for curriculum development. According to this model the following curriculum programs have been prepared by ERDD: Secondary School Biology 1-2-3, Primary School Social Life Knowledge 1-2-3, Primary School Science 1-2-3 for Combined Classes, Basic Education Mathematics 1-8, Foreign Languages ( English, French, German) 4-11, Secondary School Geography 9-11, Secondary School Turkish Language and Literature 9-11, Basic Education Turkish 1-8, Secondary School Physics 9-11, Secondary School Chemistry 9-11, Anatolian Secondary School Foreign Language( English, French, German) 9-10-11. Among these programs curriculum for Primary School Social Science 1-2-3 and Basic Education Mathe- matics 1-8 have been approved by the Board of Education and others are waiting to be approved. Not only the delay experi- enced in the period of curriculum development, but also the delay experienced in the period of approval by the Board of Edu- cation prevented from experimenting sufficient curriculum in the CLS. Although curriculum development phase of the project seems to have remained incomplete, it is possible to compensate this component of the project. 4. Training Material Development: In order to reinforce the capacity of training material other than text-books of the MONE, establishment of hardware in FRTED and Technical Consultancy and Scholarship were also foreseen for the same institution in the project. For the equipment of FRTED, International bids which were conducted on 5 October 1994, and 16 June 1995, were canceled. As a result of the international bid which was made on 15 April 1998, 5 packages (6, 8, 9, 11 and 12) could be purchased out of 12 packages. Since other packages which the firms offered were not found suitable, they were canceled. Although the WB did not admit to re-bid of these packages on the grounds of the closing date of project, the WB has declared that the Bank will accept to re-bid these packages for this institution from the Basic Education Program's funds if the MONE is eager to purchase these equipment. With this approach of the WB, it seems possible to enlarge the capacity to develop training material other than text-books through the usage of the resources of Basic Education Program.. However, the delay which occurred at this phase has to be ceased as soon as possible. 5. Testing and Evaluation : Testing and Evaluation Section was established within the ERDD. For this section so as to do the functions foreseen in the project, 4 consultants were employed for 38 n/m in the Testing and Evaluation Section of the ERDD. 9 staff were sent abroad for 72 n/m for education and they worked in this section after they completed their overseas training. The section was made a member of the "International Evaluation and Measurement Institute" whose center is in Netherlands so as to be able to compare with the other countries studies of testing and evaluation which were done or will be done across the country, and also computer and software programs were purchased for this section through project funds. 6. As a result of the reinforcement given to the Testing and Evaluation Section, this section has improved and reached a ca- pacity to make testing and evaluation at international standards. As a result of this input, this section has conducted lots of pilot tests and final tests all over the country including CLS and non CLS. These studies of the Section have been presented by both reports and briefings to the Senior Staff of the Ministry at various times. The studies which the Testing and Evalua- tion Section has done and is still doing prove that the objectives set for this section in the project have been achieved. Besides a great majority of CLS have formed their own Testing and Evaluation units within themselves. Schools are realizing and evaluating the testing and evaluation studies in digital surroundings in these units. 7. Research Studies: The followings are the studies which were done by ERDD from the beginning of the project till 30 June 1999 according to the first objective of NEDP. Educational Investigation Program for Turkey, Inventory of Educational and Instructional Materials, Enrollment Estimation/School Mapping : An Educational Planning Tool for Turkey, Demonstration for Science Laboratories, Teacher Assessment, Analysis of Teaching Load, In-service Training as a Requirement for Promo- tion, A Research for Course Promotion and Credit System, Comparison of Educational Indicators of Turkey and OECD Countries, Curriculum Studies in MONE, Student Dresses in Primary Level, Collecting the Educational Researches in Tur- key, Contemporary Teacher Profile Needed in Turkish Education System Towards 21. Century, Importance of Current Insti- tutions Giving Private Courses in Turkish Education System. First seven of these studies were conducted by the coordination of local and foreign specialists at the ERDD. The other seven were conducted by the Research Team of the ERDD. Beside these researches, the ERDD has supported a lot of researches which were conducted by master and Ph degree students of universities. The ERDD has still been doing 15 research studies, 11 of which will be finished by the end of 1999, and 4 of them will be finished in 2000. 3 8. The ERDD is planning to support research studies in the following years. All these researches show that local and foreign specialists who worked at ERDD assisted the research capacity of ERDD according to ISt objective of the project. Therefore, it can be concluded that 15' phase of the project has been succeeded. 9. Improving Quality of Text-Books: There was a positive development in improving the quality of the text-books, that is, one part of the project in processing levels of the prcject. In 1990s, books were printed in 3rd quality papers but nowadays they are printed in 15' quality papers. To improve quality of books, personnel were educated in other countries and came to Publi- cations Department with a Ministry Approval but the Board of Education do not let this authority transfer. As a result, quality control of books' content is done by the Board of Education and press quality control is done by the Publication Department. 10. Textbook Distribution and Stock Control &Lstem: This system was set up to connect Publications Department and re- gional chiefs in 21 provinces by computer network. For this purpose, an international bidding was held on 24 October 1994 and US $ 445,000.00 contract was signed for hanrware and software with the awarded firm. The firm fulfilled its responsibil- ity and turned over the system in working condition. Afterwards, a US $ 30,000.00 equipment purchased from the same firm to up-grade the system, and a training was given by this firm for staff who use the system. Despite the fact that the same training was given by the General Directorate cf Educational Technologies personnel, except for Konya, there is no other province using the system actively because they have not been able to employ the personnel with needed qualifications to use the system. 11. Curriculum Laboratory Schools (CLS): After the physical capacities for 208 CLS, which were chosen in accordance with the criteria defined in SAR ATTCH-8, were established with the resources from the general budget and local resources as a conclusion of an international bidding realized on 15 August 1995 for the equipment of these schools, the needed equipment were delivered to these schools. The related unit. under the light of the works of native and foreign experts, has determined the equipment to be used more effectively and productively in education and training, CLS Section of ERDD has prepared the guidelines of "CLS Model" and "Science Laboratory Usage Guide" and has distributed this to schools. Besides, regular in- service training activities are being organized to educate school administrators and teachers on the effective and productive usage of these equipment which have been purchased. These schools have become attractive in their surroundings and are guiding other schools nearby. Hence, it is possible to say that this phase of the project is successful. 12. Computer Experimental Schools (CES): An international bid was organized in 1994, September 27. It was organized to set up laboratories for the 53 CES which have 20+1 computers and various training material including educational software. These schools were selected according to the criteria mentioned in the SAR Attach.-8, from 7 regions whose physical capaci- ties were set up through State Budget and Local opportunities. Hardware and software have been distributed to 53 CES. It was decided to set up 182 schools with 20+1 computers and other training materials including educational software across the country with each city having at least one computer laboratory. As the implementation of CESs was found successful by the consultants, this was the result achieved through mutual agreement between the MONE and the WB and it was run through the approximately US $ 8 Million - resource which was estimated to have remained from project Loan an international bid was organized on 18 March 1998. Hardware and software in this content of the bid were distributed to the 182 schools and they are being used in education at the moment. It can be said that this phase is the least problematic and the most successful phase of the project and it can be said that object.ives which were set in the project have been achieved. The success which has been achieved by CES studies and carried out within the framework of NEDP has set the agenda for usage of computer and software in education. It has leaded new pro ects, and Basic Education Project has been planned relying on experience, success and result obtained from that. CES has taken the objective of priority role for the rest of the projects being imple- mented by the MONE. SECOND OBJECTIVE 13. In-service Teacher Training: It is stated in SAR that, according to the MONE the number of teachers that need in-service training is too much (200,000) and this need can be met via 4 In-service Training (Education Resources Centers) to be estab- lished under the project, as well as TV broadcast:, video cassette delivery, and distant-education implementations benefiting from programs organized in the place by instructors. Within this context, in Ankara, Aksaray, icel and Izmir four Education Resources Centers have been established and these centers have been equipped with the bidding conducted for both CESs and CLSs. Besides the objective to train 57,000 teachers annually by giving in-service training authority to Provincial Director- ates since 1995, the amount of in-service training courses and amount of trained teachers has been exceeded. From the begin- ning of the project till 31 December 1999, a total of 998,203 teachers have been trained in 12,542 in-service training activi- ties organized at central and local level. 420,437 teachers have completed their four year license education, therefore the rest 577,617 is the number of the teachers trained via in-service training. This number is more than the number of the teachers in Turkey (which is 523,617). Though in SAR the number of the teachers that have to take in-service training has been foreseen as 57,000 (24,000 of whom via in-service training and 33,000 of whom via Distance Education ), only in 1999, 71,319 teachers were given in-service training (13,440 of whom central, 57,879 of whom local level) except for those who have completed their license education. Therefore it is possible to say that the amount foreseen has been exceeded and this compo- nent is successful. 14. Equipping of Education Resources Center (E,): It is foreseen in SAR that ERCs should be equipped with staff and equipment so that it will become the multi-purposed branch of the MONE in regional level. After establishing the necessary physical capacity and physical places for four ERCs by the MONE, related equipment were provided witn the ICBs realized 4 firstly for 53 CES, and for 208 CLS. Moreover, "Being a Teacher with All Aspects" and "Guidelines to Prepare In-service Training Program" have been prepared by native and foreign experts working under the Technical Consultancy Services in In-service Training Department. These documents have been printed with the sources of the project and distributed. It is pos- sible to say that activities done by the MONE in order to reach the project objectives are successful. THIRD OBJECTIVE 15. Improving Management Skills and Practices: In this component of the project, Management Assessment and Develop- ment Unit operating within the General Directorate of Personnel at the beginning, and this unit has been restructured by being separated from this institution with the Ministry approval. As a result of the evaluations realized by the native and foreign experts working at the MADU within the Technical Consultancy Services of the project, terms of references for the positions in this unit and for the ministry units and performance assessment criteria have been established. The terms of references established by MADU have been sent to the State Personnel Department to be approved. These terms of references have been sent to all state organizations by the State Personnel Department. The Ministries and other organizations taking these terms of references as the basis have improved their own terms of references and had them published in the Official Gazette. Moreover MADU regularly gives in-service training to the personnel of both central and local offices and to the teachers on Total Qual- ity Management. A monthly news bulletin is being published in order to inform Ministry central and local offices about the activities of MADU, so it is possible to say that this component has been successful. 16. Combining Some of the Administrative Units With Similar Characteristics: These activities that should be realized under the project are not said to be given enough importance. In approximately nine years during which this project was being im- plemented, with respect to the Combining Some of the Administrative Units with Similar Characteristics, only legal regula- tion realized was the combining General Directorate of Computer Education and Services and Film, Radio and Television Education Center and establishing General Directorate of Education Technologies. Except for this, there are no other im- provements and no other efforts in this direction because of bureaucratic obstacles. Therefore, it is not possible to say that this component of the project has been successful. 17. Strengthening Strategically Planning and Education Research Functions: For this component of the project native and foreign experts who worked at MADU and APAK, as well as the ministry personnel have realized important activities. The activities realized by APAK are as follows: Preparing Education Master Plan, Preparing Education Master Plan Attachments, Introducing Education Master Plan to the Public, Studies to Estimate Population of the Country and Provinces in Accordance with the Age Groups by the year 2000, Research of Present Education Situation at Provincial Level, Annual Implementation Plans of the MONE, Comparative Study of International Education System, Research for the Education Needs in the Whole Country. Though Education Master Plan completed at the beginning of 1997, because of the 8-year Compulsory Education Law, this plan has to be revised. Studies to revise this report have still been going on at the APAK. Education research func- tions of the MONE have been strengthened by the overseas fellowship training and native and foreign consultants have been appointed within the MADU and APAK under the project. 18. Improving and Expanding Management Information System: By means of Project Management Information System, it has been envisaged to expand and develop present MIS of the MONE in order to improve the effectiveness, productivity and coordination in the education system management. With the approval of the MONE and the World Bank, pilot implementa- tion of Management Information System has first been applied in Ankara and qankiri provinces, and sub-provinces of An- kara, Qankaya and Polatli. Pilot implementation has been found successful as a result of the experiments realized during the software development phase and consultants' assessments; and it has been agreed to extend MIS implementations to the whole country. In order to extend MIS, an international tender was realized on 7 April 1997. MIS, which has been functioning and software of which can be used, has been completed in the provinces, studies necessary to make it work as WAN have been going on. If TT-NET agreement is signed with Turk-Telekom, the system will be put into use. Why this system is not oper- ating actively is because Turk-Telekom has left TURNET use for TT-NET. Because of this change, the MONE has not been able to sing a contract with Turk-Telekom. However, the Ministry is about to make an agreement with Turk-Telekom by using the Basic Education Project sources, in order to start this system to operate. It is possible to say that this component of the project has been successful in equipment and software procurement, though it is not possible to neglect the delays in oper- ating. The MONE is also eager to establish of Decision Support System in this scope through Basic Education Program's fund. 19. Increasing the Research and Development Adequacy of METARGEM: It has been envisaged that this unit should provide equipment that is necessary to analyze and operate basic data in order to support and strengthen this function, and also to upgrade and strengthen the education data collection and analysis via project as well as its duty to prepare and steer the proj- ects, purchase and keep the records. Research and development authority of this unit has been increased with overseas fellow- ship training, equipment provided and also with the support of native and foreign consultants gained from the project, there- fore objectives planned in the project have been reached Amendment Done In The Project Loan 20. In the budget allocated in accordance with the planning realized during the project implementations, because of the changes in needs, the categories of NEDP Loan were changed with the mutual agreement of the MONE and the WB January 29, 1996. In accordance with the needs occurred during the project Implementations, loan categories were changed on 13 February 1997. 5 Objectives Included In The Project With The Aniendment Made On 29 December 1996 21. Expanding Basic Education: Basic Education Project has been prepared in order to extend 8-year Compulsory education and for this project US $ 13.1 Million resource has been allocated from the NEDP loan. Upper level management of the min- istry has decided this project to be applied with the coordination of the General Directorate of Primary Education. Following the 8-year Compulsory Education Law # 4306, published in the Official Gazette # 23,084 dated 18 August 1997, and also after the Basic Education Project started to be applied, Basic Education Pilot Project has been stopped by the MONE. 22. Expanding Pre-school Education: Mother and Child Education Program, aiming to educate mothers and children living in disadvantaged conditions, which previously was implemented under the Non-formal Vocational Training Project, has been supported through NEDP with the mutual agreement of the Ministry and the World Bank since 1 January 1996 after the clos- ing of Non-formal Vocational Training Project on '31 December 1995. Mother and Child Education Program was applied to 3628 mothers and children by 112 teachers in 23 provinces in the 1995-1996 academic year. Within the coming years this implementation was widespread and finally in 1993-1999 academic year the number of the provinces became 59, teachers became 269 and the number of educated mothers and children became 9324. During the implementation period of this pro- gram, the number of the mothers and children has teached 26.053 This program has been implemented within the coordina- tion between General Directorate of Apprenticeship and Non-formal Vocational Education and Mother and Child Education Foundation, and success has become more than expected. 23. Feasibility Study: In order to improve Vocational Secondary Education in Turkey, by searching the background and pres- ent situations of the sector, and as a result of this research, establishing of a system in which man power would be trained to compete in the international business market in any way, and realizing a study with respect to the preparations of project proposals to establish such a system were included in the project with a loan amendment made on 29 January 1996. In order to make such a research a contract was signed with Center for Occupational Research and Development firm from the USA that was awarded with the tenders realized with Short List method on 12 December 1997. During this study implemented under the coordination of METARGEM, the background and present situation of the Vocational and Technical Education have been examined. The findings gained from these researches have been submitted to the ministry in a report. These find- ings have also been presented to senior officials of the ministry in a briefing. With respect to the data gained from these stud- ies, a "Concept Paper", prepared by the MONE so that a new project can be prepared for the Vocational Secondary Education development, has been sent to the World Bank. 24. Technical Consultancy Services: Within the scope of NEDP, foreign language training services have been purchased from International Language Center and Turkish American Association as well as Technical Consultancy Services and training services have been purchased from The Ohio State University, Bilkent University, Anadolu University, Center for Occupa- tional Research and Development and Argon International Constancy Ltd. For the Technical Consultancy Services and Fel- lowship training component of the project, a contract was signed with The Ohio State University on 11 June 1992, and studies were started. Within the scope of the contract signed with The Ohio State University, consultancies needed by the ministry units were purchased. Moreover, the necessary fellowship training taking place in the Technical Consultancy Services have been used effectively and productively and required benefit has been gained from these, nevertheless, some of the fellowship training has become a short-term study - tour, because foreign language knowledge of the staff has not been good enough. Personnel who had overseas foreign language traini rig have worked in the fields they were educated at least twice as much as their overseas training. However, these personnel h.ave not worked in the related units continuously after completing their compulsory work period. 25. Use of Loan: US $ 67,100,000.00 of the (US$ 90,200,000.00) Loan was allocated for the MONE and 48,624,752.70 has been used but US$ 18,475,247.30 has not been used. US$ 8,119,455.99 is the amount that has remained from the Basic Edu- cation Pilot Project implementations including the US$ 476,391.01 repaid amount by the UNICEF to the W.B which had been paid for the Basic Education Pilot Project to the UNICEF previously. 26. Use of Government Budget: Expenditure that should be made for the National Education Development Project has been put into the annual Budget Laws (as often as MONE budget is convenient according to the years) since 1991. At the begin- ning of the project, appropriations- added in the budaet under the name of "National Education Development Services" have been added in the budget under the name of "National Education Development Project" since 1993. Except for personal ex- penditures, which is US$ 68.2, Million the govermnent contribution is US$ 63.4 Million. Moreover, great resources have been provided to change the physical capacity and to establish physical places in the 53 CESs, 208 CLSs and 183 CESs from the local administrations and parents. In 1998 the contribution of the parents for the establishment of 182 CES's physical capacity was US $ 700,000. In short, the expenditurL realized by MONE, local administrations and parents can be said to be satisfactory. C- MAJOR FACTORS AFFECTING THE PROJECT 27. The MONE has established Project Coordination Unit in order to coordinate and implement the project financed by the World Bank and the MONE. At the beginning of 1992, PCU was restructured and reinforced with the staff, most of whom 6 completed their master degrees and know a foreign language. It has been stated in the SAR that " Implementing activities related to development, upgrading the quality in primary and secondary education, in-service teacher training, im- proving the management shall be carried out by Education Research and Development Center, In-Service Training Department and General Directorate of Personnel (Regulations Department) respectively". 28. It has been stated in Table 5 Implementation Program (A) Project Management Paragraph 2 of the Loan Agreement that" MONE and YOK each shall establish a Consultancy Comniittee to monitor the developments in project implementa- tions and to control whether these developments are proper to the project objectives and shall enable this committee to continue until the completion of the project." 29. The MONE established Project Advisory Committee with the Ministry approval # 433 dated 6 May 1991. The meetings of this Committee have been realized with difficulty most of the times because the approval for this Advisory Committee, consisting of senior officials of the MONE, unit directors under the project, and the directors of other institutions, has been taken according to the names and because of the changes occurred in the positions of these officials. That is why this Com- mittee was renewed with approval of the MONE # 1223 dated 10 May 1994 with respect to the positions but because these people are too busy, it has not been possible for them to meet. Directors of the ministry units under the project and those responsible for the project have met every month at the PCU with the coordination of PCU. 30. Project management has been realized by the MONE units under the project coordinated by the Project Coordination Unit and also with the guidance of the WB. During the implementation period of the project, 8 Ministers, 4 Undersecretaries, 3 Deputy Undersecretaries, 3 PCU Directors and 3 Project Directors have worked. During this period, there have been many changes among the positions of the directors of the units under the project and those responsible for the project. These changes have been the major factor delaying the project implementations. 31. Projects Department, previously working within the General Directorate of Boys' Technical Education, has been estab- lished again with the ministry approval under the name "Project Coordination Unit", which has been directly connected to the Undersecretary since 1990. Project Coordination Unit, which was re-structured again in 1992, consists of 1 Director, 2 Dep- uty Directors, Project Directors responsible for each project and 8 Section Directors. Because the general policy of the min- istry does not enable Project Coordination Unit to employ professional staff, teachers, who still belong to their schools, mostly knowing a foreign language and having been trained on project implementation issues abroad had to be employed. 32. It has been thought to make use of the experience of the foreign and native consultants provided in the scope of the Tech- nical Consultancy Services of NEDP in the Project Coordination Unit and other units of the MONE under the project. How- ever, units of the MONE under the project have experienced difficulties in finding the qualified staff to work as the counter- part of the consultants. D. PROJECT SUSTAINABILITY: 33. Within the scope of the project, 147 primary education schools, 53 high schools (16 of these are Anatolian High Schools) and 8 Anatolian Teachers High Schools have been chosen as CLS, the above mentioned schools have been chosen from 7 region and after the establishment of physical capacity of these schools, the necessary equipment have been procured in order to test the developed curricula. Also within the scope of the project, after completion of the necessary physical capacity of the 53 CES chosen from 7 region, a computer laboratory consisting of 20+1 computers and peripheral equipment has been es- tablished and they have been used in education. After the successful implementation of the CESs that has been registered by the reports of the specialists, the MONE proposed to the WB to use the remaining money from the project fund and the WB found the proposal acceptable. After the no-objection of the WB, 71 Primary Schools, 12 Anatolian Teacher High Schools, 98 High Schools and 1 Science High School (total of 182 schools) were chosen as CESs and equipped with computer labo- ratory consisting of 20+1 computers. From the 182 schools 73 primary education school, 31 high school, among the 208 CLS the ones with the appropriate physical capacity have been chosen and therefore the number of the computer laboratory estab- lished within the scope of the NEDP has been increased to 235. The CESs and the CLSs have been found to be the favorite among the existing schools in their region. Therefore, there is an increase in the demand of the parents of the students in order to have their children enroll in one of these schools. The central administration and the local administration are conscious about the situation of these schools and are trying to find ways to equip and spread these schools. Especially the voluntary contribution of the parents to this matter is worth appraising. 34. Although the automation system of the computers in the Publication Department and the regional offices of the State Textbooks at the 21 provinces, do not work functionally according to the project, as there shall be changes in the vision of the people, this shall enable the system to work functionally in the near future. 35. There has been no result from two ICBs organized for equipping of the FRTVD within the scope of the project. As a result of 3 tenders for 12 packages only equipment in 5 packages has been procured, and also the most important production equipment constituting this tender has still not been procured. But the Ministry is willing to complete this component of the project. Also the WB is in favor of completing this component of the project by procuring the related equipment under the Basic Education Project (BEP). The procurement procedures shall be done in 2000 and the missing component of this proj- ect is expected to be completed under BEP. 7 36. In order to reach to success in the In-service Training of the Teachers, by the help of the international tender for equip- ping the 208 CLSs and 53 CESs, the ERCs have been equipped in Ankara, Aksaray, izmnir and Mersin. At these centers, not only in-service training for project schools' teachers and principals has been given, but also regular in-service training activi- ties have been continuing. Within the period of the project, also the construction of the National Education Academny, which could not be completed and begin to work, is continuing. With the help of the local administrations, by starting the in-service training for the teachers at least once in 5 years which is anticipated in the project, has helped to quicken the objectives. 37. There has also been a development in the increasing of the efficiency of the Administrative and General Management of the system within the scope of the project. A lot of research and development studies have been conducted successfully by the specialists and by the MONE staff working within the scope of the project technical consultancy services at ETGEM, APAK, MADU and METARGEM. These research and development studies shall continue after completion of the project. The Ministry shall continue to develop and widen these studies. Education Master Plan, establishing of the Terms of Refer- ence for the personal of the Ministry, Total Quality Management and spreading the MIS throughout the country, establish- ment of Decision Support System are some exampl-s of these studies. Continuing the development of these studies is the primaty objective of the Ministry. After the Terms of Reference for Ministry's Units prepared by the Management Assessment and Development Unit (MADU) within the scope of the NEDP have been sent to the State Personal Unit in order to get ap- proval, both the State Personal Unit and the other Ministries took the Terms of Reference as a model for the preparation of their own Terms of References and published them at the Official Gazette with this example, it is easily understood that, NEDP made changes not only in the education system but also in all the State organizations. 38. With the help of the following studies; develop:mg the curriculum, testing and evaluating, developing the quality of the textbooks, increasing the success of the students by spreading the usage of technology within the education, increasing the quality of the teachers by spreading the in-service training and also establishing of country's biggest computer network, preparation of the education master plan and the other studies related to the development of the management, NEDP has given opportunity for the future studies and also by the help of the gained experiences, the MONE has a chance to use them in the future studies. The project schools chosen among the schools within the scope of the Basic Education Project, the above mentioned schools both have been continuing to be equipped just like the equipment within the scope of the NEDP benefiting from the resources of the Loan and the general budget is also another source to be taken into consideration. E- BANK PERFORMANCE 39. Education has been the most effective process within the social development because of its importance in establishing social structure and directly effecting future lives of i:he individuals. That is why countries give the necessary importance to the education and allocate the required sources, and put their investment with great patience in this field because education is a long process, the outcome of which takes too long to get the expected results. The purposes of NEDP have been deter- mined as: (a) to make student success closer to the averages in the OECD countries by increasing the quality in primary and secondary education (b) to make teacher standards similar to the norms and standards in the OECD countries by increasing the quality and accreditation in teacher trading (c) to make it possible to become more economical and more effective in source use by improving managerial and administrative skills and implementations within the MONE. That is why it is the best designed project with respect to elimination of the deficiencies in the education sector. 40. It has always been difficult and sensitive matter to add new functions and institutions to the system. Societies have always resisted against the changes. This is a general rule and not limited with this project. Although it has been agreed to make several modifications in the system with this project, some of these modifications have been realized while some have not. Sources that the project envisaged and the strategy have been insufficient in order to realize the changes. The Bank has monitored and oriented in a satisfactory way with respect to the guidance and assessment matters so as to provide a good personnel attendance and to implement project effectively. When the whole performance of the project taken into considera- tion, preparation and supervising activities can be said to be satisfactory. F- BORROWER PERFORMANCE 41. During 1980s, the Government started implementing structural modification programs in the economy and accelerated the opening of Turkish economy to the international maikets, therefore for the international competition. In order to stimulate export, it is necessary to develop and support the Small and Medium Sized Enterprises. Establishing new technological ca- pacity should be given priority and investment should be done on well-educated, qualified and skilled man power that is able to solve problems, give and carry on decisions. This is necessary to meet the needs of the industry. In the scope of this policy followed, the MONE started to adapt NEDP in 1990 in order to make general education reach at the same level as the OECD countries. For this reason the project has been prepared in accordance with the development policies of the country. In order to carry out the project effectively within the existing system, the MONE, as it is envisaged in the project, has established a Project Coordination Unit under the Undersecretary of the MONE. 42. On the other hand, within the framework of ministry personnel policy, the staff of the PCU have temporarily been ap- pointed with the Ministry approval, responsibilities of these staff have been hindered. However, since 1992, the Unit has been re-structured within itself and a Service-Directive has been prepared. Terms of Reference of the staff have been done in this 8 Directive and appointments have been realized in accordance with these references. The PCU does not have any legal support except for the Loan Agreements. As PCU does not exist in the Law # 3797 dated 1992 about the Organizations and Duties of the MONE, projects implemented under the Loan Agreements should be carried out again by the teachers temporarily ap- pointed with the Ministry approval. All the staff working at PCU, especially the key personnel are temporarily appointed teachers. These staff have been working with great effort and sacrifice though they get nothing extra but their monthly sala- ries. 43. The PCU has established a Project Follow-up Section in order to make effectively use of the equipment purchased under the project. However, as PCU does not exist in the Law #3797, it does not have its own budget, therefore it is difficult for PCU to control the use of equipment in the schools. But, by means of the in-service courses prepared by ERDD and ET- GEM, and the documents delivered during these courses as well as the guidelines prepared by these units have made it possi- ble for this equiprment purchased under the project to be effectively used for training and education. Effectively use of this equipment have also been noticed during the visits realized by the WB to the project schools and this matter has been stated in the Aide Memoires. 44. It has been envisaged that during the project implementations it is necessary to meet the expertise needs of the units under the project within the scope of the Technical Constancy Services and to support them with the training for the fellowship in order to increase the capacity of the project implementations of the Ministry. The idea to employ one native consultants with the foreign consultants has come out for the first time with the NEDP and it has been put into practice. However if some of the Ministry Units receiving consultancy support had provided qualified counterparts for these experts, productivity and the theoretical benefits demanded from this consultancy would have been as it was required. Those who were sent abroad for fellowship training, because they knew the language well, have completed their training in a year and got their "Masters" Degrees. The personnel returned from overseas training have to work in the related unit twice as much as the period they spent for their training abroad. This legal necessity has been applied for all of the NEDP fellowship. But, because of the per- sonnel policy of the MONE, some of these staff have left the units after completing their compulsory working periods. In fact the Ministry should have taken necessary precautions to keep these staff as they had received overseas training and gained the required experience by working in the fields related to the education they had. It will be possible to reach the planned level in the sector and benefit from these well-educated and experienced staff only after employing them in the places for which they had been trained. Training of fellowship first in the country, then abroad costs a lot for the borrower, and that makes it more costly for the government to let them leave (if the position of these people had been upgraded, they would not have left) with- out taking advantage of them. Very few personnel at the PCU (all of whom are teachers) have tried hard to complete the ac- tivities like purchasing equipment, counseling, and also realizing coordination between the units under the project as well as finishing all activities within the project duration. They have tried to economize as much as they can while purchasing the equipment and good quality equipment and services have been purchased with cheaper prices. With a little more of the money of the FRTVD equipment that couldn't be purchased, labs consisting of 20 +1 computers have been established in 182 CES. So it is possible to say that the performance of the borrower is satisfactory. G- ASSESSMENT OF OUTCOME 45. Within the National Education Development Project, it is possible to say all of the physical restoration and equipping objectives (except for the equipping of FRTVD) as well as some of the objectives oriented institutional development have been achieved. Students of the project schools equipped under the project have met computers and started to have better edu- cation by using technology in training and education. These schools have become the example for the other schools in their regions and school management and the parents have been trying to make their schools similar to the CLS and CES. Again within the scope of the MIS, Ministry units under the project have been equipped with the necessary computer supplies. Other Ministry units under the project have been equipped with not only computer and office equipment but also other audio-visual equipment and other necessary software. While the project was being implemented in its original design, with the Loan Agreement amendment made on 29 January 1996, Basic Education Pilot Project (US $ 13.1 Million), Mother-Child Program (US $ 3.3 Million), Vocational Secondary School Study (US $ 0.5 Million) were added to the project. The Ministry has continued to realize necessary studies to be done within the scope of Loan Agreement amendment dated 29 January 1996, besides the objectives defined in original design of the project. General Directorate of Primary Education has implemented Basic Education Pilot Project included in the project with the Loan Agreement amendment. Long term overseas and in- country training realized under the technical Consultancy Services of the personnel National Education Development Project increased the experience and knowledge of the staff on different implementations in the fields they were trained. Moreover, again under the Technical Consultancy Services of the Project, short term study tours organized for the personnel chosen from Ministry units under the project and for the directors of the project schools increased the knowledge and experience of these personnel. Some of these personnel have become successful in transferring the experience they gained to their institu- tions. 46. Mother and Child Program, that was added to the project with the Loan Agreement amendment dated 29 January 1996, was implemented in 18 provinces and 1751 mothers and children were reached, however, during the three and a half years that the program has been implemented within the National Education Development Project the number of the provinces has increased to 59 and the number of the mothers and children has become more than 26.053. For the Vocational Secondary 9 Education feasibility study included in the project with the same amendment a contract has been signed with an American Firm called CORD and within the framework of the data gained through the studies realized in the scope of this project "Con- cept Paper" has been prepared by the Ministry ancl submitted to the World Bank. 47. Evaluation tests prepared by the ERDD under the project have been regularly used and monitored both in project schools and non-project schools. Evaluation and Testing Section of ERDD has implemented pilot tests for the 64,581 students from 5h to I Ih grades in Science, Maths, Social sciences and Turkish. Results of these tests have been submitted regularly to the Bank's Missions along with the Progress Reports prepared by the MONE. A total of 3137 items chosen from 6396 items have been written in these tests. These items have been collected in 44 books. The same Section implemented "Final Tests" to 361,323 students from 5., 8., 10. and 11. Grades of CES and non-CES in fields of Science, Maths, Social sciences and Turk- ish in 22 provinces of 7 regions from May 1995 till the closing date of the project. 48. On the other hand, Development Objectives Assessment Process that has been prepared by the native and foreign experts as well as the authorities working in the Ministry units under the project and approved by the first level superiors has not been pursued and implemented well enough. It has not been yet completed to measure "the development of the quality" resulted during the implementation of the project because the related units have not practiced this performance assessment criteria as required. 49. It is observed that the project has been accelerated during the recent years and disbursement as well as the procurement activities of the project have increased because of this acceleration following the completion of the studies by the consultants worked under the Technical Consultancy Services. Because the equipment purchased has reached schools only towards the end of the project, and success of the students has been affected directly. Teachers and students of these schools have been able to use this equipment any time they want. Students and public have also used this equipment after the school time; and by opening the use of the equipment for the public and organizing courses at the weekends have made it possible for these schools to gain financial support. These improvements will affect the outcomes of the project in a positive way. However, shortening the approval period for the curricula models developed within the scope of curricula development models prepared by ERDD is another matter to improve the success in the schools and should be solved by the Ministry as soon as possible. 50. The second objective of the project, which is to increase the quality and accreditation of teacher training through in- service training has been achieved both in quantity and quality by transferring the authorization of in-service training activi- ties from the central offices to the local ones. Native and foreign experts working in the In-service Training Unit have pre- pared two books named " Being A Teacher With All Aspects" and "Guidelines of In-service Program Development" have contributed a lot to the in-service training of the teachers in a positive way. These books have been printed in a great many numbers with the own sources of the project and delivered to schools and In-service Training Centers. During the imple- mentation process of the project all teachers graduated from two-year teacher training faculties have completed their license completion education program. 51. As the third objective of the project most of the activities have been completed; however activities started within the proj- ect but not yet completed at the closing date of the project and also the activities that were completed before but have to be reviewed because of the 8-year compulsory education have still going on. It is the priority of the MONE to realize TT Net contract to be made between Turk Telekom in the scope of the MIS especially with the funds of Basic Education Program Funds and to start MIS operate actively. The MONE also has been trying to establish of Decision Support System which is second step of MIS. H- FUTURE OPERATION OF PROJECT DELIVERABLES 52. Project Coordination Unit of the MONE has been going on its studies. PCU will be able to make use of equipment pro- vided by the project in effectively by working in coordination with the Ministry units under the project in the project objec- tives direction, also PCU will be able to organize in-service training activities which will be necessary for this. Moreover the PCU will get the failure which occurred within the guarantee period of the products to be repaired and will inform the firms about the failure, and get the firms to repair the failure during the expiration of the guarantee period. 53. In order to increase the outcomes of the project in positive way it is very important to solve the teacher problem in proj- ect schools and to obey the regulations prepared by ERDD when appointing teachers and directors in these schools. In order to provide this it is necessary to work in-coordination with the local management and also with both ERDD and ETGEM. It is also important to obey the regulations about transferring teachers and directors of the CLS and CES from one project school to another. 54. Within the coordination mentioned above, in-service training should be performed in coordination with the In-service Training Department in the Education Resources Cznters equipped with the project sources, and in cases where these centers cannot meet the needs ( during the in-service traini ig activities organized by local administrations), works to meet in-service training needs regularly by using project schools as in-service training centers. Employing teachers received overseas and in- country training as formator teachers in these centers will increase the benefits of the in-service training. 10 55. It is necessary for the related units to add sufficient appropriations into their budgets in order to realize the activities mentioned above and to supply the schools with the additional education materials they may need. On the other hand, the MONE will take into consideration all reports and deliverables which have been prepared by the MONE's staff and consult- ants provided under the TA contracts of NEDP in its future operations. As a sample; a copy of OSU's consultancy services reports has been received by ERDD from the PCU. 1- LESSONS LEARNED 56. National Education Development Project has been planned as a social sector project aiming to develop the quality of the Turkish National Education and to improve the central and local management of education. In order to realize the project objectives studies have been done in 443 primary and secondary schools in all the provinces (80) as 15 General Director- ates/Head of Departments and as Local Offices at the Central offices of the Ministry. 57. A great deal of experience has been gained through these studies, besides it will be useful to take this experience into consideration for the future planning. Therefore, both the creditor and the borrower will search ways to prevent from wasting time and source in the projects to be implemented in the future by aiming to get maximum profit and by taking negative and positive aspects gained from the project into consideration e.g. the MONE has been trying to establish of Decision Support System related to the MIS, and to use the ILSIS system effectively by using the Basic Education Program's sources, and the MONE has been trying to allocate the necessary fund from the Basic Education Program in order to make an agreement with Tuirk-Telekom which allows to use the TT-Net line for ILSIS. 58. Project Manageme t: National Education Development Project has been realized following a-nine-year implementation project done by the project implementation team and project directorate formed by the Project Coordination Unit established with the approval of the Ministry; but several problems have been experienced during the implementation of the projects and establishing a coordination. Because of these problems, it seems necessary to establish of a legal Project Implementation unit, to account the government supported budgets under this unit, to identify the Terms of Reference of this unit as well as the staff working within this unit legally and to include the responsibilities of the other units under the project in details for the projects to be implemented with the loans from the World Bank. Besides the establishment of a project executive council formed by 8 (eight) people who will observe the project implementations and review them by meeting at least once a week will make these implementations more productive. Moreover, it is also important for the unit responsible for the project im- plementations to have the authority on the budget that the project activities envisage. It will be possible to reach an effective and productive conclusion with the project management and coordination for which these provided. 59. Since NEDP foresees some institutional changes (establishment of new unit, making some laws etc.) and as these institu- tional changes are important factors to achieve the aims of the project, it will be harder because of bureaucratic difficulties to reach the project aims. These difficulties have been experienced in practice by both the units of the Ministry which are within the context of the project and by also the Projects Coordination Unit which coordinates the projects among the related units. 60. Technical Consultancv Services: It has been envisaged that the studies realized by the consultants to be employed in the fields determined as a need for the sectoral development should be prepared with more details under the project. It is neces- sary to define the Ministry officials to be employed as counterparts according to the Terms of Reference to be prepared. These staff should be prepared through in-service training and should not be put in any other charges during their mission if necessary. This condition is important if the Ministry should get the expected results from the project. It will increase the productivity if well-educated staff that will able to work continuously are recruited in the units benefiting from native and foreign experts within the scope of the Technical Consultancy. 61. Overseas Fellowship Training: As the cost of abroad training of fellows is too expensive, the objective of this training should be determined more carefully in SAR. Choosing the staff to be trained is also important. Staff who will apply for planned activities within the related units should be sent for overseas training. Especially with the insufficient language knowledge, the training to be obtained abroad will not be good enough. For the overseas training of the staff, good quality and high level courses should be organized and following these courses a limit should be decided. Those who can pass this limit should be sent abroad for training and if needed, those who cannot pass should be trained by receiving services from the universities. By doing this, sources of the country would not be wasted. For this reason, it is very important to specify the qualities that the staff should have and the scope of the overseas training in parallel to the project objectives. To detail these in the project will make the fellowship training more productive. 62. Procurement Activities: In order to reach the objectives defined in the project, seven (7) international competitive bid procedures have been realized under NEDP. When equipment tenders of the Film Radio and Television Education Center that have been done three times are taken into consideration, the number of ICBs realized under NEDP becomes nine (9). Techni- cal Consultancy Tenders are not included in this number. In the scope of the project, Technical Consultancy Contracts, except for the individual consultancy contracts, have been signed between the Ohio State University, Bilkent University and Anadolu University. Furthermore, in order to equip ministry offices under the project a lot of equipment has been purchased through national bidding or direct purchase. Using technology in general education has been put on the agenda extensively for the first time with the NEDP. Almost all the equipment purchased under the NEDP has been used in project schools and ministry units under the project in accordance with the objectives of the project. 11 63. On the other hand, for the project implemented after the Loan Agreements, the Borrower has to fulfill the commitments arousing from the regulations and laws in his own country, the Borrower also has to complete all the procedures of the credi- tor in purchasing goods and services. Because of these, at least six months of time is wasted when doing each procurement for the project implementations. To remove this condition will accelerate project implementations and will prevent source and time from being wasted. 12 Appendix: C YOK IMPLEMENTATION COMPLETION REPORT REPUBLIC OF TURKEY NATIONAL EDUCATION DEVELOPMENT PROJECT PRE-SERVICE TEACHER EDUCATION COMPONENT Loan No. 3192-TU Evaluation Summary Project Objectives 1 The project development objective was to contribute to the improvement of pre- service teacher education at primary and secondary levels. It did this by (a) upgrading the facilities of all faculties of education, (b) curriculum development and materials production in the teaching of the major school curriculum areas, as well as development work on student- teacher experience in schools, (c) the provision of long-term fellowships for those beginning a career in teacher education and short-term fellowships for those in mid-career and management, (d) setting up a system of faculty-school partnerships to assist student-teacher work in schools, (e) developing and agreeing standards in teacher education and setting up an accreditation system. The last two objectives were those of the extensions. Implementation Experience and Results 2 It is YOK's opinion that this has been one of the more successful World Bank projects. During the life of the project, there was general consensus that it was making good progress, and qualitatively as well as quantitatively fulfilling its objectives in a well-planned and timely manner. The report, while discussing issues and areas of concern nevertheless presents project accomplishments in a positive light. Evidence for this opinion is summarized in the report. It may also be found in World Bank Aide- Memoires, in the Quarterly Reports and the Evaluation files, and in other reports and communications. 3 There were considerable delays in the start of implementation. The World Bank identification missions in 1988 were followed by preparation and pre-appraisal missions in 1989. The Staff Appraisal Report and the Loan Agreement are dated April and May 1990. Project implementation did not then proceed as planned. It was delayed until December 1994 because of the cancellation of the Technical Assistance (TA) bidding process in April 1992, followed by delays of nearly one and a half years as amendments to the TA contract were negotiated at the request of the Ministry of Finance. The TA contract was signed on October 3, 1994. 4 Following the establishment of the TA office on December 1, 1994, and the start of the work of the expatriate and national consultants and teams, implementation of the curriculum objective (b above) proceeded smoothly. Project books were published embodying the new approach, and four series of training workshops were sequenced with pilot activities and revision of materials. 5 The fellowship component was faced with problems of timing, and the location and language training of suitable fellows, but 42% of the masters quota and 84% of the doctoral quota were achieved. To meet developing project needs, fellowship funds were then redirected to the provision of 222 short-term fellowships, itself a major factor in sustainability. The short-term fellowships were for six groups of people: in 1997, 88 deans and department heads, then 45 rectors and YOK members had short study tours, and 51 people had a 10-week training. In 1998, there was a 4-week training for the Partnership Working Party and a study visit on accreditation for the National Committee. In 1999, the Accreditation Working Party had a 10-day study visit. 6 Procurement activity was slow in starting, due to the lack of a procurement specialist within the PCU for the first ten months of the project. However, progress, albeit slow, was made during these early months in planning and drawing up lists of equipment. An expatriate consultant was employed to assist with the essential steps in procurement in its mniddle stages. Procurement was completed by the end of the project. 7 All these three project cornponents were widened early in the project from the original planned seventeen project faculties of education, to embrace all 34 faculties. Eight new faculties of education were then included in 1997-98. They received some equipment, and other project benefits. 8 The two major extension objectives of faculty-school partnerships, and teacher education standards and accreditation were achieved in full. A national system of faculty- school partnerships was set up after nation-wide training courses in 38 faculties in 1998, run by the Partnership Working Party. T7hey had previously piloted the course, and completed a book giving full guidelines for setting up partnerships. The standards and accreditation work formed the subject of an international and national conference. The proposed approach to accreditation was successfully piloted in six faculties in February 1999, and a comprehensive book on standards and accreditation published as the basic reference for future accreditation activity. 9 The third extension work of' a distance-learning certificate program in computing, organized and partly held at Middle East Technical university, was also successfully accomplished. 10 One of the major factors affecting the project was the fact that it ran parallel with a major national change in teacher education. In 1996, YOK constructed and established a new system of teacher education, which tDok effect from the start of the 1998-99 academic year. The restructuring successfully incorporated the project's ideas, publications and existing fellowships. The terms of reference of the project's new short-term fellowships included the needs of the national developments. A second factor in the success of the extension work, particularly in faculty-school partnerships, was the increasing cooperation between YOK and the Ministry of National Education (MONE). 11 Difficulties encountered in the implementation of the curriculum work included the lack of counterpart Turkish colleagues to work with the foreign consultants. A further problem in a project where books have to be translated was language. This was also a factor inhibiting the full understanding of some of the participants of the short study visits. The final major factor considered in the text is the kind of participation PCU members in full-time jobs are available to give, and the need therefore for effective TA management. Sustainability 12 The project is well-placed wilh regard to sustainability. The capacity of the faculties of education has been significantly increased. This has occurred both through improvements in facilities and equipment, and thrcugh institutional strengthening by fellowships, course content and training. The short-term fellowships were designed with sustainability in mind, and the masters/doctoral fellows will return to be distributed across almost all faculties, offering new and up-to-date ideas to merge with the new developments in teacher education here. 13 Further, over 3,600 academic staff from all faculties cooperated in the curriculum development and partnership work, and worked with the project. Many of them wish to continue the work after the end of the project, and are committed to the developments. The accreditation studies of 1999 involved all deans and senior colleagues, plus 366 people from all faculties in the dissemination courses, and the training of 23 assessors. This work will continue as an accreditation system is set up. 14 Finally, and perhaps most importantly, project implementation has gone hand-in-hand with national developments. These changes are now established, and development will continue under YOK, thus ensuring the continuity of the work done under the project. Future Operations and Key Lessons Learned 15 The Project Director became a member of YOK Executive in May 1998, and has overall responsibility for education. In addition, YOK has set up a National Committee for Teacher Education. The future work in teacher education will be led by them, and YOK has comnnitted itself to sustain and expand the achievements of its project. YOK will also continue its co-operation with the Ministry of National Education. The Ministry representation on the National Conmmittee has been increased from one to four key people in the Ministry. 16 The principal lessons learned during project implementation have been that project resources combined with major developments, and with YOK/MONE cooperation, lead to success. Further, that building on a project's achievements, as was done in the project extension work of faculty-school partnerships and standards and accreditation, enables nation- wide contacts to be maintained, participants of short-term fellowships to be utilized, and mounting motivation and comrnitment to be hamessed to further progress. MAP SECTION lBRD 315501 'B-UtGARIA(UNT ociisopsies4 S e a 2 tlesostuoloso.Maocsadstsos '~~~~~~~~~~~~~~~GEORGIA--- >o 'F ~~ eK$kkMt B la c k U r Hssss bnocsianychsscps(additioIs e a GREECE ) ~~~~~~~~i' (.I ZosAoc5 7 0 os:l5pcicncncy$ycN 1 r4QosamcL B6 h-4~ ,79A2s '. APOascsm / -<> !qK~~1a6IA r - GREECEa~ -~~ N ,yN'in ,iL' (O4ucocaroGemini '~~f li.8tT~ %0kAc~a /t-ua-¾'4' Odo TrakiOo Mau~~~ ~ ~ ~ 2U7J1/31 iCaca ~ ~ ~ ~ ~ ~ '-xffl. l®t/h-®. -~~~~~~ / I Joule '>~~~~AT-b- &k ~~ ~ ~ janSor' 1Z ¸7--Ae,nsyat > -~~~~~~~ ¾'~~ ~ j ®~~R/ Tk,.g, yuO9~ 5- I- kOran C> ,lH i2 ~ J7Y 0fAsponri ~ ~ - ~ I JuiARMNI ,%A~~ la,' ®K-i> ~ F477 (64 j A r ~ ~ ~ ~ ~ '-1 .4 ,-, , , ~~~~~~~~~~IRAN 240 J- S)~~~~~~~~¸ /kss ~m -- 2 1 1 ~ j 02 '~~~~~~~~~ 121 ~~~~~~~~~~ ) I? Ti C'> fl~~~~~~~~~~~~~~~~~~~~17 1®M oHty ' ' \' MOXI G61 ~ 'Iu 34 OX>@h Nigd. u U 2 K-- I G.--.i-:st OuT l AI1y /I057$ RY A0 irii Xm /. 4,1 _vJ T U R K E Y~~~~~O' iT 2 2 -Jut Sea -, 5 ] ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~MINISTRY OF NATIONAL EDUCATON' [MONO): Mediterranean ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~~~I SYRIAN U~~~~~~~~~ 208 CURRICULUM LABORATORY SCHOOLS® PRVNECITL' S I ARAB REP. 0 232 COMPUTER LABORATORY SCHOOLS 35ATOALCPIA 05'-,> - , I . -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~-- RIVERS~~~~~~~~~~~~~~~~~RVR f ~~ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~A EDUCATION RESOURCE CENTERS Thssspaasd-cebyt$s*RqSsisvs,toSThWaI RaisE YPRU K N--.d----t PROVINCE BOUNDARIES" th.l'c4sris~cdoca,dsomhiasionsns4ocy-tIuseisRqpPioHonshos Numbsrs ad jocaCOtoCCOUNCIL HIRHHIGERUECUCATN'N (YOK) cosisis sop sic coticcp& ontls.pastoEThswosldOackGroapm an~~~~~~- ~~ T90 ~o ~c CYPRUS U symomis educate the -m - INTERNATIONALATIOALOBNUNDARIE io4nsctc0ItssspItatpnls~Pscitcsasosync4sssmstn KLMETnERS ~ quatity at -hc 1ocoto., fACULTIES OF EDUCATION 05' s~~~~~~~~he p-oinc capitols,5 1010 2000