Document of The World Bank Report No: 27696 - TU IMPLEMENTATION COMPLETION REPORT (SCL-43550) ON A LOAN IN THE AMOUNT OF US$300 MILLION EQUIVALENT TO THE REPUBLIC OF TURKEY FOR A BASIC EDUCATION PROJECT IN SUPPORT OF THE FIRST PHASE OF THE BASIC EDUCATION PROGRAM JUNE 24, 2004 Human Development Sector Unit Europe and Central Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 9, 2004) Currency Unit = Turkish Lira Turkish Lira 1 = US$ 0.000001 US$ 1 = Turkish Lira 1,466,472 FISCAL YEAR January 1 December 31 ABBREVIATIONS AND ACRONYMS APK Justice and Development Party APL Adaptable Program Lending BEP Basic Education Project CAS Country Assistance Strategy EGITEK General Directorate for Educational Technologies ERDD Education Research and Development Directorate EU European Union GoT Government of Turkey IBRD International Bank of Reconstruction and Development ICT Information and Communication Technology MoF Ministry of Finance MONE Ministry of National Education NGO Non-governmental Organization OECD Organization of Economic and Co-operation and Development PCC Project Coordination Center PISA Program for International Student Assessment TOR Terms of Reference Vice President: Shigeo Katsu Country Director Andrew N. Vorkink Sector Director Charles C. Griffin Sector Manager Maureen McLaughlin Task Manager: Ferda Sahmali TURKEY BASIC EDUCATION PROJECT APL I CONTENTS Page No. 1. Project Data 1 2. Principal Performance Ratings 1 3. Assessment of Development Objective and Design, and of Quality at Entry 2 4. Achievement of Objective and Outputs 6 5. Major Factors Affecting Implementation and Outcome 13 6. Sustainability 15 7. Bank and Borrower Performance 16 8. Lessons Learned 20 9. Partner Comments 21 10. Additional Information 21 Annex 1. Key Performance Indicators/Log Frame Matrix 23 Annex 2. Project Costs and Financing 25 Annex 3. Economic Costs and Benefits 28 Annex 4. Bank Inputs 29 Annex 5. Ratings for Achievement of Objectives/Outputs of Components 32 Annex 6. Ratings of Bank and Borrower Performance 33 Annex 7. List of Supporting Documents 34 Annex 8. Beneficiary Survey Results 35 Annex 9. Stakeholder Workshop Results 41 Annex 10. Borrower's Evaluation Report 43 Project ID: P009089 Project Name: BASIC ED Team Leader: Ferda Sahmali TL Unit: ECSHD ICR Type: Intensive Learning Model (ILM) of ICR Report Date: June 25, 2004 1. Project Data Name: BASIC ED L/C/TF Number: SCL-43550 Country/Department: TURKEY Region: Europe and Central Asia Region Sector/subsector: Primary education (83%); Tertiary education (11%); Central government administration (6%) Theme: Education for all (P); Rural services and infrastructure (P); Social analysis and monitoring (S); Gender (S) KEY DATES Original Revised/Actual PCD: 02/04/1998 Effective: 08/12/1998 08/12/1998 Appraisal: 05/05/1998 MTR: 05/31/2000 04/12/2000 Approval: 06/23/1998 Closing: 06/30/2001 12/31/2003 Borrower/Implementing Agency: REPUBLIC OF TURKEY/MINISTRY OF NATIONAL EDUCATION Other Partners: STAFF Current At Appraisal Vice President: Shigeo Katsu Johannes Linn Country Director: Andrew N. Vorkink Ajay Chhibber Sector Manager: Maureen McLaughlin James Socknat Team Leader at ICR: Ferda Sahmali Michael T. Mertaugh ICR Primary Author: Hong Chen 2. Principal Performance Ratings (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible) Outcome: U Sustainability: L Institutional Development Impact: N Bank Performance: U Borrower Performance: U QAG (if available) ICR Quality at Entry: S U Project at Risk at Any Time: Yes 3. Assessment of Development Objective and Design, and of Quality at Entry 3.1 Original Objective: In 1973, the Basic Law for National Education (Law 1739) redefined basic education as comprising eight years of compulsory schooling. However, the Government was unable to enforce this mandate for compulsory education mainly because of the lack of facilities and teachers. In 1983, Law 1739 was amended requiring enrollment wherever the physical infrastructure was in place. Throughout the 1990s, there was a growing public interest in expanded basic education. In 1995, the Government requested the Bank's assistance in preparing a project to support universal basic education. In August 1997, Parliament approved a new Basic Education Law (No. 4306) which established the timetable for implementing universal eight-year basic education. Most importantly, this law provided major budget resources in the form of taxes earmarked specifically for the investments necessary to achieve that goal. The new Government's expanded basic education strategy was articulated in its action program, which received a high level of private and voluntary contributions. A major expansion of school capacity was needed to reach the goal of expanded universal basic education and reduce crowding in existing schools. The constraints included inadequate classroom capacity and shortage of teachers. Parental demand for education in rural areas in the East and the Southeast appeared to be weak due to the low quality of teachers in their villages and the cost of sending their children to school. The Government's basic education program was to address the issues of raising quality, provide material incentives to children from poor households, improve incentives for teachers in rural schools, promote school-based activities to make basic education schools a resource of the community, and expand preschool education. The Government's program was to be implemented in three phases: launch, consolidation, and subsequent phase. In the policy letter sent to the Bank in 1998, the Government stated that the expansion of the five-year compulsory cycle to eight years was a centerpiece of its 7th Five-Year Development Plan (1996-2000). Government also reiterated its strategy to achieve universal attendance in eight-year basic education and improve significantly the quality of basic education. There was substantial confusion over the project development objectives (PDOs) as the Loan Agreement had a set of objectives for the basic education program and the project. However, various sections within the Project Appraisal Document (PAD) had inconsistent sets of goals for the project. For the purpose of this ICR, the original PDOs are those from the original Legal Agreement: (a) the expansion of the capacity of basic education schools; (b) training of teachers, school principals and inspectors; (c) development of central and provincial implementation capacity to carry out the program; and (d) creation of a mechanism to monitor and evaluate the outcomes of the program. The original project objectives (DOs) reflected the priorities of the Government at the time of project preparation. They were consistent with the Bank's Country Assistance Strategy (CAS) (September 1997) which called for increased administrative resources aimed at social development and poverty alleviation. The CAS advocated shifting the focus of investment activities to poorer regions. It stated clearly: "Our highest priority in education is to help the Government achieve its goal of universal schooling through the eighth grade". The Letter of Sector Policy sent to the Bank in April 2002 indicated the following goals of the Ministry of National Education (MONE) in basic education: increase school enrollments to 100%; improve the quality of education; reduce the number of students per classroom to 30; eliminate gradually the practice of combining classes in rural schools; and end double-shift teaching. The Bank's current CAS for Turkey (October 2003) for FY04-06 included key education outcome indicators: net enrollment rate in basic education maintained at not less than 99%; increased girls' enrollment, - 2 - especially in secondary education; and male/female ratio improved from 1.20 to 1.15 by 2006. The development objectives of the Government and the APL1 were neither clearly nor consistently presented in the main text of the PAD and the Annex 1 of the PAD. There were also inconsistencies when the PAD was compared to the Legal Agreement. There was a lack of clarity on the DOs for the APL1. The PAD outlined the Development Objective for the Phase I of the Basic Education Program (establish capacity to implement, monitor and evaluate the Program, initiate key investments to achieve Program objectives), but did not indicate the DOs for APL1. Even if these were clarified as the DOs for APL1, they seemed inappropriate as they tended to be more of an input than outcome nature. The logframe gave the key performance indicators for the program objectives, but only the trigger actions were listed for the key performance indicators for the project objectives. There were no clear measurable outcomes and related indicators for APL1. The implementation arrangements of the project required well-coordinated and trained MONE officials, both at the central, provincial, subprovincial and school levels. School construction, renovation, and upgrading under the project were spread over the whole country. The accountability and the competence of the implementation agencies were high risk areas for the project. The project design had appropriately put in place a set of risk minimization measures. The most significant risks were: cooperation among various elements of the MONE; lack of complementary inputs (such as training or educational software); excessively high expectations established by the Government's Basic Education Program that exceed the Ministry's implementation capacity; and delay in procurement of IT hardware and software that could delay the delivery of computer instructions. 3.2 Revised Objective: Due to the needs of the emergency created by the Marmara earthquake of August 1999, the Legal Agreement was amended in December 1999. While maintaining the broad objectives of the first phase to support the borrower's program, the second paragraph of Schedule 2 to the Loan Agreement was amended as: (a) the expansion of enrollment in basic education schools; (b) training of teachers, school principals, and inspectors; (c) development of central and provincial implementation capacity to carry out the program; (d) creation of a mechanism to monitor and evaluate the outcomes of the program; (e) reconstruction and rehabilitation of educational facilities for preschool, basic education, and secondary education which were destroyed or damaged in the Marmara earthquake of August 1999; and (f) resumption of preschool education, basic education, and secondary education in the area affected by the Marmara earthquake of August 1999. Since this was a formal amendment to the Legal Agreement, it was reported to the Executive Directors. This amendment reallocated the Proceeds of the Loan to make resources available for Marmara earthquake needs. It also changed the first objective from "expansion of the capacity" to "the expansion of enrollments". The amendment of November 24, 2000, which included the changes to some of the components and the reallocation of loan proceeds, was reported to the Executive Directors. Although it was not the appropriate document to indicate a formal change in the project development objectives, the Project Status Report (PSR) of April 2003 changed the project development objectives to read: a) the expansion of enrollments in basic education schools; b) training of teachers, school principals, and inspectors; c) improving basic education quality by providing information, communication and technology (ICT) classrooms, educational materials to schools; d) development of central and provincial implementation capacity to carry out the program; and e) creation of a mechanism to monitor and evaluate - 3 - the outcomes of the program. 3.3 Original Components: The Loan contained four components: (1) Expanded Basic Education Coverage (cost: $48.4 million): This component was to carry out: a) teacher training, recruitment, and deployment; and b) school construction, extensions, and upgrading. There was no loan financing used under the first APL for teacher training, recruitment and deployment, although an extensive program of in-service training was provided under the second component. The first APL would finance the rehabilitation and related architectural and engineering design work of about 200 village schools, 100 central village schools, and 40 urban schools in the provinces of Sirnak, Van, Mardin, Hakkari, and Bitlis. (2) Improved Basic Education Quality (cost $233.5 million): This component was to finance the upgrading of information technology (IT) classrooms and provide hardware and software for 2,828 IT classrooms throughout the country. The first APL would support in-service training for teachers, inspectors, and school principals. It would provide essential teaching and learning materials to rural schools. The Loan would also support the development and implementation of assessments of learning outcomes. (3) Program Implementation Support (cost $4.8 million): This component would provide support to the Program Coordination Team (PCT) with consulting services, study tours, and office equipment. It also aimed to strengthen educational management information systems, including school mapping. (4) Monitoring and Evaluation (cost $13.2 million): This component would assess the outcomes of the program against several categories of performance indicators, and provide continuous feedback to the Program Coordination Team (PCT) and the related line units of the Ministry to improve the effectiveness of the program in meeting its objectives as implementation was underway. This component also included support for a Monitoring Response Facility (MRF) to support school-based initiatives, and public media efforts promoting the Basic Education Program. The components were clearly linked to the PDOs. The project supported the top priorities outlined in the Government's program. It targeted the activities to assist the rural and poor urban schools. The component activities were the necessary actions to achieve the development objectives. The lessons learned from computer laboratory schools components under the National Education Development Project (NEDP) were taken into account during the design of APL1. The project also incorporated experiences from UNICEF's program on rural education. It introduced a school improvement program MRF, which was considered innovative at the time. The design of monitoring and evaluation was extensive. Despite all of these positive steps, however, the four project components posed a strong challenge to the Government's ability to implement. 3.4 Revised Components: To address the emergency needs created by the Marmara Earthquake, the Loan Agreement was amended in December 1999 to add a component, Part F: Marmara Earthquake Relief Component. This new component was mainly to: 1) reconstruct and rehabilitate preschool, basic education and secondary education facilities which were damaged by the earthquake; 2) provide educational materials and student support in the form of textbooks, educational materials, - 4 - school uniforms, basic clothing and nutritional supplements to student affected by the earthquake; 3) support homeless teachers by providing basic clothing, furniture and basic furnishings for temporary accommodations; 4) contract transport services to transport students and teachers to temporary schools; 5) provide technical assistance and consultant services to develop and provide psychological counseling and training to affected students and teachers; 6) provide consultant services and studies to monitor and evaluate the emergency recovery actions and their consequences; and 7) provide consultant services for the design and supervision of construction, rehabilitation, and retrofitting of damaged schools. The activities for extensions to about 40 basic education schools in Ankara province were deleted from the project. In the November 2000 Loan Amendment, Part F component was revised and reduced to only three types of activities: 1) contracted transport services to transport students and teachers to temporary schools; 2) provision of educational materials and student support in the form of textbooks, educational materials, schools uniforms, basic schooling, and nutritional supplements to students affected by the earthquake; and 3) consultant services and studies to monitor and evaluate the emergency recovery actions and their consequences. The amendment also reinstated the 40 Ankara basic education school extensions, which were deleted from the project in December 1999 in order to make resources available for earthquake recovery activities. The amendment added the rehabilitation of about 1,300 formerly closed rural basic education schools in provinces throughout Turkey and 11 basic education boarding schools in provinces throughout Turkey. The Loan was able to finance these reinstated and new activities because some of the earthquake recovery actions planned for Loan financing were carried out with grant funds or other assistance. In addition, the amendment added a subcomponent under the Improved Basic Education Quality Component: Expansion of the Mother/Child Preschool Education Program and strengthening of the MONE's capacity to implement the program through training of MONE staff and provision of instructional materials. 3.5 Quality at Entry: The ICR performance rating for quality at entry is unsatisfactory. The strength of the project design was that it reflected Government's priorities in the sector. There was also a high degree of ownership by the counterpart at the time of project launch. The Bank incorporated what were believed to be successful lessons from the previous National Education Development Project (NEDP), i.e. introduction of ICT into the curriculum. It also built on the available experiences from the UNICEF work in Turkey. Nevertheless, the PAD did not define clearly the development objectives. There were no measurable outcomes stipulated for APL1. This caused difficulties in monitoring activities. It also did not allow meaningful supervision by the implementation agency and the Bank. This was partly due to the fact that Turkey APL1 was one of the first APLs ever prepared in the Eastern Europe and Central Asia (ECA) region. The APL approach had just been introduced in the Bank in FY98. The logframe approach was still at the early stage of development. A Quality at Entry Assessment was conducted for the project in 1998. The overall Assessment of Quality was rated as marginally satisfactory. The report rated the project satisfactory for the project's concept, objectives and approach, environmental aspects, and poverty and social aspects. The report confirmed the national ownership for the project. It stated that the pilot activities under the NEDP and the - 5 - involvement of UNICEF were helpful to the design of APL1. However, the Assessment expressed the concern that "the Bank's financial activities focused on underserved regions, but almost 50% of Bank finances was for computers which were more likely to make economic sense in better served areas." The ratings were marginally satisfactory for technical and economic aspects, institutional capacity analysis, risk assessment and sustainability, and Bank inputs and processes. The Assessment stated that the project used a relatively simple analysis, even for the computer component of the project, to justify the technical soundness of the project/program. Options were not examined carefully regarding boarding, bussing, and double-shifts schools. There were no alternative designs seriously considered by the project. The preparation team disagreed with this finding and indicated they had urged the government explore options. On the quality and coherence of the economic rationale underpinning the project, the Assessment found that the micro justification was satisfactory, but that it was not helpful in choosing among options. The macro analysis was found to be lacking. On the institutional capacity aspects, the Assessment commented that the project built on the knowledge of the institutional framework in the past, but did not develop the framework for the implementation of the construction work. There were high expectations in the project design, but the triggers assumed low performance. In addition, the project did not attempt to strengthen performance capacity. On the risk assessment and sustainability, the Assessment indicated that there were no actions included in the design to minimize construction and especially recurrent cost risks. It mentioned further that the arrangements for monitoring to ensure timely implementation were still to be defined. Bank's inputs and processes was assessed as marginal satisfactory, adding that disbursements may be slow given the status and the preparation of the project. The Assessment rated the APL1 unsatisfactory for financial management aspects and readiness for implementation. It commented that project financial management concerns were not addressed in accordance with OP/BP 10.02 during project preparation. As a result, the limited financial management information available would hinder efficient and effective project management. Prompt attention to improve financial management arrangements was needed, particularly under the decentralized operations of this project. It stated that only strong political will would offset the large financial risk. There was no major cost recovery feature incorporated into the project design. On the readiness for implementation, the Assessment commented that the project attempted to improve the quality of the government program, but it had no project implementation plan and no planned procurement packages. 4. Achievement of Objective and Outputs 4.1 Outcome/achievement of objective: The assessment of the outcomes is based on the revised development objectives in the Amendment to the Legal Agreement dated December 1999, as stated in paragraph 3.2 of the previous section. Assessing these objectives also makes sense because the Government expressed in its Letter of Sector Policy in 1998 the objectives of universal attendance for an eight-year education and significant improvement of the quality of basic education. The overall outcome of the project is rated unsatisfactory. The first project development objective, the expansion of enrollments in basic education schools, is the only development objective this ICR rates satisfactory. The expanded capacity of basic education schools contributed significantly to the increase of enrollment. Government built 103,983 new primary education classrooms during 1997-2001, bringing to a total of 265,000 classrooms. This created additional capacity for more than three million basic education students. The number of teachers recruited for primary education schools since 1997 amounted to 103,692. The teacher-student ratio dropped to 26 in 2003 from 33 in 1996. The net enrollment rate for the eight-year primary education program was raised - 6 - from 85.63% in 1997 to 96.3% in 2003. The number of female students was 4,085,452 in 1998 and increased to 4,828,276 in 2002-2003 academic year. The transition rates from 5th to 6th grade reached 100% from 1998/99 to 2003/04 school year, except the rate was 96% in 2002/03 academic year. The schooling rate for pre-school education was 13.1% according to the 2004 data from MONE. However, these achievements were mostly attributed to the Government's effort, and the Bank's APL was only a small part of the larger program (see details below in Outputs by Components). A qualitative study conducted by a local consultant indicated shortcomings associated with this development objective. Absenteeism, varied between 1% to 10% throughout the provinces and it was about 10% higher among female students than that of male students. The bussing program introduced by the Government's Basic Education Program actually contributed to the increased class size in larger villages and city centers, and double-shift education increased from 16% of schools in 1996 to 22% of the country's 36,064 schools in 2000. In 2003, there were 8,532 schools practicing double shifts - 4,410 in urban areas and 4,122 in large villages. The bussing program had adverse effects such as female students attending schools further from home and parents being isolated from their children's' schools. On the second project development objective, training of teachers, school principals, and inspectors, the rating is unsatisfactory. The potential, critical in-service training originally planned for APL1 did not take place under the project. Major training activities were carried out under the Government's budget. The Bank did review the training course materials at the beginning of the project, but APL1 did not attempt to review the effectiveness of the training conducted by the Government after the project failed to spend the allocated $31.2 million in the first year of implementation. Out of the $31.2 million planned for the in-service training programs, only $3.7 million were spent. See details of in-service training carried out under APL1 in section 4.2, component 2, subcomponent 2, below. The third project development objective, development of central and provincial implementation capacity to carry out the program, is rated unsatisfactory. At the end of APL1, MONE's capacity to implement the project, as well as the program remained weak. Program coordination continued to be inadequate toward the end of the project. Communication and collaboration within the units of MONE had been problematic. There was no effective information flow within the Ministry regarding project, and also the program activities. The planned education management information system was not implemented under APL1. At the provincial level, there was limited capacity building in terms of promoting and coordinating education work. Schools and communities were not able to actively participate in the plans for them under the project, except for the MRF activities. The fourth project development objective, creation of a mechanism to monitor and evaluate the outcomes of the program, is rated unsatisfactory. The achievements of this objective were extremely limited. The evaluation to provide essential feedback to make course corrections and respond to changing circumstances did not take place under APL1. There were many assessments and studies included in the original project design (about 11 studies). At the request of the Government, the number of studies was later reduced to four, with Child Labor in Turkey already completed at the time. Out of all the assessments and studies, the project financed only three studies, a) cost analysis, b) social impact, and c) institutional impact of bussing schools, YIBO/PIO schools, and traditional schools; these studies were conducted after long delays. Under the APL1, MONE failed to set up a mechanism to monitor and evaluate the outcomes of the project and the program. The Ministry did not coordinate the monitoring and evaluation activities within the relevant units in MONE. The Research, Planning and Coordination Council (APK) was not involved in the project in a substantive manner. The Education Research and Development Department (ERDD) was the main unit to prepare and implement student achievement assessment tests throughout the country, but it had little cooperation with APK. Limited monitoring and evaluation of project activities was - 7 - undertaken by a small unit within the PCC, and there was no collaboration with APK nor ERDD. This lack of overall coordination undermined an effective monitoring and evaluation mechanism. As part of the monitoring and evaluation activity, the Monitoring and Response Facility was to provide the flexibility to respond to development needs at the school level. Although positive outputs were generated from the MRF projects, the subcomponent activities took a long time to launch. In February 1999, the guidelines for the MRF were finalized. Only in December 2001, six months after the original closing date, did the PCC and the Commission established at the MONE approve 81 projects for implementation. Failure to develop further the idea underlying this subcomponent through effective communication between the Bank and the MONE and capacity building at the local and school levels severely delayed implementation and limited use of allocated budget. This negatively affected the Government's view towards similar activities in subsequent cooperation with the Bank. The Government requested the elimination of such activities in a subsequent APL2 Loan, stating that it was not worth the additional effort. After many follow up discussions, the Government expressed the interest of adding school improvement activities to APL2. The project objectives of (e) and (f), which were related to the Marmara earthquake, are not being rated as some of the earthquake recovery actions planned under the APL1 were carried out with grant funds or other assistance. Concerning the achievement of the development objectives for the basic education program, the ICR provides the following assessments. The assessment took into account that these objectives were actually set for the whole APL period and the 5 1/2 years of implementation of APL, which was more than half of the planned APL program. The overall outcome of the program is rated unsatisfactory. The three program development objectives for the basic education program were: (a) to achieve universal coverage in an expanded, eight-year basic education cycle; (b) to improve the quality and relevance of basic education; and (c) to make basic education schools learning resources for the communities. On the first program development objective, the assessment is similar to the assessment of the project development objective (the expansion of enrollments in basic education schools) and is rated satisfactory. On the second program development objective, there was not enough evidence to substantiate the achievement and is rated unsatisfactory. In addition to the points made in the previous assessment of the project objectives, particularly the assessment of the second, third, and fourth project development objectives, the following can be added. As shown from recent international assessments, such as TIMSS and PIRLS, Turkish students were still well behind most of their counterparts in student learning. Computer equipment was provided under the project to 2,802 ICT classrooms and ICT formators and coordinators were trained, but insufficient computer software was provided to the schools. The actual IT courses taught by the IT formators were limited to IT laboratories that focused on basic computer skills. Subject teachers were not trained in how to integrate computers into their subject teaching. The Bank asked the Government to prepare and update the IT policy paper and formulate a strategy to integrate IT into the curriculum and teacher training. After 5 1/2 years of project implementation of APL1, including three extensions of the closing date, the IT policy paper was not completed before the project closing date. The ICT impact study was also not completed during project implementation. While computer training may have accomplished the Government's goal of achieving computer literacy (sufficient - 8 - measuring of computer literacy had not been done under the project), there was not enough evidence to show that the APL1 inputs contributed significantly to the project's development objectives of improving quality of basic education. The potential uses of computers in schools are many and implementation processes are long-term. The ICT investments that were made probably set the preconditions to achieve more ambitious goals for the future, which includes radical change of school culture. However, it is too early to draw final conclusions in this ICR, except to present a snap shot of an ongoing process. The much-needed educational materials were purchased under the project, but only after long administrative and procurement delays, and three extensions of closing date. Of all the planned social assessments, only three studies were completed at the later stage of project period, but again the MONE did not disseminate the study results to the broader audience, such as stakeholders and academics. The Monitoring and Response Facility (MRF) yielded good results in terms of increased parental and community involvement in schools, but again, this activity was seriously delayed. The approval of the first group of projects under the MRF came only after May 2001, one month before the original closing date. The actual activities implemented were far less than the planned. There were $8.4 million planned for MRF, only $0.7 million was actually spent. On the third program development objective, there were few facts to show that significant progress had been made in this regard and is rated unsatisfactory. There was evidence that some schools offered the communities the ICT laboratories for after school use. There were good examples that MRF activities also reinforced the link between communities and schools. However, for the majority of schools that APL1 supported, it was difficult to conclude that they had become the learning resources for the communities. In fact, the bussing program introduced by the Government had some adverse effects. In some teachers' views, there were no longer educational and multi-activity cultural institutions left in the villages. The Government later amended the bussing program, reopening many of the village schools. 4.2 Outputs by components: Component 1: Expanded Basic Education Coverage (US$48.4 million in PAD, US$94.4 million actual). This component is rated satisfactory. The objectives of this component were to support the training and recruitment of additional teachers and the design, construction, expansion and upgrading of basic education schools. There was no loan financing used under the project for the first subcomponent which covered teacher training, recruitment, and deployment. This ICR is not rating this subcomponent, because the project did not supervise the activities that were not loan-financed. The second subcomponent, financed by the APL1, was School Construction, Extensions, and Upgrading (US$48.4 million in PAD, US$94.4 million actual). This subcomponent is rated satisfactory. The planned outputs established in the PAD for the Phase I of the project were to construct 450,000 new student places, and rehabilitate 300 existing schools in rural areas. The project financed the renovation and extension of 199 large rural schools and 123 central village schools. It financed the infrastructure work for 3,188 ICT classrooms at 2,802 basic education schools. Also, under APL1, there were 44 urban schools in Ankara Province renovated and extended, 1,287 rural schools and 42 YIBOs and PIOs renovated all over the country. The design of this component met the urgent needs of expanding school capacity at the time of project start. The focus of construction and rehabilitation were rural and poor urban areas. The major weaknesses of the subcomponent were that the needs assessment and construction work were carried out at central and provincial/subprovincial levels, and the schools and communities had limited input regarding the design and the specifications of the civil works. The technical staff of MONE at the central and - 9 - provincial levels did not have the capacity and resources to prepare, supervise, and monitor the construction and renovation. The Bank supervision teams observed many defects in the completed works and brought them to the attention of the MONE on several occasions. Component 2: Improved Basic Education Quality (US$233.5 million in PAD, US$190.4 million actual). This component is rated unsatisfactory. The objectives of the component were to improve basic education learning outcomes and increase the relevance of basic education. It would support in-service training for teachers, inspectors, and school principals; provide textbooks and educational materials for basic education schools; provide hardware, software, and training for information technology in basic education schools; and carry out a number of other initiatives designed to improve the quality of basic education. The outputs for this component were outlined within the following five subcomponents, one of which was financed under the Government's Basic Education Program. Subcomponent 1: Information Technology (US$154.0 million in PAD, US$105.9 million actual). The subcomponent is rated unsatisfactory. The objectives of the subcomponent were related to the objectives of Component 2. The outputs set for this subcomponent were: 2800 IT rooms prepared and equipped; the number of trained IT coordinators is doubled. These outputs were achieved under APL1. The APL1 procured computer hardware/software and peripheral equipment for 22,854 rural schools, overhead projectors for 18,517 rural schools, and 6,255 data projectors for 2,802 basic education schools installed with IT classes. The project also purchased 1500 notebooks for master formators, primary education inspectors, and 130 notebooks for the Ministry's inspectors. The APL1 financed the training of 2,058 IT formators, 250 master formators training. Since there were no IT coordinators at the beginning of the project, it was not possible to conclude the number of trained IT coordinators was doubled. According to the PAD, the design of the IT activities of this project was built upon the successful Computer Experimental School component under the National Education Development Project. Under the NEDP project, computer laboratories were established and 235 primary and secondary teachers were trained. Computer-aided education programs for the vocational high schools were also developed. The Computer Experimental School component was evaluated by an international group of experts in 1996, and was found successful in integrating IT into learning and engaging community involvement in the initiative. After 5 1/2 years of APL1 implementation, it was not evident that the outputs accomplished under this subcomponent contributed significantly to the achievement of objectives under this component. Subcomponent 2: In-service Training (US$31.2 million in PAD, US$3.7 million actual). This subcomponent is rated unsatisfactory. The objective of this subcomponent was to support the Government's program on providing two weeks of in-service teacher training each year for approximately 305,000 basic education teachers and school principals. The outputs outlined in the PAD were: by the end of Phase I of the Project, there would be 300,000 existing teachers and 30,000 new teachers who had participated in in-service training programs, and 3,000 school inspectors trained. In-service training was to carry out the training based on modules designed by the MONE, covering topic areas including pedagogical methods, learning assessment, community participation, and classroom management and communication. By the end of the project, APL1 financed only the following: training for 405 teachers on active learning and special learning methods; training for 3,781 principal candidates; training for 861 inspectors on active learning and special education methods; training for 2,956 inspectors on active learning and teaching strategies and use of materials; and English language training for 150 MONE staff. Clearly, the substantial planned in-service training was not carried out under APL1, and the project had little information on Government's training activities. - 10 - Subcomponent 3: Provision of Essential Teaching and Learning Materials to Rural Schools (US$47.6 million in PAD, US$76.5 million actual). This subcomponent is rated unsatisfactory. The objective of this subcomponent was to improve learning achievement in poorly equipped rural schools. The subcomponent output was to finance the provision and distribution of a package of teaching and learning materials to 25,900 schools in rural areas. The APL1 did distribute educational materials to 22,287 rural schools and 2,993,692 students throughout the country. However, the procurement of these educational materials took an unacceptably long time to complete. The project closing date was extended three times, to a large extent, to allow the completion of procurement of these materials. Subcomponent 4: Student Social Aid (US$45 million through grant funds not financed by the project). The rating for this subcomponent is not applicable as it was not financed by the APL1. The activities were planned under the Government's budget, and the APL1 did not supervise this activity. Subcomponent 5: Assessment of Learning Achievements (US$0.7 million in PAD, US$0.1 million actual). This subcomponent is rated satisfactory. The objective of this subcomponent was to develop and implement assessments of learning outcomes. There were no specific output indicators included in the logframe of the PAD under this subcomponent. The APL1 was to support the representative surveys of approximately 70,000 students in 500 basic education schools to assess learning achievements in four subjects: Turkish, science, math and social studies. The survey was to expand to include computer literacy and foreign language proficiency in 1999 and subsequent years. The results of the assessments would be shared with the provinces to improve educational performance. The Department of Educational Research and Development (ERDD) of MONE completed and analyzed the results of the student achievement tests in Turkish, mathematics, science, and social science. An Interim Report for the Student Achievement Assessment Tests was finalized in April 2003. ERDD also prepared the first drafts of the National Reports for TIMSS 1999 and PIRLS 2001 in April 2003. The results of these tests were put on MONE's website, and the test results, both national and international, were presented to the senior level management at MONE. The results of the tests provided valuable comparative information to MONE. They were also disseminated to universities which had education faculities, provincial and subprovincial directorates of education and the relevant schools. Component 3: Program Implementation Support (US$4.8 million in PAD, US$0.6 million actual). The component is rated unsatisfactory. The component had two subcomponents, which separately explained the objectives of this component. The supporting argument for the rating, the output, and the design of this component is discussed in subcomponent 1. Subcomponent 1: Support for the Program Coordination Team (US$1.8 million in PAD, US$0.6 million actual). This subcomponent is rated unsatisfactory. This subcomponent was to provide implementation support to the Program Coordination Team (PCT). The role of the PCT was not clearly spelled out at the early stage of project implementation, and it caused confusion within the MONE. The name of PCT was later changed to Project Coordination Center (PCC). The project financed the consultants employed in the PCC. However, there was no evidence indicating that implementation was improved after the recruitment of consultants. However, program coordination remained weak throughout project implementation. The implementation capacity in the PCC continued to be limited. Subcomponent 2: Educational Management Information Systems (US$3.0 million in PAD, US$0.0 million actual). The rating for this subcomponent is not applicable as both the Bank and the borrower agreed to delete this subcomponent from the project. This subcomponent was to support the Ministry's Council for Research, Planning (APK) to improve the Ministry's educational information management - 11 - capacity, train and strengthen APK technical staff in the use of state-of-the art education management information systems, and acquire up-to-date computer hardware and software. In 2000, the Ministry expressed the interest to complete the MIS upgrades, which it initiated under the National Educational Development Project before developing an electronic school mapping system, which was planned under APL1. Component 4: Monitoring and Evaluation (US$13.3 million in PAD, US$1.0 million actual). This component is rated unsatisfactory. The objective of this component was to assess the outcomes of the program and provide continuous feedback to the PCT and the related line units of the Ministry to improve the effectiveness of the program. The supporting argument for the rating, and the discussions on the output as well as the design are included in the subcomponent sections. Subcomponent 1: Social Assessment and Monitoring (US$3.0 million in PAD, US$0.3 million actual). This subcomponent is rated unsatisfactory. The objectives of this subcomponent were to use the on-going evaluation to provide essential feedback for gauging progress, making course corrections, and responding to changing circumstances. There were no output indicators set for this subcomponent. The plan was that the results of monitoring efforts would be evaluated in a participatory manner at several levels including: PCT, relevant departments of the MONE, provincial, and subprovincial administrators. There would also be national seminars and joint evaluations with the Bank, international and national experts. Out of the many activities, the numerous planned assessments, and studies (about 11), the APL1 was able to finance only three studies: (a) cost analysis; (b) social impact; and (c) institutional impact of bussing schools, YIBO/PIO schools, and traditional schools. These studies were conducted after long delays. Details findings of these studies are available in project files. Subcomponent 2: Monitoring Response Facility (US$8.4 million in PAD, US$0.7 million actual). This subcomponent is rated unsatisfactory. The objective of this subcomponent was to provide the Basic Education Program with the flexibility to respond to developments arising during implementation, and to support the development of pilot projects and initiatives, consistent with program objectives, at the school level. The output indicators for this activity were not included in the PAD. During the supervision mission in January 2002, the Bank and the PCC agreed on the output of the "School improvement program implemented in eight provinces". By the closing of the project, APL1 financed 85 MRF projects. Although there had been no impact analysis conducted, field visits carried out by the MONE and the Bank, as well as the monthly Project Monitoring and Evaluation reports PCC received from project schools showed that there were great demands and interest at the school level for the MRF. The MRF projects took an unacceptably long time to implement. Out of the $8.4 million planned for this subcomponent, only $700,000 were spent. Subcomponent 3: Basic Education Promotion (US$1.9 million in PAD, US$0.0 million actual). The rating for this subcomponent is not applicable since the activities under this subcomponent were cancelled at the request of the MONE. The original objective was to provide specific information to the public on the development of the Basic Education Program. During implementation, MONE stated that public support for the Basic Education Program was strong, and it preferred to finance some limited promotion activities including publication of bulletins, under its own budget. The Bank agreed to this proposal. 4.3 Net Present Value/Economic rate of return: Not applicable. 4.4 Financial rate of return: - 12 - Not applicable. 4.5 Institutional development impact: Overall, the institutional impact of the project has been negligible. During this first phase of APL, the Bank project focused on providing inputs, such as improving the school infrastructure, providing educational materials, purchasing IT equipment, and training of IT coordinators/formators. This was to set the stage for future development of the system. Most of the inputs were provided to the project during the latter part of the project (after 3 extensions). As a result, there was little time for impact to develop. The substantive part of teacher training was carried out by the Government and the Bank's role in this aspect was limited to ICT training and active learning. As pointed out above (section 4.2, Components 3 and 4) and below (section 7.6), the MONE research functions were not fully coordinated among the different departments. The monitoring and evaluation unit established within the PCC under the project was only reporting the activities under the project and had little collaboration with other departments in the MONE. The supervision of project activities was not fully integrated in the MONE. For example, supervision of civil works was coordinated by the Primary Education Department of the MONE, however, MONE suggested that the Investment and Facilities Department of the MONE could have offered more expertise in this aspect. The decisions on the needs of each school were largely made from the central ministry level. MONE staff working at the provincial level were not always capable of preparing subcomponent activities and supervising them. There were many aspects of ongoing education work under the Government's Basic Education Program, but the APL1 did not contribute to streamlining the functions of all parties in the overall education system. Follow-up activities will need to address this need for further streamlining. The design of setting up the PCC away from the MONE was subject to debate. The Bank resorted often to consultants to assist with project management, procurement and financial management. The Government had expressed the need to utilize the line units instead of recruiting additional consultants for the PCC. The lack of communication and collaboration among the line units of the MONE and the PCC affected project implementation. The physical separation of the PCC from the MONE exacerbated this problem. More importantly, this project did not effectively contribute to the institutional capacity building of the Ministry. 5. Major Factors Affecting Implementation and Outcome 5.1 Factors outside the control of government or implementing agency: As mentioned earlier, the Marmara Earthquake in August 1999 significantly affected project activities. The Government had to reconstruct and rehabilitate the educational facilities for preschools, basic and secondary education schools, which were destroyed or damaged by the earthquake. Consequently, rehabilitation and upgrading of about 40 urban basic education schools in low income areas of Ankara province were dropped from the project. Although there were further changes later to delete some of the activities related to the Marmara earthquake and to add the activities planned originally, this held up the implementation of some activities. 5.2 Factors generally subject to government control: The financial crisis of late 2000 resulted in a 9.4 percent reduction in GNP, which reduced the Government's ability to finance expenditures. Although the Government was committed to maintain the share of education at 4.25 percent of GNP and counterpart funding for the project was never a problem, the total expenditure on education decreased. The Government's requirements and procedures for the approval of procurement documents caused serious delays. To obtain visas from the Ministry of Finance and later the Courts of Accounts always took a long time. Towards the end of the project, the Treasury Department - 13 - implemented a new Debt Management Law. This Law and the budget law for 2003 posed additional approval steps for procurement and the payment process. The change of government after the election of November 2002 resulted in the change of senior MONE officials, including the change of the PCC Directors. The changes caused delays since new officials had to familiarize themselves with the project. 5.3 Factors generally subject to implementing agency control: MONE's structure was very fragmented, with departments that had overlapping functions. The collaboration and coordination among the line units and the PCC had always been problematic. Information flow within the Ministry was not effective even towards the end of the project. Communication within the PCC was not effective. The training provided to the PCC staff did not meet expectations in the short-term. Procurement and financial problems were severe. Management issues also caused problems at the provincial and local levels regarding civil works. Provinces were provided with World Bank guidelines and introductory training. The provincial staff went through a lengthy and difficult process since they did not have previous experiences. However, an effective system to monitor the performance of the civil works contractors, supervising consultants and provincial staff was not put in place. The project had to extend the closing date largely due to the delay in procurement of various goods. A total of US$6 million was cancelled by the Bank during the third closing date extension. The reason was that the Bank did not expect that the project would be able to complete all the procurement activities. On the assessment studies, MONE found it hard to agree to have third parties, Ankara and Bashkent Universities, carry out the studies while the Ministry had three of its own research departments: APK, ERDD and Vocational and Technical Education Research and Development Center (METARGEM). There were also disagreements between the Bank and the MONE regarding the terms of reference. 5.4 Costs and financing: The total project cost was estimated to be US$2,825 million, out of which US$300 million was to be Bank financed, and US$2,515 million was to be Government financed. At the time of this ICR, the total disbursed Bank loan was US$286.2 million and the total amount spent from Government's budget was US$2,281.4 million. A total of US$6 million was cancelled from the Loan in June 2003. As of June 24, 2004, the government is requesting that the Bank cancel the additional undisbursed $7.81 million from the Loan. Major changes on costs include: PAD estimate Actual US$ US$ Increase in construction and upgrading of basic education schools 48.4 94.4 Reduction in in-service training programs 31.2 3.7 Increase in educational materials 47.6 76.5 Reduction in information technology 154 105.9 Reduction in assessment of learning outcomes 0.7 0.1 - 14 - Increase in bussing facility 0.0 2.3 Increase in free lunch for students 0.0 2.0 Reduction in program management team 1.8 0.6 Reduction in social assessment, monitoring and studies 3.0 0.3 Reduction in Monitoring Response Facility 8.4 0.7 Significant delays in procurement led to three extensions of the Loan closing date. Reasons for the persistent procurement problems during project implementation include: the lack of capacity in the PCC although efforts were made to recruit consultants for the procurement unit; the political instability and frequent changes of government's internal rules and regulations prevented the PCC from handling procurement matters efficiently; the insistence of MONE to be involved in the detailed review of procurement documents prior to approvals; and the lack of planning at the MONE for procurement activities. As mentioned in the previous section, the Bank decided to cancel US$6 million in 2003 because it did not feel confident that the government would be able to complete all the planned activities. 6. Sustainability 6.1 Rationale for sustainability rating: The project sustainability is rated likely. Given the limited institutional development impact the project has had, and the uncertainty of project benefits, it is difficult to argue that the benefit of the project would be sustainable. However, from the social and financial point of view, the agenda for education reform and continuous improvement is sustainable. The new Government of Justice and Development Party was committed to the Rapid Coverage for Compulsory Education Program. It has continued the implementation of the program as planned. The Government has provided free sets of textbooks to all primary school children and has launched a "Hundred-Percent Support for Education" campaign. It introduced the Law No. 4842, which allows tax deductions for donations to education, this helped to increase private donations to education five fold. The Government's 2004 budget places special emphasis on education. For the first time, education spending will exceed the defense budget. If education budgets of other public institutions and higher education were excluded, MONE's budget share represents 3% of GNP. The current Government's commitment to the Program was illustrated in details of its budget to the Planning and Budgeting Commission of the Parliament. The Government is planning to review and upgrade the primary education curriculum in 2004. A protocol between the Ministry of Transport and the MONE was signed in December 2003 to provide high speed internet connectivity to approximately 40,000 schools including most of the basic education schools and all secondary schools. According to the protocol, high speed internet connectivity will be provided to these schools for a 3-year period between 2004 and 2006. The cost of internet connectivity will be covered under the MONE's budget. The State Planning Organization has been eager to see the results of the ICT Impact Study. They would like to be informed of the effectiveness of ICT investment carried out so far, and follow up with more informed decisions. The maintenance of schools has been assigned to the municipalities and the provincial governments. - 15 - Turkey's interest in joining the European Union (EU) will be a major stimulus for further development of the basic education program. UNICEF has been engaged even more actively since 2002 in attracting girls in the poor provinces in Eastern Anatolia. It initiated a pilot project in 2002 a program called "Come on, girls, let's go to school!" which has been expanded to more provinces and is being jointly monitored by the Government and the UNICEF. Other NGOs and private sectors have launched new programs to improve education quality. It should be noted that in order to achieve the maximum benefit, an effective monitoring and evaluation system needs to be strengthened within the MONE and at the provincial level. 6.2 Transition arrangement to regular operations: In July 2002, the Bank approved the Second Basic Education Project (APL2). The follow up tasks for APL1 are included in the implementation of APL2. The Government and the Bank have also been preparing the Secondary Education Project for sometime, and it is presently scheduled for Board presentation in the next fiscal year. Both projects will reinforce the activities under APL1. Bank has also launched a Turkey Education Sector Study: Sustainable Pathways to an Equitable, Effective, and Efficient Education System. This study is to build a technical foundation for policy analysis. It will also support Turkey's goal of revitalizing its education system to deliver quality educational services both equitably and efficiently and assure that Turkey's citizens will be better prepared to compete in global labor markets and participate actively in the knowledge society. This study also aims to enhance the capacity in the relevant government agencies through data collection, workshops, international seminars, or joint preparation of studies, etc., to build up the information base on Turkish education, enhance the institutional base for carrying out related analytic work in the future, and help Turkish authorities replace the outdated paradigm of educational reform. 7. Bank and Borrower Performance Bank 7.1 Lending: The Bank's lending performance was unsatisfactory. The Bank and UNICEF were instrumental in raising the awareness of universal education in Turkey. The Government was clearly committed to the Basic Education Program and was eager to have the Bank's assistance. The actual preparation of APL1 took a long time mainly because: 1) the Government wished to prepare the project itself (demonstrating government ownership, but requiring a longer preparation period); 2) The Government and the Bank agreed that a basic education project pilot be set up within the now closed National Education Development Project that would experiment with some of the concepts that would help in the design of APL1. The Bank addressed the most urgent needs in the sector at the time and focused on equity issues, directing attention to the poor and the rural population. As pointed out earlier in the Quality at Entry section, the project design was rated unsatisfactory by QAG for financial management aspects and readiness for implementation. NEDP created 53 experimental schools on Computer Aided Education and it was confirmed by the Bank in 1996 that this project component was successful. This became the basis for a much larger scale of IT investment under the APL1. The Bank also asked the Government to prepare an IT policy paper and outline a strategy on how to integrate IT into curricula and teacher training. The Bank recognized the needs of carrying out various assessments and monitoring and evaluation during project implementation. However, the social assessments listed under Annex 1, Attachment 1 of the PAD were too fragmented. The most significant weakness to this design was that it did not focus on the outcomes of the activities. In particular, it should have concentrated on institutional capacity building at the central and local level. On the management of the program and the project, efforts to streamline the MONE during preparation were - 16 - resisted by some of the government officials. Problems on implementation were predicted by the Bank at preparation, but were not fully addressed. An implementation plan was not prepared for the project. Setting up the PCC and recruiting consultants may have helped in the short run to move the project ahead, but they did not contribute to institutional capacity building of the Ministry of Education as well as the provincial government. 7.2 Supervision: The Bank's overall supervision performance is unsatisfactory. At the beginning of project implementation, the Bank and Government focused on the whole Basic Education Program, in particular, the increase in enrollment especially for girls, the number of teachers recruited and trained, and the schools constructed or renovated. This was necessary since APL1's contribution amounted to only 11.9% of the total government's commitment for Phase I of the program. The Bank asked the Borrower to emphasize monitoring and evaluation. Major indicators, particular the indicators for IT component were to be defined by the Ministry after the project took off. It was essential for the Bank to review the project in the context of the whole program particularly since APL1 had only input and output indicators, not the impact or outcome indicators, for the project specifically. Bank supervision missions had been conscientious about ensuring that the project was performing in compliance with the legal covenants, procurement and financial policies. The Bank advocated strongly on the adherence of financial management procedures and procurement rules. Despite the above, Bank did not consistently evaluate its value-added role nor continuously assess its options to make a larger impact on the program, taking into account the changed circumstances, progress made and new challenges. In implementation, especially during the first year or two, the Bank should have focused more on correcting the deficiencies and improving the design. The Government added new courses to the curriculum and launched a massive effort in teacher training. The review of the curriculum was not carried out under the project. This was partly due to Government's unwillingness to have the Bank involved in this area and this was not part of the project. The Bank could have eventually looked for other entry points during the implementation to engage the Government in a dialogue in this very important area, particularly as the Loan was financing civil works, computer equipment and education materials. APLs are designed to be more flexible compared to the ordinary investment loans, but the inflexibilities of the triggers constrained the teams. When reviewing the Bank's input, extensive emphasis was placed on resolving bottlenecks related to procurement and financial issues instead of focusing on outcomes. The IT training for formators and IT coordinators was conducted under the project, but the IT equipment could not be procured for a long time. Refresher training courses had to be conducted for the formators. The Bank repeatedly tried to convince the Government to complete the ICT impact study before loan closing, but failed. While APL1 experienced periodically interrupted progress, the Government's program moved ahead. Frequent changes of Bank task team leaders disrupted continuity of supervision. There was also no implementation plan for a long time after the project started. The ratings in the earlier Project Status Report (PSRs) were mostly satisfactory for project implementation reflecting Government's progress, but in a few cases, the PSRs failed to weigh the potential APL1 problems. Aide-Memoires and Project Status Reports were structured around procurement activities. Measurement of progress by the Bank, followed by the Borrowers, was on completing procurement activities. Partnership with EU, other international agencies, and civil society could also have been a more concerted and effective effort in working with the Government. The sometimes weak communication between the Bank and the Government delayed project implementation. There were issues regarding program and project management and coordination, which - 17 - the Bank and MONE were not clear on at various times of implementation. The role of the line units in the MONE and the PCC caused confusion among all parties throughout the project. It was noted that the MONE did not understand the concept and ideas associated with the Monitoring Response Facility and this led to a slow start in the submission of appropriate proposals. At the very early stage of project implementation, the Bank had suggested that the PCC select facilitators, or that the Mother and Child Foundation or the Middle East Technical University assist with the preparation of MRF proposals. However, according to the Bank, the Government initially took a different approach by asking provincial education authorities to generate proposals for the schools, without the involvement of schools, teachers, students, and community members. The Government stated in its own assessment (see Annex 10) that the Bank did not provide enough guidance on this subcomponent. The MRF staff claimed that they had requested training and documents on the subject but couldn't get any response from the Bank. The goal and the mechanism of this subcomponent were not clearly defined at the start of the project. Towards the second half of the original implementation period, the Bank made efforts to assist the PCC with organizing the provincial staff and helping the schools prepare proposals with the participation of students, parents, school staff, and local leaders. Eventually, all the 85 MRF projects were prepared in collaboration with the school communities, parents, teachers, students. The Bank's field office did play a critical role in communicating with the PCC and in ensuring the coordination and collaboration within the MONE. Given years of poor project performance and a huge backlog, during the last year of implementation, the Bank's supervision concentrated nearly exclusively on procurement. The Bank intensified supervision efforts after February 2001, following the January supervision of APL1 and preappraisal of APL2. The Bank worked closely with the Borrower in detailed implementation plans for every activity, and instituted very close monitoring. This close monitoring included weekly detailed meetings between the Bank and the Borrower, following every procurement and activity on a step by step basis. It was this effort that made it possible to close the project with most of the basic activities completed, at least in quantitative terms. Nevertheless, there was the need to enlist the Government's effort more rigorously. Government had staff in the MONE to carry out the activities, what the Bank could have assisted was that to effectively utilize MONE's manpower as well as efficiently organize the various departments within MONE to carry out a national program. 7.3 Overall Bank performance: The overall rating for Bank performance is unsatisfactory. The Bank did not continuously seek opportunities, or entry points, to engage in dialogue with the Government on other aspects of education quality (although they were not necessarily in the original project design), such as review of the curriculum and teacher training content. In the 1997 CAS, it was mentioned that the previous National Education Development Project had a difficult experience politically to put national level quality improvement into place, thereby the Bank resorted to focus on a pilot operation to lay the basis for what would be a sizeable investment to meet the acute shortages in school capacity. Nevertheless, continuous Bank involvement, particularly the substantial physical investment planned for APL1, justified the need for much more substance in the dialogue between the Bank and the borrower. In addition, the Bank proceeded with the preparation of APL2 without seriously looking at the critical lessons learned from APL1, such as outcome focused implementation, capacity building of the MONE, and a sustainable monitoring and evaluation system. Borrower 7.4 Preparation: The Borrower's overall performance in preparation was satisfactory. Government expressed strong commitment to the Basic Education Program throughout the preparation process. Major efforts were made by the Government in all aspects of education to realize the expansion of basic education. At the time of - 18 - project launch in 1998, Government had already committed $1.5 million for the new program. Government established the Basic Education Coordination Council in June 1998. The Council consisted of 9 units and over 40 members. It was envisaged that the line units of the MONE would implement the program and the project with the Council playing the coordination role. Government placed extreme importance to the preparation work and appointed key staff to work with the Bank to formulate the APL1. However, the project design did not take into account the complexity of the project, and the government's capacity to implement the project. Appropriate project management and implementation structure were not established. 7.5 Government implementation performance: The overall rating for Government implementation performance is unsatisfactory. The Government was very committed to the program and the project throughout implementation. There had always been good relations between the Government and the Bank. Frequent meetings were held with senior Bank officials in the Bank's Ankara office. Even during the financial crisis, the Government never failed to provide the counterpart funding. However, the extensive and complicated Government approval procedures delayed project implementation. The visas required by the Ministry of Finance and later the Court of Accounts proved to be very time consuming. Often times, officials in these organizations had to review the documentation themselves, including procurement documents. The clearance from the Ministry of Finance budget personnel who was stationed in the MONE also posed additional bureaucratic hurdles. 7.6 Implementing Agency: The overall rating for implementing agency performance is unsatisfactory. At the beginning of the project, Government established the Basic Education Coordination Council. In December 1999, Government set up the Basic Education Steering Committee to formulate policies for the program, and to plan, direct, and implement project related activities. The agreement had always been that project activities would be carried out by the units of the MONE in coordination with the Basic Education Coordination Council. MONE and the Council did carry out studies of their programs, which led to constructive proposals to improve the efficiency, quality, coverage, and responsiveness of the Program. The PCC had consistently prepared comprehensive progress reports, covering the overall view of the Government's program, although the reports towards the later stage of the project were more oriented to procurement activities. Despite the above-mentioned efforts, MONE continued to be as fragmented in its roles and functions as it had been when the project started. The units (ERDD, APK, and METAGEM) were responsible for various aspects of research. ERDD and the Board of Education did not always work closely together. Monitoring and evaluation of the project and the program was not sufficient within the MONE, including the PCC. The approval process within the MONE continued to be lengthy. The assessments and studies were reduced to four during implementation, with the Child Labor in Turkey study already completed. Nevertheless, MONE still took a long time to actually launch the three remaining studies. This was partly due to MONE's concern that the studies would be carried out by universities rather than the three research units of MONE. The Bank had urged the MONE to update the IT issue paper as well as prepare an E-learning strategy paper. These were sent to the Bank for comments by late 2002. However, after the Bank commented on the papers, the MONE senior management had not taken positions on them. The civil works implementation in the more remote provinces suffered from lack of timely and frequent supervision visits by the MONE. The provinces lacked the capacity to fully assume the responsibility for construction and rehabilitation. There were concerns that teacher training was not synchronized with the educational materials. This matter requires further examination. On the procurement of computer equipment, there was also a delay when the Government insisted on having a particular brand of computers. The Bank had to reaffirm its position related to its procurement principles. - 19 - 7.7 Overall Borrower performance: The overall rating for Borrower performance for the project is unsatisfactory. The Government achieved great progress under the Basic Education Program. As pointed out earlier, there had been major increase in enrollment rates, in the number of teachers recruited and trained and in the availability of schools. The Government always regarded basic education as a high priority. Funding for the sector and the project was emphasized and protected. The Government complied with the loan covenants, as well as the financial and procurement procedures. However, the implementation of the APL1 had been sluggish. The previous sections provide elaborations on the many reasons for the slow implementation. 8. Lessons Learned The Bank needs to improve its monitoring and course correcting during implementation and this process begins with a clear and unambiguous statement of project objectives. Project objectives need to be clearly defined to enable the borrower and the Bank to supervise the project. There is need to focus on the "big picture" in the sector, and assess the effects of Bank's contribution on a regular basis, especially in the evolving country situation. Supervisions, especially at the initial project implementation stage should correct any inadequacies from project preparation. Supervision Aide Memoires and PSRs need to be structured to reflect progress on output and outcomes and not only on activities. Concrete follow up actions on the recommendations from the "Quality at Entry" or "Quality Enhancement Review" need to be ensured. Bank teams should address all major issues highlighted in these reports and justify to the management one way or another on all major concerns. The QER panel provided many valuable comments for APL1, but serious, concrete actions were never taken. Focus on institutional capacity building in monitoring and evaluation of projects is critical. It is important to set up a good system of M&E for a project. It is even more essential to help a client build an M&E system which can be sustainable after the project closes. To explore the potential of government capacity and build upon it often yields better ownership and sustainability. Failure to do this in APL1 led to a breakdown in M&E. The Bank needs to review and assess all of the components which contribute to development objectives, including the ones that are not financed by the Bank. In APL1, there were activities that were not financed by Bank funds, such as student social aid and teacher training (APL1 was only to finance in-service teacher training). These activities had significant impact on the overall program, but since they were not financed by the Bank, they were not reviewed by the Bank. Triggers put into APLs should not be created to be rigid, constraining the government or the Bank team to fully to carry on the project agendas. Triggers should be policy-oriented and results-focused. In APL1, inappropriate triggers that focused on procurement actions were put in which later led the task team concentrate solely on procurement activities. The Bank and the borrower should agree on the outcomes and the monitorable performance measures for ICT investments. ICT investments are complicated and there is ongoing experience in this area of work all over the world. For large investments, the Bank needs to conduct serious cost effectiveness analysis. The evaluation of these kinds of large investments should be constantly assessed. Establishment of a project management structure is a pre-requisite for implementation. It is important to assess the institutional capacity of the implementation agency during preparation of the - 20 - project, and make arrangements to ensure the management structure would meet the demand of the project implementation. Milestones and targets for implementation need to be agreed and strictly followed by the borrower and the Bank. Each quarterly report should report on the achievement or reasons why the targets were not obtainable. These milestones and targets could be made public to all stakeholders as well as beneficiaries. Bank management would also need to monitor the progress of these milestones/targets and review them in light of the outputs the project achieved. Realistic risk assessment in project design and measures to mitigate them are essential. Projects need to work out the worst case scenarios to ensure adequate implementation plans are in place to avoid delays and setbacks. Lessons from APL1 were not assessed in time in order to incorporate them during the preparation of the APL2. This has led to deficiencies in the design of APL2. Important assessments and evaluations under APL1 were not completed, and therefore could not contribute to the design of APL2. For the Bank to prepare subsequent APLs, apart from meeting the triggers, there should be reviews of the lessons learned from the ongoing APL. 9. Partner Comments (a) Borrower/implementing agency: Detailed evaluation from the borrower is included in Annex 10. (b) Cofinanciers: There were no co-financiers in APL1. (c) Other partners (NGOs/private sector): The NGOs participated in the MRF activities under the project. Private sector provided substantial support to the education sector, but the project did not work with them directly. 10. Additional Information The Government prepared its contribution to this ICR. Government is generally very positive about the APL1. This obviously is a big contrast to the Bank's evaluation of the project. The explanation for the differences of opinion includes: 1) The APL1 was supervised mainly based on the inputs. Government's evaluation is a reflection of how both the borrower and the Bank review the project. The Bank's supervisions, which were also followed by the borrower, were inputs- and activities-focused. Project development objectives and outcomes were not consistently discussed. The inputs the Borrower referred to in its report do not necessarily translate to the achievement of outcomes. For example, under the increasing of basic education quality, the report stressed the various inputs the project was able to provide, but did not comment on the linkage of these inputs to outcomes and objectives. 2) Government is reviewing the APL1 in light of the whole basic education program and what APL1 was able to contribute. Whereas the Bank's evaluation concentrated on the possible impact the project could have or would have on the sector in Turkey. A beneficiary assessment was conducted for this ICR that was based on school visits, and interviews with students, parents, teachers, and schools administrators. The main findings from the assessment were consistent with those of this completion report. Administrators, teachers and parents indicated that they were not consulted on the project activities, however, they stated that what the project provided was actually what the schools needed. The assessment also pointed out that the poor construction - 21 - of the school buildings was a serious problem. It also indicated that the project activities did contribute to the increase of student enrollment. One of the major concerns the assessment discovered was the lack of maintenance for the school buildings, facilities, and equipment. Detailed report of the beneficiary assessment is available in Annex 8 and in the project file. A stakeholder workshop was conducted to discuss the draft ICR. Participants shared the concerns that the APL1 was not implemented as well as it could have been. Some people expressed the need to draw lessons from this experience in order to improve implementation for the on-going APL2. Other representatives stressed that the government's basic education program achieved a lot, and did not understand the reason for some of the unsatisfactory ratings given by this ICR. The Bank team explained that this evaluation was only for the APL1, not for the whole government's program. An important point was made during this workshop which was: under APL1, exchange of international experiences did not take place. A teacher from an Ankara school expressed the concern that the schools had many other problems, such as lack of water and electricity, and that the computers were purchased for the schools, and the project expected the schools to solve the related problems. They received some substandard videotapes, but the TVs provided to the school could not be fully utilized for educational purposes. In general, participants were eager to learn the lessons and improve the effectiveness of any follow up projects. A summary of the stakeholder workshop is included in annex 9. - 22 - Annex 1. Key Performance Indicators/Log Frame Matrix Outcome / Impact Indicators: Indicator/Matrix Projected in last PSR1 Actual/Latest Estimate 450,000 new student places No specific baseline numbers indicated in Under the Government's Basic Education the Loan documents. Program, 12,186 new classrooms and 6,072 additional classrooms were constructed. Rehabilitation 300 existing schools in No specific baseline numbers indicated in Under the APL1, 199 large rural schools rural areas the Loan documents. and 124 central village schools were renovated and extended. Works for 3,188 ICT classrooms at 2,802 schools completed. 1,287 rural schools and 42 YIBOs and PIOs were renovated. 44 Ankara urban schools renovated and extension works completed. 300,000 existing teachers plus 30,000 221,000 teachers received training on ICT Under the Government;s Basic Education new teachers participate in the in-service literacy; 250 master ICT formators Program, 221,000 teachers were trained training programs trained; 405 teachers received training in on the most effective and efficient use of Active Learning and Teaching Method; IT rooms, hardware and software 100 MONE staff received English products. training. Under the APL I, 405 teachers were trained on Active Learning and Special Learning Methods; 3,781 received training for candidate principals; 150 MONE staff received English language course training. 3,000 school inspectors trained 3,000 inspectors trained. 2,956 inspectors received training on Active Learning and Teaching Strategies and Use of Material; 861 inspectors received training on Active Learning and Special Education Methods. Educational materials package distributed Globes, world maps, atlas, math sets, These educational materials were to 25,900 rural schools science materials, story books, children distributed to 22, 287 rural schools and literature distributed. 2,993,692 students throughout the country. Social assistance program implemented in Students bussed. This was part of the government's village schools as well as centralized program. It is implemented, but was not bussing schools financed by the Bank. 2,800 IT rooms prepared and equipped 2,802 ICT rooms equipped, 1,630 Computer hardware/software and notebooks, 22,854 PCs, 6,255 projectors peripheral equipment have been procured provided; overhead projectors to 18,517 for 22,854 rural schools and overhead schools. projectors were provided for 18,517 rural schools and 6,255 data projectors for 2,802 basic education schools installed with IT classes. 1,500 Notebooks for Master Formators, primary education inspectors and 130 notebooks plus 1 server for MONE's inspectors. - 23 - The number of trained IT coordinators is There were no IT coordinators in the basic Under the Government's Basic Education doubled education schools before the project Program, 221,000 teachers were trained started. on the most effective and efficient use of IT rooms, hardware and software products. Under the APL I, 2,058 IT formators were trained on IT coordination; 250 master formators trained on master IT formator training; 250 master formators again received refresher courses. Output Indicators: Indicator/Matrix Projected in last PSR1 Actual/Latest Estimate Monitoring and Evaluation capacity for Small but weak M & E Unit established The project originally aimed at a the program established at PCC; M & E consultant not hired; M & mechanism that would be created in E Unit at the PCC did not work with MONE to monitor and evaluate the ERDD nor with APK. outcomes of the Project. M & E Unit was set up at the PCC but the Unit has not worked closely with other research and development units of the Ministry, i.e., Research Planning and Coordination Council (APK) and Education Research and Development Department (ERDD), and capacity of MONE has not increased significantly. There is still not clear and mutual understanding at the Ministry regarding the definition of roles and responsibilities of these units and the ways of collaboration among them. APK's MIS system for school planning n/a This activity was not financed under the and management updated and staff Loan. trained 1End of project - 24 - Annex 2. Project Costs and Financing Project Cost by Component (in US$ million equivalent) Appraisal Actual/Latest Percentage of Estimate Estimate Appraisal Component US$ million US$ million 1) Expanded Basic Education Coverage Construction and upgrading of basic education schools 48.40 94.40 195.04 Teacher training and recruitment 2) Improve Basic Education Quality In-service training program 31.20 3.70 11.86 Educational materials 47.60 76.50 140.76 Information technology 154.00 105.90 63.57 Assessment of learning outcomes 0.70 0.10 14.29 Student social aid 3) Program Implementation 0.00 4.30 Program management team 1.80 0.60 33.33 EMIS/school mapping 3.00 0.00 0 4) Monitoring and Evaluation Social assessment, monitoring and studies 3.00 0.30 10 Monitoring response facility 8.40 0.70 8.33 Basic Education Promotion 1.90 0.00 0 Total Baseline Cost 300.00 286.50 Total Project Costs 300.00 286.50 Total Financing Required 300.00 286.50 Amounts may not add up to exact totals due to rounding. Project Costs by Procurement Arrangements (Appraisal Estimate) (US$ million equivalent) 1 Procurement Method Expenditure Category ICB NCB 2 N.B.F. Total Cost Other 1. Works 0.00 25.10 62.60 1238.70 1326.40 (0.00) (21.10) (52.80) (0.00) (73.90) 2. Goods 204.30 5.80 2.00 188.30 400.40 (172.20) (4.90) (1.60) (0.00) (178.70) 3. Services 0.00 0.00 56.40 42.00 98.40 (0.00) (0.00) (47.40) (0.00) (47.40) 4. Incremental recurrent 0.00 0.00 0.00 690.00 690.00 costs (0.00) (0.00) (0.00) (0.00) (0.00) Total 204.30 30.90 121.00 2159.00 2515.20 (172.20) (26.00) (101.80) (0.00) (300.00) Amounts may not add up to exact totals due to rounding. Figures in parentheses are the respective amounts financed by the World Bank. - 25 - Project Costs by Procurement Arrangements (Actual/Latest Estimate) (US$ million equivalent) 1 Procurement Method Expenditure Category ICB NCB 2 N.B.F. Total Cost Other 1. Works 0.00 23.11 0.53 0.00 23.64 (0.00) (19.57) (74.87) (0.00) (94.44) 2. Goods 181.23 0.56 0.00 0.00 181.78 (179.35) (1.02) (0.67) (0.00) (181.04) 3. Services 0.00 0.00 1.00 0.00 1.00 (0.00) (0.00) (4.60) (0.00) (4.60) 4. Incremental recurrent 0.00 0.00 0.00 0.00 0.00 costs (0.00) (0.00) (0.48) (0.00) (0.48) Total 181.23 23.67 1.52 0.00 206.42 (179.35) (20.59) (80.62) (0.00) (280.56) Services included the training costs. Goods, works and services for subprojects under Part E of the project (MRF) amounted to $0.70 million. Goods under Part F.2 of the project (school lunches in earthquake zone) amounted to $2.29 million. Contracted transport services (student bussing in earthquake zone) amounted to $1.98 million. 1/Figures in parenthesis are the amounts to be financed by the Bank Loan. All costs include contingencies. 2/Includes civil works and goods to be procured through national shopping, consulting services, services of contracted staff of the project management office, training, technical assistance services, and incremental operating costs related to (i) managing the project, and (ii) re-lending project funds to local government units. Project Financing by Component (in US$ million equivalent) Percentage of Appraisal Component Appraisal Estimate Actual/Latest Estimate Bank Govt. CoF. Bank Govt. CoF. Bank Govt. CoF. 1) Expanded Basic Education Coverage Construction and 48.40 1897.40 94.40 1624.20 195.0 85.6 upgrading of basic education schools Teacher training and recruitment 2) Improve Basic Education Quality In-service training 31.20 5.80 3.70 0.10 11.9 1.7 program Educational materials 47.60 44.10 76.50 267.90 160.7 607.5 Information technology 154.00 28.80 105.90 6.70 68.8 23.3 Assessment of learning 0.70 0.10 0.10 0.00 14.3 0.0 outcomes 3) Program Implementation Program management 1.80 42.40 0.60 0.00 33.3 0.0 team EMIS/school mapping 3.00 0.60 0.00 0.00 0.0 0.0 - 26 - 4) Monitoring and Evaluation Social assessment, 3.00 0.60 0.30 0.10 10.0 16.7 monitoring and studies Monitoring response 8.40 1.60 0.70 0.10 8.3 6.3 facility Basic Education 1.90 0.30 0.00 0.00 0.0 0.0 Promotion Additional cost include: Student Social Aid: Appraisal estimate: Bank - $0.0 million; Government - $0.0 million Actual: Bank $4.3 million; Government $382.3 million - 27 - Annex 3. Economic Costs and Benefits No economic costs and benefits analysis was carried out at the time of the ICR. As stated in the Quality at Entry section, the Quality at Entry Assessment rated marginal satisfactory for technical and economic aspects. The Assessment stated that used a relatively simple system, even for the computer component of the project, to justify the technical soundness of the project/program. Options were not examined carefully regarding boarding, bussing, and double-shifts schools. There were no alternative designs seriously considered by the project. The preparation team disagreed with this statement and indicated they had urged the government explore options. On the quality and coherence of economic rationale underpinning the project, the Assessment pointed out that micro justification was okay, but it was not helpful in choosing among options and that macro analysis was lacking. - 28 - Annex 4. Bank Inputs (a) Missions: Stage of Project Cycle No. of Persons and Specialty Performance Rating (e.g. 2 Economists, 1 FMS, etc.) Implementation Development Month/Year Count Specialty Progress Objective Identification/Preparation 02/04/98 10 Principal Education Economist (1); Counsel (1); Disbursement Officer (1); Social Development Sector Leader (1); Senior Implementation Specialist (1); Senior Informatics Specialist (1); Senior Country Officer (1); Human Development Specialist (1); Policy Analyst (1); Senior Textbook Specialist (1) Appraisal/Negotiation 05/05/98 9 Principal Education Economist (1); Counsel (1); Disbursement Officer (1); Social Development Sector Leader (1); Senior Implementation Specialist (1); Senior Informatics Specialist (1); Senior Country Officer (1); Human Development Specialist (1); Policy Analyst (1) 05/18/98 9 Principal Education Economist (1); Counsel (1); Disbursement Officer (1); Social Development Sector Leader (1); Senior Implementation Specialist (1); Senior Informatics Specialist (1); Senior Country Officer (1); Human Development Specialist (1); Policy Analyst (1) Supervision 12/18/1998 9 MISSION LEADER, ECONOM S S (1); HUMAN DEV. SPECIALIST (1); SR. IMPLEMENT. SPEC. (1); SR. EDUCATION SPEC. (1); FINANCIAL MNGMT. SPEC. (2); SOC.DEV. SECTOR LEADER (1); SOCIAL SCIENTIST (1); COMM. DEVEL. SPEC. (1) - 29 - 07/16/1999 9 MISSION LEADER, PR ECO S HS (1); HUMAN DEVELOPMENT SPEC (1); SR IMPLEMENTATION SPEC (1); SR EDUCATION SPEC (1); SR INFORMATICS SPEC (1); LEAD INFO SPECIALIST (1); IT SPECIALIST (2); PROCUREMENT SPECIALIST (1) 12/08/2000 8 PROGRAM TEAM LEADER S HS (1); SR. EDUCATION SPEC. (1); OPERATIONS ANALYST (1); OPERATIONS OFFICER (2); PROCUREMENT SPECIALIST (1); SR. MGT. FINANC. SPEC. (1); SR. INFORMATICS SPEC. (1) 02/28/2001 9 TTL, LEAD EDU. SPEC. (1); S S PTL, CO-TTL (1); SR. OPERATIONS OFFICER (1); PROCUREMENT SPECIALIST (1); FINANCIAL MAN. SPEC. (1); OPERATIONS ANALYST (1); WORLDLINKS COORD. (1); CONSULTANT IT (1); CONSULTANT ARCHITECT (1) 09/20/2002 10 TTL, LEAD EDU. SPEC. (1); S S SR. EDUCATION ECON. (1); SR. OPERATIONS OFF. (1); SR. EDUCATION SPEC. (1); LEAD INFORMATICS SPEC. (1); OPERATIONS ANALYST (1); FINANCIAL MANA. SPEC. (1); PROCUREMENT SPEC. (1); CONSULTANT ICT (1); CONSULTANT ARCHITECT (1) 10/24/2003 6 TTL & SR. OPERAT. OFF. (1); S S LEAD EDCUATION SPEC. (1); LEAD INFORMATICS SPEC. (1); OPERATIONS OFFICER (1); PROCUREMENT SPECIALIST (1); FMS SPECIALIST (1) ICR (b) Staff: - 30 - Stage of Project Cycle Actual/Latest Estimate No. Staff weeks US$ ('000) Identification/Preparation Appraisal/Negotiation 158 395 Supervision 462 1155 ICR 16 44 Total - 31 - Annex 5. Ratings for Achievement of Objectives/Outputs of Components (H=High, SU=Substantial, M=Modest, N=Negligible, NA=Not Applicable) Rating Macro policies H SU M N NA Sector Policies H SU M N NA Physical H SU M N NA Financial H SU M N NA Institutional Development H SU M N NA Environmental H SU M N NA Social Poverty Reduction H SU M N NA Gender H SU M N NA Other (Please specify) H SU M N NA Private sector development H SU M N NA Public sector management H SU M N NA Other (Please specify) H SU M N NA - 32 - Annex 6. Ratings of Bank and Borrower Performance (HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HU=Highly Unsatisfactory) 6.1 Bank performance Rating Lending HS S U HU Supervision HS S U HU Overall HS S U HU 6.2 Borrower performance Rating Preparation HS S U HU Government implementation performance HS S U HU Implementation agency performance HS S U HU Overall HS S U HU - 33 - Annex 7. List of Supporting Documents The Institutional Evaluation Research on Primary Education Schools. Ministry of Education, and Baskent University, Ankara August 2002. Research on Social Impact Assessment of Basic Education Schools. Ministry of Education Project Coordination Center, and Ankara University, Ankara July 2002. The Cost Analysis Research on Primary Education Schools. Ministry of Education, and Baskent University, Ankara August 2002. Turkey - Education Sector Study: Sustainability Pathways to an Equitable, Effective, and Efficient Education System. A Programmatic Sector Study Concept Note. World Bank, Washington DC. July 2003. Turkey: Rapid Coverage for Compulsory Education Program. Case Study. Ankara January 2004. Beneficiary Assessment for Implementation Completion Report. Ankara May 2004. - 34 - Annex 8. Beneficiary Survey Results BENEFICIARY ASSESSMENT FOR IMPLEMENTATION COMPLETION REPORT Project Background The Basic Education Project under that World Bank financed Project was to support the first phase of the Turkish Government's Basic Education Program which aims to: (a) achieve universal coverage in an expanded, eight-year Basic Education cycle; (b) improve the quality and relevance of Basic Education; and (c) make Basic Education schools learning resources for the community. The World Bank financed Project planned to achieve these objectives by: (a) the expansion of the capacity of Basic Education schools; (b) training of teachers, school principals and inspectors; (c) development of central and provincial implementation capacity to carry out the Program; and (d) creation of a mechanism to monitor and evaluate the outcome of the program. Specifically, the Project was to: a) expanding the capacity of basic education schools throughout the country, (b) facilitating school attendance of children who are at least likely to attend school, c) reducing classroom overcrowding and double shifting, d) improving training and incentives for teachers, e) improving the supply of educational materials to basic schools, f) introducing computer-aid learning, and increasing parental and community involvement in schools. The World Bank financed Project consisted of four components of which particular interest to this assessment are the following activities: 1) Expanded Basic Education Coverage. The component financed the rehabilitation of about 200 village schools, 100 central village schools and 40 urban schools. The project also financed the rehabilitation of about 1300 formerly closed rural basic education schools in provinces and 11 basic education boarding schools in provinces throughout Turkey. 2) Improved Basic Education Quality. This component financed the upgrading of computer rooms and for the provisions of computer equipment and software. It also provided in-service teacher training of computer teachers (formators). The component provided essential teaching and learning materials to rural schools. The Purpose of the Beneficiary Assessment The Project activities has been closed end December 2003 and an Implementation Completion Report is being prepared by the World Bank. An Intensive Learning Completion procedure is being followed to ensure that lessons learned from the Project are fully explored and incorporated into the World Bank's future work. The present beneficiary assessment is part of that process of discussion with the beneficiaries on the benefits of the project. The purpose of this beneficiary assessment is to discuss with the participants in the Project, and those affected by it, the impact of the project activities on the educational system, and its relevance to the stated Project goals. Beneficiary Assessment Methodology Qualitative research method was used for this assessment study. Data were gathered through conventional interviews, focus group discussions, and observations. Provincial officials, sub-provincial officials, community leaders, school principals and assistant principals, parents, and students in the benefited schools were used as sources of data. A semi-structured interview form was developed for interview. During the interviews the - 35 - interviewers took notes or used the tape recorder with the consent of the related person. Content analysis was used to describe the data gathered with open ended questions directed to the participants. Selection of Provinces and Schools The provinces have been classified into two categories as developed and undeveloped according to their level of development determined by State Planning Organization (SPO or DPT). Two provinces from each category were taken. As developed provinces Ankara and Antalya were selected, while Mardin and Diyarbakır were selected as undeveloped provinces. The schools which received the World Bank financed aids intensively in these provinces were determined. Two schools in Ankara, two schools in Antalya, four schools in Mardin, and five schools in Diyarbakır were visited. The list of schools is provided in Table 1. What the World Bank Financed Project Provided The schools that benefited from the Project have received different types of aids. What the World Bank financed project provided to these schools are shown in Table 2. Main Findings of the Study Administrators, teachers and parents indicated that they were not asked about the project aids. Their opinions were not taken. However, they commonly stated that what the Project provided for the schools is what they needed at the schools. School construction and renovation was listed as the first priority. Then, the computers and other educational equipment were mentioned. They believe that what they receive from the project are the basics for the good quality of education. Community contributions were not demanded. Most of the parents, teachers even administrators do not know the exact sources of the aids. It can be considered as a strong indicator of lack of beneficiaries' involvement during decision making, planning, and implementing processes. They also had no role to supervise the project activities such as construction and renovation of the buildings. The poor construction and work of the school building is evaluated one of the serious problem. Gülveren Elementary School in Antalya, Timur Elementary School and Kayı Elementary School in Ankara can be shown as examples of poor construction. However, the renovation of Gazi YIBO in Mardin was completed effectively. Parents, teachers, and administrators complain that their warnings about the construction design deficiencies or poor construction were not taken into consideration. In some schools even fire escapes were not included in the design of the school building. - 36 - TABLE 1 SCHOOLS SELECTED AND VISITED Province Sub-Province Schools Date of Visit Visited by Whom Ankara Kazan Kayı Elementary April 29, 2004 Ali Ekber ªahin (Bussing School) May 3, 2004 Çankaya Timur Elementary May 5, 2004 (Urban School) May 6, 2004 Antalya Elmalı Atatürk March 26, 2004 Elementary (Rural School) Kepez Gülveren March 25, 2004 Elementary March 27, 2004 (Urban School) Mardin Merkez Ortaköy April 17, 2004 Selahattin Gelbal Elementary (Central Village Sch.) Merkez Atatürk April 18, 2004 Elementary (Central Village Sch.) Merkez Cumhuriyet April 19, 2004 Element. (Urban School) Merkez Gazi YİBO April 19, 2004 (Boarding School) Diyarbakır Merkez ªehit Polis Mehmet April 20, 2004 Nuri Doğan Erçin Elementary (Urban School) Merkez Mustafa Kemal April 22, 2004 Elem. (Urban School) Merkez Kayapınar Elem. April 21, 2004 (Central Village) Merkez Arif Eminoğlu Elem. Merkez İskenderpaşa Elem. - 37 - TABLE 2 WHAT THE WORLD BANK FINANCED PROJECT PROVIDED Province Sub-Province Schools Aids from the Project Ankara Kazan Kayı Elementary Construction of the school building with 10 (Bussing School) classrooms. Renovation of the previous building. ICT room without computers Çankaya Timur Elementary Construction of the school building for 960 (Urban School) students, an ICT room with 15+1 computers, UPS, switch, 2 overhead projectors, 1 data projector, 30 loudspeakers, 15 headphones, audio-visual aids, in-service training for teachers Antalya Elmalı Atatürk Elementary An ICT Room with 18+2 computers, UPS, (Rural School) 2 printers, network, 2 overhead projectors, and audio visual aids. Kepez Gülveren Elementary Construction of the school building for 240 (Urban School) students, an ICT room with 18 computers, UPS, 2 printers, network, 2 overhead projectors, and audio visual aids. Mardin Merkez Ortaköy Elementary Data projector and overhead projector (Central Village Sch.) Merkez Atatürk Elementary Data projector, overhead projector, 5 (Central Village Sch.) computers, printer, fax machine, photocopy machine, audio visual aids, science set. Merkez Cumhuriyet Element. Two ICT rooms wit the total of 30 (Urban School) computers, data projector, overhead projectors, science lab supplies and audio visual aids. Merkez Gazi YİBO Renovation of the school building, an ICT (Boarding School) room with 21 computers,. Diyarbakır Merkez ªehit Polis Mehmet Three ICT rooms with the total of 63 Erçin Elementary computers, overhead projector, TV, VCD (Urban School) player, and printer Merkez Mustafa Kemal Elem. Two ICT rooms with the total of 43 (Urban School) computers, overhead projector, TV, VCD player, and printer Merkez Kayapınar Elem. An ICT room with 20 computers, printer, (Central Village) overhead projector, TV, and VCD player Merkez Arif Eminoğlu Elem. An ICT room with 20 computers, overhead projector, TV, and VCD player Merkez İskenderpaşa Elem. An ICT room with 20 computers, printer, overhead projector, TV, and VCD player One of the main purposes of the Project is to reducing classroom overcrowding and double shifting. However, it is determined that the number of students was increased very quickly because of the Project - 38 - activities and aids. For example, Antalya Gulveren Elementary School was designed for 240 students but in 2003-2004 academic year the school has an enrolment of approximately 850 and it became double shifting school. In Diyarbakır and Mardin, parents begin to prefer the schools because of the existence of the computers. Therefore the average class size is 50 to 60 students. Another serious problem is about the maintenance work for the school buildings, facilities and the equipment. In addition to the poor construction, the misuse of the buildings and the facilities damaged what has done. There is no school wide discipline plan to keep the facilities safe. Actually, there is no personnel to do maintenance work. Schools hire contracted servants and parents pay for it. Cleanness was seen insufficient in school buildings. In several schools, it is mentioned that the school administrations signed an agreement with private companies to service computers in case of malfunction or to upgrade. The lack of the security concerns is another problem that determined through the observations and interviews. Seven computers and mainframe have been stolen from Gülveren Elementary School. 15 computers, mainframe, 100 educational CDs have been stolen from Timur Elementary School in Ankara. The schools were insured, an alarm system was installed, and the basement floor windows have been barred after the thefts happened. A teacher at a village school in Mardin had taken the computer to his house because of the security reasons, so the students were not able to use the computer at all. The project aids related to the educational technologies are attractive and motivative factors for the students. A major increase in the number of girls in the villages was noticed. It was observed that in the first, second, third, fourth, fifth, and sixth grades the number of girls and boys very close to each other, but in the seventh and eight grades the number of girls becomes dramatically fewer. Students have positive attitude toward school education. In boarding school in Mardin, computer lab is open during evening ours and weekends in order to enable more students use and therefore some students do not go their homes in weekends to use computers. Organization and distribution of the project aids is poor. Administrators state that some schools which are chosen to give the supplied aids are wrong. Moreover, priorities are listed wrongly. In Diyarbakir and Mardin, central heating system in the villages is the first priority and then school facility construction comes next. It is reported that computers are not used intensively in the courses other than computer courses. There are several reasons of it. First, the teachers are not competent to use computers. Second, computers are not allowed to be taken from the computer laboratories. Third, most of the teachers avoid using technological tools and insist on classical teaching methods. Teachers who attended 70 hours in-service computer training state it is not sufficient and it should be extensive. The instructors who gave the training were incompetent. Teachers and administrators believe in-service training activities do not increase the quality of education due to the fact that given education is poor quality. Conclusion 1. The quality of school constructions is unsatisfactory. 2. The quality of in-service education is unsatisfactory. - 39 - 3. The use of ICT technologies by the computer teachers is satisfactory. 4. The use of ICT technologies by the teachers is unsatisfactory. 5. Beneficiaries' involvement and contribution in the project is unsatisfactory. 6. Security and maintenance of the aids is unsatisfactory. Detailed reports on school visits are available in project files. - 40 - Annex 9. Stakeholder Workshop Results A stakeholder workshop was conducted via video conference on June 1, 2004, to discuss the draft ICR. Participants included the officials from various departments of MONE, representatives from universities, basic education schools, the provincial education general directorates, and the PCC representatives. Teachers, staff from school protection association, principals, representatives from the parent and teachers associations were present at the workshop. The consultants who have conducted the beneficiary assessment for the ICR were also at the workshop to share their findings. A full list of participants is attached to this summary. The Bank informed the session on the purpose of the workshop. Hong Chen, operations officer and primary author for the implementation completion report gave an overview of the purpose and the nature of an ICR. She also made a power point presentation on the draft ICR. The workshop was later open for discussions. The director for the PCC expressed the view that the project went through a lot of difficulties. It was difficult to recruit qualified staff, who had relevant experiences and could speak English, to implement the project. He pointed out that the project failed to facilitate an exchange of international experiences. It would be very important for both the borrower and the Bank to learn relevant lessons and improve the performance of APL2. Other participants shared concerns that the APL1 was not implemented as well as it could have been. MONE representatives stressed that the government's basic education program achieved a lot, and could not understand the reason for some of unsatisfactory ratings given in the ICR. In fact, the representative strongly objected to the unsatisfactory rating for the in-service training subcomponent. After the video connection was cut off, the Bank team in Ankara explained that this evaluation was only for the APL1, not for the government's overall program. A teacher from an Ankara school expressed the concern that the schools had many other problems, such as lack of water and electricity, and that although the computers were purchased for the schools, and the schools had to solve the practical problems. They received some substandard videotapes, and the TVs provided to the school could not be fully utilized for educational purposes. Participants stated that what APL1 provided were useful for the schools and students. They indicated the need for additional support. They were eager to learn the lessons and improve the effectiveness of any follow up projects. Again, after the video connection was cut off, the consultants who had prepared the beneficiary assessments also shared their findings in a continuous discussion of the stakeholder workshop off line. The assessments revealed the problematic areas of the APL1 and hoped the findings would inform out future work. In addition, the consultants pointed out: 1) Under Component 1, subcomponent 2, the ICR rated satisfactory. However, three schools that the consultant visited had been poorly constructed; 2) the fact that leadership at the school level to maintain and improve the project benefits was omitted in the ICR. The school principals do not take any action get parents and communities participate and improve their conditions. They have not plans to protect what were already invested; 3) As the principals indicated: "Only the walls would not create the schools themselves". School planning for the facilities is important, construction of the schools should be based on academic needs. - 41 - Participant List for Stakeholders Workshop, June 1, 2004 Name Organization Title Assoc. Prof.Ali Ekber Sahin Hacettepe University Faculty member Mr. Faik Kazanci Primary Edu.General Directorate Div. Chief Mr. Abdullah Aydogan Ankara National Education Directorate Deputy Gen. Director Dr. Nuri Dogan Hacettegpe Unit Education Faculty Mr.Ali Dede Altindag-Mehmet Memisogullari Primary School Principal Ms. Rukiye Karciga Altindag-Mehmet Menisogullari Primary School Sch.Prot. Assoc. Ms. Isil Boyacioglu Altindag-Mehmet Memisogullari Primary School Teacher Mr.Mehmet Kazim Ozcan Cankaya Kutukcu Alibey Primary School Principal Ms. Sengul Ozdemir Cankaya Kutukcu Alibey Primary School President of PTA Ms. Rabia Guday Cankaya Kutukcu Alibey Primary School V.President of PTA Mr. Mustafa Cengiz Ozan Cankaya Kutukcu Alibey Primary School Teacher Ms. Damla Guney PCC/MONE Translator Mr. Erdogan Serdengecti PCC/MONE Director Ms. Ayseful Tonuc PCC/MONE Unit Head Mr. Zahir Bedir PCC/MONE Deputy Director Mr. Zahir Bedir MONE/in-service training Deputy Head Assoc.Prof. Selahattin Gelbal Hacettepe University Education Faculty Mr. David Fretwell World Bank Lead Emply&Trng Spec Ms. Ferda Sahmali World Bank Senior Operations Officer Ms. Aysu Serpen World Bank Temporary Ms. Hong Chen World Bank Operations Officer Mr. Robin Horn World Bank Lead Education Specialist - 42 - Additional Annex 10. Government's Contribution IMPLEMENTATION COMPLETION REPORT BASIC EDUCATION PROJECT APL I 1. INTRODUCTION Education is the change. Education helps children and the youth acquire necessary knowledge, skills and understandings required for being a part of the society and to develop their own personalities. It is also an investment for human capital. People both learn how to live in the societies they belong to and make livings through their own professions and skills that they acquired at the end of their educational period. People increase their future incomes by investing on their own through education and other resources. All kind of resources improving the creation capacity of individuals can be considered within the framework of this concept. All expenditures for improving this capital, on the other hand, are included under investment expenditures. Education has also returns for the society, other than those obtained by independent decision makers such as students and parents. These returns of education are comprised of such inputs as more effective democracy and democratic institutions, more efficient markets, low level of crime rating, decreasing the penalty system costs, decreasing social aid costs, eliminating deficiencies of capital markets, increasing the number of institutions providing community services, better neighborhood relations, decrease in the birth rates and increasing returns for local and international regions. Taking into consideration these returns which have nothing to do with monetary benefits, education has become much more important and effective not only in economical growth and societal development but also in the modernization of the community. Although returns of education in all levels are always positive, societal returns of basic education are always much higher than those of upper levels of education. In general, while the levels of education get higher, both societal and individual returns get decreased. However, individual returns, especially at the university level, are much higher than societal returns. Both people and governments are making investments in education due to its returns for individuals and societies. One of the most crucial reasons for making basic education be much more important than any other levels of education and be free and compulsory at schools, is the fact that the societal returns of basic education is much higher than those of other levels of education. Due to this reason, governments gives much more due importance on basic education than any other educational levels and makes much more investments on this educational level. The main purpose here is to ensure all age population in the country access and complete basic education level. As in many other countries, education has been given at a great importance in Turkey. Atatürk, the Great leader of Turkey, has never given up his attempts for increasing the quality in education even during the Independent War period and always given the first priority to education in Development Plans. Basic education was given comprehensive coverage in the first five-year development plan (1963-67) developed in 1963 in which "planning era" has been started. Education was the first and the most crucial problem for Turkey in this period. Education has always been given the same importance, and in each period educational problems were discussed and educational targets were revealed in the following - 43 - development plans. In all development plans, schooling rates have always been a common problem The most radical change for educational problems was the recommendation for transferring into 8 year compulsory education implementation which was defined under the 7th Five Year Development Plan for the years between 1996 and 2000 With this recommendation, 8 year compulsory continuous education has been put into implementation under the Law no. 4306 which was become effective after being published in the Official Gazette dated August 18, 1997 with ref. No. 12084. Following this implementation, the Catching up the Age in Education Project was commenced for the effective implementation of necessary arrangements. The `Basic Education Program' which is an action program for ensuring implementation of compulsory eight-year basic education was commenced through the passage of the Basic Education Law no. 4306. The Government of Turkey has provided the necessary support to this Program through a Loan Agreement signed with the International Bank for Reconstruction and Development on June 25, 1998. 2. BASIC EDUCATION PROGRAM The fundamental structure of the National Education System has been defined under the National Education Law no.1739. Education system is comprised of sub systems to meet the needs of individuals. Turkish Education system is comprised of two integrated sections. (TURK; 1999:110). Formal education is the regular education carried out within the framework of the schools with the programs prepared for the individuals at a certain age group and the same level. Formal education includes the pre-school education, primary education, secondary education and higher education. Non formal education, on the other hand, includes all kinds of educational activities arranged for the support of formal education or those other than formal education for the individuals who has never participated within formal system and/or left the school at any educational level (TURK; 1999:110). Pre-school education includes the education, on a volunteer basis, of the children between the ages of 3 and 5. Pre-school education institutions can be opened either as independent kindergartens or as kindergarten classes within the primary schools or as practice rooms within the other training institutions, as necessary. (TURK; 1999:111). Although an important progress was obtained after the passage of 1924 Constitution in which primary education was defined as compulsory, the desired level was not achieved at that period due to such reasons as the need for teachers, the lack of school buildings and the less number of students in some villages. Despite all positive attempts of the governments, no law structuring primary education has been put into force till 1960s. Therefore, the Primary Education and Training Law which was put into effective in first years of the planning era, is a milestone for education system in Turkey in terms of both its general provisions and its specific provisions on primary education (TURK; 1999:112-113). The definition, scope, period, type of instruction, academicians, attendance, land and facility works of schools, construction and equipment works, incomes and outcomes regarding primary education and planning works for primary education are determined by the decree in law and all these reveal the fact that primary education with a great number of social returns, is the fundamental step of the Turkish Education System (TURK; 1999:112-113). Primary education includes the training of children between the ages of 6 and 14. Primary education is compulsory for both male and female citizens and free in public schools (TURK; 1999:113). - 44 - The objectives of the primary education are to make each Turkish child acquire the basic knowledge, ability, attitude and skills necessary for being a good citizen, to grow him/her in compliance with the national moral approach, and to make him/her ready for the following education and life in accordance with his/her interests, ability and skills, in compliance with general targets and basic principles of National Education (TURK; 1999:113). 8 eight year compulsory continuous education has been put into implementation under the Law No.4306 which has become effective by being published in the Official Gazette dated August 18, 1997 with ref. no. 12084. Innovations brought by the passage of this Law are as follows (MEB;2002:67-68): · the basic education institutions shall be comprised of eight-year basic education schools, continuous education shall be implemented in these schools, and the students to enroll and finish these schools shall be granted a primary education diploma, · the terms "primary school" and "secondary school" defined within the Laws No. 222, No. 1739 and No. 3308 both together and independently shall be hereinafter called under one name as "basic education school", · both the students at the 6th, 7th and 8th grades in the secondary schools and those in apprenticeship level in vocational training centers shall complete their education within these institutions mentioned above, and no students shall be enrolled in these institutions as from the beginning of the 1997-1998 Academic Year, · both the successful students of the preparatory classrooms in basic education schools in which some of the lessons are in foreign language, and the ones who have right to be educated in these schools for the 1997-98 Academic Year shall complete their compulsory education within the schools mentioned above, · the National Education System shall be arranged in order to guide the students as to their own interests, abilities and capabilities, and the arrangements of the preparatory classes shall be conducted for the secondary education institutions in compliance with the curricula of the secondary education, · that necessary studies should be carried out by the guidance services in order to submit instructive information on which schools and programs provide better advantages in business life and on life standards provided by these professions, and to assist students in their occupational selection at the second term of the last academic year in basic education, · the graduates can be trained in business fields as candidates for apprenticeship, the formal age for apprenticeship shall be determined as 14, and those above 19 years old will be provided with the apprenticeship opportunity and the candidates shall have at least basic education diploma under the article No.9 of the Law No.4702 adopted on 19.06.2001. Procedures regarding imposing levies on some papers and transactions as a contribution to education have been carried out between the dates of September 1st 1997 and December 31st 2000. The Law No. 4702 has been extended till December 31, 2010 in a way to include secondary education (MEB;2002:68). - 45 - For the effective implementation of these arrangements, Catching Up The Age in Education Project has been put into force (MEB;2002:68). The main targets of this Project were (a) to reach into 100% schooling rate in basic education, (b) to eliminate gradually multi grade classroom implementation, (c) to decrease the class size into 30 students, (d) to transfer from double shifting to normal education, (e) and to increase the quality in education (MEB;2002:68). With the policies for achieving the targets, the following were planned: (a) increasing the numbers of schools and classrooms, (b) expanding bussing education implementation, (c) increasing the boarding capacities of YIBOs and PIOs, (d) meeting the needs of poor students and (e) providing schools with necessary physical infra-structure and equipment compliant with contemporary criteria (MEB;2002:68). The Basic Education Program is the Government's action program to apply its new basic education strategy developed as a result of the Law No.4306 which commenced eight year compulsory continuous basic education. The objectives of the Program) are: a) to achieve universal coverage in an expanded, eight-year basic education, b) to improve the quality and interest of the society to schools, and c) to make basic education schools a learning resource for the community (MEB;2002:87). BEP aims at (a) increasing the school enrollment rates to 100% and (b) increasing student achievement levels for expanding basic education coverage (Resmi Gazete;1998:38). Within this scope, BEP aims (a) to eliminate the gap between female and male enrollment rates, (b) to increase transition rates from 5th to 6th grade to 100% and (c) to increase the preschool education enrollment rates from 7% to 16% (Resmi Gazete;1998:38). The Program aims to achieve these objectives by a) expanding the capacity of basic education schools throughout the country, b) facilitating school attendance of children who are least likely to attend school, c) reducing classroom overcrowding and double shifting, d) improving training and incentives for teachers, e) improving the supply of educational materials to basic education schools, f) introducing computer-aided learning, and g) increasing parental and community involvement in schools (Rapor; 1998: 2). 3. BASIC EDUCATION PROJECT The Children's Education Right executed with the approval of the Declaration of Human Rights by the United Nations of 1948 stating that "Each individual has a right to receive education. It is necessary that free education should be provided for each individual at least at primary and basic education level. And basic education is compulsory..." (Article No. 26/1), and the Declaration of Children Rights of 1959, along with the activities related with these rights, are still carried on the international level, with the assistance of the Agreement on the Fight For Discrimination in Education by UNESCO of 1962, the International Agreement on the Economical, Social and Cultural Rights by the European Council of 1976 and the Agreement on Children's Rights by the United Nations of 1990. The negotiations regarding the support of the World Bank for the program on the enhancement of the basic education in Turkey between the Turkish Government and the World Bank have been launched following the "Jomtien Declaration of Education For All", and "the Activity Plan For Meeting The Needs of Basic Education" held in cooperation with the UNICEF, UNDP, UNESCO and the World Bank in 1990. At the end of these negotiations, it has been agreed upon that the World Bank shall compensate a part of the financial support for the Basic Education Program, which has been launched with the execution of the Basic Education Law No. 4306 and for which the total cost has been estimated as 11.3 billion US Dollars (Progress Report No. 2, PCC, August 1997-December1999). - 46 - The Government of Turkey committed itself to implement Basic Education Program (BEP) for reforming and improving the basic education system in Turkey through its official Letter to International Bank for Reconstruction and Development (IBRD) on May 26, 1998. In its Letter, the Government of Turkey requested from the Bank a loan for the support of this Program and the assistance of the Bank in terms of financing the Project which constitutes of the first phase of the Program. The Bank's Board approved this request and the related Loan Agreement was signed by/between the Government of Turkey and the IBRD on June 25, 1998. Targeted population of this Program would be all individuals including parents, students and teachers together with public and/or private institutions. Through this Loan Agreement, the Bank provided the Government of Turkey with a loan in the amount equivalent to USD 300,000,000. The Project supports the first phase of the Basic Education Program aiming to a) achieve universal coverage in an expanded, eight-year basic education, b) improve the quality and interest of the society to schools, and c) make basic education schools a learning resource for the community (Resmi Gazete;1998:8). The targets of the Project are as follows (Resmi Gazete;1998:8).: (a) improving the capacity of basic education schools, (b) training of teachers, school principals and inspectors, (c) enhancing the implementation capacity of the central and provincial/sub-provincial Ministry units (d) Creating a mechanism for the monitoring and evaluation of the Program outcomes. This Project is comprised of five components (Resmi Gazete;1998:8-9,38-40).: A) Expanding Basic Education Coverage (a) Increasing the numbers of basic education teachers and inspectors including employment of 150,000 teachers and inspectors (b) implementing activities for increasing the flexibility in teacher assignments and effectiveness in employment/distribution of teachers including decentralization of teacher assignment process. (b) of additional facilities and equipment of basic education schools, YIBOs and PIOs where appropriate. (i) Rehabilitation of approximately 200 village schools in Şırnak, Van, Hakkari, Bitlis and Mardin provinces (ii) Renovation of approximately 100 central village schools in Kocaeli, Yalova, Batman, Iğdır and Bartın provinces; and (iii) renovation and construction of additional facilities for approximately 40 basic education schools in Ankara provinces and those to be mutually agreed by the Government and the Bank. B) Increasing the Basic Education Quality (a) Improving the capacity of basic education teachers, school principals, inspectors and provincial national education directorate staff by providing a comprehensive in-service training program including the use of Information Technologies. (b) Providing necessary educational materials and equipments to basic education schools in rural areas. (c) Enhancing information technologies in basic education through the provision of computers and educational software. - 47 - C) Implementation Support (a) Enhancing the institutional capacity of PCC through the provision of consultancy services, training, study tours and equipments, (b) Enhancing the institutional capacity of Research, Planning and Coordination Council through the provision of consultancy services, training and equipments for the effective use of MIS and improving school mapping process. D) Monitoring and Evaluation (a) Providing technical assistance to MONE in implementing, monitoring and evaluation the Project (b) Monitoring Response Facility (MRF) * Developing a Monitoring Response Facility under the MONE and providing necessary goods, civil works and consultancy services for the beneficiaries in implementing sub-projects under MRF. Despite three extensions in the Project closing date due to several reasons, considerable progress has been accomplished in the implementation of the first APL of USD 300 million to support the Implementation of the Government's Basic Education Program. 4. ACHIEVEMENTS IN BASIC EDUCATION PROGRAM (i) General Targets of the Basic Education Program a. to achieve universal coverage in an expanded, eight-year basic education, b. to improve the quality and interest of the society to schools, c. to make basic education schools a learning resource for the community (MEB;2002:87). (ii) Specific Objectives of the Basic Education Program (a) to eliminate the gap between female and male enrollment rates. (b) to increase transition rates from 5th to 6th grade to 100% (c)to increase the preschool education enrollment rates from 7% to 16% (Resmi Gazete;1998:38) The Program aims to achieve these objectives by a) expanding the capacity of basic education schools throughout the country, b) facilitating school attendance of children who are least likely to attend school, c) reducing classroom overcrowding and double shifting, d) improving training and incentives for teachers, e) improving the supply of educational materials to basic education schools, f) introducing computer-aided learning, and g) increasing parental and community involvement in schools (Rapor; 1998: 2). Up-to-now, there has been major progress in implementing the Basic Education Program and achieving its objectives. Through the implementation of the Program, the Ministry of National Education (MONE) constructed 106.993 new basic education classrooms during the five years 1998 ­ 2002; and provided annual cycles of in-service training for teachers, school administrators, provincial/sub-provincial staff and basic education inspectors. Accordingly, while the number of teachers, schools and students were respectively 302,982, 45,649 and 9,102,074 in 1997-1998 academic year, today totally 390,275 teachers are working in 35,168 basic education schools with 10,331,619 students. During this period, the number of female students which was - 48 - 4,085,452 in 1998 has increased to 4,828,276 in 2002-2003 Academic Year. This increase in the enrollments of female students to basic education level simply indicates one of the most important outputs of the Program for the country. On the other hand, totally 10,481 schools were closed during this period, as a result of bussing implementation and terror attacks especially in South Eastern and Eastern parts of Turkey. The transition rates from 5th to 6th grade by academic years are as follows: YEARS Number of 5th grade Students Number of 6th grade Students Transition Rates by years MALE FEMALE TOTAL MALE FEMALE TOTAL 1997-1998 637,752 573,421 1,211,173 632,086 405,876 1,037,962 1998-1999 656,650 590,331 1,246,981 738,040 574,225 1,312,265 100 1999-2000 657,028 589,289 1,246,317 700,767 587,246 1,288,013 100 2000-2001 670,567 603,397 1,273,964 718,859 609,848 1,328,707 100 2001-2002 697,276 623,562 1,320,838 683,657 601,680 1,285,337 100 2002-2003 663,374 595,516 1,258,890 679,944 590,828 1,270,772 96 2003-2004 664,764 604,058 1,268,822 669,944 586,827 1,256,585 100 Please note that the open education data are not included in this Table. Resource: Figurative Data for National Education Series between 1998-1999 and 2003-2004 academic years by Research, Planning and Coordination Council, MONE./ 1997-1998 National education data by State Statistics Institute. However, in terms of pre-school enrollment, the specific objective to increase the ratio from 7 to 16, has not been achieved totally yet. The schooling ratio for pre-school education level was 8.9 in 1997-1998 academic year. According to the official data by Research, Planning and Coordination Council (APK) under the MONE, during the 2000-2001 Academic year, the schooling rate for pre-school education has reached to 10.3%, in basic education to %100.7, in secondary education to 64% (22.2% in vocational training and 41.8% in general secondary education) and in higher education 17.8%.Today the schooling rates for pre-school education and basic education have increased to 13.1% and 98.2% respectively. (Data from APK, MONE, 2004) In order to provide social support for the needy basic education students in rural areas with poor families, the MONE has commenced distribution of free textbooks to those students starting from 1999-2000 academic year. In 2002-2003 academic year, totally 12,472,292 free textbooks were distributed to the students throughout the country. Accordingly, the numbers of free textbooks distributed by the MONE to needy children by years are as follows: - 49 - Academic Year # of Textbooks 1999-2000 4.090.000 2000-2001 3.913.442 2001-2002 1.631.375 2002-2003 2.397.769 Resource: Data from Research, Planning and Coordination Council, 2003 In 1999-2000 academic year, after the Marmara Earthquake, 55,201 free textbooks were provided for the students in Düzce province while 384,505 were provided for those in Sakarya province. In 2003 -2004 Academic year however, totally 81,835,281 free textbooks were provided for all students in basic education throughout the country. Besides, for improving the quality of educational opportunities for the students at schools with multi-grade class Implementation, together with the children living in such areas where no basic education school is available, and those at the age of basic education living in places with low and dispersed population, bussing basic education implementation has been put into implementation by the DG for Primary Education under the MONE. Bussing Basic Education, which was carried out in nine provinces in 1990-1991 school year, rapidly became widespread on the demand and reaction, and launched to be carried out in 75 provinces in 1998-1999; and in 79 provinces in 1999-2000 academic year. Through Bussing Basic Education system, the students at schools with multi-grade class implementation, together with the children living in such areas where no basic education school is available, and those at the age of basic education living in places with low and dispersed population, are transported daily to basic education schools in selected centers, which heals their educational opportunities. Methods and transactions on Bussing Basic Education implementation are arranged under "Bussing Basic Education Directive under the Ministry of National Education " which was put into force on 24.06.1994. Today, totally 646,457 students in 79 provinces are being transported to central bussing schools daily and all transportation and lunch expenditures of these students are compensated by the Government under the Law No.4306. The textbook and stationary needs of the bussed students are completely compensated by the MONE for free. Please see table below for the bussing implementation status by academic years. ACADEMI #of Provinces #of #of Central #of Bussed #of Bussed C YEARS Sub-Provinces Schools Schools Students 1989-1990 2 5 12 305 1990-1991 9 35 78 258 3.289 1991-1992 29 160 408 1.094 18.256 1992-1993 43 325 938 2.371 53.676 1993-1994 56 537 1.653 4.416 84.263 1994-1995 57 493 1.630 4.683 74.981 1995-1996 62 557 2.182 5.994 95.554 1996-1997 64 576 2.336 7.502 120.998 1997-1998 72 740 4.803 18.213 281.833 1998-1999 75 810 5.697 27.081 521.218 1999-2000 75 812 5.633 27.994 635.041 2000-2001 76 791 5.249 25.967 607.918 2001-2002 78 801 5.373 27.665 636.508 2002-2003 79 814 5.424 28.044 661.757 2003-2004 79 807 5.559 28.716 647.921 Resource: Data from Research, Planning and Coordination Council, 2003 - 50 - In order to expand basic education coverage, the Government of Turkey has also given due importance on expanding Regional Boarding (YIBO) and Pensioned (PIO) Basic Education Schools. All expenditures in YIBOs and PIOs are compensated by the Government. While the number of YIBOs and PIOs was 153 in 1997-1998 academic year, this number has been increased by 272% to 561 in 2003-2004 academic year with the boarding capacity of 198,181 students. Despite all these achievements under the Basic Education Program, there is a lot much to be done in basic education system based on gradually increasing the needs of students, parents, and community together with those of labor market in Turkey. 5. ACHIEVEMENTS IN KEY PERFORMANCE INDICATORS UNDER THE FIRST PHASE OF THE BASIC EDUCATION PROJECT Performance success of the Basic Education Project can be evaluated under 4 components: 1.Expanding Basic Education Coverage, 2. Increasing Basic Education Quality, 3. Implementation Support and 4. Monitoring and Evaluation. Accordingly, the realization of the following outputs as defined in line with the Project objectives. Accordingly, the key indicators defined under the Project Appraisal Document for the Basic Education Project I, are as follows: A) Expanding Basic Education Coverage i. Construction of 450,000 new student places in total. ii. Rehabilitation of 200 existing schools in rural areas B) Increasing Basic Education Quality i. In-service Training of 300.000 existing and 30.000 new teachers ii. Training of 3.000 school inspectors iii. Provision of educational materials for 25.900 rural schools iv. Implementation of social assistance program in centralized bussing schools and village schools v. Provision of free lunch for children with poor families vi. Preparation and equipment of 2.800 IT rooms vii. Doubled number of IT formators C) Implementation Support i. Establishment of monitoring and evaluation capacity for the Program. ii. Provision of equipment and technical assistance for implementation iii. Updating APK's MIS and training of its staff D) Monitoring and Evaluation i Establishment of monitoring and evaluation capacity for the Program implementation ii Public Promotion Campaign for Basic Education iii Monitoring Response Facility 1. Expanding Basic Education Coverage For expanding basic education coverage, the construction of 450.000 new student places and renovation of 200 rural schools were planned to be completed by the end of Phase I. Accordingly, under the Basic - 51 - Education Program, totally 106.993 new students places were constructed under the Law No.4306 (APK, 2003), which means a capacity of 3,209,790 students. In terms of number of renovation under the BEP APL I, totally 1690 rural schools, YIBOs and PIOs were renovated (including 124 central primary education schools in 6 provinces (Bartın, Batman, Iğdır, Şırnak, Kocaeli and Yalova) while 166 schools were constructed with additional facilities and infra structural works were carried out for 2802 IT Rooms during project implementation (Progress Report, 2003), which reflects a great progress in achieving component targets. 2. Increasing Basic Education Quality In-service Training of 300.000 existing and 30.000 new teachers, training of 3.000 school inspectors, provision of educational materials for 25.900 rural schools, implementation of social assistance program in centralized bussing schools and village schools, provision of free lunch for children with poor families, preparation and equipment of 2.800 IT rooms were planned to be completed under this Component. Accordingly, fully-equipped IT Rooms were installed in 2,802 primary education schools and Internet access was provided to these schools; 130 Notebooks and 1 Server were provided for the Ministry's Basic Education Inspectors and 1500 Notebooks for Master Formators, Primary Education Inspectors and Ministry's Inspectors. These activities are far beyond the expected outcomes. Besides, Computer Hardware-Software and Peripheral Equipment have been procured for 22,854 Rural Schools and overhead projectors were provided for 18,517 rural schools and 6255 data projectors for 2802 basic education schools installed with IT classes and for 3453 basic education schools to be installed with ICT classes under the APL II has been procured under the First APL. Under the First Phase of the Basic Education Project, the following in-service trainings were provided for the school administrators, teachers and inspectors by the end of the Project. Totally 221,000 teachers were provided with in-service training for the most effective and efficient use of IT rooms, hardware and software products under the Government's Basic Education Program. 3000 primary education inspectors were provided with in-service trainings on different subjects. Other than those provided by the Ministry, some training were provided by the awarded hardware and software firms. The numbers of teachers participated in trainings provided by hardware and software firms, are respectively 6,677 and 9,251 in total. Foreign Language (English) Training was provided for 100 Ministry Staff in order to improve foreign language capacity of its staff over two sessions. Refresher In-Service Training Activities for 500 Master IT Formators were arranged over two sessions. Moreover, in-Service Training on Active Learning and Teaching Methods was provided for 405 Basic Education Teachers (Progress Report No:10, MONE PCC) Under the First Phase of the Basic Education Project, the Ministry provided educational materials for 22,287 rural schools and 2,993,692 students throughout the country (progress Report No:10, MONE PCC) . Within this scope, the number of school provided with educational materials is much less than the planned since the number of rural schools were decreased to 22,287 through these years. 3. Implementation Support Establishment of monitoring and evaluation capacity for the Program, provision of equipment and technical assistance for implementation and updating of APK's MIS and training of APK staff were planned to be completed under Project Component 3: Implementation Support. - 52 - Since monitoring and evaluation capacity of the Program has been under the responsibility of Research, Planning and Coordination Council and DG for Primary Education under the MONE, the Ministry did not prefer to establish an independent M&E capacity for the Program. However, required individual consultancy services were provided for the PCC and line units of the MONE under this Component of the Project to strengthen the physical and institutional capacity of the MONE and the PCC. Besides, all costs of the seminars and workshops to strengthen the physical capacity and to improve the qualifications of the relevant staff were allocated from this Component of the Project. 4. Monitoring and Evaluation Under the M&E component of the Project, establishment of monitoring and evaluation capacity for the Program implementation, public promotion campaign for Basic Education and Monitoring Response Facility activities were planned to be completed by the end of BEP APL I. Since Monitoring& Evaluation is an indispensable tool for the effective and efficient implementation of the Program/Project, both to support the implementation of the Program and Project and to design new initiatives to make the timely necessary adjustments in the operation of the Program/Project and is also an accountability tool in the use of public resources by providing the greater transparency, the PCC has established a M&E Unit comprised of statisticians and measurement and evaluation specialists. Monitoring and evaluation activities were included under the Basic Education Project to assess the implementation of the Project as planned and the achievement of the Project's performance goals. These activities were the key elements in guiding the Program implementation and in determining the modalities of partnership between the Government and the Bank through various phases of Bank's support to the Basic Education Program. Under the Basic Education Project, monitoring and evaluation activities were closely related with each other. Project activities and goals were planned to be evaluated at all stages of the Project cycle to help strengthen Project and Program design and implementation since in their entirety, these blocks of activities played a critical role in guiding the BEP and in sharing its achievements. MONE carried out monitoring and evaluation of the first phase of the APL effectively, but under a quite different system based upon limited use of consultants and data from the Ministry permanent documentation system. The Consultancy Services aiming at 3-phased Evaluation of the Program in terms of Social, Institutional and Cost Analysis aspects, were procured and the related contracts were conducted to monitor impacts of the Project. Under the Monitoring Response Facility sub-component of the Project which provides financial support to school-based projects, 85 school projects were conducted successfully in İzmir, Van, Bitlis, Ankara and Mardin provinces. Regarding PR Promotion Campaign, upon the request of the Ministry, the World Bank has cancelled this activity and trigger condition. PR promotion activities aiming to introduce the Basic Education Program and Eight Year Compulsory Education to public were also given due importance during project implementation. For the effectiveness of these activities, such publications as books, bulletins and brochures on the BEP were prepared and distributed by the PCC/MONE; and such important arrangements as symposiums, meetings, and seminars concerning Basic Education Program, were organized by the MONE. Besides, in order to find out the impact of the ICT rooms on basic education, the MONE signed a contract with a consultancy to conduct ICT Impact Study throughout the country. - 53 - Most important achievements of M&E were the social assessments and Monitoring Response Facility implementations under the Project: a. SOCIAL ASSESSMENTS Social Assessment Studies made under the Project are as follows: (a) Social Impact Study for Basic Education Schools (b) Cost Analysis Study for Basic Education Schools (c) Institutional Impact Study for Basic Education Schools a.1. Social Impact Study for Primary Education Schools This Study was conducted by Educational Research and Implementation Center under the Education Faculty of Ankara University starting from July, 2002. The main purpose of this Study was to determine social impact of the basic education schools based upon the comments of school principals, teachers, students, parents, educational leaders (provincial, sub-provincial national education directors, basic education inspectors) and community leaders (muhtars and imams). The core of this Study, in which both qualitative and quantitative research methods were followed, were the school principals, teachers, students, parents in bussing basic education schools, PIOs and YIBOs and the community leaders in these areas such as imams and muhtars together with provincial-sub-provincial national education directors and inspectors in those areas. At the end of this Study, it was determined that YIBOs, PIOs and Bussing schools are useful for the community and these schools were perceived as an important incentive in terms of increasing the quality in education, ensuring equity in educational opportunities and developing the capacity of students. Moreover, it has been also determined that these schools increased literacy and schooling rates; that they prevented marriages at early ages and child labor and child crimes, and that they ensured awareness for the community in terms of health, population growth and disease protection. In line with this, the facts that these schools created necessary awareness about democratic rights and freedoms; led to positive changes in perception and life styles of the community; affected traveling, clothing and reading hobbies of the community and increased the community participation in social, artistic and educational facilities, were revealed as a result of the social impact results of these schools.. a.2. Cost Analysis for Basic Education Schools In this research study, the focus was on how unit student costs vary among basic education implementations in terms of investment expenditure and current costs. The study was run on the data collected from 144,299 students from 219 primary education schools selected from Turkey. While selecting these schools, the results of socio-economic regions study performed by state Planning Organization and the geographic regions in Turkey, were taken into account and special emphasis was given to Eastern and Southeastern regions. In the form up of sampling methodology, regions and school types that is, 8-year basic education schools, YIBOs , PIOs and bussing schools were taken into consideration. The data was collected by assigned staff of Baskent University who visited the selected schools and provincial and sub-provincial national - 54 - education directorates. The results of the research can be summarized as follows; (a) Total cost per student in Turkey as a whole is TL 490,345,000. Staff expenses are the most major component of this. The share of personnel expenses within total cost varies according to regions (between 65% and 85%). A similar difference was observed to exist among schools types. (b) Staff expenses excluded from the other items in total cost are: a. Supply expenses constitutes 87 of the total cost; while 28% of this ratio is heating expenses, and 46 is nutrition expenses. The share of total nutrition expenses is naturally the highest at YIBOs. b. The share of investment costs within total cost is 35. This ratio varies according to regions and schools types. This ratio is 26 for primary education schools whereas it is 56 for YIBOs c. Expenses generated from the revenues supported through other finances such as donations and parents' contributions to schools and school protection associations seem to constitute 47 of the total cost. However, this share was observed to be higher. (c) Total cost per student varies among regions. The lowest cost belongs to the Marmara region with TL 359,271,836 and the Aegean region with TL 672,834,366. The cost of the Eastern, the Western Black Sea regions and Central Anatolia region are higher than cost of the other regions. Moreover, there are important differences among schools in the same region in terms of cost per student. (d) Cost per student indicates significant differences according to school types as well. While the cost per student is TL 872,987,127 at YIBOs, it is TL 466,316,175 at 8-year primary education schools without boarding students. (e) The availability of boarding students makes difference in cost per student. At 8-year primary education schools without bussing student, a cost per student is TL 389,819,000. This value is TL 654,210,000 at the same type of schools with bussing students. This is something normal since the difference occurs due to the transportation and food expenses and the amount allocated for bussing students. (f) The schools with the highest costs are YIBOs, followed by PIOs and normal primary education schools. (g) The reason why staffing cost is much more higher in YIBOs is that all students in these schools are boarding students, which requires more variety in staff and more number of staff than other school types. Also, being a boarding school makes it necessary to be heated longer hours, which causes an increase in heating expenses. (h) With respect to investment cost, YIBOs seems to be at the first rank. The main reason for this situation is the higher number of additional facilities such as restaurants, dormitories, and laundry rooms and sports center at this type of schools than others. (i) Food costs at YIBOs are higher than bussing schools as meals are prepared in their own facilities. (j) According to the calculations for bussing education, the cost per a bussing student corresponds to the cost of 1.6 non-bussing student. - 55 - a.3. Institutional Impact Study for Basic Education Schools The main purpose of this Study is to find out institutional impacts of the basic education schools by evaluating the physical rehabilitations obtained at the end of the First Phase of the Basic Education Project and determining the comments of school principals, teachers, students, parents and other stakeholders on the impacts of these rehabilitations on educational outcomes For this aim, firstly 24 sub-provinces from 10 provinces (2 or 3 from each province) were selected focusing on East and South Eastern Anatolia provinces as defined under the Specifications. The developmental indicators determined in accordance with the results of the SPO's `Socio-Economic Developmental Ranking of the Provinces in Turkey' were taking as a basis in the selection process of the provinces and sub-provinces. Then, totally 219 basic education schools (158 normal schools, 39 YIBOs and 29 PIOs) were selected by taking into consideration such features as the type of school, teaching type (full day, double shifting or bussing), the number of students, its location (urban, rural) etc. And related information was filed from these schools. The summary of the results of this Study are as follows: a) Bussing education implementation gets more common in basic education. Some serious problems that may cause adverse affects on objectives were determined in bussing implementation. The system and implementation need to be analyzed in details and re-structured by taking into account the local conditions of the areas. b) In most of the basic education schools other than YIBOs, pre-classes were opened. It has been determined that these pre-classes were opened by the school administrations as per the requests of the parents. This situation indicates that the importance of the pre-school education is well understood by the parents in most areas. However, the existing number of students needs to be increased and the existing places need to be improved. Moreover, the pre-school education needs to be re-structured in a way to include all age population by developing different models and determining the burden it will cost to education system in details. c) In 27% of basic education schools, double shifting has been determined to be ongoing. Many principals, teachers and parents complained about double shifting Necessary incentives need to be brought into life to eliminate double shifting implementation. d) It has been observed that a great number of schools have been renovated under the Basic Education Project and many schools (projected) have been constructed since 1957. The rate for the reconstructed or rehabilitated schools is 53%. e) In all school types, the numbers of students were increased in 2001-2002 academic year compared with those in previous academic year. The female students' ratio in total is 42% in basic education schools, while that of those in normal basic education and PIOs is 44% in total. f) According to the results of this Study, 2% of the basic education schools do not have school gardens. The ratios for YIBOs, PIOs and normal basic education schools with adequate school area and gardens are 66%, 59%, 46% respectively. - 56 - b. MONITORING RESPONSE FACILITY (MRF) `Monitoring Response Facility' was developed under the Basic Education Program in 1999 in order to meet the needs that may occur during project implementation and to support school based pilot projects and incentives. Monitoring Response Facility supported financially the projects of the schools for which they have no budget and ensured educational needs of rural and gecekondu schools be determined by the school protection associations, school principals, teachers, students and parents. The Purpose of the MRF: To finance school-based innovative projects prepared by school principals, teachers, students and parents through a participatory approach for increasing school attendance and basic education quality. Scope of the MRF: The MRF was implemented in totally 85 schools settled in gecekondu areas of Ankara, İstanbul and İzmir provinces and rural schools settled in rural areas of Hakkari, Bitlis, Şırnak, Mardin and Van provinces. In 68 schools of this total Multi-purpose Hall Projects and Technology Rooms were brought into life while in 7 of them Sports Facilities Projects, in 4 of them Contemporary Library Projects, in 2 of them Guidance Projects, in 2 of them Pre-school Education Projects and in 1 of them Arts Atelier Project and 1 of them Contemporary Study Room for PIO. In the selection of these innovative projects and schools, those which will increase basic education quality; promote school attendance; take at least 20% community support in financing and those with 60% ratio for classroom/students were given the priority 6. PROJECT FINANCING A. Amendments to the Loan Agreement Since the beginning, the Basic Education Project has been amended six times. The first amendment of the Project was made due to Marmara Earthquake disaster on December 3, 1999 to include Marmara Earthquake Relief Component including rehabilitation and reconstruction of pre-school, basic education and secondary education schools which were damaged in the Marmara earthquake of August 1999; providing support for homeless teachers in the form of textbooks, educational materials, school uniforms, basic clothing and nutritional supplements to students affected by the earthquake, contracted transport services to effected teachers and students, technical assistance and consultancy services to develop psychological counseling and training including those to monitor and evaluate the emergency recovery actions and their consequences. (Loan Amendment Letter dated 03.12.1999, the World Bank). The others were simply related with the reallocation of the loan proceedings. - 57 - The sixth amendment has been mutually agreed on December 11, 2003 with the cancellation of USD 6,000,000 from the Loan. According the last amendment made to the Basic Education Project I, the allocation of the loan proceedings is as follows: Category Amount of the Loan % of Expenditures Allocated (USD) to be Financed 1. Civil Works under Parts A.3 and B.3 94,200,000 87% of the Project 2. Equipment and materials under Parts 189,820,000 100% of foreign B.2, B.3 and C of the Project expenditures, 100% of local expenditures (ex- factory cost) and 84% of local expenditures for other items procured locally 3. Consultants' services under Parts A.3, 1,350,000 100% C and D of the Project 4.Training, fellowships and study tours 3,600,000 87% under Parts B.1 and C of the Project 5.Goods, works and services for sub- 750,000 87% projects under Part E of the Project 6.Civil Works under PArt F of the 0 Project 7.Goods under Part F2 of the Project 2,300,000 100% of foreign expenditures, 100% of local expenditures (ex- factory cost) and 84% of local expenditures for other items procured locally 8.Contracted Transport Services 1,980,000 87% 9.Consultant Services 0 10.Unallocated 0 TOTAL 294,000,000 B. Use of the Loan The Basic Education Project was scheduled to last for three years, starting in June 1998 and ending in July 2001. Upon the request of the Turkish Government, the Project has been extended to December 2003. Although total loan amount allocated for the Basic Education Project was USD 300,000,000 till the last year of the Project, the Bank has decided to cancel USD 6,000,000 from the loan through its Letter dated June 3, 2003 due to the time constraints and the loan amount has been decreased to USD 294,000,000 in total. - 58 - As of February 18, 2004, total amount used from the loan is USD 260,968,478.75, while USD 33,031,521.25* (including cancelled amount in equivalent to USD 6,000,000) has not been used. Category Amount of Loan Used Amount (USD) Remaining Amount Allocated (USD) (USD) 1. Civil Works under 94,200,000 93,636,880.53 563,119.47 Parts A.3 and B.3 of the Project 3. 2. Equipment 189,820,000 157,557,278.98 32,262,721.02 and materials under Parts B.2, B.3 and C of the Project 3. Consultants' services 1,350,000 1,096,936.73 253,063.27 under Parts A.3, C and D of the Project 4. Training, fellowships 3,600,000 3,701,212.23 -101,212.23 and study tours under Parts B.1 and C of the Project 5. Goods, works and 750,000 706,858.47 43,141.53 services for sub-projects under Part E of the Project 6. Civil Works under 0 0 0 Part F of the Project 7. Goods under Part F2 of the Project 8. Contracted Transport 1,980,000 1,978,701.51 1,298.49 Services 9. Consultant Services 0 0 0 10. Unallocated 0 0 0 TOTAL 294,000,000 260,968,478.75 33,031,521.25 The following table indicates the category-based used amounts: * As of January 12, 2004. The total amount will decrease after all remaining payments are completed within 4 months. - 59 - C. Use of the Government Funds under the Law 4306 Total amount allocated by the Government from its budgets for the Basic Education Project was USD 2,215,000,000 in total at the beginning. However, due to the educational changes and extensions, the Government has used USD 2,274,740,621.31 for a five year Project period. Category Amount of the Loan Used Amount (USD) Remaining Amount Allocated (USD) (USD) 1. Civil Works under 1,624,076,080.69 1,624,076,080.69 0 Parts A.3 and B.3 of the Project 3. 2. Equipment 268,018,684.58 268,018,684.58 0 and materials under Parts B.2, B.3 and C of the Project 3. Consultants' services 164,303.67 164,303.67 0 under Parts A.3, C and D of the Project 4. Training, fellowships 93,872.05 93,872.05 0 and study tours under Parts B.1 and C of the Project 5. Goods, works and 147,677.50 147,677.50 0 services for sub-projects under Part E of the Project 6. Civil Works under 0 0 0 Part F of the Project 7. Goods under Part F2 353,291.09 353,291.09 0 of the Project 8. Contracted Transport 381,886,711.73 381,886,711.73 0 Services 9. Consultant Services 0 0 0 10. Unallocated 0 0 0 TOTAL 2,274,740,621.31 2,274,740,621.31 0 7. MAJOR FACTORS AFFECTING PROJECT IMPLEMENTATION The First Phase of the Basic Education Project has obtained a great achievement especially in the provision and distribution of equipments and materials to schools. The Project has played a crucial leader role in equipping basic education schools with IT Rooms with the aim of introducing and expanding ICT use in schools under the 8th Five Year Development Plan. However, the Project has faced with delays in achieving all expectations and objectives due to the following reasons: 1. During five year project implementation period, the Project and its staff experienced turnovers of 4 PCC heads and 4 WB team leaders. These staff turnovers have always been the most important factors affecting project implementation. - 60 - 2. The staff working in this Center was comprised of civil servants "teachers" and contracted staff under the MONE. Despite their experience in teaching and other educational issues, these staff had to work really hard to experience and implement Bank's Guidelines. The MONE has preferred to hire individual consultants at every stage of the project implementation. However, most of these consultants were hired for a short period of time. At the last year of project implementation, the Ministry realized that there was a need to further strengthen both Ministry's and PCC's together with ERDD's project management, procurement management and financial management and reporting and communication capacity. Therefore, four consultants in the areas of project management, civil works procurement, procurement management and financial management were hired to support the PCC during the last year of project implementation. 3. The two earthquakes that took place in the Marmara region in 1999 ruined the economical balances in Turkey. The Government had to transfer a significant number of allocations to these regions following these disasters which adversely affected most of the sectors in Turkey including education sector. Within this framework, an amount has been transferred into Marmara Relief Component from the Loan Proceeds for the renovation construction of schools. 4. During the project implementation period, Turkey has faced with two major economic crises which brought about a sharp decrease in GNP. These crises have led to increase in the need for external debts and inflation rate together with number of unemployed. Due to these crises, the value of Turkish Liras and economical power decreased sharply, which led to fluctuation in foreign exchange rates in the bidding processes which caused problems for many of the bidders at that time and led to long bidding procedures. Due to primary surplus target of 6% on the average for the last three years of implementation of the project, the allocation of the General Budget for the investments was restricted by the Government. Accordingly, the general budget allocated for the Ministry of National Education was reduced by 3% and disbursements under the Basic Education Program by 15%, which brought about reduction in educational investments throughout the country. 5. Uncertainties related to the war in Iraq had an adverse effect on Turkish economy, and led to a 10% reduction in the investment expenditures of the Government, causing insufficient allocations particularly in civil works. 6. The procurements at the provincial level are subject to the State Bidding Law and determined through the use of local laws. One of the innovations brought by the Basic Education Project Implementation was the authority of bidding procedures given to the provincial directorates. Therefore, civil works and IT infra-structural works were carried out by the local authorities based upon the short term informative meetings on WB Procurement Guidelines by the PCC. However, due to the differences between the Public Tender Law and the Bank's Procurement Guidelines the Project implementation has faced with some problems at certain periods, which led to some problems such as cancellation of some contract packages and inadequate use of loan funds by the MONE during project implementation period due to unfamiliarity of local authorities with the WB Guidelines at provincial level. 7. Major deviations were faced during the completion of the bidding processes due to both the Ministry's and the World Bank's long internal approval periods. This led to greater work intensity and extension of biddings and activities. 3 times Extension requests for the Project closing date are the best indicator for this situation. Besides, the visa period of the Ministry of Finance (before contract signing) and Court of Accounts (after contract signing) has taken a long time during procurement phases, which led to extensions of procurement, distribution and payment processes. - 61 - 8. The experience gained at the First Phase of the Basic Education Project showed existing problems of the Project could not be determined effectively due to inadequate monitoring and evaluation and irregularity in preparation of monthly, quarterly, semi-annual and annual reporting of the Project implementation. The most important part of the project implementation is the monitoring and evaluation. The monitoring and evaluation capacity of the Ministry needs to be strengthened for a better and powerful implementation at the second phase. 8. PROJECT SUSTAINABILITY The Minister of National Education simply declares how much the Government of Turkey gives due importance on education system in his speech for "100% Support to Education" Project by stating that "As we are all aware, education is a sine qua non process which ensures development of individuals and societies; supports economical development of the country; and improves and protects material and moral cultural values of the country and transfers these values to the next generations. Therefore, the most important indicator in defining the country itself and in declaring what kind of a future it prepares for its people, is the education system of that country and the realization ratios of its educational policies" ("100% Support to Education" Message of the Minister of National Education, 2004). These words indicate not only the importance of effective educational systems but also that of realization ratios of educational objectives for a better future of that country. In relation to the main goals of the Government's Basic Education Program, the Government of Turkey and the World Bank signed an APL (Adaptable Program Loan) agreement to finance a supportive project called "the Basic Education Project", with the loans over two phases at an amount of USD 300 million for each. The first Basic Education Project has been closed as of December 31, 2003 with a considerable progress and the Loan Agreement for the Second Basic Education Project has been signed by/between the Government of Turkey as of July 26, 2002 and will become effective starting from January 2004 with a one year period of delay. Since the Basic Education Program is a whole package for reforming education system, the Government supports its attempts through 3 more projects with EU financing so-called "Support to Basic Education Project", "Project to Modernize Vocational and Technical Education", and "Strengthening the Vocational Education Training System in Turkey" and 1 more project with WB financing called "Secondary Education Project". Additionally, European Investment Bank also supports the Government's education system through a "Framework Education Project". Moreover, a Cooperation Protocol has been signed by/between the Ministry of National Education and the National Assembly on January 13, 2004 for the Democracy Education and School Assemblies Project, with the aims of improving a democracy culture in children and expanding the democracy approach to a baseline by ensuring an election/ being elected experience for children. Besides, the Ministry of National Education, in cooperation with UNICEF, has commenced Educational Maps Project at the end of December 2003 for the preparation of Educational Maps by provinces/sub-provinces, Educational Data Handbooks and of Terminological Dictionary for Educational Statistics. The Research, Planning and Coordination Council has arranged a meeting with the participation of 50 staff both from the Ministry line units and from other organizations/ministries such as State Planning Organization, State Statistics Institute, Ministry of Interior Affairs and Ministry of Justice etc. to define the scope and features of the educational maps to be developed. Educational Maps System is planned to be associated with the ILSIS Portal of the Ministry and Eurydice Network which consist OECD; EU; EFTA - 62 - And EDA countries. By this way, educational data regarding provinces, sub-provinces and even schools including the number of teachers, students, schools, age population, the distances between schools etc. will be shared throughout the country and the world. This implementation will also ensure quick updating of educational data which will be definite and easy to be looked at. The Research, Planning and Coordination Council is nowadays working on the development of Eurydice network in the Ministry of National Education. The installation of this network is planned to be completed within 3-4 months. Both field trips and the requests forwarded to the Ministry indicate that the public attention and interest to the Basic Education Program is increasing gradually through the country. The requests of the community for better educational conditions and technology-based education are constantly forwarded to the Ministry of National Education. At this point with full of success in terms of basic education system in Turkey, there is still much to be done in basic education system by taking into account large geographical and demographic features of Turkey. The Government of Turkey once again supports its commitment for sustaining Basic Education Program and sub-projects through the use of government budget, educational laws and educational mobilization activities and regulations commenced throughout the country. 9. BANK PERFORMANCE: The Basic Education Project assisted in achieving the Government's broader development program that is "Basic Education Program". The Project covers a certain part of a comprehensive program. Projects Coordination Center which was established in June 22, 1998 performed the coordination for Basic Education Project. PCC managed the coordination with the Bank and the Ministry's line units and other partners. The role of PCC was crucial during the implementation period. The staff turnover concerning WB task team leaders adversely affected project implementation. WB Task Team Leaders of the project have been changed four times during the life of BEP APL I, which caused deficiencies in detecting administrative and technical problems timely directly involved in the implementation of the Project. Despite variances in the performances of WB Team leaders, a better and closer cooperation and communication with both the World Bank side and the Ministry is required for the success of the Second APL. Long approval procedure is also one of the adverse effects caused problems in project implementation period. For instance, for MRF subcomponent, the Bank's approval procedure for the first three projects caused considerable delay in the process. Required no-objection for the first three projects which were prepared for the First Phase of the Project and submitted to the Bank's prior review has been submitted to the Ministry after a long time following the commencement of the program. However, the Bank has monitored and oriented the Project implementation in a satisfactory way in terms of guidance and assessment matters. Supervision activities are performed on a regular basis as well. Taking into consideration the whole performance of the Bank, the Ministry deems the Bank performance at project implementation including preparation and supervising activities as "satisfactory". 10. BORROWER PERFORMANCE: Projects Coordination Center which was established in June 22, 1998 had a crucial role during the whole process. Despite the important role of the PCC, it doesn't have any legal support except for the Loan Agreements. As PCC does not exist in the law no. 3797: Law on Organizational Structure and Duties of the MONE, projects under the Loan Agreements should be carried out by the teachers temporarily appointed with Ministry Approval. Most of the staff, especially key staff, working at PCC, is temporarily - 63 - appointed teachers. In order to support the implementations, the Ministry recruited contracted civil servants pursuant to the Law for Civil Servants paragraph no. 657/b. The changes in priorities have necessitated Loan Amendments and caused delay in the process as well. Since the beginning, the Basic Education Project has been amended six times. In spite of the factors that adversely affected the implementations, considering that project was a comprehensive one and most of the performance indicators were achieved, it is possible to say that the performance of the borrower is satisfactory. 11. ASSESSMENT OF OUTCOME: It is possible to say that most of the development objectives have been achieved under the First Phase of the Basic Education Project. For expanding the basic education coverage, lots of additional facilities and renovation of existing schools have been completed and a considerable number of students were benefited throughout Turkey. The quality of civil works was generally satisfactory but sometimes low in some regions. In terms of civil works, the vitality of supervision activities was remarkable. Under First Phase of the Basic Education Project, 3188 ICT classrooms were set up in 2802 schools. With the provision of access to Internet and IT teachers, ICT classrooms were utilized more efficiently and effectively under the BEP APL I. Despite delays in procurement activities, all equipments delivered to schools are in use. However, concrete outcome of these activities seems to take long to get the expected results in terms of increasing the quality. Regarding in-service training, due to the legal constraints, schools faced with some problems in appointing ICT teachers. However, training activities have been achieved in terms of both quality and quantity. Implementation support in the form of specialized consulting services, office equipment and technical assistance supported project implementation. However, limited study tours and training have been served to the staff during the process. Monitoring and social impact assessments were the key instruments for guiding Program Implementation and development. The three assessment studies regarding social and institutional impacts and cost analysis of Project have been completed by universities and distributed to MONE line units and relevant institutions. Despite the efforts, ICT impact study has begun with a considerable delay. The study couldn't be completed until Project closing date. The results of the study will be available after the second phase of the implementation is launched. MRF implementation seems to be "truly satisfactory". At the beginning of the Project, due to the lack of guidance and experience regarding the subject and long approval procedures of the Bank, applications started with a significant delay. During the process, a high interest and participation of school community have been observed. The overall rating of the project could be regarded as "partially satisfactory". 11. LESSONS LEARNED The Basic Education Project has been planned as a support to the comprehensive Basic Education Program of Turkey aiming to promote progress in coverage of eight-year basic education and to improve the quality of basic education. The Project provided lots of facilities throughout Turkey particularly for age groups in - 64 - rural areas with low income families. Besides, a significant experience gained by Provincial Directorates and all implementing agencies in terms of Project implementation. It will be useful to take this experience into consideration for the future planning in particular in the Second Basic Education Project. Both the Bank and the Borrower will take into consideration negative and positive experience gained from the Project implementation and will get maximum profit for the upcoming projects i.e "the Second Basic Education Project" and "Secondary Education Project". Project Management: Several problems have been experienced in terms of project management and coordination. It will be useful to identify the responsibilities of the units under the project in details. Arrangement of regular meetings with PCC and other line units will accelerate the collaboration and coordination. A closer collaboration between the Bank the Ministry and quick response of the Bank was required in most cases. Bureaucratic difficulties have been experienced in practice by both line units of the Ministry and Projects Coordination Center as well. Elimination of these bureaucratic difficulties by delegation of authority to the Deputy Undersecretary and PCC Director will "probably accelerate the implementations. Procurement Activities: During BEP APL I, the procurement activities were rather slow than expected due to the long approval procedures and lack of experience regarding the World Bank Rules and Regulations. There seems to be a remarkable requirement for training on procurement activities including PCC staff. In most cases technical consultancy services were insufficient to handle difficulties. Financial Management: Financial Management Unit has used MS office for budget activities and recording until the year 2001. At the end of the year 2001, due to the requirement, FMS system has been established. All data concerning BEP APL I has been transferred into the FMS. The system facilitated Project implementation in terms of monitoring, recording and elimination of probable errors. The same system is agreed to be used for the Second Basic Education Project and Secondary Education Project as well. Monitoring and Evaluation: The experience gained at the First Phase of the Basic Education Project showed existing problems of the Project could not be determined effectively due to inadequate monitoring and evaluation and irregularity in preparation of monthly, quarterly, semi-annual and annual reporting of the Project implementation. The most important part of the project implementation is the monitoring and evaluation. The Unit has been restructured with a clear TOR at the last period of the Project. Monitoring and Evaluation activities are expected to be more effective and successful in the Second Basic Education and Secondary Education Projects. RESOURCES MEB, 2002 yılı Başında Milli Eğitim, Aralık, 2001, Ankara Official Gazette dated August 1, 1998 with Ref. No. 23420, Başbakanlık Mevzuatı Geliştirme ve Yayın Genel Müdürlüğü, 1998, Ankara. Rapor No: 17877-TU, Project Appraisal Document on a proposed adaptable program loan in the amount of US$ 300 Million equivalent to the republic of turkey for a basic education project in support of first phase of the basic education program, June 1, 1998. Ercan TÜRK, Türk Eğitim Sistemi, Nobel Yayın Dağıtım, Ankara, 1999. - 65 - Progress Report No.2, Projects Coordination Center under the MONE, August 1997-December1999. Progress Report No.10, Projects Coordination Center under the MONE, January 31, 2003-July 31, 2003 Data from DG for Pre-school Education, MONE, 2004 Data from Research, Planning and Coordination Council, MONE, 2004 Figurative Data for National Education Series between 1998-1999 and 2003-2004 academic years by Research, Planning and Coordination Council, MONE./ 1997-1998 National education data by State Statistics Institute 100% Support to Education" Message of the Minister of National Education - 66 - - 67 -