Document of The World Bank FOR OFFICIAL USE ONLY Report No. 22491 PERFORMANCE AUDIT REPORT MADAGASCAR ACCOUNTING AND MANAGEMENT TRAINING PROJECT (Credit 1661-MAG) EDUCATION SECTOR REINFORCEMENT PROJECT (Credit 2094-MAG) MANPOWER TRAINING PROJECT (Credit 2382-MAG) June 28, 2001 Sector and Thematic Evaluation Group Operations Evaluation Department This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Currency Equivalents (annual averages) Currency Unit = Malagasy franc US$1.00 = MGF 6543 (March 2001) Abbreviations and Acronyms AfDB African Development Bank APL Adaptable Program Loan BEPC Brevet d' Enseignement du Premier Cycle CNFTP Conseil National de Formation Technique et Professionnelle CRESED Cr6dit pour le Renforcement du Secteur de I'Education (Education Sector Reinforcement Project) GDP Gross Domestic Product HIPC Heavily Indebted Poor Countries ICR Implementation Completion Report IMATEP Institut Malgache des Techniques de Planification (Malagasy Institute of Planning Techniques) INSCAE Institut National des Sciences Comptables et de l'Administration des Enterprises (Accounting and Management Institute) MIS Management information system NGO Nongovernmental organization OED Operations Evaluation Department PAR Performance Audit Report PCR Project Completion Report PIU Project implementation unit PRAGAP Programme de Renforcement et d' Am6lioration de la Gestion Administrative et P6dagogique PREFTEC Projet de Renforcement de l'Education Technique et Professionnelle (Manpower Training Project) SAR Staff Appraisal Report TVET Technical and Vocational Education and Training UNESCO United Nations Educational, Scientific, and Cultural Organization Fiscal Year January 1-December 31 Director-General, Operations Evaluation Mr. Robert Piccioto Director, Operations Evaluation Department Mr. Gregory K. Ingram Manager, Sector and Thematic Evaluation Mr. Alain Barbu Task Manager Ms. Helen Abadzi FOR OFFICIAL USE ONLY The World Bank Washington, D.C. 20433 U.S.A. Office of the Director-General Operations Evaluation June 28, 2001 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: Performance Audit Report on Madagascar Accounting and Management Training Project (Credit 1661-MAG); Education Sector Reinforcement Project (Credit 2094-MAG); and Manpower Training Project (Credit 2382-MAG) This is a Performance Audit Report (PAR) on three education projects in Madagascar: * Education Sector Reinforcement Project (Cr. 2094-MAG; CRESED) for US$39 million equivalent, which was approved in FY90 and made effective on July 24, 1990. After extensions totaling two years, the credit closed on June 30, 1998, and a balance of US$.65 million was cancelled. * Manpower Training Project (Cr. 2382-MAG; PREFTEC) for US$22.8 million equivalent, which was approved in FY93 and made effective on January 15, 1993. After an extension of 12 months it closed on June 30, 1999; US$0.63 million was cancelled. * Accounting and Management Training Project (Cr. 1661 -MAG; INSCAE) for US$10.3 million equivalent, which was approved in FY86 and made effective on October 8, 1986. After extensions totaling two years, the credit closed on December 31, 1994, and a balance of SDR 2.8 million was cancelled. France, Canada, and USAID provided parallel financing of US$2.50 million, US$1.15 million, and US$0.17 million respectively. Total project cost at completion was US$12.17 million. Relevance. The projects aimed to improve access to, quality, and management of education for primary students, vocational trainees, and accountants. All three projects financed civil works, technical assistance, teacher training, curricular improvements, and studies. The projects were complex, consisting of multiple unrelated components with little coordination. Efficacy. The accounting training project (INSCAE) achieved its objectives, although the procurement component of the project did not. The primary education (CRESED) and vocational training (PREFTEC) projects did not achieve their objectives. They financed many activities, but these were oriented towards procedural and administrative issues; project activities were peripheral to learners' acquisition of useful information. Thus, although the projects were relevant to the economic needs of the country, their outcomes were not sufficient in bringing about desired improvements in the education sector. Efficiency. CRESED and PREFTEC did not use resources efficiently. The credits disbursed almost completely, but project objectives were not met. Designs focusing on the true issues that the projects were trying to resolve rather than on ancillary services would have been much more efficient. The INCSCAE project used resources more efficiently. However, IDA and other donors spent US$21.8 million over 10 years to develop a single institution that serves about 300 full-time students per year and still lacks its own building. This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. 2 The audit ratings for each project are shown below, together with the original ICR ratings. The outcomes of Credits nos C2904-MAG and C2382-MAG were rated as unsatisfactory because project objectives were not achieved. Weaknesses in project design and execution were the reasons why Bank and borrower performance were rated unsatisfactory for these two projects. Education Sector Reinforcement Manpower Training Project Accounting and Management Criteria Project (C2904-MAG) (C2382-MAG) Training Project (C1661-MAG) ICR Audit ICR Audit ICR Audit Outcome Satisfactory Unsatisfactory Satisfactory Unsatisfactory Satisfactory Satisfactory Sustainability Likely Unlikely Uncertain Unlikely Likely. Highly likely Institutional Substantial Negligible Modest Negligible Substantial Substantial Development Bank Satisfactory Unsatisfactory Unsatisfactory Unsatisfactory Satisfactory Satisfactory performance Borrower Satisfactory Unsatisfactory Unsatisfactory Unsatisfactory Satisfactory Satisfactory Performance Lessons Experience with the projects confirms a number of OED lessons: * Attention to the management and procedural aspects of projects is necessary but insufficient to improve the quality of education. Attention to instructional delivery is necessary at all levels of education. * Preservice and inservice teacher training often does not result in behavioral change. Much more support and supervision is needed than is often available. Rather than being lectured, teachers must learn desirable behaviors through means that are more effective in producing behavioral change. * For small countries with large needs, it is tempting to attach various components to a single project rather than do multiple small projects that might not be viable. Very often this strategy has not proved effective. When the commitment of an institution is low or when significant changes take place, some components may be neglected. The Bank does not have the resources to appraise multiple small projects, but supervision missions should be more intensive when multiple components are involved. * It is very difficult to ascertain what a project has accomplished without a monitoring and evaluation system. Baseline data are needed to gauge progress in outcome indicators. * Children learn information best in a language they know well. Lower-income children with parents who have limited education and who are in schools where individual attention is limited, may not master the foreign language fast or well enough to process information efficiently in it. Though political considerations usually drive decisions on language of instruction, the substantive issue is ability to learn important material through a language. Attachment 1 Contents PRINCIPAL RATIN G S ............................................................................................................................................... III P R EF A C E ....................................................................................................................................................................... V 1. BA C K G R O U N D .................................................................................................................................................. 1 2. IMPLEMENTATION EXPERIENCE OF THE AUDITED PROJECTS..................................................3 THE EDUCATION SECTOR REINFORCEMENT PROJECT (CRESED; C2094-MAG; ANNEX A T. 1) ............ 3 THE MANPOWER TRAINING PROJECT (PREFTEC; CR. 2382-MAG; ANNEX A TABLE 2) ...................... 7 THE ACCOUNTING AND MANAGEMENT TRAINING PROJECT (INSCAE; CR. 1661-MAG; ANNEX A, TABLE 3) ........ ....................................................................... 9 PROJECT OUTCOMES .............................................................11 RELEVANCE AND EFFICIENCY..................................................... 1 INSTITUTIONAL DEVELOPMENT IMPACT.......................... ...................12 SUSTAINABILITY ................................................................12 BANK PERFORMANCE ............................................................12 BORROWER PERFORMANCE....................................................... 13 3. ISSUES FOR FUTURE CONSIDERATION........................................... 14 INCREASING TIME ON TASK IN CLASSROOMS......................................... 15 HELPING STUDENTS TO GRADUATE RATHER THAN FAIL................................17 VOCATIONAL TRAINF NG-ENABLING CRAFTSMEN TO TEACH TO WILLING STUDENTS.................. 17 LESSON S UREO................. ....... ................................................................................................... 18 ANNEX A. PROJECT ACTIVITIES ...... ........................................................... 21 ANNEX B. BA SIC DATA ............................................................................................................................................ 25 MADAGASCAR-ACCOUNTING AND MANAGEMENT TRAINING PROJECT (C1661-MAG)............... 25 MADAGASCAR-EDUCATION SECTOR REINFORCEMENT PROJECT (C2094-MAG) .......................... 27 MADAGASCAR-MANPOWER TRAINING PROJECT (C2382-MAG) ................. . .......... 31 ANNEX C. COMMENTS FROM THE BORROWER ....................................................................................... 33  111 Principal Ratings Education Sector Reinforcement Manpower Training Project Accounting and Management Criteria Project (C2904) (C2382) Training Project (C1661) ICR Audit ICR Audit ICR Audit Outcome Satisfactory Unsatisfactory Satisfactory Unsatisfactory Satisfactory Satisfactory Sustainability Likely Unlikely Uncertain Unlikely Likely. Highly likely Institutional Substantial Negligible Modest Negligible Substantial Substantial Development Bank Satisfactory Unsatisfactory Unsatisfactory Unsatisfactory Satisfactory Satisfactory performance Borrower Satisfactory Unsatisfactory Unsatisfactory Unsatisfactory Satisfactory Satisfactory Performance Key Staff Responsible Project Task Manager/Leader Division Chief/ Country Director Sector Director Accounting and Management Training Project (Cr. 1661-MAG) Appraisal Paul Blay Alain Colliou Francisco Aguirre Sacasa Completion Pierre Mersier Michael Sarris Calisto Madavo Education Sector Reinforcement Project (Cr. 2094-MAG) Appraisal Paul Blay Alain Colliou Francisco Aguirre Sacasa Completion Daniel Viens Michael Sarris Calisto Madavo Manpower Training Project (Cr. 2382-MAG) Appraisal Paul Blay Alain Colliou Francisco Aguirre Sacasa Completion Daniel Viens Michael Sarris Calisto Madavo  V Preface This Performance Audit Report (PAR) covers the three most recently completed education projects in Madagascar: * Education Sector Reinforcement Project (Cr. 2094-MAG) for US$39 million equivalent, which was approved in FY90 and made effective on July 24, 1990. After extensions totaling two years, the credit closed on June 30, 1998, and a balance of US$.65 million was cancelled. * Manpower Training Project (Cr. 2382-MAG) for US$22.8 million equivalent, which was approved in FY93 and made effective on January 15, 1993. After an extension of 12 months it closed on June 30, 1999; US$0.63 million was cancelled. * Accounting and Management Training Project (Cr. 1661-MAG) for US$10.3 million equivalent, which was approved in FY86 and made effective on October 8, 1986. After extensions totaling two years, the credit closed on December 31, 1994, and a balance of SDR 2.8 million was cancelled. France, Canada, and USAID provided parallel financing of US$2.50 million, US$1.15 million, and US$0.17 million respectively. Total project cost at completion was US$12.17 million. The audits were conducted to study the effectiveness of IDA strategy in an extremely poor country with complex cultural and economic problems. The PAR is based on the following sources: Implementation Completion Reports (ICRs), Staff Appraisal Reports (SARs), Credit Agreements for the projects, and project files, particularly the supervision reports. An OED mission visited Madagascar in March 2001 to collect other pertinent information. The author thanks the many government officials who received the mission for their extensive cooperation. Thanks are also given to the country office staff who helped the mission. Following standard OED procedures, copies of the draft PAR were sent to the relevant government officials and agencies for their review and comments. A number of observations were made, which have been incorporated into the PAR as Annex C.  1 1. Background 1.1 Madagascar is an agricultural country with a per capita income of about US$210, where social demand for education has been high; in almost all provinces, entries to grade 1 average 75% of the relevant age group or above,' and schools exist in almost all of the country's 11,000 villages. Girls are as likely to enroll as boys, and there is a large private and religious school presence at all levels. Yet, repetition rates are very high, averaging 31% in the primary years. In the universities, practically all students get scholarships, yet most fail course examinations. The performance of the Malagasy system is dismal, even compared to sub-Saharan Africa. To improve the system, a large share of the savings created by the Heavily Indebted Poor Countries (HIPC) initiative has been earmarked for education. 1.2 About 10 years after its independence from France in 1960, the government changed all instruction from French to the Malagasy language (spoken by all inhabitants) and tried to create scientific terminology in it. In 1993, it reversed course and decreed instruction through French for all students starting in grade 3. The changeover coincided with a time of severe economic and social upheaval, which resulted in very limited financing for schools at a time when large resources were needed to train teachers and print books. The educational system has not fully recovered financially. Public school enrollments temporarily dropped in the 1990s, because many schools closed; teachers were unwilling to work for extremely low salaries in rural areas some security problems and with limited opportunities for transfer. The families that can, send children to private schools, whose enrollments tend to increase.2 1.3 The Bank has made few direct investments in primary education. The first project in the 1970s built three technical schools and two teacher training institutes, while the second in the 1980s built a learning materials center and five regional academic centers, procured 118 vehicles, and imported a still functional printing press. Simultaneously an institute for accountant and auditor training was developed through two technical assistance projects, and in the 1990s a manpower training project was implemented. The Bank's long-term strategy on the education of Madagascar and the rationale for this lending pattern are unclear; no papers clearly outlining country strategy were found before 2001.3 In contrast to many low-income countries where large numbers of rural primary schools were built or extended in the 1980s and 1990s, IDA education 1. Education and Training in Madagascar: Towards a Policy Agenda for Growth and Poverty Reduction. Draft January 2001. 2. The 2001 sector strategy paper reports that, though very low, student learning in primary education compares favorably with outcomes in other low-income countries, with Malagasy children scoring above the sample averages in language, mathematics and life skills (p. xv). Since the poorer students drop out of school and those who can enter private school, the comparison does not represent the true state of the school system. 3. The three education Ministries also planned an education sector development program (ESDP) in the late 1990s with IDA encouragement. Subsequent sector work was built on this framework: Education and Training in Madagascar: Towards a Policy Agenda for Growth and Poverty Reduction. Draft January 2001. Sector work not found in the Bank's archives was carried out in the 1980s with the support of UNESCO (Programme National d'Amdlioration de lI'ducation) and was approved by the government in 1988. Other related papers in the archives were: Colletta, N. and Gillian Perkins. Participation in Education. 1995. Report No. 18182, Environment Department Papers. Facilitator's Guide: A Seminar on Improving the Quality of Education. Human Resources Division, Technical Note no. 19 report No. 15055. (Annex 1, p. 10) 2 projects did not finance primary school construction in Madagascar. Only the Emergency Social Fund (Fonds Social d'Urgence III, Supplemental Credit for Cyclone Rehabilitation Activities - Cr. 3180-MAG) has built about 1,000 schools at the request of communities. Aside from churches, some NGOs (like Aide et Action) have built schools with community help. As a result of limited investment, however, schools are small, furniture is old and unsuitable for multiple activities, and classes are very crowded. Table 1. IDA-Financed Education Projects in Madagascar Approval Fiscal Credit amount Project name Credit no. A year Final closing date Cr mil aon) Completed Projects Tananarive education project Cr. 510-MAG 1968 1974 4.8 Second Education Project Cr. 663-MAG 1977 1984 14 Accounting and audit Cr. 1155-MAG 1982 1989 11.5 organization training project Accounting and Management Cr. 1661-MAG 1986 1994 10.3 Training Project Education Sector Reinforcement Cr. 2094-MAG 1990 1998 39 Project Manpower Training Project Cr. 2382-MAG 1993 1999 22.8 Ongoing Pmjects Education Sector Development Cr. 3046-MAG 1997 2003 65 Project (FADES) 1.4 IDA has strongly encouraged the government to reform higher education finance and devote more of the education budget to primary education. The government did make progress in eliminating chronic students and returning some order to the university,4 but large-scale reform and reduction of expenditures have been hard to achieve. This audit examines the outcomes of the last three completed projects, which belong to three different subsectors of education. They are here distinguished by their French acronyms. (See project objectives in Table 2.) Table 2: Main Objectives of the Audited Projects Quality Access-Equity Finance-Management Education Improve the quality of basic and Regulate student flows through Improve the sector Sector general secondary education by the system to encourage the administration, Reinforcement setting up a pedagogical unit, qualitative and quantitative management, and Project by increasing the role of development of basic planning, by purchasing Cr6dit pour le inspectors, education advisers, education while improving computer and office Renforcement and headmasters, as well as quality at other levels, equipment, by conducting du Secteur de pre-service and inservice studies and training, by I'Education training for teachers, by reorganizing planning and providing textbooks and statistics services, and by (CRESED) teachers' guides, and by establishing project upgrading secondary school coordination. laboratories. Reorganize the vocational Strengthen higher education by training and technical establishing two short-cycle education systems under a higher institutes of technology, coordinating structure developing plans to modernize which will establish a close university curricula, improve link between training and university administration and employment requirements. costd financing balance. 4. Madagascar: A Decade of Reform and Innovation in Higher Education. Human Development No. 159, May 2000. 3 Quality Access-Equity Finance-Management Manpower Rehabilitate and reform the Establish a system to provide Establish a system to Training Project public technical-vocational information about labor market develop technical Projet de training system, increase quality needs. vocational education and Renforcement and external efficiency. Develop a training system for training (TVET) policy and de I'Education Commence the process of senior civil servants, promote, coordinate, and Technique et agricultural education and Professionnelle training reform, and public training (PREFTEC) institutions and employers. Accounting and Assist in the modernization and Train qualified accountants, Provide technical Management improvement of the auditors, and managers assistance and training for Project management and operation of those responsible for Institut National public and private enterprises procurement des Sciences and would support the Comptables et government in the de I' implementation of policies Administration geared to the revitalization of the des Enterprises economy. (INSCAE) I___________ I_____I_____i_ 1.5 Madagascar has received the benefit of much donor financing. France, UNDP, UNESCO, the African Development Bank, Canada, the US, and OPEC have provided parallel financing of some IDA projects. Donor coordination was limited in earlier years but has recently improved, and the consortium meets regularly. 2. Implementation Experience of the Audited Projects 2.1 CRESED and PREFTEC were centralized and complex operations with many unrelated components that involved different agencies and were expected to reform multiple aspects of the educational system. The IthSCAE project was simpler but included a procurement component that was unrelated to the other activities. The projects included several studies and years of technical assistance, created new departments and organizations, and recruited several civil servants. None of the projects had monitoring indicators. Not surprisingly, all three projects suffered various delays. Supervision reports indicate that the multiple agencies that participated in the same project often did not coordinate activities and did not achieve the synergy expected. Implementation highlights are given below; detailed targets and outcomes are in Annex A Tables 1-3. 2.2 Mission Visits. The mission visited various project sites in Toliara and Antananarivo (including Tsiroanimandidy). These included the Higher Institute for Technical Education of Antananarivo, NSCAE headquarters and classrooms, headquarters for the Council for Technical and Vocational Training (CNFTP), the technical teacher training college, businesses that had used CNFTP training services, and rural schools. The Education Sector Reinforcement Project (CR SED; C2094-MAG; Annex A T. 1) 2.3 This centralized project was to commence a major sector reform that would last 10-15 years, which included: (i) control of recurrent costs at all levels to free resources for quality improvement and increased primary enrollment; (ii) a rolling three-year sector public expenditure program to be reviewed annually, reflecting increased spending on primary education and on quality at all levels; and (iii) a progressive reorientation to employment needs of vocational education and training. After a difficult and slow start, the project came to a standstill because of 4 the political events and civil disturbances of 1991. By mid-term in 1993, the curricula had not changed, the universities were dysfunctional, expected action plans had not been done, and policy reforms had not progressed. Quality of education was deteriorating, and about 2,500 schools closed due to a lack of teachers. 2.4 Advised by the Bank, a team of Malagasy professionals studied the issue and developed elaborate school effects models in 1993.5 Causes for the high dropout rate were considered to be teachers' professional behavior (absenteeism, lack of commitment, preference for hiring teachers belonging to certain political parties) coupled with a lack of school leadership, and a shortage of physical facilities. A pilot project (PRAGAP) was developed in 1994 in 20 districts to devolve school management to communities so that children could be helped to acquire basic skills. 6 Communities were to decide what schools needed and become more involved in their children's education. After discussion, contracts were drawn up between communities and schools outlining responsibilities. 2.5 A 1997 evaluation of PRAGAP three years later showed mixed results. The project implementation unit (PIU) spent two years negotiating with communities. The contracts between schools and community were complex, sometimes 80 pages long. The main outcome was school buildings, some of which were of low quality. There was limited teacher housing and few bathrooms for children. Parents had to walk for miles to bring in stone and concrete, and if they did not have materials ready, construction fell behind schedule. Given the large effort necessary by village volunteers in comparison to better equipped contractors, construction was not a good use of poor people's scarce time. Furthermore, the pilot did not improve classroom instructional inefficiencies, so its main objective was not achieved. 5. Madagascar: Towards a School-Based Strategy for Improving Primary and Secondary Education. Report No. 13450-MAG., April 7, 1995. (by Ward Heneveld). 6. Evaluation du Programme de Renforcement et d' Amelioration de la Gestion Administrative et Pedagogique (PRAGAP). Madagascar Ministry of Education: Bureau Projet Education, September 1997. Madagascar: A Developmental Approach to Community-Based School Management. Human Development No. 185, January 2001. 5 Box 1. After 20 Years of IDA Involvement, Still Few Textbooks The Second Education Project (1977-1984) financed a unit for textbook production, complete with paper and printing press. CRESED (effective in 1990) financed textbook production, and Germany provided technical assistance to develop books in Malagasy. When the government decreed in 1993 instruction through French for all students starting in grade 3, Germany stopped the collaboration, but the French government agreed to print French-language textbooks. The subsequent economic and political difficulties caused many delays. CRESED financed a pedagogical unit for textbooks and educational research, but its activities have been limited; several French-language textbooks were still being edited in 2001. After two projects with textbook components, there are only seven titles in Malagasy for primary school: reading for five grades, and arithmetic for only two. (No science for any grade.) In French, only the language instruction textbooks exist, financed by France. For secondary schools there are French, English, and math textbooks for four grades. The textbooks found in schools are old, with pages missing. They are relatively expensive, so poor parents cannot afford them. Parent-teacher associations were expected to buy them, but demands on poor communities have proved excessive. In one school visited by the audit mission, there was one first-grade reader for every three students. Some NGO schools, like those of the Catholic Church and Aide et Action give students textbooks. Others which are less experienced (e.g. the Orthodox Church) use no textbooks at all. The texts have no evaluation questions for homework or group work. The result is that rural fifth-grade children were observed learning geometry in French by copying from the blackboard. At the same time, the Malagasy textbooks that offer this content lie unfinished. 2.6 CRESED was expected to improve the quality of instruction. However, the 16 schools (regular as well as PRAGAP) visited by the mission had the following conditions: * Parental help built schools partly financed by the Aide et Action. These are often small and of poor workmanship, in a country where preventive maintenance is nonexistent. Many have no latrines. In many areas larger schools are needed, but there are no plans for expansion. A lower-secondary school had four grades and three classrooms, with students rotating. * Where parents are involved and monitor supplies, school feeding programs (financed by the World Food Program) may be feeding students efficiently. One PRAGAP school visited has rice fields, and students take home some rice. * Lower-grade classrooms were typically packed with 45 students or more; higher grades had 25-35 students. Teachers easily lecture to 45 students but cannot effectively interact with them or pay attention to their learning needs. * Classes had few textbooks and almost no instructional materials, not even pictures or signs on the walls; though the project financed the reproduction of 89,000 charts and maps, none of these schools had any. Though science labs had been financed for secondary schools, the schools visited did not have any. (Perhaps they exist in other schools.) Students had almost no opportunity to analyze or synthesize the subject matter being taught. * The students get very little time on task (possibly only 20% of classroom time) and process very little academic information, because: - The time available for learning in poor public schools is extremely limited (2.5 hours per day when the teacher is not absent). 6 - Without textbooks or other materials, students spend much time copying from the blackboard and have no means to do class work or meaningful homework. - Students are not kept busy all the time, as they should be. In all classes visited, one student worked on the blackboard, while the rest were unoccupied. This is a common activity in the classrooms of the world, but it should only last a few minutes. Students instead could be doing individual work or group work by turning around and forming groups with the students at the desks behind them, but they were not. - Teacher interaction methods limit student participation. In all classes visited, teachers standing by the blackboard asked questions to the general class and accepted volunteer group answers from a few students in the front rows. The rest were unresponsive, and likely to fail. - The limited class time must be designated on a priority basis to French instruction starting in grades 2-3 rather than to the acquisition of basic skills. For example, the mission observed teaching conditional sentences in grade 4 to students who could hardly do basic arithmetic. 2.7 Not surprisingly, student achievement in rural schools is low. For example, third graders read haltingly and without comprehension, unsure whether to read vowels using the French or the Malagasy phonology. Those in the middle of grade 1 had no reading skills (whereas in other countries with phonetic alphabets, children may already read by then). Repetition rates in the areas visited hover around 31-40%.' Though the repetition grades are lower in later grades, after the weaker students have dropped out, students in secondary school of small towns do not fare much better. For example, in Andrianovory (Toliara) no student in the past year had passed the 9th grade final examinations for the BEPC (Brevet d' Enseignement du Premier Cycle). The 115 students of the small lower secondary school do not have textbooks or laboratories; they study from notes, and secondary schoolteachers have limited education and high absenteeism. 2.8 The project had expected to change management styles, but school principals' role has not expanded, and they were not providing leadership or supervising other teachers' work. The supervisory chain that can effectively produce improved instructional delivery is broken. An administrative zone officer is in charge of groups of schools and is expected to visit them on a bicycle. But regulations decree that the officer must visit only twice a year, and this person is not closely supervised. The pedagogical advisors who should monitor learning often have no means of going to the field. About 1300 bicycles were financed by the project, but they were given to the zone chiefs (chefs ZAP). The cars provided by two IDA projects are either used for general purposes or have broken down, and the project motorcycles are too large for women (who in Madagascar do not typically ride them). 2.9 The administrative zone chief is also in charge of training teachers. Apparently teacher training takes place, but without supervision; teachers do not practice the more interactive techniques they have been taught, partly because these are complex and may require some preparation. Also, teachers were themselves taught by teachers who lectured to a silent class, and they really do not understand why they should do any differently. Without textbooks or materials, the activities students can do are limited. Teachers do learn to make instructional materials (e.g., a set questions and answers that students can use by themselves), but it is up to them to duplicate them later, and without extra means they do not. Preservice teacher training in 7. Sector study 2000, table 2.8, p. 23. 7 Madagascar also functions through lectures, and with a duration of one year, it is not sufficient to change people's perceptions of what effective teaching is like. 2.10 During vacations, school districts (such as Toliara) offer inservice training to teachers. These (particularly the community-paid teachers who do not have a baccalaureate) may be deficient in subject matter, French, and pedagogy. The government is thinking of introducing distance education for teacher training, but the problem to resolve is how to supervise and motivate teachers to teach well rather than how to offer them voluntary educational opportunities. Also, teaching behaviors may change much more easily through role modeling than through lecturing and reading. 2.11 Some motivating activities were going on. In Tsiroanimandidy there was a contest for the model school, and the zone officer was visiting many. Teachers had been trained to ask their students to interview their parents and bring the results to class. New textbooks on environmental education appeared with poems in Malagasy about the environment. However, no effort was being made to retain in class girls whose parents wanted to marry them early. Despite statistics that show almost equal numbers of girls going to school, one fifth grade had only 18% girls. The mission received the explanation that by age 14 most girls were married; most fifth-grade students, having started late or repeated years, were already 14. 2.12 A large problem identified through the CRESED studies (Annex A Table 1) was teacher absenteeism, partly due to the fact that teachers must go to towns every month to get paid. Zone officers are expected to take salaries to teachers, but apparently this does not work well. It is also cumbersome because they must be accompanied by the police or army. If a system of rural banks is put in place through a proposed microfinance project, this may become easier. 2.13 The government gives no attention to NGO-run schools, since it is expected that the NGOs will provide all inputs. But this is not necessarily the case. Some merely build schools and hire a teacher (e.g. the Orthodox Church). Others demand fees, which students cannot pay. For example, a Catholic rural school (the only school in Andabanabo, Toliara) demanded MGF 4000 (US$.6) per month, so fewer than half the residents went to school. Compared to large schools and full classes everywhere, that school had 56 students. Yet, a government official was under the impression that those students had access to school. 2.14 Due to the lack of monitoring indicators, quality of instruction in the schools visited prior to CRESED is unknown; but the level of education observed was so low that it is hard to discern what the project benefits have been. The Manpower Training Project (PREFTEC; Cr. 2382-MAG; Annex A Table 2) 2.15 Even by 1992 when PREFTEC became effective, the concept of attempting to project manpower needs and train accordingly was obsolete, yet the project implemented centrally planned training activities. The multiple and unrelated objectives, from training of civil servants to agricultural reform, negatively impacted on the outcomes of this project. Their coordination depended on a committee that did not function well throughout most of the project. 2.16 Mission interviews with some members of the business community brought to the fore a perception that there is a continuing lack of skilled laborers and artisans, such as electricians, builders, plumbers, and carpenters.! The interviewed businessmen stated that the few existing 8. Labor data were not available to support this position, partly because the labor 'observatory' that would have collected them has not functioned. 8 skilled workers are in much demand and often move to companies paying higher salaries. Some businessmen expressed the concern that there are few institutions that train skilled or semi-skilled workers badly needed in the economy. Vocational raining centers exist but may not respond well to the decision-making processes of the very poor; persons with limited education may decide to learn a skill only when they look for work and have not attended vocational schools in advance. Though many businesses prefer to train their own staff, some skill level is a prerequisite, and many cannot afford to spend the time needed and hire unskilled laborers willing to learn on the job. The low level of education provided in schools makes quick training difficult. 2.17 Yet, IDA has not directly helped vocational training centers to become more effective. PREFTEC, like the first IDA-assisted education project (1968-1974), focused on technical secondary schools (lycees) in terms of preservice education.' These lead to the Baccaulaureate, which enables access to the university, and graduates are unlikely to become artisans. The project also supported the National Institute for Teacher Training of Technical and Vocational Institutions, and financed training of trainers. However, the teachers who teach vocational training must have a Baccalaureate plus at least two additional years of study; this level of education is typically attained by the middle class, so instructors rarely have industry experience. (The project attempted to provide industry experience, but results were limited.) At the same time, master artisans are ineligible to teach in vocational training centers. Also, teaching salaries are low and equipment is outdated, making these centers not very desirable places of training. The project did not directly deal with these issues and conditions in vocational centers; it equipped and refurbished only technical secondary schools and supported the Department of Technical Education that would make policy decisions for the lower-level institutions. Accordingly, a research center was set up within the National Institute for Teacher Training of Technical and Vocational Institutions, which developed curricula, hired 29 trainers of trainers, and trained teachers of technical-vocational institutions in various specialties. However, the system was not reformed, teacher salaries remained low, qualifications did not match teaching needs, and needed new specialties were not opened. The strategy was ineffective: an operation audit of 62 vocational centers upon project completion showed that they continued to provide a low level of training. The only institutions considered by the business community as credible are those of NGOs, such as the Catholic church. 2.18 To advise the government and the private sector regarding inservice training needs of employees and groups of independent workers, the project established the National Council for Technical and Vocational Training (CNFTP), which consists of private-sector members. The council has had limited activity in various project periods, but it was functional at the time of the OED mission, and a meeting was held with members. The Council believed that the private sector should take over the inservice training system with government support and with a payment of an obligatory tax by employers to support activities. (This contribution is expected to start in 2003.) However, the related legislation did not pass before the project was ended, and training activities were sharply reduced. Inclusion of more prominent businessmen might help sustain the activities of this council. 2.19 The inservice training fund overseen by the Council was the most successful component of the project. A unit of nine technicians (supported by 24 mainly auxiliary staff) contacted enterprises and workers' associations, assessed needs for inservice training, and located suitable trainers. Ultimately, 638 subprojects were carried out, which gave training to about 18,000 workers. Examples are training for fishing, shrimp culture, fish processing, car mechanics, pt de fois gras production, business administration, bookkeeping, salt gatherers to improve quality of salt, food hygiene in a cooperative, French language for an accounting firm, and computer 9. CRESED was also expected to establish a link, but this did not happen. 9 training. For women in particular, training for traditionally female occupations was provided in sewing, embroidery, and fish processing. (Because of the tourist trade, embroidery does produce some income in Madagascar.) The fund also gave equipment in some occasions, such as sewing machines to a Catholic vocational center (Don Bosco) in Toliara. The agencies requesting training paid about 40% of training costs. Partly to assess training needs, CNFTP financed 31 studies. Some were of low quality; others resulted in the provision of training courses. Overall, the study results did not give a clear strategy direction and may not have been worth the money spent on them. 2.20 Enterprises and agencies visited by the mission at the recommendation of CNFTP stated satisfaction with the quality of training. They stated that CNFTP staff came to observe training and carried out evaluations before disbursing the final payment to providers. Ultimately the extent to which the 18,000 workers benefited is unknown. It would have been useful to evaluate more extensively the actual use of training and resulting productivity increases at a later time, but existing evaluations are quite limited. At any rate, information on the CNFTP and its activities is has not been widely disseminated; for example, none of the nine businessmen informally interviewed by the mission had heard of CNFTP. Providing demand-based training to the private sector is a complex task and needs much more government direction and priority. Nevertheless, the project succeeded in putting together the rudiments of a network to group small operators together for the purpose of training and could get effective instructors. 2.21 Upon completion of PREFTEC, a project preparation fund (PPF) was implemented for a project that would develop competencies in four areas (essential oils, fruits and vegetables, foie gra, and raphia). Based on the findings of the 2000 sector study, the idea for this project was cancelled, and CNFTP obtained financing through the follow-on primary education project. 2.22 The project also financed the development of a labor "observatory", for which the International Labor Organization (ILO) was to provide training. But technical assistance was delayed, and the observatory never became functional. (A similar component to provide job information for university graduates under CRESED also had limited impact.) The project created positions for the director and staff, which use up financial resources, and would have created more civil servant positions if it had functioned. It was unclear what benefits these components would provide, given their cost. 2.23 Very few activities were carried out in a component that aimed to reform agricultural education. Pilot projects with agriculture were carried out in two primary schools, which were also rehabilitated. Mid-level technicians taught students to plant gardens, raise chickens, and plant trees in one of the schools. 2.24 PREFTEC also financed furniture, equipment, vehicles, and a dormitory for the National School of Malagasy Administration (ENAM). Contrary to expectations, however, no training system was established. Furthermore, the mission heard complaints about the construction quality of the dormitory. Though completed three years earlier, plumbing was dysfunctional, and roof was falling. The director refused to receive it officially. Staff expressed concerns that the contractor had mismanaged the project and asked if IDA could help resolve the situation. ENAM has to finance from its own budget the repair of this building. The Accounting and Management Training Project (INSCAE; Cr. 1661 -MAG; Annex A, Table 3) 2.25 This is the second project that supported the development of INSCAE (Table 1); the support was given because during the structural adjustment credits of the early 1980s, IDA found 10 out that the country had practically no accountants and wanted to strengthen institutions that could provide accountability. The project depended heavily on foreign technical assistance, which was expensive. Overall, IDA and bilateral donors (France, Canada) spent $21.8 million in credits and grants for the two INSCAE projects. The second IDA credit was underutilized for a long time because of the more generous bilateral grants from which INSCAE has benefited. For unclear reasons, the projects did not build a building, though a lot was acquired. Since there was no space in rented premises, the second project refurbished a building that belongs to a nearby secondary school. Most of its staff went overseas (France and Canada) to obtain advanced degrees. All but 2 or 3 returned and have been teaching at the institution, where several have developed textbooks. 2.26 INSCAE is considered a first-class, independent institution that has a rapid rhythm of work and operates without being affected by university political upheavals. It is governed by a board consisting of bank representatives and other private sector people. Though the members could raise funds for a building, they do not because that is not the local custom. However, INSCAE has applied for some funds from the follow-on project to the CRESED (Cr. 3046). 2.27 In accordance with IDA expectations, INSCAE charges relatively high fees, about MGF 2.2 million (about US$350) per month. These pay for about 80% of recurrent expenditures. The institution has about MGF 20 million to spend on scholarships, and they mainly give them to high-scoring rather than needy students. Those who cannot pay usually are unable to attend. (In any case, the very poor are eliminated from the school system long before they become eligible to attend.) 2.28 Younger people study full-time and older ones part-time. Continuing education non- degree courses are also offered. All students get French Canadian textbooks in marketing, economics, and accounting (getting outdated) that were financed by the project and that are issued by the library for the semester. For the full-time students, INSCAE has limited places, is very selective, and admits students through entrance examinations. Only half the class of 120 graduates; the failure rate in the three years of studies is about 50%, and happens mostly in the first year. Those who score below 12 (of 12) in accounting at that time are expelled and told that they may retake the entrance examinations and start all over. Those failing courses in subsequent years are told that they may become part-time students. To avoid chronic students, as is frequent in the university, INSCAE allows very few repetitions of courses. 2.29 Professors defend these policies as a way to maintain quality and the elite reputation of the institution. However, the high repetition and push-out wastes parental and government resources and raises equity concerns. Classes must still be carried out for higher years, but they serve fewer students. Students interested in becoming accountants can be helped to succeed without lowering course-level standards, and without being forced to leave or to become part- time students. 2.30 Since January 2001 the institution is taking some steps to reduce failure. Student orientation has been increased from three hours to one week, during which professors explain what will happen in their courses and where students find difficulties. Professors must reserve time to see students, and those with difficulties are urged to attend. 2.31 Since the project was completed, the Bank has emphasized poverty alleviation and equal opportunity. Continuing in some manner to finance through CRESED an institution that focuses on elites and creates artificial obstacles to reduce student numbers seems contrary to country strategy. The scholarships given to high-performing students who can pay could be devoted to enable attendance for the poor who manage to pass the examinations. To help the poor, tuition 11 on the rest might also be raised. (The issue is whether the institution can identify who is truly poor.) 2.32 The project had an unrelated procurement component attached to it that had limited outcomes. Training took place mostly as planned, but government changes to the procurement code were unsatisfactory to IDA until long after the project was completed. Also, IDA did not assign a knowledgeable staff member to follow this component. To facilitate coordinated implementation, IDA could have made efforts to connect it to INSCAE. Courses might have been offered in procurement to preservice or inservice students, so that INSCAE could have been responsible for the instruction. Project Outcomes 2.33 The Accounting and Management Training Project met its main objective of training qualified accountants, managers, and procurement specialists, and belatedly the government passed procurement laws that are satisfactory to IDA. Project outcome is rated satisfactory. 2.34 The outcomes of both CRESED and PREFTEC, however, are rated unsatisfactory. The projects carried out many activities, but these were oriented towards procedural and administrative issues (such as production of studies with no impact); project activities were peripheral to learners' acquisition of useful information. 2.35 Only one of the five CRESED objectives (Table 2) has clearly been achieved, the strengthening of higher education; besides establishing short-cycle higher institutes of technology, the government made serious (though only partly successful) efforts to make universities more functional. But higher education costs were not contained, and the financial objectives were not achieved. Of the 10 policy measures envisaged in the SAR, only three were clearly met. 2.36 PREFTEC completely missed the mark. Although the Staff Appraisal Report discussed the vocational needs of the less educated population, the project was to a significant extent oriented towards technical secondary schools. Despite the achievements of the CNFTP training fund, a system of developing technical and vocational education policy was not established, and none of the project's five objectives was clearly achieved. 2.37 PREFTEC had an unforeseen negative outcome of creating many new civil service positions in the Ministry of Technical and Vocational Training and recruiting staff, which burden the national budget. As the mission took place, discussions were under way on how to maintain some of the staff involved in offering private-sector courses, given that there is no budget for them. Relevance and Efficiency 2.38 The overall human resource development strategy that IDA had developed for Madagascar is unclear. The goals of all three projects have been relevant to the economic needs of the country but not necessarily attuned to its social and economic priorities. CRESED and PREFTEC did not use resources efficiently; the changeover to French created training and materials expenditures that the country could hardly afford. The credits disbursed almost completely, but project objectives were not met. Designs focusing on the true issues that the projects were trying to resolve rather than on ancillary services would have been much more efficient. 12 2.39 The INCSCAE project used resources more efficiently. However, IDA and other donors spent considerable funds over 10 years to develop a single institution that serves about 300 full- time students per year and still lacks its own building. In hindsight, it seems that INSCAE could have been part of a larger project directed towards various aspects of higher education. Institutional Development Impact 2.40 The institutional development impact of the INSCAE project has been substantial, but the impact of CRESED and PREFTEC on the development of sectoral institutions has been negligible. The latter projects created several departments in the Ministry of Primary and Basic Education as well as in the Ministry of Technical and Vocational Training. However, these departments carry out very little relevant work and have very limited communication with each other. The fact that the personnel must be paid through the national budget without a significant benefit to the country makes the institutional development impact negative. Sustainability 2.41 Thus far, the INSCAE outcomes have proved sustainable in terms of resilience to risk of net benefits over time, and the sustainability of this project is rated likely. However, the sustainability of CRESED and PREFTEC is rated unlikely. The activities of these projects have had very limited effects on the education system; even the CNFTP training fund is unsustainable without donor financing and a tax paid by private-sector employers. Upon completion of currently planned activities, the operations may cease. Bank Performance 2.42 Overall, IDA performance leaves much to be desired, in project performance and in country strategy development. The reasons are the following: * None of the projects had baseline data and evaluation indicators, and it was therefore hard to the present state with the past. Expected targets were given on physical activities (such as civil works) but not on quality of education issues (e.g. amount of time on task, no of textbooks per student). * Although relevant, the lending portfolio did not fit well with country needs. Despite the preponderance of agricultural activities and smallholder trades, IDA did not directly finance agricultural or vocational education at a time when such investments were frequent. Paradoxically, IDA financed the INSCAE project at a time of strict austerity measures. Accountants are useful, but it is hard to believe that they could be the focus of two projects worth US$21.8 million in loans and grants. This is a case where the better-off got the lion's share of the lending despite bigger social priorities. * IDA financed few civil works in poor areas. In the 1980s and early 1990s, when the Bank financed large numbers of primary schools in many countries, none was built in Madagascar, and this task was left up to the communities. * As described in the Staff Appraisal Reports, the various departments and agencies created by CRESED and PREFTEC have complex organizational relationships and vague, overlapping roles that did not prove workable during implementation. (For example, three different departments in the Ministry of Technical and Vocational Training do identification of 13 needs.) These administrative arrangements may have been developed by Bank staff with minimal borrower consultation. The documents are long and time-consuming to read, and it is possible that borrower staff did not read them very well. At the same time, the archived documents do not show IDA's reaction to the creation of a separate Ministry for technical- vocational training, which added considerable recurrent expenditures to the budget. * IDA created complex projects, whose components were not sufficiently well prepared. Subsequently the supervision missions did not follow them closely. For example, IDA did not follow up on the functioning of the PREFTEC coordination committee and did not improve the scattered actions of the project. As a result, money was spent on sundry activities that did not reform the technical-vocational system as expected, and the country lost a valuable opportunity to develop functional vocational education. To the contrary, it was saddled with a loan and with recurrent expenditures that provided little benefit. * The project documents focus on procedural issues and offer very little information on instructional problems. Even in the successful INSCAE project, IDA did not enter into instructional issues, and the policies that result in artificial student rejection did not receive any attention. The very detailed and innovative sector work that was completed in 2001 also excluded at the outset quality of education and made recommendations without taking instructional delivery issues into account. * The projects were supervised adequately (see tables in Annex B). However, IDA seems to have focused on implementation details rather than question how various components helped achieve project goals. The big picture was missed in a multitude of discrete tasks. * IDA may have missed opportunities to help the government and the French Cooperation to assess the effects of teaching poor rural students through a foreign language. IDA could have promoted pilot projects, research, workshops, and donor coordination. Regarding the argument that the Malagasy language was insufficient to translate technological subjects, IDA could have brought in expertise from Malaysia and Indonesia, which developed their language so that it could be used all the way through higher education. Because of inattentiveness to language of instruction problems, the already weak rural education system faces an additional obstacle to the acquisition of basic skills. 2.43 Overall, IDA performance is considered satisfactory for the INSCAE project. But for the reasons discussed above, IDA performance in the CRESED and PREFTEC projects are rated unsatisfactory. Borrower Performance 2.44 Borrower performance was satisfactory for the INSCAE project but unsatisfactory for the other two. CRESED and PREFTEC suffered from shortage of counterpart funds and delays. Government commitment to INSCAE and PREFTEC was sometimes limited. The government showed a greater willingness to implement activities that would benefit the urban middle class than the rural poor. It did not increase budgets for primary education, but it allowed increased expenditures for higher education. 10. Merely analyzing enrollment statistics gives only part of the picture. The sector work could have been more valuable if it had included quantitative analyses of classroom observations from a sample of schools using a questionnaire that measured time on task. 14 2.45 The Minister of Primary and Basic Education expressed frustration with IDA advice that he perceived as conflicting with advice from the IMF. He attributed delays and slow disbursements to some extent to donors' complex requirements. For example, to prevent the construction of schools that may not open, IDA in a follow-on project specifies that the government should recruit a teacher before building a school. However, the IMF says the government should not recruit teachers before they have enough revenue to pay them. The government finds itself in a difficult position. Delays may be a result of systemic inefficiencies, and perhaps schools should not be built unless adequate budgets are available to operate them; however policies to increase the supply of teachers apparently are not coordinated with public sector management reforms designed to limit public expenditures on salaries. 3. Issues for Future Consideration 3.1 Below are the salient issues arising in the audited projects. Table 3 outlines more specific issues and recommendations. Foreign-Language Instruction May Impede the Learning of Information in Poor Schools 3.2 In Madagascar as in sub-Saharan countries, there is a high social demand for the mastery of French at all socioeconomic levels. Middle-class students exposed to audiovisual stimuli and aided by parents and caring teachers in private schools, may easily acquire the vocabulary needed to learn the information imparted in schools. Decisionmakers who watch their own children do well with French in private schools may be misled regarding the problems facing poor children and the social and financial costs of foreign-language immersion. 3.3 As discussed earlier, students' learning opportunities are very limited in poor areas. Students are in school only 2.5 hours per day (if the teacher is present), large classes limit teacher interactions, pedagogical advice is poor, absenteeism limits the number of school days. Therefore, much class time (perhaps two or three school years) is lost teaching students to understand the language well enough to acquire information using it. To acquire basic skills in French, priority has to be given to this subject rather than reading and math, or science. This lost time translates into low achievement levels in all subjects. 3.4 Children in multilingual countries often get some exposure to the foreign language of instruction through the media and the necessity to communicate in a lingua franca. In Madagascar, French is not needed to communicate with anyone, and the children who do not regularly deal with tourists get essentially no exposure. With a limited vocabulary available, classroom interactions suffer. Children cannot ask the questions they have and may not understand the explanations given to them. The poor command that many rural teachers have of French limits their interaction with students. There is an urgent need to improve these teachers' command of subject matter and methodology, yet the government must spend its scarce resources making them more fluent in French, instead. Overall, Madagascar cannot finance the intensive language exposure and instructional support needed for successful foreign-language immersion. 3.5 The existing educational research in developing countries favors mother-tongue instruction, particularly for lower grades and children of uneducated families. 11 In particular, students who study science in their mother tongue perform much better than those who study it in 11. e.g., H. Patrinos and E. Velez. 1996. Costs and Benefits of Bilingual Education in Guatemala, Human Capital Development Working Papers 74, Washington, D.C.: World Bank. 15 the official language and drop out less than other students. Experiences from Bank and other donor projects in Papua New Guinea, Mali, and Burkina Faso also indicate that the official language is learned better if students study the essentials in their mother tongue. For example, in Mali the French Cooperation sponsors pedagogie convergente, use of African mother tongues in the beginning and gradual transition to French. The mostly poor and rural children who attend the schools score better in French and math at the end of Grade 6 than do the mostly wealthier, urban kids (especially the children of civil servants) who go to French-medium-only schools." 3.6 It is well recognized that language of instruction is a politically sensitive issue in Madagascar. However, low quality of education is an equally politically sensitive subject. IDA should encourage the government to fully evaluate the impact of this policy, particularly on illiteracy among the poor and the rural-urban inequity in terms of basic skills. The donor community urgently needs to reach consensus and help the government establish a language policy that is fair to the poor populations. Pilot projects could be carried out to study the effects of Malagasy versus French-language instruction as well as the mastery of French on the various population segments of Madagascar. Based on increased student retention and achievement in countries with successful mother-language programs, the donors could help the government calculate the cost of foreign-language immersion in terms of (a) dropout, (b) repetition, (c) lost subsequent wages due to limited schooling. 3.7 Changes in policies to consider could include: * Malagasy instruction in primary and at least lower secondary grades of rural schools coupled with instruction of French as a second language for the current number of hours. In this way, basic information would be acquired independently of foreign-language mastery. * Instruction in French of less important subjects in lower secondary schools. Gradual introduction of subjects in French (p6dagogie convergente). * Development of Malagasy textbooks (or republication of older and private editions) as was originally planned, but with vocabulary that maintains rather than translate the French technical and scientific terminology. IDA could help provide technical assistance from countries that made such language adjustments, such as Malaysia and Indonesia. * Bilingual textbooks for primary and lower secondary grades, with French and Malagasy on opposite pages. Though these textbooks may have a higher cost, student dropout costs even more. Increasing Time on Task in Classrooms 3.8 Teaching and supervision of primary schools are difficult tasks; they are repetitive and may not be challenging to adults. Some teachers may make little effort to reach their students, while some pedagogical advisors find reasons not to visit classrooms. It is important to structure teaching and supervision tasks so that they can be manageable, clear, and possible to achieve. In response to the government's request for specific remedies to the situation, some measures to improve instructional delivery are presented below: * Every child should have textbooks in a clearly comprehensible language to take home for the duration of the school year. Textbooks should be given to religious and other NGO- supported schools that may have weak instructional supervision. 12. Research by Penelope Bender, World Bank, currently in process. 16 * Organized group work can increase time on task and use of information learned, particularly for the large classes that are characteristic of Madagascar. Teachers could learn a clear series of group and individual tasks for the various subjects, which students can do while other students are writing on the board or reciting. * Student attention must be directed by asking questions and requesting answers from students at random rather than from those who volunteer. Teacher should focus on the students at the back of the class, who may be more likely to drop out. * Inservice training must be focused on improving classroom behaviors. Lectures cannot achieve this; behavior is more easily modified through role modeling and direct feedback, so instructional videotapes on classroom processes should be produced and used where possible. Teachers need the time, encouragement, and means to develop their own instructional aids. Training sessions may last a day longer, and materials may be given to teachers during training so that they can prepare basic instructional aids all together. Under the follow-on project to CRESED, the schools have some money they can use to buy paper, glue, colorful pens, and other items. * Regular and skills-oriented supervision is necessary to increase time on task. For example, the administrative zone officer (Chef ZAP) could focus on improving teachers in two or three specific items each year. (e.g., asking questions to students at random, organizing group work while one student works at the blackboard, using instructional aids that were bought or developed). * The incentives needed to make the administrative zone officers and pedagogical advisors visit schools must increase, and their work must become more structured. For example, they may be asked consistently by their superiors to report on the achievement of the few specific teacher behaviors targeted for improvement. * A simple testing system to monitor students' learning with various changes must be established. A series of pilot tests (as originally conceived for the CRESED pedagogical unit) should clarify which concrete actions maximize achievement. * Teachers who have a Baccaulaureate are often unwilling to work in rural areas. The government has been recruiting teachers in specific localities, but even in such cases they prefer to work in smaller towns rather than villages. The government might experiment with assistant teachers, local women who may have 8+ years of education. These should receive training, which must also be given to the less educated teachers who are often hired by communities. * Schools exist in most villages, but they need improvement and expansion, including teachers' quarters to reduce absenteeism. The Bank or other donors should finance the expansion of the many small schools that have huge classes. CRESED asked communities to build or refurbish several schools. However, the time of community members may best be utilized in obtaining qualitative improvements and safeguard teacher attendance building. IDA should finance the construction and repair of schools, even if they are done with community labor. * With community help, school feeding programs, financed through the World Food Organization may be feasible with minimal losses. 17 Helping Students to Graduate Rather than Fail 3.9 Sometimes high failure rates imply that schools are good. This tradition may be in part responsible for the push-out policies of INSCAE. But it has similar consequences in primary and secondary schools, where large numbers of students drop out or repeat classes. 3.10 Automatic promotion has been used in many countries, but it merely postpones the problems to higher grades. Given the language, limited number of school hours, limited time on task, and dearth of textbooks, rural students may need remedial teaching to pass classes. NGOs and communities may be asked if they can organize these. Remedial teaching may be a much better use of community resources than building schools. Vocational Training-Enabling Craftsmen to Teach to Willing Students 3.11 Despite the many studies conducted on vocational training, the decisionmaking processes of students and employers are still not well understood. Social status issues are often ignored when making financing decisions. For example, students destined to receive the Baccalaureate in technical secondary schools are likely to be unwilling to exercise professions that are perceived to have lower status, such as masonry and plumbing. The people who are willing to learn these professions may do so only after they start working and, in effect, need inservice rather than preservice training. Therefore, investments in technical secondary schools may be targeted to the wrong populations. Investment should be focused on providing skills to those who will actually use them, possibly in short-term modular courses. 3.12 In addition, suitable teachers are not always available. Aging artisans may be very willing to impart their skills in classes, but they are often academically unqualified. Rather than attempt to provide industrial experience to academically qualified teachers who did not choose to exercise a low-status profession, it may be more useful to train the aging artisans in basic instructional delivery. Staff at the Technical and Vocational Training Ministry have recognized this need but have not yet taken action. 3.13 The donor community must help learners obtain the needed skills in whatever modes these are necessary. They may finance the refurbishing and staffing of vocational training schools to facilitate short-term training provided by artisans. 18 Table 3: Some Specific Sectoral Issues Main Issues Recommendations Primary and lower High dropout and repetition rates Instruction needs to become much more effective (see secondary below). Education Very limited time on task - students spend Teachers must schedule group work, simultaneous very little time learning activities Very few textbooks Each student should have a set of understandable textbooks to take home and study Crowded classrooms IDA should finance classroom construction Hire and train local residents, possibly less qualified women, as assistant teachers Many rural or poor children do not Teach French as a foreign language but consider and understand sufficient French pilot reintroduction of Malagasy as a language of instruction for public schools. Community involvement was limited to civil Parents' time is better used in monitoring and helping works teachers rather than build schools Weak school supervision A few concrete and clear goals should be assigned to supervisors, who should be held accountable Suitable motorbikes, more cycles needed Technical/Vocation Vocational training centers do not respond Curricula and equipment need effective updating for al Training to training needs market demands Academically qualified teachers are poorly Hire as part-time teachers experienced artisans who paid and inexperienced in needed trades may lack formal qualifications and provide focused teacher training to them. Technical secondary graduates often go to Focus training in secondary schools to sophisticated the university rather than the labor market technological careers, e.g. computers, biomedical, chemical, etc. Many staff occupied by the Ministry of Give the various technical and vocational institutions Technical and Vocational Training to financial and administrative independence and regulate training centers with very limited accountability so that they respond to their clients' needs results Training to businesses and small groups is Invite more prominent businessmen to council limited and spotty Create stronger and more extensive linkages with businesses Accountant Expensive program catered to a limited Refocus scholarships to reach those too poor to be Training number of elite students admitted. Regulations push out many first-year Enable students to meet standards rather than leave or students repeat entire years Lessons 3.14 The projects did not pay attention to instructional processes. The design of teacher training in CRESED did not sufficiently take into account the classroom conditions of teachers or the teacher training literature regarding effective behavioral changes. Many of the multiple activities of the CRESED and PREFTEC projects were not carried out, partly because of limited government commitment to the project design IDA presented. The difficulty of keeping track of the multiple activities and their actual outcomes were compounded by a lack of baseline data. For these reasons, the following lessons may be abstracted. * Attention to the management and procedural aspects of projects is necessary but insufficient to improve the quality of education. Attention to instructional delivery is necessary at all levels of education. * Preservice and inservice teacher training often does not result in behavioral change. Much more support and supervision is needed than is often available. Rather than being lectured, teachers must learn desirable behaviors through means that are more effective in producing behavioral change. * For small countries with large needs, it is tempting to attach various components to a single project rather than do multiple small projects that might not be viable. Very often this 19 strategy has not proved effective. When the commitment of an institution is low or when significant changes take place, some components may be neglected. The Bank does not have the resources to appraise multiple small projects, but supervision missions should be more intensive when multiple components are involved. * It is hard to carry out extensive systemic reforms when government commitment is low. * It is very difficult to ascertain what a project has accomplished without a monitoring and evaluation system. Baseline data are needed to gauge progress in outcome indicators. * Children learn information best in a language they know well. Lower-income children with parents who have limited education and who are in schools where individual attention is limited, may not master the foreign language fast or well enough to process information efficiently in it. Though political considerations usually drive decisions on language of instruction, the substantive issue is ability to learn important material through a language.  21 Annex A Annex A. Project Activities Table 1. Education Sector Reinforcement Project Components Activities Targets to be Outputs Outcomes Subcomponents AciiisAchieved Improve sector administration, management, and planning Provide computer and Provided Equipment is functional office equipment Refurbish schools 0 112 Schools are functional damaged by cyclones Training Various courses Results unknown provided to staff Reorganize planning Train education Curriculum for Done, effect on and statistics services planners and educational planning educational planning statisticians locally established at uncertain. and abroad. IMATEP 49 persons trained Effect of local training abroad is uncertain 2600 staff days local training Establishing provincial 6 6 In principle, closer directorates supervision of schools [after midterm Improve community Establish PRAGAP 2600 schools in 20 Communities built review] control of schools sub-provinces schools, little else (CiScos) happened. Improve the quality of basic and general secondary education Set up a pedagogical Established Develops programs but unit, institute of no textbooks, has little educational training activity. Increase the role of 58 primary-level Staff trained, results inspectors, education advisors not evident; teaching advisors, headmasters 97 secondary behaviors unchanged. Improve preservice 6 teacher training 5 primary, 1 for lower Trained teachers often training colleges to be secondary unwilling to go to established 864 primary teachers villages; those trained at preservice affordable by level communities often less educated improve inservice 33 centers 17 centers Methods taught at training 22,050 secondary training often not used teachers trained in class. Provide textbooks 9 textbooks 18 primary and Essential textbooks Malagasy, French, secondary textbooks missing for many math, science 2,978,000 copies grades 2 in math, no science Many rural students still have no textbooks Provide teachers' 7 12 guides Teachers use guides, maps, charts 204,480 copes textbooks rather than guides Introduce Materials printed and Unknown if materials environmental, distributed, some can change the 22 Annex A Componentsl Activities Targets to be Outputs Outcomes Subcomponents Achieved nutritional, and consisting of behavior of the population education environmental poetry population in school curricula Upgrade secondary Very few upgraded Schools in small towns school laboratories have no laboratories Strengthen higher level education Establish short-cycle Refurbish and equip 2 institutes Steady enrollments, higher institutes of 2 institutes in existing established, Bac+2 low dropout; industrial, technology buildings years secretarial specialties no textbooks, some Survey showed that furniture poor 68% had found work, About 510 students 23% were looking trained Develop plan to A few specialties Limited impact modernize university improved curricula Develop plan to National program for Chronic students improve university the improvement of removed from dorms, administration and education but finances have not finance improved Strengthen the Prepare master plan Master plan Plan has not been administrative and prepared, 5-year implemented; used to planning capabilities plan through working develop higher of the Ministry of groups of national education component Higher Education and consultants of follow-on project the universities. establishment of an Establishment of 4'Technopole Information provided to Employment centers of centers established; students is Information System to information and occasionally useful for provide students with vocational training aie job search. information and advice on employment and career possibilities. Studies Book distribution and Done Despite study, cost recovery, distribution is not good Preventive Done Virtually no preventive maintenance of school maintenance is done and administrative buildings Better use of sector Done Some auxiliary personnel and personnel sent to improve sector classrooms, most management. others in non-teaching _________________ ________________ ________________ positions Quality of education First mentioned in Partially carried out Results unknown mid-term review 23 Annex A Table 2. Manpower Training Project Components Activities Targets to be Outputs Outcomes Subcomponents AciiisAchieved Develop National Council became Little effect on quality Council for operational, and planning of the Technical and continues to work system Vocational Limited authority Education Institutional framework Decentralize Few functions The 62 vocational for preservice management to the decentralized training establishments education school level have weaknesses, inappropriate training, insufficient resources Civil works Refurbish and equip 8 refurbished They continue to technical secondary provide inadequate schools technical training Restructure, strengthen the TVET system Establish training fund 638 subprojects Much training executed successfully given, 18277 workers effects on productivity benefited unknown Set up 8? 8 Associations remained interprofessional weak, suspended after regional associations end of project for inservice vocational training Promote private 700 operators listed Variable quality, main provision of vocational in tertiary-sector training training Improve quality of the 8 schools 8 established and Graduates may attend system equipped, total 26 university rather than Establish technical existing work higher secondary schools Training trainers Unknown numbers of Teachers may not be staff received training technically competent Introduce a labor Technical assistance Technical assistance Component not market through ILO was delayed executed information No monitoring system system established Reform Extend the reach of 2 pilot projects in No significant activities agricultural agricultural education agricultural primary have taken place education and education training Train senior civil Strengthen National Build dormitory, Material and building Material underutilized servants School of provide furniture, provided Dormitory badly Administration equipment, cars 30 received training constructed (ENAM) Studies Women's 31 studies Overall, results on participation, various conducted, some productivity of needs assessments resulted in courses enterprises are uncertain. 24 Annex A Table 3. Accounting and Management Training Project Componentsl Activities Targets to be Outputs Outcomes Subcomponents Achieved Train managers Graduate students 653 in cycle 1 100% placement and auditors from INSCAE by 174 in cycle 2 project end. 101 in cycle 3 Establish and Establish institution Achieve self- Established as Not financially self- operate the capable of training sufficiency expected sufficient, institute National Institute 360 full-time students Government pays functional for Accounting 20% of expenses and Management Provide fumiture, Acquired as needed Furniture and equipment provision Library has 3000 equipment used for rented premises books Provide adequate Building a building Land available, IDA Institute remains in premises decided not to rented premises finance the building Acquire vehicles Acquired Vehicles still in service Provide technical Curricular Achieved Curricula currently assistance from U. of development used Quebec Provide staff Study abroad, Staff studied in Well-trained professors fellowships advanced degrees France and Canada now available at INSCAE Procurement assistance Establish a Unit originally Not functional; procurement unit established Central Tender Board attached to the was abolished Central Tender Board Train staff in procurement Update procurement Code was revised in A market law passed legislation some details rather around 1999 updated than reformed. legislation. 25 Annex B Annex B. Basic Data MADAGASCAR-ACCOUNTING AND MANAGEMENT TRAINING PROJECT (C1661-MAG) Key Amounts ($US million) Original commitment 10.3 Total cancellation 4.09 Total project cost Original 13.1 Latest 12.7 Date physical completion: December 31, 1994 Cumulative Estimated and Actual Disbursements (US$ million) FY87 FY88 FY89 FY90 FY9 FY92 FY93 FY94 FY95 Appraisal estimate 2.0 4.0 6.2 8.2 9.2 10.0 10.3 Actual 0.1 0.3 1.0 1.6 2.3 3.1 4.3 5.3 8.1 Actual as% of estimate 5 8 16 20 25 31 42 52 79 Date of final disbursement: May 22, 1995 Project Dates Steps in project cycle Original Actual Identification (Executive Project Summary) April 1984 Preparation August 1984 Appraisal January/February 1985 Negotiations December 1985 Board presentation February 25, 1986 Signing April 16,1986 Effectiveness July 15, 1986 October 8, 1986 Midterm Review N/A N/A Project Completion June 30, 1992 December 31, 1994 Credit closing December 31, 1992 December 31, 1994 26 Annex B Staff Inputs (staff weeks) Stage of project cycle Planned Revised Actual Weeks US$ Weeks US$ Weeks US$ Preparation to appraisal N/A N/A N/A N/A 28.7 N/A Appraisal/Negotiations N/A N/A N/A N/A 26.2 N/A Negotiations through Board approval N/A N/A N/A N/A 9.3 N/A Supervision N/A N/A N/A N/A 71.5 N/A Completion N/A N/A N/A N/A 6.0 N/A Total N/A N/A N/A N/A 141.7 N/A Mission Data Duration of Performance ratingsb Stage of project Date No. of staff mission Specializations Types of cycle (month/year) in field represented' Implement. Develop, problems' (# of days) Status Objectives Through Appraisal Dec.-84 2 7 EC, PS Appraisal through Jan/Feb-85 4 20 EC, SA, MTS, PS Board Approval Supervision 1 Mar-86 3 9 EC, SED Supervision 2 Jul-86 2 9 EC, SED Supervision 3 May-87 1 6 PS Supervision 4 Oct.-87 2 13 EC, SED 2 1 PMP Supervision 5 Jan.-88 1 N/A ACTS Supervision 6 Sep./Oct.-88 1 17 EC 2 1 PMP Supervision 7 Jan.-89 1 17 ACMTS 2 1 PMP Supervision 8 Jul.-89 1 9 ACMTS 2 1 CLC, PMP AF, PP, TAP Supervision 9 Oct.-89 2 8 EC, ACMTS N/A N/A Supervision 10 Jul.-90 1 18 ACMTS 2 1 CLC, PMP, AF, PP, TAP Supervision 11 Jan.-91 1 14 ACMTS N/A N/A Supervision 12 Jul.-91 1 2 EC 2 1 CLC, PMP, AF, PP, TAP Supervision 13 Jul.-92 2 14 TVS, AIS 2 1 CLC, PMP, AF, PP, SP Supervision 14 May 93 2 14 TVS, FA 2 1 CLC, PMP, AF, PP, SP, FP Supervision 15 Nov./Dec.-93 2 16 TVS, FA 2 1 PMP, AF, PP, SP, FP Supervision 16 Mar/Apro.-94 1 16 TVS 2 1 PMP, AF, PP, SP, FP Supervision 17 Oct.-94 1 7 TVS S HS Completion Apr./May-95 2 EC, IS S HS a. AIS = Architect/Implementation Specialist; EC Economist; FA = Financial Analyst; IS = Implementation Specialist; MTS = Management Training Specialist; PS = Procurement Specialist; SA = Senior Architect; SED = Senior Educator; TVS = TechnicalNocational Specialist. b. 1 = Problem Free; 2 = Moderate problems; 3 = Major problems; 4 = Major problems - Corrective action to be taken. c. AF = Availability of funds; CLC = Compliance with legal covenants; FP = Financial Performance; PP = Procurement progress; SP = Studies Progress; TAP = Technical Assistance progress. PMP = Project Management Performance 27 Annex B MADAGASCAR-EDUCATION SECTOR REINFORCEMENT PROJECT (C2094-MAG) Key Amounts ($US million) Original commitment 39 Total cancellation 0.64 Total project cost Original 55 Latest 61 Datepyia opein ue3,19 Cumulative Estimated and Actual Disbursements (US$ million) FY90 FY91 FY92 FY93 FY94 FY95 FY96 FY97 FY98 FY99 Appraisal estimate 3.0 9.0 17.0 32.5 35.0 39.0 39.0 39.0 39.0 39.0 Actual -- 2.8 5.7 9.5 12.5 20.6 29.9 38.9 42.9 42.9 Actual as % of estimate - 31.5 33.7 34.1 35.8 52.7 76.7 99.9 109.9 109.9 Date of final disbursement: August 25, 1998 Project Dates Steps in project cycle Original Actual Identification -- July 1986 Preparation -- 1987-1988 Appraisal -- April 1989 Negotiations -- December 1989 Board presentation' -- February 13, 1990 Signing -- April 4, 1990 Effectiveness May 1990 July 24, 1990 Midterm Review -- October/November 1992 Project Completion December 31, 1995 June 30, 1998 Credit closing June 30, 1996 June 30, 1998 28 Annex B Mission Data Stage of project Date No. of staff Duration of Specializations Types of cycle (month/year) in field representecf Implement. Develop. problemsc (# of days) Status Objectives Through Appraisal 7/86 to 3/89 Appraisal through 4/89 5 26 ED, EC, IMP, TEX Board Approval Supervision 1 2-3/90 4 21 EC, ED, T. ED, PLA Supervision 2 7190 1 15 IMP Supervision 3 10-11/90 1 10 ED 2 Supervision 4 6/91 1 12 ED 2 Supervision 5 2192 2 11 EC, T.ED 2 2 Supervision 6 2-3/92 4 21 ED, PLA, AR, ED 2 1 PMP, CLC Supervision 7 6/92 2 7 ED, FA 2 1 PP, CLC Supervision 8 10-11/92 6 16 ED, PLA, HED, 2 1 PP, CLC T.ED, AR Supervision 9 7/93 7 15 ED, PLA, H.ED, 2 1 PP, CLC AR, IMP, FA Supervision 10 11-12193 6 20 ED, H.ED, IMP, AR, 2 2 PMP, PP, T.ED, EC CLC Supervision 11 3194 2 12 ED, H.ED 2 2 PP Supervision 12 4/94 1 15 CO S S PP Supervision 13 6-7/94 3 10 ED, IMP S S PP Supervision 14 9-10/94 5 16 ED, H.ED, IMP, MA S S PP Supervision 15 1-2/95 5 17 ED, IMP, MA, AR U S PMP, PP Supervision 16 5-6195 3 14 ED, T.ED S S PMP, PP Supervision 17 10/95 1 8 TEX S S PMP, PP, AF Supervision 18 11-12/95 3 14 ED, H.ED, AR S S PMP, PP, AF Supervision 19 4-5/96 3 16 ED, H.ED,IMP S S PMP, PP, AF Supervision 20 11-12/96 6 28 ED, H.ED, IMP, EC S S PMP, PP Supervision 21 3-4/97 6 20 ED, H.ED, IMP, EC S S Supervision 22 6-7/97 7 16 ED, H.ED, EC, IMP, S S TEX Supervision 23 10-11/97 7 16 ED, H.ED, EC, IMP, S S Completion 5-6/98 3 29 ED, H.ED, IMP S S a. AR = Architect; CO = Country Officer; EC Economist; ED = Education Specialist; FA = Financial Analyst; H.ED = Higher Education Specialist; IMP = Implementation Specialist; PLA = Education Planner; T.ED = Technical Education Specialist; TEX = Textbook Specialist b. 1 = Problem Free; 2 = Moderate Problems; 3 = Major Problems; 4 = Major problems = Corrective action to be taken; S = Satisfactory; U = Unsatisfactory. c. AF = Availability of funds; = CLC = Compliance with legal covenants; FP = Financial Performance; PDO = Project Development Objectives; PMP = Project Management Performance; PP = Procurement Progress; SP Studies Progress. 29 Annex B Staff Inputs (staff weeks) Stage of project cycle Actual Weeks US$ Preparation to appraisal 168.3 213.1 Appraisal 19.4 37.2 Negotiations through Board approval 9.5 21.7 Supervision 192.2 469.2 Completion 7.6 .4 Total 397 741.6 Related Bank Credits Credit Purpose Year of Status approval 2474 Food Security and Nutrition (SECALINE) 1993 Completed 2778 Social Fund II (FID) 1995 Ongoing 3046 Education Sector Development 1998 Ongoing 3060 Community Nutrition 11 1998 Ongoing  31 Annex B MADAGASCAR-MANPOWER TRAINING PROJECT (C2382-MAG) Key Amounts ($US million) Original commitment 22.8 Total cancellation 0 Total project cost Original 55 Latest 621.3 Date hsclcmpletion: June 30, 1999 Cumulative Estimated and Actual Disbursements (US$ million) FY93 FY94 FY95 FY96 FY97 FY98 FY99 FY00 Appraisal estimate 1.50 5.00 10.00 16.00 20.00 22.80 22.80 22.80 Actual 1.54 2.56 6.01 10.12 13.81 18.93 22.39 23.04 Actual as % of estimate 103 51 60 63 69 83 98 101 Date of final disbursement: October 27, 1999 Project Dates Steps in project cycle Original Actual Identification -- February 1989 Preparation -- 1989-1990 Appraisal -- November 1990 Negotiations April 1992 Board presentation -- June 9, 1992 Signing -- June 25, 1992 Effectiveness September 25, 1992 January 15,1993 Midterm Review October 31, 1995 June/July 1995 Project Completion December 31, 1997 June 30, 1999 Credit closing June 30, 1998 June 20, 1999 32 Annex B Staff Inputs (staff weeks) Stage of project cycle Actual Weeks US$ Preparation to appraisal 73.2 n.a Appraisal 27.3 n.a Negotiations through Board approval 18.7 n.a Supervision 163.9 241.8 Completion 7.0 34.0 Total 290.1 275.8 Mission Data Duration of Performance ratingsb Stage of project Date No. of staff mission Specializations g Types of cycle (month/year) in field represented' Implement. Develop problemsc (# of days) Status Objectives Through Appraisal Up to 11/90 Appraisal through 2/92 2 12 PO, MT Board Supervision 1 7/92 2 14 IMP, MT, DO 2 2 ACF, PMP, CLC Supervision 2 12/92 3 12 Imp 2 2 PMP, PP, CLC Supervision 3 5/93 2 18 Imp, mt 2 2 Act, pmp, pp, cIC Supervision 4 11-12/93 3 16 DC, IMP, MT 2 2 Act, pmp, pp, cIc Supervision 5 3-4/94 1 17 IMP 2 2 ACF, PMP, PP, CLC Supervision 6 10-11/94 3 7 IMP, MT, FA S S CLC Supervision 7 1-2/95 2 7 IMP, MT S S CLC Supervision 8 6-7/95 2 14 IMP, MTE HS S Supervision 9 10-11/95 2 14 IMP, MT HS S Supervision 10 2-3/96 2 8 IMP, MT HS S Supervision 11 10/96 1 6 IMP HS S Supervision 12 11-12/96 3 16 ED, IMP, MT HS S Supervision 13 6-7/97 2 15 ED. O,J JS S Supervision 14 10-11/97 2 17 ED, IMP HS S Supervision 15 2-3/98 3 7 ED, IMP, MT S S Supervision 16 5-6/98 3 30 ED, IMP, MT S S Supervision 17 10/98 4 21 ED, IMP, FA, MT U S PMP, ACF Supervision 18 12/98 5 15 ED, IMP, EC, MT, U S PMP, ACF FA Supervision 19 5/99 4 16 DC, ED, IMP, EC S S Completion 10/99 1 10 HDE S S a. DC = Division Chief; DO = Disbursement Officer; EC = Economist; ED = Education Specialist; FA= Financial Analyst; HDE = Human Development Economist; IMP = Implementation Specialist; MT = Manpower Training Specialist; MTE = Manpower Training Economist; PO= Project Officer. b. 1 = Problem Free; 2 = Moderate Problems; 3 = Major Problems; 4 Major Problems - Corrective action to be taken. S = Satisfactory; U = Unsatisfactory; HS = Highly Satisfactory; HU = Highly Unsatisfactory; NR = Not rated c. ACF = Availability of counterpart fund; CLC = Compliance with Legal Covenants; FP = Financial Performance; PDO Project Development Objectives; PMP = Project Management Performance; PP = Procurement Progress; = SP = Studies Progress. 33 Annex C Annex C. Comments from the Borrower REPUBLIC OF MADAGASCAR MINISTRY OF TECHNICAL EDUCATION AND VOCATIONAL TRAINING GENERAL SECRETARIAT Comments concerning the Madagascar Performance Audit Report dated June 6, 2001 (Sector and Thematic Evaluations Group, Operations Evaluation Department) 1. Audit procedure and formulation of the report We fully endorse the mission's decision to visit stakeholders in the field. However, its observations would be more useful if more meaningful references to the Implementation Completion Report (ICR) and the Education Sector Study were included along with the examples drawn from the mission's brief visits (cf., page 3, para. 2.2), particularly since the mission was fairly short, considering its objectives. 2. Content of the report 2.1 Para. 1, Background (pp. 1-3) For the many reasons already indicted in the Malagasy PAR, the observations in the report regarding strategy shortcomings (1.3), the projects' complexity (2.1), and the multiple and unrelated objectives (2.15) need no further comment. However, we would like the report to state, for example, that these factors also had an adverse impact on the social and economic actors (enterprises, local communities, NGOs) which were our principal partners, as well as on the project actors. 34 Annex C - Regarding paragraph 1.5 (p. 3), the report should specify which donors contributed to which project(s). In addition to IDA, PREFTEC also received support from France and UNDP, although they withdrew before the project was completed 2.2 Para.2: Implementation Experience of the Audited Projects Manpower Training Project (PREFTEC, Cr. 2382-MAG; Annex A, Table 2) Reference Language used in report Comments Proposed language 2.2 -p. 3 "The mission visited various project The mission did not visit all PREFTEC "The mission visited a number sites in Antananarivo, Toliara, and sites or organizations, even in of project sites in Antsiranana (Tsiroanimandidi). These Antananarivo. Antananarivo (including included... the technical teacher Tsiroanimandidi). These training college, businesses that had Tsiroanimandidi is a rural municipal included.. the National used CNFTP training services,..." district in Antananarivo province; it is Technical and Vocational not part of Antsiranana. Education Teacher Training Institute, formerly the It is assumed that by "technical teacher Resource Center for training college" the authors mean the Technical Education National Technical and Vocational Personnel, CERES, (Credit Education Teacher Training Institute, Agreement 2382-MAG), a formerly the Resource Center for number of businesses that had Technical Education Personnel, CERES used CNFPT training (Credit Agreement 2382-MAG). services,..." 2.15-p. 7 "Even by 1992 when PREFTEC It is true that the project was conceived " Even by 1992, when 2.16-p. 7 became effective, the concept of (under the guidance of international PREFTEC became effective, attempting to project manpower needs consultants) largely in order to provide a manpower needs assessment and train accordingly was obsolete,..." majIor boost to Madagascar's private and the concept of training 35 Annex C sector (job creation, demand for skilled accordingly were obsolete. workers), which explains the choice of interventions such as the focus "on "This shortcoming was technical secondary schools in terms of partially overcome following preservice education"(p.8-2.17,). the mid-term review when Indeed, inasmuch as Malagasy texts PREFTEC activities were stipulate that it is the responsibility of focused on offering basic technical secondary schools and colleges skills classes targeting the to train qualified workers, the choice very poor, examples of which does not seem entirely inappropriate are mentioned in the report (2.15, first sentence), given the concern (p.8-2.16). All technical expressed by some businessmen secondary schools and interviewed by the mission "that there colleges receiving PREFTEC are few institutions that train skilled or assistance have introduced an semi-skilled workers badly needed in the in-service training component economy" (p.8- 2.16). incorporating short-term training focusing in particular on self- employment and jobs in the informal and traditional sector (artisans), thereby expanding public access to training. CNFTP, through the inservice training fund, provided many of these training courses, as well as others in cooperation with NGOs and private vocational training centers (cfs 2.19-p. 8)." 36 Annex C 2.17-p. 8 "Yet, IDA has not directly helped The fact that institutions receiving vocational schools to become more support from PREFTEC have not effective. PREFTEC, like the first become more effective has been IDA-assisted education project (1968- discussed in a number of analyses 1974), focused on technical secondary (including the analysis of the Education schools in terms of preservice Sector Study). education. These lead to the Baccalaureate, which enables access Nevertheless, this paragraph further to the university, and graduates are reinforces the value of having both unlikely to become artisans." parties agree from the very outset on the monitoring and evaluation indicators to be used. 2.17-p. 8 The technical teachers training college "The primary responsibilities and the research center mentioned in this of INFOR (formerly CERJES) paragraph are one and the same entity: are to: the National Technical and Vocational - develop and update Education Teacher Training Institute, training programs and INFOR (formerly CERES). teaching methods, and - train trainers (initial and The link the paragraph establishes inservice training)." between the level of education of the teachers and their social class (middle The remainder of the paragraph class), on the one hand, and the fact that should be amended they rarely have industry experience, on accordingly. the other hand, is unclear. INFOR is not responsible for reviewing teachers' salaries. A comprehensive reform of staff regulations is under way in the Ministry of the Civil Service (with World Bank support). Any salary 37 Annex C reform would obviously have to evolve from this process. 2.18-p. 8 "To advise.., the project established CNFTP consists of three bodies: a "To advise..., the project the... (CNFTP), which consists of General Assembly, a Governing Council, established the... (CNFTP). private-sector members." and an Executive Secretariat. The Governing Council of this body consists mainly of The Governing Council with which the representatives from the mission met has 12 members: two private sector (10 of the representing the State and 10 Council's 12 members). representing the private sector. CNFTP was of the view that the private sector should take No consideration has been given yet to over the inservice training having workers help finance inservice system with government training. support in the form of mandatory contributions from employers to finance activities. These contributions are scheduled to start in 2003. However, the corresponding legislation did not pass before the project was ended, and training activities were sharply reduced." 2.19-p. 8 "A unit of ssix technicians (supported By virtue of its mandate, the Executive "Nine technicians contacted by 24 mainly auxiliary staff)" Secretariat of CNFTP carries out enterprises and workers' important technical activities (providing associations, assessed needs for advice and support to businesses and to inservice training, located the Ministry). However, it also carries suitable trainers, and processed out vitally important administrative and, requests for cofinancing from especially, financial management tasks the training fund. y (such as managing the inservice training 38 Annex C fund and ensuring that the appropriate "Three individuals are procedures are followed, depending on responsible for accounting the sources of financing). and financial management. Three others are responsible Owing to mission and leave schedules, for administrative some key CNFTP personnel were management (personnel, unavailable during the mission's visit, logistics, and procurement). "The remaining personnel are suport staff." 2.20-p.9 "...It would have been useful to The audit was prepared with the World "...It would have been useful to evaluate the actual use of training and Bank at project completion and during evaluate the actual use of resulting productivity increases at a the initial phase of the project training and resulting later time, but such an evaluation was preparation fund (PPF) established productivity increases more not done. At any rate, these activities pursuant to an economic analysis. The thoroughly, but the did not become widely known; none analysis contained recommendations that evaluations were somewhat of the nine businessmen informally led to the development of the Skills limited. At any rate, interviewed by the mission had heard Development Project. Support for the information concerning of CNFTP." preparation of the project was provided CNFTP and its activities was by Advance Agreement Q-179 (August not disseminated widely. For 1999). (Cf. "Skills Development in example, none of the nine Madagascar: businessmen informally Background and Potential Approaches," interviewed by the mission had J. Lane and J.-P. Peresson, February heard of CNFTP." 2000.) A large-scale effort to communicate with all businessmen would have been too expensive, and such a campaign would have accentuated the scattered impact of the activities. Information was provided 39 Annex C communicated mainly to targeted contacts, in keeping with the various issues addressed in project activities. 2.21 -p. 9 "The CNFTP obtained financing This paragraph is very confusing. ...The CNFTP obtained PPF through the follow-on primary financing under an Advance education project for a period of two Agreement (Q-179 dated years. Financing was reduced, and it August 23, 1999) for the was decided to support only the four preparation of a Skills sectors deemed government priorities: Development Project focusing mines, clothing, fishing-agriculture, mainly on four areas that tourism. Requests for training in other were considered strong: sectors are rejected. This is essential oils, fruits and unfortunate, because the network of vegetables, foie gras, and contacts in various sectors is still raffia. fragile, and should not be abandoned." "Soon before PPF completion, and on the basis of the results of the Education Sector Study, the Skills Development Project idea was dropped. CNFTP is in the process of developing a technical and vocational education and training (TVET) component for inclusion in CRESED H, which is currently under way. The component's objective is to strengthen TVET capacity to offer educational services and develop a future 40 Annex C education policy for TVET." 2.23-p.9 "Very few activities were carried out Activities were carried out in eight focal "Few activities were carried out in a component that aimed to reform areas under the EFA reform component, in a component that aimed to agricultural education. All that was including rehabilitation of two pilot reform agricultural education. done were two pilot projects with agricultural technical vocational schools The main accomplishments agriculture in primary education in in order to test the foundation for EFA were: two provinces. Students were taught reform. - Enrichment of the general to plant gardens, raise chickens, and education curriculum by plant trees." In order to reform rural education including material on systems, attention must be focused on agriculture, along with field basic education, the aim being to work and teaching aids incorporate the agricultural sector in the (kitchen gardens, henhouses, general education curriculum and find etc.), followed by teacher ways to integrate the marginalized and training. The CRESED reduce the primary school drop-out rate. project's pedagogical unit organized these activities in two pilot public primary schools. - Rehabilitation of two pilot technical and vocational training schools to test the foundation for EFA reform; and - Various training activities provided with CNFTP support for rural youth in one of the FTP pilot schools." 2.24-p.9 "PREFTEC .. .(ENAM). .). At the time Immediately after the mission's "The mission commented on this report was being written, the issue departure, and before seeking redress, the problem before its had not been resolved." METFP asked TRANSTECHNIQUES - departure. Appropriate the company responsible for corrective measures have ben 41 Annex C supervising the ENAM dormitory taken (by both METFP and rehabilitation - to identify the ENAM)." construction defects, and the Ministry so informed the World Bank. To date ENAM itself has taken the action required to rehabilitate the buildings. RELEVANCE AND EFFICIENCY Reference Language used in report Comments Proposed language 2.38-p. 11 "... .Designs focusing on the true The Credit Agreement called for the issues that the projects were trying to establishment of bodies such as ONEF resolve rather than on ancillary (formerly ONCE), CNFTP, and INFOR services would have been much more (formerly CERES). efficient." The intention was to have these entities and the training centers provide guidance to and conduct activities for the ultimate beneficiaries (including employers, workers, and self-employed persons). Institutional Development Impact Reference Language used in report Comments Proposed language 2.40-p. 12 "....The latter projects created several The Ministry of Vocational Training no "...The latter projects created departments in the Ministry of longer exists. Virtually from the outset, several departments in the Primary and Basic Education as well PREFTEC was under the technical Ministry of Primary and Basic as in the Ministry of Vocational supervision of the Department of Education as well as in the Training." Technical Education and Vocational Ministry of Technical 42 Annex C Training (a special unit reporting to the Education and Vocational Prime Minister, or a component of the Training." Ministry). Sustainability Reference Language used in report Comments Proposed language 2.4 1-p. 12 "....However, the sustainability of Although it is true that some activities ... the CNFTP training fund is CRESED and PREFTEC is have slowed down, the bodies created unsustainable without donor unlikely .... even the CNFTP training under PREFTEC have continued to financing (particularly in the fund is unsustainable without donor discharge their mandates, with primary case of activities focusing on financing." support from Malagasy Government poverty reduction) and subsidies. The Government definitely without the establishment of wishes to ensure the project's a national system in which sustainability. However, given the private sector employers obvious magnitude of the task ahead, it finance inservice vocational naturally hopes that donor financing will training." be forthcoming. The sustainability of the training fund vis--vis the private sector can be assured by requiring employers to make mandatory contributions for inservice vocational training. Complementary activities are planned under the CRESED uu TVET component so that the contribution can start in 2003 and public resources can be directed mainly pto_poverty reduction efforts. 43 Annex C 2.3 Paragraph 3: Issues for Future Consideration Reference Language used in report Comments Proposed language 3.13-p. 17 "The donor community must help Plans for this option are included in the learners obtain the needed skills in TVET component of CRESED II. whatever modes these are necessary. They may finance the refurbishing and staffing of vocational training schools to facilitate short-term training provided by artisans."