21243 Working Paper Series on Impact Evaluation of Education Reforms Paper No. 4 El Salvador's EDUCO Program A First Report On Parents' Participation in School-Based Management El Salvador Evaluation Team** July 1997 National Research and Evaluation Division Ministry of Education, El Salvador Development Economics Research Group Poverty and Human Resources The World Bank ** The evaluation team consists of Sandra de Umanzor, Isis Soriano, and Marta Rosa Vega (Ministry of Education, El Salvador) and Emmanuel Jimenez, Laura Rawlings, and Diane Steele (World Bank) This paper is a product of the research project, "Impact Evaluation of Education Projects Involving Decentralization and Privatization" (RPO 679-18) being carried out under the guidance of Elizabeth King by the Development Economics Research Group of the World Bank. The paper was written by the National Research and Evaluation Division, Ministry of Education, El Salvador, in collaboration with the World Bank. The findings, interpretations, and conclusions are the authors' own and should not be attributed to the World Bank, its Board of Directors, or any of its member countries. Comments are welcome and should be sent directly to the author(s). For copies, please send request to Selina KIan at SKhan8@Worldbank.org I 4% -0 1 SUMMARY In 1991, El Salvador's Ministry of Education created a new model of school-based management to serve pre-school and primary school students in the most rural and remote areas of the country that had suffered disproportionately in the country's civil war. The aim of the Community Managed Schools Program, better known by its Spanish-language acronym, EDUCO, was to decentralize the management of public education by increasing parents' involvement in and responsibility for the running of these schools. The philosophy behind the EDUCO program is that local people can run the schools in their communities more efficiently and effectively than can a centralized bureaucracy. The driving force behind the EDUCO school strategy is the Community Education Association, an elected parents' group responsible for the administration of the school. This report is the first in a series of studies being carried out by the Ministry of Education's National Research and Evaluation Division in collaboration with the World Bank aimed at evaluating the impact of EDUCO and other education reform programs. This study focuses on parents' participation in school management and its effects on schooling outcomes, including achievement, attendance, and repetition and dropout rates, all of which will be explored in greater depth in subsequent analyses. For this first study which was authored by the Ministry of Education based on analyses conducted in conjunction with the World Bank, we compared EDUCO schools and traditional public schools in rural areas, which are the most similar to EDUCO schools in terms of size, location, and student population. In the case of both EDUCO and traditional rural public schools, we examined associations between school and family characteristics and the outcomes observed in the school and then developed some hypotheses about how parents' participation affects school outcomes. We compared those parents who are members of the Community Education Association (ACE) in EDUCO schools with those parents who are members of Parents' Associations in traditional public schools. We found that ACE parents are more active participants in school affairs and that they feel that they have more personal influence over decisionmaking and have a more direct relationship with the school's teachers than do parents who serve on Parents' Associations. However, our analysis also revealed that the physical condition of EDUCO schools is worse than that of traditional rural schools and that the socioeconomic status of the households of the EDUCO students is lower than those of their counterparts in traditional rural schools. Also, the parents of EDUCO students tend to have attained a lower level of education than the parents of their counterparts. However, EDUCO third-grade classrooms have more teaching materials than do classrooms in traditional rural schools. Nevertheless, the study demonstrated that there are no significant differences in the academic performance of students in both types of schools. In addition, it appears that the objectives of the EDUCO program are being met particularly with regard to targeting poorer communities and promoting parents' participation in school management. ii The data used in this report were collected in October 1996 from a sample of 311 schools. The evaluation team consists of Emmanuel Jimenez, Laura Rawlings, and Diane Steele (World Bank) and Sandra de Umanzor, Isis Soriano, and Marta Rosa Vega (Ministry of Education, El Salvador). iii STRUCTURE OF THE REPORT I. INTRODU CTION................................................................................................. II. EVALU ATION M E TH OD OLOGY ...................................................................... 3 D efining the Population.......................................................................................... 3 Selecting the Sample .............................................................................................. 4 Choosing the Respondents ..................................................................................... 4 D ata Collection Procedures.................................................................................... 6 The M odel used to Evaluate EDU CO .................................................................... 6 The M ethodology used to Evaluate EDU CO .......................................................... 7 111. THE MAIN RESULTS OF THE EVALUATION ................................................. 8 School Inputs......................................................................................................... 8 Parents' Role in Management: ACEs and Parents' Associations ............................ 12 Parents' D irect Relationship with the School.......................................................... 14 Parents' Influence on their Children's Academic Performance ................................ 16 Educational Outcom es ........................................................................................... 20 IV . CON CLU SION S ................................................................................................... 21 V . RECOM M END ATION S.......................................................................................25 BIBLIOGRAPH .................................................................................................27 iv L Introduction In 1991, El Salvador's first post-war government set out to rehabilitate the public sector's capacity to deliver basic health and education services effectively and efficiently, notably to the poorer, rural areas of the country that have been under-served throughout Salvadoran history. As part of this initiative, the government launched a reform program in the education sector that has become internationally recognized as an innovative and progressive approach to primary schooling. One of the most interesting aspects of this reform program was that the government chose to decentralize responsibility for providing and managing educational services to local communities through the Community-Managed Schools Program, better known by its Spanish- language acronym, EDUCO (Educaci6n con Participaci6n de la Comunidad). This program was envisioned as a way to extend pre-primary and primary educational services to the rural areas, particularly to the poorest areas and those with little access to existing educational services. The three main aims of the EDUCO Program are to: * Increase the supply of educational services in the poorest rural communities * Promote the participation of local community members in their children's education * Improve the quality of pre-primary and primary schooling The main tenet of EDUCO's philosophy is the need for parents to be directly involved in their children's education. This was the rationale behind the creation as part of the EDUCO program of each EDUCO school's Asociaci6n Comunal para la Educaci6n or Community Education Association (ACE). These bodies consist of elected members who are all drawn from the community and are usually parents of the children who attend the school in question. ACEs are legally responsible for the running of EDUCO schools, including their budgets and personnel. ACEs can hire and fire teachers and are responsible for supervising their performance and attendance. The Ministry of Education transfers funds directly to the ACE's bank account. The ACE is responsible for administering these funds according to its assessment of the educational needs of the school. The bulk of the transfer is used to pay teachers' salaries in accordance with their contract with the ACE. ACEs are able to raise additional revenues by negotiating with other government agencies and international donors as well as by mobilizing local support, including in- kind support. ACEs, therefore, serve not just as administrators, but as important focal points for involving parents and local residents in supporting the school. Parents' Associations (Directiva de Padres de Familia) are found in traditional schools and have a more limited role that that played by the ACE. They have no jurisdiction over either the school's personnel or budget beyond the amounts they are able to raise locally through school fairs and other findraising activities. 1 During the early years of the EDUCO program, several studies were done that assessed what progress was being made towards meeting the program's aims.' These studies reached several interesting conclusions, including: * First-grade teachers in the EDUCO schools had more responsibility than first-grade teachers in non-EDUCO schools and their teaching was of a higher quality. Also, the ACE in EDUCO schools played a bigger role in monitoring the teacher's work than did Parents' Associations in non-EDUCO schools. * There was a greater awareness of educational issues in communities that were served by EDUCO schools. * The annual cost per EDUCO student was c/744 (equivalent to $85) in 1992, compared to c/541 (equivalent to $73) per traditional primary school student. However, several years have passed since these studies were published, which meant that there was a need for an up-to-date assessment of the success of the EDUCO program. In consequence, the Ministry of Education's National Research and Evaluation Division and the World Bank decided in 1996 to collaborate in conducting the first longitudinal evaluation of the EDUCO Program. The aim of this evaluation is to determine how the EDUCO program has affected parents' participation in running the schools and the quality and efficiency of the education provided. The evaluation is also testing the hypothesis behind the EDUCO program -- that local people can run the schools in their communities more efficiently and effectively than can a centralized bureaucracy. The study is based on the comparative analysis of a representative sample of EDUCO schools and a representative sample of traditional schools. This report is the first in a series that will be produced as part of this evaluation exercise. The subject of this first report is parents' participation in school management both as members of ACEs and Parents' Associations and as individuals directly involved with their children's schooling. In it, we provide up-to-date information to those policymakers who are responsible for administering the EDUCO program and to others with an interest in this innovative school- based management reform. Like the other reports in this series, our analysis is based on the Ministry of Education's National Research and Evaluation Division's baseline data from 311 EDUCO and non-EDUCO schools in El Salvador. This baseline data set is also being used by the Ministry of Education to develop a profile of traditional urban schools before the implementation of the urban school- based management model which is being introduced in 1997. The National Research and "A Comparative Analysis of EDUCO First Grade Sections and NO EDUCO First Grade Sections" Julia Mora Mora. San Salvador. October 1992, "An Evaluation of the Execution of the Established Goals for the First Year and the Development of the Organization and Participation Processes" Hugo Diaz Diaz. September 1992, "The Systematization of the EDUCO Program" German Marino Solano. San Salvador. December 1993, and "El Salvador Community Education Strategy: Decentralized School Management" The World Bank. December 1994. 2 Evaluation Division in collaboration with Francisco Gavidia University will carry out a follow-up survey of the same sample of EDUCO and traditional rural schools in October 1997, the data from which will be used to continue the evaluation of the EDUCO program. A separate follow- up survey will be carried out in 1998 of the urban schools in the sample in order to examine the impact of a new administrative model being introduced in urban areas in 1997. Our aim in this first study is to analyze the 1996 baseline data and to develop some hypotheses about how parents' participation in school affairs can affect the academic performance of students. Realizing that parents also directly influence the academic performance of their own children outside the context of the school and conversely, that the school and educational decisionmakers also have an influence on how students perform, we also examine the characteristics of the homes and schools of the EDUCO students. The associations that we observe do not strictly imply causality. However, they are suggestive and we use them to develop hypotheses that we then test more rigorously using multivariate analysis as part of a second study to find out whether there is an 'EDUCO effect" on student outcomes. A separate report examining the efficiency of the EDUCO model by closely analyzing the costs of EDUCO versus traditional schools, taking the costs of parents' contributions into account, is also being prepared. IL Evaluation Methodology The data used in this report were collected in October 1996 from a sample of 311 EDUCO and non-EDUCO schools in El Salvador by the Ministry of Education's National Research and Evaluation Division in collaboration with Don Bosco University. This section describes the methodology used to collect t-ese data and the criteria that we used to analyze them for the purposes of this study. Defining the Population The sample was drawn from the 3,634 primary schools with at least one third-grade section with at least 10 students enrolled in it. Each one of the four different types of El Salvadoran schools are included in this study - pure EDUCO, mixed ( which include both EDUCO and traditional sections), traditional public and private (see Table 1). Pure EDUCO schools are those with only EDUCO sections and that are run entirely under the jurisdiction of the ACE. Mixed schools have both EDUCO and traditional school sections, each of which are administered independently, with the EDUCO sections being run by the ACE and the traditional school sections as schools in the Ministry of Education's system. Generally, in these mixed schools, EDUCO sections lease space from an existing school. Because this particular study focused on parents' participation in the running of the whole school rather than of individual sections of schools, mixed schools were not considered in this study. All EDUCO schools are located in rural areas and, as our study illustrates, are targeted to the poorer segments of the population. 3 Traditional schools are public schools under the administration of a school director with little or no community input in the management and school-level decisionmaking process. At the time of the study, traditiaonal schools were located in both rural and urban areas.2 Private schools can be either state-subsidized, non-profit, or non-subsidized private schools. Private schools were not considered in this study, nor were traditional public schools located in urban areas. Therefore, the primary sampling unit for this study consisted of all third-grade sections with at least 10 students. Since enrollment data from EDUCO schools are of limited availability in the Ministry of Education, in the case of pure EDUCO schools, the primary sampling unit consisted of all schools with at least one third-grade section. Selecting the Sample To select the sample, the 3,364 primary schools were ordered first by type of school, then by department', location within the department, and the number of third-grade sections in the school. The initial ordering by type of school was carried out to facilitate the selection of the sample by type of school using a systematic sampling approach. Accordingly, after the ordering took place, a systematic random sample of 311 education schools was selected. The population and selected sample are presented in Table 1. Table 1: Sample Statistics Type of School Population Sample Number % Traditional 2,283 193 62 Mixed 500 69 22 Pure EDUCO 395 32 10 Private 456 17 5 Total 3,634 311 99 Choosing the Respondents This study is based on data generated from interviews with five types of participants in the education community: the director or person in charge of the sampled school, the teacher of the third- grade section selected for the study, five randomly selected students in that same section, and the parents or guardians of those students. In addition, we interviewed two members of the parent's committee --either the ACE or the Parents' Association- of the sampled school. 2 All traditional rural schools will be converted to EDUCO schools by the year 2000. 3 Departments are administrative government levels directly below the national level. 4 A separate questionnaire was administered to each of the respondents. Each questionnaire collected information on the informant's background, their role in the school, as well as basic data on the unit of interest for the analysis be it the school, classroom or household (see Table 2). In addition, language and mathematics achievement tests were administered to all of the students in the selected third-grade sections. Table 2. Factors Explored in Different Qestionnaires by Different Respondents Respondent Parents Parents of in Factors Director Teacher Student Student Parent Group Identification of center and respondent X X X X X School statistics (repetition, dropout, etc.) X X Infrastructure and resources in the school X Administrative management X Technical education management X X Finances X X Level of influence in decisionmaking X X X Labor conditions X Physical characteristics of the classroom X Teacher's view of student characteristics X Pedagogical conditions in the classroom X Identification afthe household X X Family characteristics x x Perceptions about the school x Characteristics of the family group x Household expenditures X Housing X Individual student characteristics X X Parents' relationship with the school X Parents' participation in school X management Parents' opinion about the school X In order to merge the data from the achievement tests with the interview data, the student codes assigned on the achievement tests were also used in the student and parent interview instruments. 4 This study is part of a larger effort in which achievement tests in language, mathematics, natural science, and social science were administered in the thirl, fourth, fi, and sixth grades to a nationally representative sample of schools These achievement tests are applied annually by the Ministry of Education with the technical assistance of the Intercultural Center for Research in Education through the financial support of the US Agency for International Development. 5 Data Collection Procedures Each school was visited twice. In the first visit, the language and math achievement tests were administered to all students in the selected third-grade section. The second visit was made a week later when the respondents were interviewed by a team consisting of three interviewers and one supervisor. The interviews were administered in two locations. The interviews with the person in charge of the school, the teachers, and the students were done inside the school, while the interviews with the parents of the five students and with the two members of the parents committee took place in their home or workplace. In each case, attempts were made to speak with the president and treasurer of the parent's committee, but, when these individuals could not be found, other members of the committee were interviewed instead. The Model used to Evaluate EDUCO The model that we used for evaluating EDUCO is based on the idea that parents' participation in their children's schools can affect these children's academic achievement. The model draws from the production function literature in the field of the economics of education, which views the schooling process as the result of a series of input mixes determined by a variety of actors. As illustrated in Figure 1, the decisionmakers in our model are the school managers -- the Ministry of Education and the ACEs or Parents' Associations - and individual parents. These decisionmakers determine the input mix in the school that shapes the students' academic performance. Additionally, the model recognizes that family characteristics also affect these outcomes directly and often significantly. Using this model, we first examined whether there were any differences between the inputs to EDUCO schools versus the inputs to traditional rural schools. We compared the mix of inputs at both the school level and the classroom level, looking at both human and physical resources. Next we examined whether there were any differences between EDUCO and traditional rural schools in terms of the management of the schools and the parents' role in the schools, both through the formal channel of the ACE or the Parents' Association and informally through parents' direct involvement with the school. Finally, we explored the outcomes presented in the box on the right in Figure 1. 6 Figure 1: Evaluation Model Management Results: * Academic ACE School Inuts achievement PA ScholInut *Repetition *Attendance Parents 1 ........ S*Dropout rate For this study, we measured impact in terms of academic achievement, repetition rates, absenteeism, and dropout rates. Each of these factors can be influenced by school inputs and by the home conditions of the families of the students. The data analyzed in this study consisted of: (i) language and math achievement tests in the third grade; (ii) school data, and (iii) data from the households Finally, the results were processed and analyzed using SPSS (Statistical Program for the Social Sciences), a statistical software that allows for data analysis. The Methodology used to Evaluate EDUCO The methodology we used for this evaluation was an ex-post matched comparison, which makes comparisons between a population that is covered by the program in question and another population that is not served by it (the comparison group). The idei behind the methodology is that the comparison group, which in this case consisted of the traditional rural schools, is used to establish what would have occurred had the EDUCO program not been implemented. The associations examined in this study are based on comparisons between the two types of schools and students without controlling for other factors. The follow-up econometric study presently being carried out will control for differences in household characteristics. Additionally, because the outcomes observed are 7 related to choices made by government officials concerning program placement and by parents concerning attendance, the econometric study will address the selection issue by examining the criteria used by the Government of El Salvador to decide where to locate EDUCO schools and the way in which families decide whether to send their children to an EDUCO or a traditional school. s HL The Main Results of the Evaluation This section contains a description of the results of the evaluation in terms of school inputs, the management of schools by parents' groups, the relation of individual parents with schools, and the relationship between the parents of students and their children. School Inputs The conditions that prevail in schools are thought to have an important influence on the achievement of the students and are of particular interest to the Ministry of Education. For this reason, we explored some of these conditions, including the availability of basic services, the condition of the school's physical infrastructure, the availability of various educational resources, and the characteristics of the teachers. Figure 2: Physical Condition of the School, 45%- 44% According to the Director 40%-- Deficient 36% 35%-- C- Very Good or Excellent 30%-- 25% 26% 25%-- 20% - - 15%- 10%- 5%-- Pure EDUCO Traditional Rural Among the directors or head teachers of the EDUCO schools, 44 percent thought that the physical infrastructure of their schools was deficient. In contrast, 35 percent of the directors of traditional rural schools thought that the physical condition of their schools was very good or 5 For additional information see the World Bank DRGPH Working Paper by Jimenez and Sawada, "Do Community-Managed Schools Work? An Evaluation of El Salvador's EDUCO Program?", processed 1997 8 excellent. The remainder answered that their schools were in 'good" condition. These differences in opinion are not statistically significant.' Figure 3: Actual Physical Characteristics of the School 100% 91% 93% 90%- 781% 80% 70% 60% -. h*ine 50% --1 . tricity . 38% OPped Water 40%34% 30%- 20% - - 13% 10% 0%/ Pure EDUCO Traditional Rural The data corroborated this finding by showing that the EDUCO schools have less basic services than their traditional rural counterparts. The water available to 84 percent of the EDUCO schools is not potable as, according to the directors and head teachers, it comes from unhealthy, often polluted sources such as rivers and wells. These sources are known to have caused a series of gastrointestinal illnesses in the population. The majority of EDUCO schools also lack electrical energy (70 percent), which severely limits the possibility of using illumination and audiovisual resources in the classroom. While EDUCO schools are consistently worse-off than traditional schools in a number of different measures, the only area in which pure EDUCO schools are significantly worse off statistically speaking, is in terms of their access to electricity. 6 All of the significance tests applied to the results presented in this paper were conducted at the 5 percent level of significance. 9 Figure 4: Percentage of Classrooms Where Every Student Has His or Her Own Textbook 100%- 90%-- 82% aMath 80%-- 78% Language 69% 70%-- 64% 60% 50% 40%- 30%- 20%- 10%- 0% Pure EDUCO Traditional Rural Textbooks are the main source of information on curriculum subjects for both for teachers and students and are often the only source of information in the school other than the teachers themselves, particularly in low-income rural areas. The EDUCO program has emphasized the need to provide every EDUCO student with his or her own textbook, and Figure 4 reveals that, according to teachers, more third-grade sections in pure EDUCO schools have math and language textbooks for all of their students than do third-grade sections in traditional rural schools. These differences were compared at the classroom level using the selected third-grade sections as the unit of analysis and are not statistically significant. We also examined the existence of classroom libraries and found that 77 percent of pure EDUCO third-grade classrooms have access to this resource compared to only 45 percent of traditional rural third-grade classrooms. These differences are statistically significant at the 5 percent level. 10 Figure 5: Teachers' Education 100% 90% 80% 38% 70% 60% 74% 50% -- University 40% 45% 0 Technical School 30 -WPrinary or Seconday 20% 16% 116% 10% I17 Pure Traditional EDUCO Rural In terms of teachers' characteristics, we found that 74 percent of EDUCO teachers have university training whereas the most common academic training among traditional school teachers is technical secondary education, which is offered as type of vocational training for primary school teachers. This diference is statistically significant, although the effect that these different kinds of training have on teachers' performance and students' learning needs to be more fully investigated. The teachers with university training have had more overall years of schooling, but the technical school graduates' training in pedagogical techniques, particularly at the primary level, may have prepared them better for the specific job of teaching in the classroom. Figure 6: Years of Teaching Experience by Type of School 9Private D Traditional 7- 6EDUCO 5 4-- 0.8 7.6 3 0.7 2- 12.7 0 0.2 Pure EDUCO Traditional Rural 11 Third-grade teachers in EDUCO schools have significantly fewer years of teaching experience than their traditional rural school counterparts, as illustrated in Figure 6. Figure 6 also illustrates that while there are quite a few EDUCO teachers with traditional public school and private school experience, very few traditional public school teachers have taught in other types of schools. We also found that the EDUCO teachers are younger on average; the mean age of EDUCO third-grade teachers is 27, while the mean age of rural traditional school third-grade teachers is 34. This brief examination of the school and classroom characteristics of EDUCO schools and traditional rural schools led us to conclude that pure EDUCO schools are in worse physical condition but that, at the classroom level, pure EDUCO schools have access to more resources such as textbooks, libraries, and work zones (areas set up to promote collaborative work among students). Whereas EDUCO teachers have higher levels of formal education, the impact that this difference has in the classroom is unclear and requires further examination. Parents' Role in Management: ACEs and Parents' Associations We went on to examine differences in decisionmaking between the two types of schools. First, we looked at the locus of decisionmaking to see whether parents' responsibility for key areas of decisionmaking in EDUCO schools means that more decisions are actually made directly by the school, as opposed to by the Ministry of Education, than in their traditional school counterparts. Second, we looked at the role of the ACE or Parents' Association in the school, particularly with respect to their contact with teachers and their classrooms. Figure 7: Number of Decisions Made by the School (of 24) 14-- 11.8 11.8 12-- 8 _ 6-- 4- 2_ 0I Pure EDUCO Traditional Rural 12 The school survey included a series of questions about 24 key decisions in the areas of pedagogy, school administration, personnel, and incentives. First, members of the parents' committees were asked which entity within the education system was the principal decisionmaker for each of the 24 decisions -- the central office of the Ministry of Education, the Ministry's departmental office, or the school itself. The results presented in Figure 7 reveal that the school is the principal decisionmaker both in the traditional rural schools and in the EDUCO schools and that there is no significant difference in the mean number of decisions taken at the school level across both types of schools. This finding is surprising given that EDUCO schools are legally vested with greater authority than their traditional public school counterparts and may point to a need for further training on the ACE's roles and responsibilities. Figure 8: Perception by Members of Parents' Committees of their Personal Influence 2- in School Decisionmaking 1.5 -1.4 0.7 0.5 0 I Pure EDUCO Traditional Rural Second, the parents' committee members were asked about the degree of personal influence they perceived themselves as exercising over each of these 24 decisions on a scale of zero to three from 'ho influence" to 'I great deal of influence". As Figure 8 illustrates, parents who are ACE members in EDUCO schools, on average, feel that they have significantly more personal influence over decisions than do parents who are members of Parents' Associations in traditional schools. These findings suggest that, despite the fact that there is no difference between the two types of schools in the mean number of decisions made at the school level, parents in management positions in EDUCO schools perceive themselves as having more influence over these decisions than do their counterparts on Parents' Associations in traditional rural schools. 13 Figure 9: Presence of Parent 6 -5.7 Committees in the Classroom 5 0 Number of classroom visits 4 per month 3 2-- 1.6 1 0 I_ _I Pure EDUCO Traditional Rural Figures 9 and 10 reveal that ACEs are much more involved than Parents' Associations in the day-to-day activities of the classroom. According to teachers -- a more objective source than the parents' committee members themselves -- ACE representatives are both more present in the classroom and collaborate more with teachers and students across a whole range of activities carried out in the classroom. The difference between the presence of ACE members and Parents' Association members in the classroom is statistically significant and suggests that EDUCO has been successful in mobilizing parents' participation in the school beyond their legislated role in the ACE into the realm of the classroom. Parents' Direct Relationship with the School We then turned to the relationship that individual parents have with the school, regardless of whether they are also members of a parents' committee. Although this relationship is not part of the formal mechanism of the ACE or the Parents' Association, these relationships are critically important in the EDUCO model because this model is premised on the direct involvement of parents in the education of their children. To the extent that 'fegular"non-ACE member parents are more involved in their children's education, the more we can assume that EDUCO has been successful in achieving a true integration between the school and the community. 14 Figure 10: Parent Committees' Involvement in Schools 100%- 90% -- N Sending students to school 80%-- 68% U Maintainig furture 70%-- S0% 0 Controlling attendance 50% 45% 40%-- 20% 20% 14% 9%0 10% -- Pure EDUCO Traditional Rural Figure 11. Teachers' Time Devoted to Meeting with Parents 6-- 4.9 O Hours per week 4 2.9 3 --________ 2-- 1 -. O I 0 Pure EDUCO Traditional Rural The relationship that individual parents have with their children's school can best be characterized by the amount of contact that they have with their child's teacher to discuss issues such as discipline, academic performance, and attendance. As illustrated in Figure 11, teachers in EDUCO schools spend significantly more time meeting with parents to address these types of issues than do teachers in traditional rural schools. In order to get a further measure of teachers' accountability and sense of responsibility, we asked teachers what measures they take when a student is absent for more than two weeks. As 15 illustrated in Figure 12, 73 percent of EDUCO teachers visit the student's home if he or she has had a prolonged absence from school. In contrast, only 40 percent of traditional rural school teachers choose to do this, preferring instead to ask the parents to come to a meeting at the school. The difference in the tendency to summon the family is statistically significant, whereas the difference in the tendency to visit the family is not. Figure 12: Teachers' Actions when the Student is Absent 100% - Visits the Family 0 Summons the Family 80%-- 73% 770 70%-- 60% 50% 46% 41% 40% 30% 20% 10%- 0% Pure EDUCO Traditional Rural The fact that EDUCO teachers spend more time meeting with parents and are more likely to visit a student's home in the face of his or her prolonged absence indicates that EDUCO teachers have a higher degree of contact with their students' families than do their counterparts in traditional rural schools. This may lead to more appropriate or sensitive approaches to the students' learning processes given that the teachers are more familiar with their students' home environments. Parents' Influence on their Children's Academic Performance Much to the consternation of policymakers in the education field, research on the determinants of students' academic performance has shown that the characteristics of a student's home are often more important than the characteristics of a student's school. This tendency and a desire to understand more fully the characteristics of EDUCO students households prompted us to look at the socioeconomic characteristics of the households of EDUCO students and compared these to those of traditional rural school students. 16 Figure 13: Physical Characteristics of Students' Homes 100%- 90% - W Electricity 800h0 -a Contncted floor 70%- W Gas/electric stove 66% 60% -0 Piped water 48% 50%- 40%-- 35% 29% 29% 30% 25% 202 20%- - 11% 10% Pure EDUCO Traditional Rural The homes of EDUCO students are less likely to have basic services such as running water and electricity than are the homes of traditional school students in rural areas. They are also built with lower quality materials. This is illustrated by the 'tonstructed floor" variable, which indicates the percentage of homes with a floor constructed of a material other than dirt. The indicators illustrated in Figure 13 underscore that the EDUCO households are poorer and more prone to disease, notably diseases related to poor hygiene, which is common in houses with dirt floors and no direct access to water. In each of these four categorieg, the homes of EDUCO's students are significantly worse-off compared to those of traditional students. 17 Figure 14: Family Composition 70%-- 66% 66% 60% 50% 43% 43% 40%-- U Both Parents 30% . OOne or Neither 20%- 10% I% - -H Pure EDUCO Traditional Rural Bearing in mind the difficulties that are associated with households that are headed by single parents, we also looked at the composition of students' households. Although there is no significant difference between EDUCO households and traditional rural school households regarding the presence of one of both parents, 44 percent of students in both types of schools live either with only one or with neither of their parents, which is an important effect of El Salvador's civil war and the country's high rates of migration (Figure 14). Figure 15. Mothers' Education 100% 90% 80% 70% 6 Pure EDUCO 60%- 50% 47% 0 Traditional Rural 40% 30% 32% 30% 20%1% 10% __ 1% 4% 0%I I Grade Grades Grades Grade 3 or 4-6 7-9 10 or less more 18 Figure 15 presents data on mothers' education, given that this factor has been shown to be an important determinant of students' academic performance and given the cultural traditions in rural El Salvador where mothers spend more time than fathers do with their children. Figure 15 reveals that the majority of the mothers of EDUCO students have little or no formal education and less than the mothers of traditional rural school students. Figure 16: Parents' Attention to their Children's Studies 100%- 900%- 83% 80%/ 80%- 70%-- 60%-- U Father helps with homework 50%-- 0 Mother helps with homework 40% 290 35% 0 Parents review notebooks 30% 5% 25% 21% 10% ..... 20% - Pure EDUCO Traditional Rural Parents make critical contributions to their children's education both by participating in the school's affairs and by helping with their children's schoolwork at home. For this reason, we examined the extent to which parents help iNith their children's homework and found that mothers are more involved than fathers. This finding can be attributed to the fact that fathers are more likely to work outside the home in El Salvador, while mothers tend to, stay at home with their children. Second, while just under 50 percent of all parents in the sample helped their children with their homework, we observed no differences in this variable between the parents of children attending EDUCO schools and the parents of those attending traditional rural schools, despite the fact that EDUCO parents have less formal education. We concluded that, despite the fact that EDUCO households are worse-off than traditional school households, EDUCO parents are more involved with their children's teachers than are the parents of children attending traditional rural schools (which makes EDUCO teachers more accountable) and are equally involved in assisting their children with their homework. 19 Educational Outcomes In order to evaluate the impact of EDUCO, we developed a series of indicators of educational efficiency and effectiveness, including daily attendance, dropout rates, repetition rates, and academic achievement. These impact indicators were studied in relation to school inputs, the conditions of the students' households, and parents' participation in school affairs. Figure 17: Educational Statistics from Grade 3 Classrooms 18%-- 16% 16% W Daily Absenteeism 16%- 14%-- 1 Dropout during the year 12% 12% 0 Repetition 10%- 8% -% 7% 7% 6% -- 4% 2% 0% 1 Pure EDUCO Traditional Rural With respect to educational efficiency, we found that EDUCO schools have higher dropout rates, whereas repetition rates and daily absenteeism are virtually identical in both types of schools. Dropout rates were defined as the number of students who had abandoned school during the course of the academic year as a percentage of the total who matriculated. Thus, we did not factor in any students who may have dropped out between one academic year and the next. There are no significant differences between pure EDUCO and traditional rural schools in any of these three variables. Assessing these results in light of the socioeconomic condition of the students' households, we found that EDUCO students are more likely than their counterparts to become ill, to perform household chores, and to work outside of the home. Our hypothesis is that these factors are related not only to the higher dropout rates among EDUCO students, but to rural students' high overall absenteeism and dropout rates, regardless of the type of school they attend. These statistics call for comprehensive strategies that reach beyond a particular type of school and beyond the education sector. 20 Figure 18: Third-grade Tests Number of Objectives Achieved in Math and Language 4 - - 3.6 3.7 3.5 UPureEDUCO 3 0 Traditional Rural 2.5 2-- 1.7 1.5 1- 0.5 0 - -- Language (9 total) Math (10 total) Figure 18 presents the results of the third-grade language and mathematics achievement tests given in 1996 to the sample of students selected for this evaluation. The achievement tests evaluated nine objectives set forth in the language curriculum and ten objectives set forth in the mathematics curriculum. In terms of the percentage of objectives achieved, students in both pure EDUCO and traditional rural schools performed better in mathematics than in language. However, it is important to point out that the results of the achievement tests were low across all of the types of schools nationally. The results presented in Figure 18 reveal that there are no statistically significant differences between the mathematics and language scores of pure EDUCO students and traditional rural school students. IV. Conclusions If we were to rank the population served by the education system according to the socioeconomic status of their households and the characteristics of the schools, the result would be the situation illustrated in Figure 19, where EDUCO students can be found on the bottom rung of the staircase. 21 Figure 19: Anticipated Results Private Traditional urban Traditional rural Mixed EDUCO Pure EDUCO Parents' Socioeconomic Status School Characteristics Given the pattern in Figure 19, it could be assumed that the students' achievement test scores would follow a pattern similar to that presented by the data on students' socioeconomic status and school characteristics. However, this is not the case. As illustrated in Figure 18 there are no statistically significant differences between the performance of EDUCO students (in either pure EDUCO or mixed schools) and that of students in traditional rural schools. 22 Figure 20: Observed Results Private Traditional urban Traditional rural ParerMixed EDUCO participation Pure EDUCO Classroom conditions Academic Results The results of this evaluation may explain the reasons for this phenomenon. This study confirmed that: * The academic performance students in EDUCO schools is not significantly different from the academic performance of students in traditional rural schools. * The socioeconomic circumstances that prevail in the households of EDUCO students are inferior to those in traditional school students' households. * EDUCO schools are worse-off than traditional rural schools in terms of their infrastructure and basic services. * More teaching materials are available in third-grade classrooms in EDUCO schools than in third-grade classrooms in traditional rural schools. * The parents of EDUCO students are more involved in school activities than are their counterparts in traditional rural schools. 23 Thus, it would seem that these are the determining factors that explain why the academic results achieved by students from EDUCO schools are no lower than those of students from traditional rural schools despite the lower socioeconomic status of the EDUCO students' households and the poorer physical condition of the schools. As Figure 20 illustrates, the two factors that seem to be most influential in equalizing student achievement across types of schools are: * The fact that the parents of EDUCO students participate more in the affairs of their children's schools than do the parents of traditional rural school students, either through the ACE or directly in interactions with teachers * The fact that there are more and higher quality classroom inputs in EDUCO schools than in traditional rural schools. When the results of the fuller analysis of the themes developed in this first report are available, it is likely that the causal relationships between these two variables and academic achievement will be clearer. In the meantime, the results from this first evaluation and other recent research supports the following conclusions with regard to the original EDUCO program objectives: Objective 1 - Coverage * The EDUCO program has expanded pre-primary and primary school coverage in rural areas. According to Ministry of Education statistics, enrollment in the EDUCO pre-primary program has grown from 3,072 students in 1991 when the EDUCO program was initiated to 43,168 in 1996. Enrollment in the EDUCO primary schools (Grades I through 9) has grown from 5,344 students in 1991 to 125,760 in 1995. Furthermore, the household level socioeconomic data collected in this study confirms that the EDUCO program is succeeding in its aim of targeting poorer segments of the population. Objective 2 - Parents' Participation * ACEs in EDUCO schools are more involved than are Parents' Associations in traditional schools in helping to maintain school infrastructure and equipment, procuring materials, sending children to school, monitoring attendance, visiting the classroom, devising annual plans for the school, and fundraising. * Although EDUCO parents are poorer and less well-educated than the parents of children in traditional rural schools, they give the same amount of attention to their children's homework and coursework. 24 * Because of the direct relationship between teachers and parents in EDUCO schools, EDUCO teachers are more accountable than are teachers in traditional rural schools. * There is virtually no diference between pure EDUCO and traditional rural schools in the mean number of decisions taken at the school level. However, EDUCO parents who are ACE members feel that they have more personal influence over the decisions made at their child's schools than do parents who are members of Parents' Associations in traditional rural schools. Objective 3 -- Quality * There are no significant differences between the achievement test scores of EDUCO students and those of traditional rural school students, despite the poorer physical condition of EDUCO schools and the lower socioeconomic status of the households of EDUCO students. V. Recommendations These results lead to the following recommendations: * EDUCO is a good strategy for reaching poor communities quickly and should be supported. * Because the achievement test scores of students in both EDUCO and traditional schools are low, policymakers should focus on improving the poor academic performance of all students in the public education system. * This preliminary analysis suggests that illness and the need for students to contribute their labor and earnings to the household have a strong effect on absenteeism and on repetition and dropout rates. * The possible effect that a multigrade classroom setting is having on students' attendance, repetition, dropout and overall academic performance is an issue that could be explored with the existing data and should be carried out as part of the EDUCO evaluation, given the prevalence of multigrade classrooms in EDUCO schools. * Our hypothesis as a result of our analysis is that the fact that EDUCO classrooms have more resources than traditional rural school classrooms is positively associated with their better than expected academic performance. This hypothesis should be tested in a robust econometric analysis in order to determine whether the Ministry of Education should make certain types of classroom-level interventions a high priority. * A second hypothesis we put forward is that parents' increased involvement in their children's schools (in terms both of their participation in parent's associations and of their contact with teachers) has a positive influence on students' academic performance. This hypothesis, which 25 lies at the heart of the EDUCO experiment, needs to be thoroughly tested using the proper econometric models. 26 Bibliography Ministry of Education, EDUCO Coordination Office. 1992. A Comparative Analysis of First- grade EDUCO and Non-EDUCO Sections. (Julia Mora Mora, Consultant) San Salvador, El Salvador. Ministry of Education, EDUCO Coordination Office. 1992. An Evaluation of the EDUCO Program. (Hugo Diaz Diaz, Consultant) July 1991- September 1992. Ministry of Education, EDUCO Coordination Office. 1996. Orientations for the ACE's work. First Edition, San Salvador, El Salvador. Ministry of Education, EDUCO Coordination Office. 1994 The EDUCO Program. An Experience on the Road. (Mirian Basso, Consultant) San Salvador, El Salvador. Ministry of Education, EDUCO Coordination Office. 1993. The Systematization of the EDUCO Program. (German Marifo, Consultant) San Salvador, El Salvador. World Bank. 1994. El Salvador Community Education Strategy: Decentralized School Management. Country Department II. Human Resources Operations Division. Latin American and the Caribbean Regional Office. Washington D.C. 27