THE WORLD BANK Discussion Paper EDUCATION AND TRAINING SERIES Report No. EDT53 How Texf books AffecV/ chievement Marlaine E. Lockheed Stephen C. Vail Bruce Fuller { January 1987 Education and Training Department Operations Policy Staff The views presented here are those of the author(s), and they should not be interpreted as reflecting those of the World Bank.' Discussion Paper Education and Training Series Report No. EDT53 HOW TEXTJO00S AFFECT ACHIEVEMENT IN DEVELOPING COUNTRIES: EVIDENCE FROM THAILAND Marlaine E. Lockheed Stephen C. Vail Bruce Fuller Research Division Education and Training Department January 1987 The World Bank does not accept responsibility for the views expressed herein, which are those of the author(s) and should not be attributed to the World Bank or its affiliated organizations. The findings, interpretations, and conclusions are the results of research or analysis supported by the Bank; they do not necessarily represent official policy of the Bank. Copyright © 1987 The International Bank for Reconstruc,ion and Development/ The World Bank ABSTRACT This paper analyzes lorngitudinal data from a national sample of eighth-grade mathematics classrooms (99 teachers and 4030 students) in Thailand to elxplore the mechanisms whereby textbooks affect student achievement :gain. The results indicate that textbooks may affect achievement by (a) substituting for additional post-secondary mathematics education of teachers, and (b) deliv.ering a more comprehensive curriculum. The data showed little evidence that textbooks led to better use of classroom tim,e or increased homeiwork. The authors also conclude that policies promoting extensive post-secondary teacher education in favor of investments in essential teaching materials may be inappropriate, particularly at the lower-secondary level. How Textbooks Affect Achievement in Developing Countries: Evidence from Thailand Marlaine E. Lockheed, Stephen C. Vail and Bruce Fuller In both developed and developing countries, schools and classrooms contribute to student cognitive skills (Hannaway & Lockheed, 1986; Fuller, 1986). Higher and lower income countries differ, however, in the degree to which school resources such as textbooks affect achievement. Specifically, while education expenditures on material inputs are largely unrelated to achievement gains in industrialized countries, they have positive effects in less developed countries. In an exhaustive review of 144 studies relating expenditure parameters to student achiievement in the United States, Hanushek reports that "there appears to be no strong or systematic relationship between school expenditures and student performance" (1986). By comparison, in a review of 142 analyses conducted in 72 studies of expenditure parameters in developing countries, Fuller (1986) reports that 109 (77%) of the analyses confirm the effect of expenditures on student achievement. Although the effects of expenditures and material inputs on achievement in developing countries are well established, little is known about the mechanisms underlying the contribution. A case in point is the effect of textbooks on achievement. Vor the past decade, researchers have documented the effect of textbooks on student achievement in developing countries (Heyneman and Loxley, 1983). A review of this research notes that of 18 correlational -2- studies of textbook effects on student learning, 15 (83%) report statistically significant positive results (Heyneman, Farrell, and Sepulveda-Stuardo, 1981). Two studies with experimental assignment of students to "textbook" conditions also report significant effects of textbooks on achievement (Heyneman, Jamison & Montenegro, 1984; Jamison, Searle, Galda & Heyneman, 1981). As Altbach (1983) notes: "Nothing has ever replaced the printed word as the key element in the educational process and, as a result, textbooks are central to schooling at all levels." Research findings that student achievement is positively related to the presence of textbooks in the classroom have influenced both national educational policies and the lending programs of international organizations such as the World Bank. For example, of the 232 World Bank education projects approved between 1970-1983, 48 (21%) included support for the preparation, provision or distribution of educational materials and textbooks (Searle, 1985). An analysis of 25 primary education projects or components of projects indicated that the objectives of the majority of the textbook provision components was to decrease the ratio of pupils to textbooks (Romain, 1986). Yet little research has explored the question of precisely how textbooks affect achievement. Several suggestions have been made, however, regarding possible mechanisms most emphasizinge the textbook as a portable source of information for both teachers and students. -3- For Teachers, Textbooks may: (a) either substitute for gaps in teacher knowledge and skills (Altbach, 1983), or complement existing skills by providing more able teachers with a resource that increases their effectiveness (Beeby, 1986; Murnane and Nelson, 1984); (b) promote delivery of more complete and coherently organized curricula, particularly in situations where there is a shortage of teachers and where teacher training is limited in scope (Altbach, 1983; Sepulveda-Stuardo & Farrell, 1983); (c) enable the teacher to make better use of time spent teaching (Walberg, 1984) and (d) enable the teacher to assign higher quality homework (Featherstone, 1985; Walberg, 1985). For students, textbooks may: (a) provide a basic exposure to written material otherwise unavailable in the environment (Heyneman, Farrell & Sepulveda-Stuardo, 1981) and (b) enable students to learn independently of the teacher, particularly through completion of homework (Rohlen, 19133). In this paper, we test several of these hypotheses using longitudinal data from the Second International Mathematics Study (SIMS) conducted by the International Association for the Evaluation of Educational Achievement (IEA) in Thailand during the 1981-1982 academic yea. We first describe the data and how the basic variables were measured. Then we report on the effects of textbooks on student achievement, controlling for initial level of achievement and other student - 4 - characteristics. Next we attempt to estimate the effects on achievement gain of various classroom processes and their interaction with the use of textbooks. Finally, we summarize what we have learned. Data -Sample The sample comprised 99 mathematics teachers and their 4030 eighth-grade studentsl/ and was derived from a two-stage, stratified random sample of classrooms. The primary sampling units were the twelve national educational regions of Thailand plus Bangkok. Within each region, a random sample of lower secondary schools was selected, with replacements. At the second stage. a random sample of one class per school was selected from a list of all eighth grade mathematics classes within the school. The resulting sample represented a 1% sample of eighth grade mathematics classrooms within each region.2/ Because the sampling scheme selected students with unequal probabilities (Wattanawaha (1986) notes, for example, that the study sample underrepresents students attending school in Bangkok and overrepresents those attending rural schools), sampling weights were included with the data. The weights were inversely proportional to the probability of sample selection, and the sum of the weights was equal to the sample size. The analyses we perform here are weighted to ensure that conclusions are generalizable to the Thai population. Because ten teachers failed to complete one of the teacher questionnaires (related to students' "opportunity to learn"), all analyses using this instrument are based on a reduced sample size. -5- Procedure At both the beginning and end of the school year, students were administered a mathematics test covering five curriculum content areas (arithmetic, algebra, geometry, statistics and measurement). Students also completed a short background questionnaire at the pretest and a longer one at the posttest administration. Teachers completed several instruments at the posttest, including a background questionnaire, a general classroom process questionaire, and an "opportunity to learn" questionnaire. Teachers prov.ded information about teaching practices and characteristics of their randomly selected "target" class. Data about the school was provided by a school administrator. In the following sections, a description of each of the variables we analyze in this paper is provided; summary statistics are presented in Table 1. Mathematics Achievement The IEA developed five mathematics tests for use in SIMS. One of the tests was a forty-item instrument called the core test. The remaininig four tests were thirty-five item instruments called rotated forms and designated A through D. The five test instruments contained roughly equal proportions of items from each of the five curriculum content areas, except that the core test contained no statistics items (Wattanawaha, 1986). For purposes of this analysis we regard the instruments as parallel forms with respect to mathematics content. The IEA longitudinal design called for students to be administered both the core form and one rotated form chosen at random at pretest and posttest. In Thailand students wrere pretested using the -6- core test and one rotated form. At posttest students again took the core test and one rotated form, but were prevented from repeating the rotated form taken at pretest. Approximately equal numbers of students took each of the rotated forms in both administrations. Basic test statistics (rights, wrongs, means and standard deviations) for the core test and each rotated form are displayed in Annex A. The mean number right varied between 29 and 48 percent of the test total, indicating that these were difficult tests for these students. Moreover, the sum of mean number right and mean number wrong was very close to the total number of test items, students omitting on average only about one item per test. Many students apparently guessed on the bulk of the items. One goal of our analysis was to predict posttest achievement as a function of pretest performance plus other determinants. Since students took the core form twice, the core form posttest score reflects, to some degree, familiarity with the core test items. Instead of using the core test, therefore, we analysed scores obtained from the rotated forms, after they were equated to adjust for differences in test length and difficulty. Since every SIMS cognitive item had five choices, we first computed student math scores using a formula that corrected for guessing: S = R - W/4 where S = formula score R = number of items answered correctly W = number of items answered incorrectly, excluding omits. -7- This score estimates the number of items the student knows, but has the slight interpretive drawback of sometimes producing negative scores (Gulliksen, 1950). A formula score was computed for the core test and for each rotated from. We next equated rotated form formula scores to core form formula scores using pretest data only. We used location-scale equating to derive linear equating coefficients for each rotated form (see Gulliksen, 1950 for a thorough explanation of this procedure). Equated rotated form scores Ei were obtained from formula scores Xi by the linear adjustment Ei= aXi + b where a sy sx b = Y -aX X - mean rotated formula score Y = mean core formula score sx= standard deviation of rotated formula score sy = standard deviation of core formula score. i = rotated forms 1-4 The pretest means and variances of the equated rotated scores are equal to the core pretest means and variances and are presented in Table 2. We applied the equating coefficients to pre- and posttest rotated form formula scores to create equated rotated form formula scores, which are the measures of mathematics achievement used in all analyses reported in this paper. Student Background Basic information about each student included his or her sex, age, number of older siblings, paternal and maternal education, paternal -8- and maternal occupational status, and home language. Parental occupation was classified into four international categories: (a) unskilled or semi-skilled worker, (b) skilled worker, (c) clerical or sales worker, and (d) professional or managerial worker. Because paternal and maternal occupational. status were highly correlated (r = .39), we analyzed the effects of paternal occupational status only in this paper. Highest parental educaxtion was also classified into four categories: (a) very little or no schooling, (b) primary school, (c) secondary school, and (d) college, university or some form of tertiary education. Because paternal and maternal educational attainment were also hkighly correlated (r = .58), we analyzed the effects of maternal educational attainment only. Fifty-two percent of the students were male; the average age of the sample was 1711 months (14.25 years). On average, they had 2.7 older siblings; 22% were first borns. 31% of the fathers and 16% of the mothers had attended school beyond the primary level, 30% of the mothers had no occupation other than housewife, and 41% of the fathers' occupations were classified as clerical, sales, professional or managerial. 49% of the parents spoke the language of instruction at home. Textbook Use Teachers reported how often they used published textbooks in their instruction of the target class. Forced-choice options were: rarely or never, sometimes, and often. Sixty-two percent of the teachers reported using textbooks "often;" 29 percent reported using textbooks "sometimes;" and eight percent reported using textbooks "rarely or never." In this analysis, we created a dummy variable for textbook use, in which "often" was coded as l and "rarely or never" and "sometimes" were coded as 0. -9- The IEA SIMS data do not include any indicators of textbook availability. Teacher Education Teachers indicated the number of semesters in mathematics that rere included in their post-secondary education. On average, teachers reported receiving 5 semesters of post-secondary mathematics. We note, however, that 21 teachers failed to answer this question, and that analyses employing this variable are consequently based on a reduced sample size.3/ Opportunity to Learn (OTL) The teachers completed questionnaires designed to measure their students' opportunity to learn the tested mathematics curriculum. For each item of the core and rotated forms of the mathematics achievement test, teachers indicated whether, during the current school year, they had taught or reviewed the mathematics needed to answer the item correctly. If they had not taught the material, they were asked to indicate whether: (a) it had been taught prior to the current school year, (b) it would be taught later, (c) it was not part of the school curriculum, or (d) some other reason. (Refer to Annex B for OTL item text). The complete teacher OTL questionnaire is long and repetitive, likely to fatigue teachers, and may produce invalid responses toward the end of the instrument. To minimize incorporating response set bias produced by fatigue into our OTL measure, we constructed an OTL measure based on the first 40 core items only. This measure consisted of the number of times the teacher answered "yes" to the question; "During this school year did you teach or review the mathematics needed to answer the - 10 - item correcLly?" A more complicated OTL measure, similar to that reported by Smith (1986) was constructed, but had negligable predictive value for these data. Mathematics Time Teachers indicated the number of hours of mathematics instruction the target class would have received by the end of the school year. On average, students received 106 hours of mathematics instruction per year. Assigned Homework Teachers indicated the number of hours per week they thought would be needed by a typical student in the target class to complete the assigned homework outside of class. The measure we used was the teacher report of the number of hours assigned for the previous week. On average, 4.3 hours of homework were assigned weekly. Student Homework Students indicated the number of hours of homework for mathematics, outside of formal class time, they did each week. On average, they reported doing 3.4 hours of homework in mathematics weekly. Results Our basic hypothesis was that students of teachers who used textbooks regularly ("often") would learn more over the course of a school year than would students of teachers who did not use textbooks regularly. Second, we hypothesized that the use of textbooks would interact with other elements of the classroom, so as to either substitute for or complement In order to test our hypotheses, we conducted our analyses in three stages. First, we examined the effect of textbooks on achievement, holding constant prior achievement and student background characteristics. Second, we estimated the effects on student achievement of opportunity to learn, teacher education, time spent teaching, assignment of homework, and student time with homework, holding constant prior achievement, student background characteristics, and teacher textbook use. Third, we estimated the effects on student achievement of providing textbooks to those teachers who reported not using them. All analyses were conducted with the student as the unit of analysis. Effects of Textbooks on Achievement Using ordinary least squares (OLS) with listwise deletion of missing data, we regressed post-test mathematics achievement score on: (a) pretest score, (b) pretest score and student background characteristics, and (c) pretest score, student background characteristics, and textbook use by teacher. The results of these regressions are presented in Table 3. As expected, posttest score was largely determined by pretest score, which explained 48% of the variance in posttest score. Family background variables contributed little to posttest score, after the effects of pretest score were held constant. Pretest score, of course, included the effects of such exogenous variables as family background and innate ability, as well as prior schooling effects. Specifically, sex, birth order and home use of instructional language were unrelated to posttest score, holding constant pretest and considered simultaneously with other student characteristics. However, - 12 - older students gained less from pretest to posttest--with each year of age subtracting about .84 of a point on the posttest---while maternal education was positively related to posttest score, with eachi level of education worth about .31 points on the posttest. Students of teachers who used textbooks scored about .52 points higher on the posttest, holding constant pretest score and student demographic characteristics. While the size of this effect amounts to only about 5% of a standard deviation on the posttest, it also is equivalent to one-sixth of the average gain for the entire school year (3.16 points; S.D. = 6.77), or--expressed differently--the equivalent of 1.61 more months of school. Thus, our findings confirm previous research on the effect of textbooks on achievement. Our question was not whether or how much textbooks affect achievement, however, but rather what accounts for their effect. For this, we turn to our analysis of selected teacher characteristics and practices and their interaction with textbooks. Textbooks and Teacher Education One mechanism whereby textbooks could affect student learning is by either substituting for or complementing teacher education. That is, the effect of teacher education could be significant in classes lacking textbooks and insignificant in classes with textbooks (substitution), or the effect of teacher education could be greater in classes with textbooks than in classes lacking textbooks (complementarity). To explore either hypothesis entailed examining the interaction between textbooks and teacher education. - 13 - We first examined the combined effects of teacher education and textbooks on student achievement, controlling for student pretest and background variables (Table 4, column 1). As expected, the number of semesters of postsecondary mathematics courses completed by the teacher was positively and statistically significantly related to student posttest score. However, the size of the effect was small, with each semester worth only about .05 of a point on the posttest. We next examined the interaction between textbooks and teacher education (Table 4, column 2) and found it to be quite significant, affecting both the size and significance of the coefficients for textbooks and for teacher education. Because of this significant interaction, we estimated the effect of teacher education on achievement of students in textbook and non-textbook classes separately e4/ The results of these regressions (Table 4, columns 3 and 4) provide evidence that textbooks substitute for teacher education. In classes in which the teacher used textbooks often, the effect of teacher education on student posttest achievement was not statistically significan-t and was quantitatively negligible. In classes lacking textbooks, however, teacher education was significantly related&to student posttest achievement. We estimated the separate effect of teacher education and textbooks from the coefficients provided in Table 4, column 2. Textbooks contributed .54 of a point to the student mathematics postte;t score, and each semester of teacher education contributed .01 of a point.5/ Comparing the effects of textbooks to the effects of teacher education, it is noteworthy that the regular use of textbooks in - 14 - mathematics was equivalent to about fifty semesters of teacher postsecondary mathematics, other things equal. It is also possible to compute an effectiveness-cost ratio for both teacher education and textbooks. If each semester of postsecondary mathematics training in Thailand costs 422 baht in public tertiary institutions or 525 baht in teacher training schools and teachers teach for about seven years on average, and if textbooks cost 23 baht and last one year on average, the effectiveness-cost ratio is 9.4 posttest score points for 100 baht spent on textbooks, 2.8 points for 100 baht spent ori an additional semester of postsecondary mathematics in a public university, and 2.3 points per 100 baht spent on postsecondary mathematics in a teacher training institution. It appears that textbooks are three to four times as cost effective as postsecondary teacher education. (Annex C presents the assumptions underlying these estimates; the cost estimates for teacher training are based on average yearly costs unadjusted for real inflation.) Textbooks and Opportunity to Learn A second obvious mechanism whereby textbooks affect achievement is their capacity to deliver consistent, comprehensive and logically sequenced curricula. Because of this, students whose teachers use textbooks should have greater opportunity to learn the material than students of teachers who do not use textbooks. To test this hypothesis, we entered a measure of opportunity to learn (OTL) into our regression; the results are presented in Table 5, column 1. The inclusion of opportunity to learn in the regression reduced the size and significance of the coefficient for textbooks, suggesting that - 15 - one explanation for the effectiveness of textbooks is, in fact, that they are carriers of the curriculum. Although the size of the coefficient for opportunity to learn was small, it indicatea that for every item taught by the teacher, the student gained .03 of an item. Students of teachers who taught material relevant to the successful completion of all 40 core items, therefore, scored 1.2 point higher than students of teachers who did not cover the material; this is one-third of the average score gain for the entire year. A test for textbook by opportunity to learn interaction effects indicated that there was no such interaction. Textbooks and Instructional Time Another hypothesis we explored was the relationship between textbook use and instructional time. We speculated that time use would be more effective in classrooms in which the teacher used textbooks, and tested this by including in our regression a measure of the number of hours of mathematics instruction provided during the year. The results are presented in Table 5, column 2. Here, instructional time had no effect on student achievement, and did not substantially change the size of significa.ce of the coefficient for textbook use. The interaction between textbooks and instructional time had no appreciable effect on achievement either, leading us to conclude that textbooks and instructional time neither complemented nor substituted for each other. Textbooks and Assigned Homework Most research on the effects of homework has examined the effects of student self-reported homework, rather thana assigned homework (Murnane, 1984). Since assigned homework is a policy viariable, while self-reported - 16. - homework is better considered an indicator of student effort, we examined first the effects of assigned homework. We speculated that textbooks might encourage teachers to assign more homework, which could contribute to greater student achievement. Again, we tested this hypothesis by entering the number of hours of assigned homework in the regression; this time, the effects were statistically significant, but in an unexpected direction (Table 5, column 3). Teachers who assigned more homework had students whose posttest achievement was lower than that of students whose teachers assigned less homework, possibly because teachers of low-achieving classes assigned more homework. Each hour of assigned homework was worth -.04 points on the posttest, holding constant pretest. Including assigned homework in the equation did not change the effect of textbooks on achievement. The interaction between textbooks and assigned homework was not significant, although its inclusion in the equation changed the significance of the coefficient for textbooks. Textbooks and Student Homework As expected, the weekly number of hours of homework a student reported completing was positively associated with posttest score, other things held constant (Table 5, column 4). Completing one hour of math homework daily, or five hours per week, had roughly the same quantitative effect (.45 of a point on the posttest) as having a teacher who used textbooks (.53 of a point on the posttest); there was no textbook by student homework interaction effect0 Thus, students who had teachers who used textbooks were not more likely than other students to complete more homework during the week. - 17 - The Effect of Adding Textbooks to Classes Lacking Them We next estimated the effect on posttest score of adding textbooks to classrooms lacking them; this estimate was calculated from separate regressions for textbook and non-textbook classes. Parameter estimates for family background, pretest score, teacher characteristics and practices obtained from regressions for textbook classes were applied to observed mean values of those variables in non-textbook classes. While this technique has been criticized when used in reference to significantly different subsamples, such as for estimating the effects of private schools on student achievement (Coleman, Hoffer & Kilgore, 1982; Noell, 1982), it may be appropriate when groups differ only modestly. However, since we were aware of this possible criticism, we conducted an analysis to test for differences. To estimate the degree to which students in textbook-using classes differed from students in non-textbook classes, we made multivariate comparisons of the two groups on the same predictor variables used in the regressions.6/ We first tested the similarity of the covariance matrices for the two groups using a likelihood ratio test. Under the hypothesis of no difference, this statistic has a chi-square distribution on 78 degrees of freedom. The value we obtained was 2479.9 which is significant at the .0001 level. We next computed the Mahalanobis distance D2 between the vectors of means for the two groups D2= ( x - y ) S'1 ( x - y) - 18 - using for S the pooled estimate of the covariance matrix. We obtained the value D2 = 0.8349, whose square root is about .9 'standard units' of multivariate distance. The distance D2 is related to Hotelling's T2, which may in turn be compared with critical values from an P distribution (see K.V. Mardia, et al., 1979, pages 76-77). Our value was significant at the .01 level (or less), indicating a significant difference in means. We would like to predict the posttest scores non-textbook students would achieve were we to give them textbooks. From the foregoing discussion it is apparent that textbook and non-textbook students were somewhat, but not radically, different with respect to the va'riables used in the regressions. This calls into question the validity of such a prediction using parameter estimates from the textbook group. However, inspection of the means for the two groups (Annex D) revealed very small differences overall, suggesting that the statistically significant results described above are largely attributable to the large sample sizes involved. In Table 6, we report the actual and estimated posttest score for the non-textbook group; the estimated score is computed from the parameter estimates and mean values reported in Annex,D. Overall, adding textbooks to non-textbook classes would increase the posttest score from 11.78 to 12.55, nearly one-third of the average gain for the year. - 19 - Summary and Discussion This paper has confirmed that textbooks contribute to student learning in developing countries, and has identified two important mechanisms whereby this contribution may be made: (a) by substituting for postsecondary teacher education and (b) by delivering a more comprehensive curriculum. We found little evidence that textbooks enabled teachers to make better use of classroom time, however, or that they encouraged the assignment or completion of homework. The finding that textbook use by teachers is related to more comprehensive delivery of the curriculum is consistent with previous research. For example, Sepulveda-Stuardo and Farrell (1983) note that teachers who used textbooks in Chile were somewhat more likely to be content-oriented (as opposed to student-oriented) in their teaching style. Moreover, teacher textbook use may increase the efficiency with which the teachers use classroom time, since--as we have shown--greater content coverage is related to higher levels of achievement. The finding that cextbooks substitute for postsecondary teacher education suggests that their use in developing countries can help alleviate financial burdens associated with the provision of teachers educated beyond the secondary level. We hasten to note, however, that this appears to be an unusually well-educated teacher sample, with the average teacher having studied postsecondary mathematics for at least 3.5 semesters. It is precisely for this reason that textbooks can substitute - 20 for teacher education. Thus, our findings have few implications for situations in which teachers are educated well below secondary school completion. Although teacher qualifications, experience, and education are positively related to student achievement (Husen, Saha and Noonan, 1978), the effects of post-secondary teacher education are more pronounced on secondary student achievement than on primary student achievement (Fuller, 1985). In many developing countries, however, certification requirements for primary and lower secondary teachers include post-secondary education. In Thailand, for example, teachers of lower secondary school are required to complete a two-year, post-secondary teacher education course, during which they receive general education and pedagogy courses. Investing in teacher education, however, is costly anid decisions to do so should be examined carefully. The findings of this paper suggest that educational policies favoring post-secondary education for teachers of lower secondary schools may not be appropriate under conditions in which essential, teaching materials are lacking. - 21 - Author Notes We are indebted to the IEA and to Pote Sapianchai of the National Education Commission in Thailand and Samrerng Boonruangrutana of Srinakharinwirot University for conducting the Second International Mathematics Study and making this paper possible. We acknowledge the useful suggestions of Joseph Farrell, Stephen Heyneman, Emman:-el Jimenez, T. Neville Postlethwaite and three anonymous reviewers on an earlier version of this paper and the assistance of Judy Ruzicka, Ken Travers and Larry Suter, in making the IEA SIMS data available. The World Bank does not accept responsibility for the views expressed herein, which are those of the authors and should not be attribtLted to the World Bank or to its affiliated organizations. The findings, interpretations, and conclusions are the results of research or analys.is supported by the Bank; they do not necessarily represent official policy of the Bank. The designations employed, the presentationl of material, and any maps used in this document are solely for the convenience of the reader and do not imply the expression of any opinion whatsoever on the part of the World Bank or its affiliates concerning the legal status of aniy country. Footnotes 1. According to Wattanawaha (1986), 32.94% of the 14-year-old age cohort were enrolled in grade eight. 2. The structure of Thailand's reformed education system includes six primary school grades, three lower secondary school grades, three upper secondary school grades, plus tertiary education. In 1981, the total secondary school enrollment was 1,860,615 students, of -whom approximately 23% (427,941) were enrolled in grade eight. The SIMS sample in Thailand was 4030 students. 3. Seventy-eight teachers responded to a question regarding their post-secondary mathematics course participation; on average, they reported having taken 4.92 semesters of post-secondary mathematics. (This mean includes one teacher who reported having taken 45 semesters of mathematics; omitting this teacher lowered the mean to 4.40 semesters.) Recoding the 21 omits as zero lowered the mean to 3.88 semesters, while excluding the outlier yielded a mean of 3.46 semesters of post-secondary mathematics. The average teacher education reported in Table 1 is a weighted mean based on 78 teachers. 4. The variable TEDMATH measures the number of semester mathematics courses a teacher has had. The distribution of responses to this variable was skewed highly positive. A large number of teachers (20%) did not answer the question, even though items adjacent to it on the questionnaire received high response rates. On the assumption that the missing values might really be zeros, we recoded them to zeros. Then we took the natural log of (TEDMATH + 1/3) to get a more symmetric distribution of teacher mathematics education and reran our regression analyses. The regressions using the log variable showed approximately the same relationships as those using the original TEDMATH variable, although they were not quite as strong. 5. The effect of textbooks = (1.30 - .15(5.04)) = .54, and the effect of teacher education = (.11 - .15(.64)) = .01. 6. These analyses were performed using DISCRIK, the multivariate discrimination procedure of SAS. Since DISCR:IM does not allow for weighted analyses, these results are of unweighted analyses. The variables were pretest score, student and parent demographics, and teacher characteristics and practices. References Altbach, P. G. (1983). Key issues of textbook provision in the Third World. Prospects, 13 (3), Beeby, C.E. (1986). The stages of growth in educational systems. In S.P. Heyneman and D.S. White (Edo.) The quality of education and economic development. Washington, DC: The World Bank. Coleman, J.S., Hoffer, T. & Kilgore, S. (1982). Cognitive outcomes in public and private schools. Sociology of Education, 55 (2/3), 65-76. Featherstone, H. (1985, February). Homework. The Harvard Education Letter. Fuller, B. (1985). Raising school quality in developing countries: what investments boost learning? (Discussion Paper EDT 7). Washington, DC: World Bank, Education and Training Department. Gulliksen, H. (1950). Theory of Mental Tests. New York, NY: John Wiley & Sons. Hannaway, J. & Lockheed, M.E. (1986). The contribution of social science to education policy and practice, 1965-1985. Berkeley, CA: McCutchan. Hanushek, E.A. (1986). The economics of schooling: production and efficiency in public schools. Journal of Economic Literature. Heyneman, S.P., Farrell, J. P. & Sepulveda-Stuardo, M.A. (1981). Textbooks and achievement in developing countries: what we know. Journal of Curriculum Studies, 13 (3), 227-246. Heyneman, S.P., Jamison, D.T., & Montenegro, X. (1984). Textbooks in the Philippines: Evaluation of the pedagogical impact of a nationwide investment. Educational Evaluation and Policy Analysis, 6 (2), 139-150. Husen, T., Saha, L.J. & Noonan, R. (1978). Teacher training and student achievement in less developed countries. (Staff Working Paper No. 310.) Washington, D.C.: The World Bank. Jamison, D.T., 'Searle, B., Galda, K, & Heyneman, S.P. (1981). Improving elementaly mathematics education in Nicaragua: An experimental study of the impact of textbooks and radio on achievement, Journal of Educational Psychology, 73 (4), 556-567. Mardia, K.V, Kent, J.T. and Bibby, J.M. (1979). Multivariate Analysis, New York, NY: Academic Press. Murnane, R.J. (1984). A review essay-- comparison. of public and private schools: lessons from the uproar. Journal of Auman Resources, 14 (2), 263-277. Murnane, R.J. & Nelson, R.R. (1984). Production and innovation when techniques are tacit. Journal of Economic Behavior Organization, , 369-377. Noell, J. (1982). Public and Catholic schools: A reanalysis of "Public and private schools." Sociology of Education, 55 (2/3), 123-132. Searle, B. (1985). General operational review of textbooks (Discussion Paper EDT 1). Washington, DC: World Bank, Education and Training Department Sepulveda-Stuardo, M., and Farrell, J.P. (1983). The use of textbooks by teachers and students in learning and teaching. In E. Schiefelbein, J.P. Farrell and M. Sepulveda-Stuardo, The' influence of school resources in Chile: Their effect on educational achievement and occupational attainment (Staff Working Paper No. 530). Washington, D.C: Wor'ld Bank. Smith, M. (1986, April). Policy implications of differences in mathematics coverage and expectations for achievement. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA. Rohlen, T.P. (1983). Japan's High Schools. Berkeley, CA: University of California Press. Romain, R. (1986). Lending in primary education: Bank performance review, 1962-1983 (Discussion Paper EDT 20). Washington, DC: World Bank, Education and Training Department. Walberg, H. J. (1984). What makes schooling effective? A synthesis and a critique of three national studies. Contemporary Education: A Journal of Reviews, 1 (1), 22-24. Walberg, H. J. (1985). Homework's powerful effects on learning. Educational Leadership, 42 (7), 76-79. Wattanawaha, N. (1986). A study of equity in niˇthematics teaching and learning in lower secondary school in Thailand. Unpublished doctoral dissertation, University of Illinois, Urbana-Champaign. Table 1: Variable Names, Descriptions, Means and Standard aviatioa Variable Description N Mean S.D. XROT Pretest equated scale score 4014 8.59 7.75 YROT Post-test equated scale score 3809 11.89 9.18 XSEX Sex (female = 1, male = 2) 4013 1.53 0.50 XAGE Age in months 3"93 171.13 9.16 XELDEST Birth order (1 = firstborn, 0 = else) 3967 0.22 0.41 YFOCCI Father's occupational status 3749 2.39 0.92 YMEDUC Mother's educational attainment 3762 1.95 0.79 YHLANG Language of instruction spoken at home 3777 0.52 0.50 (1 = yes, 0 = nio) TXTBK Textbooks used often in class by teacher 3990 0.64 0.48 (1 = yes, 0 = no) TEDMATH Post-secondary semesters of mathenatics 3167 5.04 6.95 completed by teacher TYESCOR Opportunity to learn (0-40 items) 3635 29.98 7.42 THPYZAR Hours of mathematics instruction per year 3820 106.42 52.65 THWRKL Hours of homework assigned by teacher last week 3665 4.32 7.68 YMHWKL Hours of homework completed by student last week 3745 3.43 3.92 Table 2: Pretest Scores and Equating Coefficients Core Form Rotated Form Equating RGtated Formula Scores Formula Scores Coefficients Form Mean S.D. Mean S.D. a b A 8.82 7.74 5.08 5.80 1.335 2.039 B 8.35 7.61 5.56 5.61 1.355 0.825 C 8.73 7.98 7.97 6.45 1.237 -1.135 D 8.49 7.67 9.91 6.52 1.176 -3.169 Table 3: Student Achievement As Determinede By Prior Achievement, Background Characteristics and Classroom Use of Textbooks Alternativre Specifications Indep. Vbls (1)a (2)D (3)C XROT .82*** .80*** .80*** (59.04) (54.31) (54.63) XSEX .29 .23 (1.31) (1.02) XAGE -e07*** -.07*** (5.47) (5.47) XELDEST -.07 -.17 (0.27) (0.63) YFOCCI .08 .12 (0.64) (0.95) YMEDUC .31* .29 (2.07) (1.88) YHLANG .16 .05 (0.72) (0.22) TXTBK .53* (2.29) Constant 4.71 15.81 15.56 (29.01) (6.76) (6.60) r2 .48 .48 .49 Note. Dependent variable is mathematics post-test score. Numbers are parameter estimates, with t-statistics in parentheses. a/ N = 3801 b/ N = 3577 c/ N = 3547 *** p < .001 * p < .05 Table 4: Student Achievement As Determined By Prior Achievement, Background Characteristics, Textbooks and Teacher Education Students Students All Studentsa With Without Indep. Vbls (1) (2) Textbooksb TextbooksC XROT .80*** .79*** .81*** .78*** (48.12) (48.08) (35.29) (32.85) XSEX .10 .15 -.02 .46 (0.40) (0.59) (0.05) (1.21) XAGE -.08*** -.08*** -.05** - 11*** (5.23) (5.29) (2.61) (5.15) XELDEST -.24 -.26 -.43 .01 (0.79) (0.84) (1.04) (0.03) YFOCCI .12 .12 .28 -.14 (0.83) (0.83) (1.47) (0.66) YMEDUC .26 .23 .30 14 (1.47) (1.33) (1.27) (0.58) YHLANG .05 .17 -.09 .61 (0.18) (0.66) (0.27) (1.58) TXTBK .57* 1.30*** (2.21) (4.17) TEDMATH .05** .11*** -.04 11*** (3.01) (4.88) (1.23) (5.33) ZEDMATH - .15*** (4.*07) Constant 16.55 16.30 13.23 21.60 (6.27) (6.19) (3.62) (5.77) r2 .49 .49 .47 .52 Note. Dependent variable is mathematics post-test score. Numbers are parameter estimates, with t-statistics in parentheses. a/ N = 2796 b/ N = 1580 cl N = 1215 *** p < .001 ** p < .01 * p < .05 Table 5: Student Achievement As Determined By Prior Achievement, Background Characteristics, Textbooks, and Possible Mediating Factors Alternative Specifications Indep. Vbls (1)a (2)D (3)c (4)dI XROT .81*** .80*** ,,79,** .80*** (52.08) (52.72) (51.45) (53.79) XSEX .28 .14 .08 .27 (1.19) (0.60) (0.35) (1.20) XAGE -.07*** -.08*** -.08*** -.G7*** (4.93) (5.84) (5.84) (5.36) XELDEST -.14 -.29 -.33 -.18 (0.51) (1.06) (1.17) (0.68) XFOCCI .16 .13 .11 .18 (1.21) (1.00) (0.80) (1.37) YMEDUC .36* .27 .21 .24 (2.21) (1.71) (1.31) (1.60) YHLANG .05 .14 .01 .08 (0.22) (0.59) (0.34) (0.33) TXTBK .34 .42 .62** .55* (1.37) (1.79) (2.58) (2.39) TYESCOR .03* (1.98) THPYEAR -.00 1 (0.93) THWRKL -.04** (3.04) YMHWRKL .13*** (4.38) Constant 13.97 17.42 17.90 14.89 (5.46) (7.07) (7.23) (6.27) r2 .50 .49 .49 .49 Note. Dependent variable is mathematics post-test score. Numbers are parameter estimates, with t-statistics in parentheses. a/ N = 3199; b/ N = 3349; c/ N = 3213; d/ N = 3478 *** p < .001; ** p < .01; * p < .05 Table 6: Estlmated Effects on Mathematics Achievement of Adding Textbooks to Non-Textbook Classes Pretest Posttest Gaina Non-textbook 8.92 11.78 2.86 Textbook 8.45 11.91 3.46 Non-textbook with textbooks added 8.45 12.55b 4.10 a/ Gain = posttest minus pretest. b/ Annex D provides computational details for this estimate. Annex A Basic Test Statistics wtd N Mean S.D. min max Pretest Core Right 4029.4 14.7 6.2 1.0 37.0 Wrong 4029.4 24.5 6.3 2.0 38.0 Form A Right 1010.2 10.8 4.6 2.0 29.0 Wrong 1010.2 22.9 5.1 6.0 33.0 Form B Right 1026.5 11.1 4.5 1.0 29.0 Wrong 1026.5 22.5 5.0 6.0 34.0 Form C Right 995.5 13A1 5.2 3.0 32.0 Wrong 995.5 20.7 5.4 2.0 32.0 Form D Right 1000.1 14.6 5.2 1.0 31.0 Wrong 1000.1 19.0 5.5 4.0 33.0 Posttest Core Right 3822.5 18.1 7.5 0.0 40.0 Wrong 3822.5 21.6 7.5 0.0 36.0 Form A Right 934.4 i3.1 5.7 0.0 299.0 Wrong 934.4 21.4 5.8 3.0 34.0 Form B Right 938.6 12.7 5.3 1.0 30.0 Wrong 938.6 21.8 5.3 4.0 34.0 Form C Right 967.4 16.1 6.0 2.0 34.0 Wrong 967.4 18.5 6.0 1.0 33.0 Form D Right 969.6 16.7 5.8 1.0 32.0 Wrong 969.6 17.7 5.8 3.0 34.0 Note: Number wrong does not include omits. Annex B Teacher Opportunity-To-Learn Questionnaire What percentage of the students from the target class do you estimate will get the item correct without guessing? 1 = virtually none 2 = 6-40% 3 = 41-60% 4 = 61-94% 5 = virtually all 9 = no response During this school. year did you teach or review the mathematics needed to answer the item correctly? I = no 2 = yes 9 = no respense If in this school year you did not review the mathematics needed to answer this item correctly, was it mainly because 1 = it had been taught prior to this school year 2 = it will be taught later (this year or later) 3 = it Is not in the school curriculum at all 4 = for other reasons 9 = no response Note: These questions were repeated for each of the 180 cognitive items. Annex C Assumptions for Computing Effectiveness-Cost Ratios Public expenditures on tertiary education, 1982a 3,563,430,000 baht Students enrolled in tertiary education, 1982b 1,056,809 Per-student expenditure, 1982 3,372 baht Public expenditures on teacher training, 1975a 194,384,000 baht Students enrolled in tertiary education, 1975b 46,248 Per-student expenditure, 1975a 4,203 baht Semester courses per year (per student) 8 Per-student per-course expenditure, 1982 tertiary 422 baht Per-student per-course expenditure, 1975 teacher Lraining 525 baht Average years teachingc 7.25 Per-student per-course expenditure, 1982 tertiary, per teaching year 58 baht' Per-student per-course expenditure, 1975 teacher training, per teaching year 70 baht Students taught per year (per teacher)c 160 Average teacher education expenditure per student, 1982 tertiary .36 baht Aiverage teacher education expenditure per student, 1975 teacher training .43 baht a/ Source: 1985 Statistical Yearbook, Paris: Unesco, Table 4.3. b/ Source: 1985 Statistical Yearbook, Paris: Unesco, Table 3.11. c/ Source: Unpublished data, SIPIS Thailand. Annex D Estimating the Effects of Adding Textbooks to Nlon-Textbook Classes Mean Value Mean Value Param. Est. Variable Textbook Gp. Non-Textbook Gp. Textbook Gp. Name (1) (2) (3) (2) x (3) INTERCEPT 15.93505 15.98 XROT 8.45 8.92 0.76202 6.80 XSEX 1.52 154 -0.42897 -0.66 XAGE 171.00 17li-55 -0.07783 -13.34 XELDEST 0.21 0u22 -0.45282 -0.10 YFOCCI 2.37 2.45 0.37762 0.93 YMEDUC 1.9,4 1.97 -0.06219 -0.12 YHLANG 0.54 0.46 -0.33556 -0.15 TYESCOR 29.62 31.09 0.11041 3.43 THPYEAR 104.58 108.36 0.00104 0.11 THWRKL 4.43 4.18 -0.10430 -0.44 YMHWKL 3.38 3.46 0.06126 0.21 TEDMATH 4.62 5.66 -0.01776 -0.10 Total 12.55