THE WORLD BANK
Discussion Paper
EDUCATION AND TRAINING SERIES
Report No. EDT53
How Texf books AffecV/ chievement
Marlaine E. Lockheed
Stephen C. Vail
Bruce Fuller
{ January 1987
Education and Training Department Operations Policy Staff
The views presented here are those of the author(s), and they should not be interpreted as reflecting those of the World Bank.'
Discussion Paper
Education and Training Series
Report No. EDT53
HOW TEXTJO00S AFFECT ACHIEVEMENT IN DEVELOPING COUNTRIES:
EVIDENCE FROM THAILAND
Marlaine E. Lockheed
Stephen C. Vail
Bruce Fuller
Research Division
Education and Training Department
January 1987
The World Bank does not accept responsibility for the views expressed
herein, which are those of the author(s) and should not be attributed to
the World Bank or its affiliated organizations. The findings,
interpretations, and conclusions are the results of research or analysis
supported by the Bank; they do not necessarily represent official policy of
the Bank.
Copyright © 1987 The International Bank for Reconstruc,ion and Development/
The World Bank
ABSTRACT
This paper analyzes lorngitudinal data from a national sample of
eighth-grade mathematics classrooms (99 teachers and 4030 students) in
Thailand to elxplore the mechanisms whereby textbooks affect student
achievement :gain. The results indicate that textbooks may affect
achievement by (a) substituting for additional post-secondary mathematics
education of teachers, and (b) deliv.ering a more comprehensive curriculum.
The data showed little evidence that textbooks led to better use of
classroom tim,e or increased homeiwork.
The authors also conclude that policies promoting extensive
post-secondary teacher education in favor of investments in essential
teaching materials may be inappropriate, particularly at the
lower-secondary level.
How Textbooks Affect Achievement in Developing Countries:
Evidence from Thailand
Marlaine E. Lockheed, Stephen C. Vail and Bruce Fuller
In both developed and developing countries, schools and
classrooms contribute to student cognitive skills (Hannaway & Lockheed,
1986; Fuller, 1986). Higher and lower income countries differ, however, in
the degree to which school resources such as textbooks affect achievement.
Specifically, while education expenditures on material inputs are largely
unrelated to achievement gains in industrialized countries, they have
positive effects in less developed countries. In an exhaustive review of
144 studies relating expenditure parameters to student achiievement in the
United States, Hanushek reports that "there appears to be no strong or
systematic relationship between school expenditures and student
performance" (1986). By comparison, in a review of 142 analyses conducted
in 72 studies of expenditure parameters in developing countries, Fuller
(1986) reports that 109 (77%) of the analyses confirm the effect of
expenditures on student achievement.
Although the effects of expenditures and material inputs on
achievement in developing countries are well established, little is known
about the mechanisms underlying the contribution. A case in point is the
effect of textbooks on achievement.
Vor the past decade, researchers have documented the effect of
textbooks on student achievement in developing countries (Heyneman and
Loxley, 1983). A review of this research notes that of 18 correlational
-2-
studies of textbook effects on student learning, 15 (83%) report
statistically significant positive results (Heyneman, Farrell, and
Sepulveda-Stuardo, 1981). Two studies with experimental assignment of
students to "textbook" conditions also report significant effects of
textbooks on achievement (Heyneman, Jamison & Montenegro, 1984; Jamison,
Searle, Galda & Heyneman, 1981). As Altbach (1983) notes: "Nothing has
ever replaced the printed word as the key element in the educational
process and, as a result, textbooks are central to schooling at all
levels."
Research findings that student achievement is positively related
to the presence of textbooks in the classroom have influenced both national
educational policies and the lending programs of international
organizations such as the World Bank. For example, of the 232 World Bank
education projects approved between 1970-1983, 48 (21%) included support
for the preparation, provision or distribution of educational materials and
textbooks (Searle, 1985). An analysis of 25 primary education projects or
components of projects indicated that the objectives of the majority of the
textbook provision components was to decrease the ratio of pupils to
textbooks (Romain, 1986).
Yet little research has explored the question of precisely how
textbooks affect achievement. Several suggestions have been made, however,
regarding possible mechanisms most emphasizinge the textbook as a portable
source of information for both teachers and students.
-3-
For Teachers, Textbooks may:
(a) either substitute for gaps in teacher knowledge and skills
(Altbach, 1983), or complement existing skills by providing more
able teachers with a resource that increases their effectiveness
(Beeby, 1986; Murnane and Nelson, 1984);
(b) promote delivery of more complete and coherently organized
curricula, particularly in situations where there is a shortage
of teachers and where teacher training is limited in scope
(Altbach, 1983; Sepulveda-Stuardo & Farrell, 1983);
(c) enable the teacher to make better use of time spent teaching
(Walberg, 1984) and
(d) enable the teacher to assign higher quality homework
(Featherstone, 1985; Walberg, 1985).
For students, textbooks may:
(a) provide a basic exposure to written material otherwise
unavailable in the environment (Heyneman, Farrell &
Sepulveda-Stuardo, 1981) and
(b) enable students to learn independently of the teacher,
particularly through completion of homework (Rohlen, 19133).
In this paper, we test several of these hypotheses using
longitudinal data from the Second International Mathematics Study (SIMS)
conducted by the International Association for the Evaluation of
Educational Achievement (IEA) in Thailand during the 1981-1982 academic
yea.
We first describe the data and how the basic variables were
measured. Then we report on the effects of textbooks on student
achievement, controlling for initial level of achievement and other student
- 4 -
characteristics. Next we attempt to estimate the effects on achievement
gain of various classroom processes and their interaction with the use of
textbooks. Finally, we summarize what we have learned.
Data
-Sample
The sample comprised 99 mathematics teachers and their 4030
eighth-grade studentsl/ and was derived from a two-stage, stratified random
sample of classrooms. The primary sampling units were the twelve national
educational regions of Thailand plus Bangkok. Within each region, a random
sample of lower secondary schools was selected, with replacements. At the
second stage. a random sample of one class per school was selected from a
list of all eighth grade mathematics classes within the school. The
resulting sample represented a 1% sample of eighth grade mathematics
classrooms within each region.2/ Because the sampling scheme selected
students with unequal probabilities (Wattanawaha (1986) notes, for example,
that the study sample underrepresents students attending school in Bangkok
and overrepresents those attending rural schools), sampling weights were
included with the data. The weights were inversely proportional to the
probability of sample selection, and the sum of the weights was equal to
the sample size. The analyses we perform here are weighted to ensure that
conclusions are generalizable to the Thai population.
Because ten teachers failed to complete one of the teacher
questionnaires (related to students' "opportunity to learn"), all analyses
using this instrument are based on a reduced sample size.
-5-
Procedure
At both the beginning and end of the school year, students were
administered a mathematics test covering five curriculum content areas
(arithmetic, algebra, geometry, statistics and measurement). Students also
completed a short background questionnaire at the pretest and a longer one
at the posttest administration. Teachers completed several instruments at
the posttest, including a background questionnaire, a general classroom
process questionaire, and an "opportunity to learn" questionnaire.
Teachers prov.ded information about teaching practices and characteristics
of their randomly selected "target" class. Data about the school was
provided by a school administrator. In the following sections, a
description of each of the variables we analyze in this paper is provided;
summary statistics are presented in Table 1.
Mathematics Achievement
The IEA developed five mathematics tests for use in SIMS. One of
the tests was a forty-item instrument called the core test. The remaininig
four tests were thirty-five item instruments called rotated forms and
designated A through D. The five test instruments contained roughly equal
proportions of items from each of the five curriculum content areas, except
that the core test contained no statistics items (Wattanawaha, 1986). For
purposes of this analysis we regard the instruments as parallel forms with
respect to mathematics content.
The IEA longitudinal design called for students to be
administered both the core form and one rotated form chosen at random at
pretest and posttest. In Thailand students wrere pretested using the
-6-
core test and one rotated form. At posttest students again took the core
test and one rotated form, but were prevented from repeating the rotated
form taken at pretest. Approximately equal numbers of students took each
of the rotated forms in both administrations.
Basic test statistics (rights, wrongs, means and standard
deviations) for the core test and each rotated form are displayed in Annex
A. The mean number right varied between 29 and 48 percent of the test
total, indicating that these were difficult tests for these students.
Moreover, the sum of mean number right and mean number wrong was very close
to the total number of test items, students omitting on average only about
one item per test. Many students apparently guessed on the bulk of the
items.
One goal of our analysis was to predict posttest achievement as a
function of pretest performance plus other determinants. Since students
took the core form twice, the core form posttest score reflects, to some
degree, familiarity with the core test items. Instead of using the core
test, therefore, we analysed scores obtained from the rotated forms, after
they were equated to adjust for differences in test length and difficulty.
Since every SIMS cognitive item had five choices, we first
computed student math scores using a formula that corrected for guessing:
S = R - W/4
where
S = formula score
R = number of items answered correctly
W = number of items answered incorrectly,
excluding omits.
-7-
This score estimates the number of items the student knows, but
has the slight interpretive drawback of sometimes producing
negative scores (Gulliksen, 1950). A formula score was computed for the
core test and for each rotated from.
We next equated rotated form formula scores to core form formula
scores using pretest data only. We used location-scale equating to derive
linear equating coefficients for each rotated form (see Gulliksen, 1950 for
a thorough explanation of this procedure). Equated rotated form scores Ei
were obtained from formula scores Xi by the linear adjustment
Ei= aXi + b
where
a sy sx
b = Y -aX
X - mean rotated formula score
Y = mean core formula score
sx= standard deviation of rotated formula score
sy = standard deviation of core formula score.
i = rotated forms 1-4
The pretest means and variances of the equated rotated scores are equal to
the core pretest means and variances and are presented in Table 2. We
applied the equating coefficients to pre- and posttest rotated form formula
scores to create equated rotated form formula scores, which are the
measures of mathematics achievement used in all analyses reported in this
paper.
Student Background
Basic information about each student included his or her sex,
age, number of older siblings, paternal and maternal education, paternal
-8-
and maternal occupational status, and home language. Parental occupation
was classified into four international categories: (a) unskilled or
semi-skilled worker, (b) skilled worker, (c) clerical or sales worker, and
(d) professional or managerial worker. Because paternal and maternal
occupational. status were highly correlated (r = .39), we analyzed the
effects of paternal occupational status only in this paper. Highest
parental educaxtion was also classified into four categories: (a) very
little or no schooling, (b) primary school, (c) secondary school, and (d)
college, university or some form of tertiary education. Because paternal
and maternal educational attainment were also hkighly correlated (r = .58),
we analyzed the effects of maternal educational attainment only.
Fifty-two percent of the students were male; the average age of
the sample was 1711 months (14.25 years). On average, they had 2.7 older
siblings; 22% were first borns. 31% of the fathers and 16% of the mothers
had attended school beyond the primary level, 30% of the mothers had no
occupation other than housewife, and 41% of the fathers' occupations were
classified as clerical, sales, professional or managerial. 49% of the
parents spoke the language of instruction at home.
Textbook Use
Teachers reported how often they used published textbooks in
their instruction of the target class. Forced-choice options were: rarely
or never, sometimes, and often. Sixty-two percent of the teachers reported
using textbooks "often;" 29 percent reported using textbooks "sometimes;"
and eight percent reported using textbooks "rarely or never." In this
analysis, we created a dummy variable for textbook use, in which "often"
was coded as l and "rarely or never" and "sometimes" were coded as 0.
-9-
The IEA SIMS data do not include any indicators of textbook
availability.
Teacher Education
Teachers indicated the number of semesters in mathematics that
rere included in their post-secondary education. On average, teachers
reported receiving 5 semesters of post-secondary mathematics. We note,
however, that 21 teachers failed to answer this question, and that analyses
employing this variable are consequently based on a reduced sample size.3/
Opportunity to Learn (OTL)
The teachers completed questionnaires designed to measure their
students' opportunity to learn the tested mathematics curriculum. For each
item of the core and rotated forms of the mathematics achievement test,
teachers indicated whether, during the current school year, they had taught
or reviewed the mathematics needed to answer the item correctly. If they
had not taught the material, they were asked to indicate whether: (a) it
had been taught prior to the current school year, (b) it would be taught
later, (c) it was not part of the school curriculum, or (d) some other
reason. (Refer to Annex B for OTL item text).
The complete teacher OTL questionnaire is long and repetitive,
likely to fatigue teachers, and may produce invalid responses toward the
end of the instrument. To minimize incorporating response set bias
produced by fatigue into our OTL measure, we constructed an OTL measure
based on the first 40 core items only. This measure consisted of the
number of times the teacher answered "yes" to the question; "During this
school year did you teach or review the mathematics needed to answer the
- 10 -
item correcLly?" A more complicated OTL measure, similar to that reported
by Smith (1986) was constructed, but had negligable predictive value for
these data.
Mathematics Time
Teachers indicated the number of hours of mathematics instruction
the target class would have received by the end of the school year. On
average, students received 106 hours of mathematics instruction per year.
Assigned Homework
Teachers indicated the number of hours per week they thought
would be needed by a typical student in the target class to complete the
assigned homework outside of class. The measure we used was the teacher
report of the number of hours assigned for the previous week. On average,
4.3 hours of homework were assigned weekly.
Student Homework
Students indicated the number of hours of homework for
mathematics, outside of formal class time, they did each week. On average,
they reported doing 3.4 hours of homework in mathematics weekly.
Results
Our basic hypothesis was that students of teachers who used
textbooks regularly ("often") would learn more over the course of a school
year than would students of teachers who did not use textbooks regularly.
Second, we hypothesized that the use of textbooks would interact with other
elements of the classroom, so as to either substitute for or complement
In order to test our hypotheses, we conducted our analyses in
three stages. First, we examined the effect of textbooks on achievement,
holding constant prior achievement and student background characteristics.
Second, we estimated the effects on student achievement of opportunity to
learn, teacher education, time spent teaching, assignment of homework, and
student time with homework, holding constant prior achievement, student
background characteristics, and teacher textbook use. Third, we estimated
the effects on student achievement of providing textbooks to those teachers
who reported not using them. All analyses were conducted with the student
as the unit of analysis.
Effects of Textbooks on Achievement
Using ordinary least squares (OLS) with listwise deletion of
missing data, we regressed post-test mathematics achievement score on: (a)
pretest score, (b) pretest score and student background characteristics,
and (c) pretest score, student background characteristics, and textbook use
by teacher. The results of these regressions are presented in Table 3. As
expected, posttest score was largely determined by pretest score, which
explained 48% of the variance in posttest score. Family background
variables contributed little to posttest score, after the effects of
pretest score were held constant. Pretest score, of course, included the
effects of such exogenous variables as family background and innate
ability, as well as prior schooling effects.
Specifically, sex, birth order and home use of instructional
language were unrelated to posttest score, holding constant pretest and
considered simultaneously with other student characteristics. However,
- 12 -
older students gained less from pretest to posttest--with each year of age
subtracting about .84 of a point on the posttest---while maternal education
was positively related to posttest score, with eachi level of education
worth about .31 points on the posttest.
Students of teachers who used textbooks scored about .52 points
higher on the posttest, holding constant pretest score and student
demographic characteristics. While the size of this effect amounts to only
about 5% of a standard deviation on the posttest, it also is equivalent to
one-sixth of the average gain for the entire school year (3.16 points;
S.D. = 6.77), or--expressed differently--the equivalent of 1.61 more months
of school. Thus, our findings confirm previous research on the effect of
textbooks on achievement. Our question was not whether or how much
textbooks affect achievement, however, but rather what accounts for their
effect. For this, we turn to our analysis of selected teacher
characteristics and practices and their interaction with textbooks.
Textbooks and Teacher Education
One mechanism whereby textbooks could affect student learning is
by either substituting for or complementing teacher education. That is,
the effect of teacher education could be significant in classes lacking
textbooks and insignificant in classes with textbooks (substitution), or
the effect of teacher education could be greater in classes with textbooks
than in classes lacking textbooks (complementarity). To explore either
hypothesis entailed examining the interaction between textbooks and teacher
education.
- 13 -
We first examined the combined effects of teacher education and
textbooks on student achievement, controlling for student pretest and
background variables (Table 4, column 1). As expected, the number of
semesters of postsecondary mathematics courses completed by the teacher was
positively and statistically significantly related to student posttest
score. However, the size of the effect was small, with each semester worth
only about .05 of a point on the posttest. We next examined the
interaction between textbooks and teacher education (Table 4, column 2) and
found it to be quite significant, affecting both the size and significance
of the coefficients for textbooks and for teacher education. Because of
this significant interaction, we estimated the effect of teacher education
on achievement of students in textbook and non-textbook classes
separately e4/
The results of these regressions (Table 4, columns 3 and 4)
provide evidence that textbooks substitute for teacher education. In
classes in which the teacher used textbooks often, the effect of teacher
education on student posttest achievement was not statistically significan-t
and was quantitatively negligible. In classes lacking textbooks, however,
teacher education was significantly related&to student posttest
achievement. We estimated the separate effect of teacher education and
textbooks from the coefficients provided in Table 4, column 2. Textbooks
contributed .54 of a point to the student mathematics postte;t score, and
each semester of teacher education contributed .01 of a point.5/
Comparing the effects of textbooks to the effects of teacher
education, it is noteworthy that the regular use of textbooks in
- 14 -
mathematics was equivalent to about fifty semesters of teacher
postsecondary mathematics, other things equal. It is also possible to
compute an effectiveness-cost ratio for both teacher education and
textbooks. If each semester of postsecondary mathematics training in
Thailand costs 422 baht in public tertiary institutions or 525 baht in
teacher training schools and teachers teach for about seven years on
average, and if textbooks cost 23 baht and last one year on average, the
effectiveness-cost ratio is 9.4 posttest score points for 100 baht spent on
textbooks, 2.8 points for 100 baht spent ori an additional semester of
postsecondary mathematics in a public university, and 2.3 points per 100
baht spent on postsecondary mathematics in a teacher training institution.
It appears that textbooks are three to four times as cost effective as
postsecondary teacher education. (Annex C presents the assumptions
underlying these estimates; the cost estimates for teacher training are
based on average yearly costs unadjusted for real inflation.)
Textbooks and Opportunity to Learn
A second obvious mechanism whereby textbooks affect achievement
is their capacity to deliver consistent, comprehensive and logically
sequenced curricula. Because of this, students whose teachers use
textbooks should have greater opportunity to learn the material than
students of teachers who do not use textbooks. To test this hypothesis, we
entered a measure of opportunity to learn (OTL) into our regression; the
results are presented in Table 5, column 1.
The inclusion of opportunity to learn in the regression reduced
the size and significance of the coefficient for textbooks, suggesting that
- 15 -
one explanation for the effectiveness of textbooks is, in fact, that they
are carriers of the curriculum. Although the size of the coefficient for
opportunity to learn was small, it indicatea that for every item taught by
the teacher, the student gained .03 of an item. Students of teachers who
taught material relevant to the successful completion of all 40 core items,
therefore, scored 1.2 point higher than students of teachers who did not
cover the material; this is one-third of the average score gain for the
entire year. A test for textbook by opportunity to learn interaction
effects indicated that there was no such interaction.
Textbooks and Instructional Time
Another hypothesis we explored was the relationship between
textbook use and instructional time. We speculated that time use would be
more effective in classrooms in which the teacher used textbooks, and
tested this by including in our regression a measure of the number of hours
of mathematics instruction provided during the year. The results are
presented in Table 5, column 2. Here, instructional time had no effect on
student achievement, and did not substantially change the size of
significa.ce of the coefficient for textbook use. The interaction between
textbooks and instructional time had no appreciable effect on achievement
either, leading us to conclude that textbooks and instructional time
neither complemented nor substituted for each other.
Textbooks and Assigned Homework
Most research on the effects of homework has examined the effects
of student self-reported homework, rather thana assigned homework (Murnane,
1984). Since assigned homework is a policy viariable, while self-reported
- 16. -
homework is better considered an indicator of student effort, we examined
first the effects of assigned homework. We speculated that textbooks might
encourage teachers to assign more homework, which could contribute to
greater student achievement.
Again, we tested this hypothesis by entering the number of hours
of assigned homework in the regression; this time, the effects were
statistically significant, but in an unexpected direction (Table 5, column
3). Teachers who assigned more homework had students whose posttest
achievement was lower than that of students whose teachers assigned less
homework, possibly because teachers of low-achieving classes assigned more
homework. Each hour of assigned homework was worth -.04 points on the
posttest, holding constant pretest. Including assigned homework in the
equation did not change the effect of textbooks on achievement. The
interaction between textbooks and assigned homework was not significant,
although its inclusion in the equation changed the significance of the
coefficient for textbooks.
Textbooks and Student Homework
As expected, the weekly number of hours of homework a student
reported completing was positively associated with posttest score, other
things held constant (Table 5, column 4). Completing one hour of math
homework daily, or five hours per week, had roughly the same quantitative
effect (.45 of a point on the posttest) as having a teacher who used
textbooks (.53 of a point on the posttest); there was no textbook by
student homework interaction effect0 Thus, students who had teachers who
used textbooks were not more likely than other students to complete more
homework during the week.
- 17 -
The Effect of Adding Textbooks to Classes Lacking Them
We next estimated the effect on posttest score of adding
textbooks to classrooms lacking them; this estimate was calculated from
separate regressions for textbook and non-textbook classes. Parameter
estimates for family background, pretest score, teacher characteristics and
practices obtained from regressions for textbook classes were applied to
observed mean values of those variables in non-textbook classes. While
this technique has been criticized when used in reference to significantly
different subsamples, such as for estimating the effects of private schools
on student achievement (Coleman, Hoffer & Kilgore, 1982; Noell, 1982), it
may be appropriate when groups differ only modestly. However, since we
were aware of this possible criticism, we conducted an analysis to test for
differences.
To estimate the degree to which students in textbook-using
classes differed from students in non-textbook classes, we made
multivariate comparisons of the two groups on the same predictor variables
used in the regressions.6/ We first tested the similarity of the
covariance matrices for the two groups using a likelihood ratio test.
Under the hypothesis of no difference, this statistic has a chi-square
distribution on 78 degrees of freedom. The value we obtained was 2479.9
which is significant at the .0001 level.
We next computed the Mahalanobis distance D2 between the vectors
of means for the two groups
D2= ( x - y ) S'1 ( x - y)
- 18 -
using for S the pooled estimate of the covariance matrix. We obtained the
value D2 = 0.8349, whose square root is about .9 'standard units' of
multivariate distance. The distance D2 is related to Hotelling's T2, which
may in turn be compared with critical values from an P distribution (see
K.V. Mardia, et al., 1979, pages 76-77). Our value was significant at the
.01 level (or less), indicating a significant difference in means.
We would like to predict the posttest scores non-textbook
students would achieve were we to give them textbooks. From the foregoing
discussion it is apparent that textbook and non-textbook students were
somewhat, but not radically, different with respect to the va'riables used
in the regressions. This calls into question the validity of such a
prediction using parameter estimates from the textbook group. However,
inspection of the means for the two groups (Annex D) revealed very small
differences overall, suggesting that the statistically significant results
described above are largely attributable to the large sample sizes
involved.
In Table 6, we report the actual and estimated posttest score for
the non-textbook group; the estimated score is computed from the parameter
estimates and mean values reported in Annex,D. Overall, adding textbooks
to non-textbook classes would increase the posttest score from 11.78 to
12.55, nearly one-third of the average gain for the year.
- 19 -
Summary and Discussion
This paper has confirmed that textbooks contribute to student
learning in developing countries, and has identified two important
mechanisms whereby this contribution may be made:
(a) by substituting for postsecondary teacher education and
(b) by delivering a more comprehensive curriculum.
We found little evidence that textbooks enabled teachers to make better use
of classroom time, however, or that they encouraged the assignment or
completion of homework.
The finding that textbook use by teachers is related to more
comprehensive delivery of the curriculum is consistent with previous
research. For example, Sepulveda-Stuardo and Farrell (1983) note that
teachers who used textbooks in Chile were somewhat more likely to be
content-oriented (as opposed to student-oriented) in their teaching style.
Moreover, teacher textbook use may increase the efficiency with which the
teachers use classroom time, since--as we have shown--greater content
coverage is related to higher levels of achievement.
The finding that cextbooks substitute for postsecondary teacher
education suggests that their use in developing countries can help
alleviate financial burdens associated with the provision of teachers
educated beyond the secondary level. We hasten to note, however, that this
appears to be an unusually well-educated teacher sample, with the average
teacher having studied postsecondary mathematics for at least 3.5
semesters. It is precisely for this reason that textbooks can substitute
- 20
for teacher education. Thus, our findings have few implications for
situations in which teachers are educated well below secondary school
completion.
Although teacher qualifications, experience, and education are
positively related to student achievement (Husen, Saha and Noonan, 1978),
the effects of post-secondary teacher education are more pronounced on
secondary student achievement than on primary student achievement (Fuller,
1985). In many developing countries, however, certification requirements
for primary and lower secondary teachers include post-secondary education.
In Thailand, for example, teachers of lower secondary school are required
to complete a two-year, post-secondary teacher education course, during
which they receive general education and pedagogy courses. Investing in
teacher education, however, is costly anid decisions to do so should be
examined carefully. The findings of this paper suggest that educational
policies favoring post-secondary education for teachers of lower secondary
schools may not be appropriate under conditions in which essential, teaching
materials are lacking.
- 21 -
Author Notes
We are indebted to the IEA and to Pote Sapianchai of the National
Education Commission in Thailand and Samrerng Boonruangrutana of
Srinakharinwirot University for conducting the Second International
Mathematics Study and making this paper possible.
We acknowledge the useful suggestions of Joseph Farrell, Stephen
Heyneman, Emman:-el Jimenez, T. Neville Postlethwaite and three anonymous
reviewers on an earlier version of this paper and the assistance of Judy
Ruzicka, Ken Travers and Larry Suter, in making the IEA SIMS data
available.
The World Bank does not accept responsibility for the views
expressed herein, which are those of the authors and should not be
attribtLted to the World Bank or to its affiliated organizations. The
findings, interpretations, and conclusions are the results of research or
analys.is supported by the Bank; they do not necessarily represent official
policy of the Bank. The designations employed, the presentationl of
material, and any maps used in this document are solely for the convenience
of the reader and do not imply the expression of any opinion whatsoever on
the part of the World Bank or its affiliates concerning the legal status of
aniy country.
Footnotes
1. According to Wattanawaha (1986), 32.94% of the 14-year-old age cohort
were enrolled in grade eight.
2. The structure of Thailand's reformed education system includes six
primary school grades, three lower secondary school grades, three upper
secondary school grades, plus tertiary education. In 1981, the total
secondary school enrollment was 1,860,615 students, of -whom approximately
23% (427,941) were enrolled in grade eight. The SIMS sample in Thailand
was 4030 students.
3. Seventy-eight teachers responded to a question regarding their
post-secondary mathematics course participation; on average, they reported
having taken 4.92 semesters of post-secondary mathematics. (This mean
includes one teacher who reported having taken 45 semesters of mathematics;
omitting this teacher lowered the mean to 4.40 semesters.) Recoding the 21
omits as zero lowered the mean to 3.88 semesters, while excluding the
outlier yielded a mean of 3.46 semesters of post-secondary mathematics.
The average teacher education reported in Table 1 is a weighted mean based
on 78 teachers.
4. The variable TEDMATH measures the number of semester mathematics
courses a teacher has had. The distribution of responses to this variable
was skewed highly positive. A large number of teachers (20%) did not
answer the question, even though items adjacent to it on the questionnaire
received high response rates. On the assumption that the missing values
might really be zeros, we recoded them to zeros. Then we took the natural
log of (TEDMATH + 1/3) to get a more symmetric distribution of teacher
mathematics education and reran our regression analyses. The regressions
using the log variable showed approximately the same relationships as those
using the original TEDMATH variable, although they were not quite as
strong.
5. The effect of textbooks = (1.30 - .15(5.04)) = .54, and the effect of
teacher education = (.11 - .15(.64)) = .01.
6. These analyses were performed using DISCRIK, the multivariate
discrimination procedure of SAS. Since DISCR:IM does not allow for weighted
analyses, these results are of unweighted analyses. The variables were
pretest score, student and parent demographics, and teacher characteristics
and practices.
References
Altbach, P. G. (1983). Key issues of textbook provision in the Third
World. Prospects, 13 (3),
Beeby, C.E. (1986). The stages of growth in educational systems. In S.P.
Heyneman and D.S. White (Edo.) The quality of education and economic
development. Washington, DC: The World Bank.
Coleman, J.S., Hoffer, T. & Kilgore, S. (1982). Cognitive outcomes in
public and private schools. Sociology of Education, 55 (2/3),
65-76.
Featherstone, H. (1985, February). Homework. The Harvard Education Letter.
Fuller, B. (1985). Raising school quality in developing countries: what
investments boost learning? (Discussion Paper EDT 7). Washington,
DC: World Bank, Education and Training Department.
Gulliksen, H. (1950). Theory of Mental Tests. New York, NY: John Wiley
& Sons.
Hannaway, J. & Lockheed, M.E. (1986). The contribution of social science to
education policy and practice, 1965-1985. Berkeley, CA: McCutchan.
Hanushek, E.A. (1986). The economics of schooling: production and
efficiency in public schools. Journal of Economic Literature.
Heyneman, S.P., Farrell, J. P. & Sepulveda-Stuardo, M.A. (1981). Textbooks
and achievement in developing countries: what we know. Journal of
Curriculum Studies, 13 (3), 227-246.
Heyneman, S.P., Jamison, D.T., & Montenegro, X. (1984). Textbooks in the
Philippines: Evaluation of the pedagogical impact of a nationwide
investment. Educational Evaluation and Policy Analysis, 6 (2),
139-150.
Husen, T., Saha, L.J. & Noonan, R. (1978). Teacher training and student
achievement in less developed countries. (Staff Working Paper No.
310.) Washington, D.C.: The World Bank.
Jamison, D.T., 'Searle, B., Galda, K, & Heyneman, S.P. (1981). Improving
elementaly mathematics education in Nicaragua: An experimental study
of the impact of textbooks and radio on achievement, Journal of
Educational Psychology, 73 (4), 556-567.
Mardia, K.V, Kent, J.T. and Bibby, J.M. (1979). Multivariate Analysis, New
York, NY: Academic Press.
Murnane, R.J. (1984). A review essay-- comparison. of public and private
schools: lessons from the uproar. Journal of Auman Resources, 14
(2), 263-277.
Murnane, R.J. & Nelson, R.R. (1984). Production and innovation when
techniques are tacit. Journal of Economic Behavior Organization, ,
369-377.
Noell, J. (1982). Public and Catholic schools: A reanalysis of "Public and
private schools." Sociology of Education, 55 (2/3), 123-132.
Searle, B. (1985). General operational review of textbooks (Discussion
Paper EDT 1). Washington, DC: World Bank, Education and Training
Department
Sepulveda-Stuardo, M., and Farrell, J.P. (1983). The use of textbooks by
teachers and students in learning and teaching. In E. Schiefelbein,
J.P. Farrell and M. Sepulveda-Stuardo, The' influence of school
resources in Chile: Their effect on educational achievement and
occupational attainment (Staff Working Paper No. 530). Washington,
D.C: Wor'ld Bank.
Smith, M. (1986, April). Policy implications of differences in mathematics
coverage and expectations for achievement. Paper presented at the
Annual Meeting of the American Educational Research Association, San
Francisco, CA.
Rohlen, T.P. (1983). Japan's High Schools. Berkeley, CA: University of
California Press.
Romain, R. (1986). Lending in primary education: Bank performance review,
1962-1983 (Discussion Paper EDT 20). Washington, DC: World Bank,
Education and Training Department.
Walberg, H. J. (1984). What makes schooling effective? A synthesis
and a critique of three national studies. Contemporary Education: A
Journal of Reviews, 1 (1), 22-24.
Walberg, H. J. (1985). Homework's powerful effects on learning.
Educational Leadership, 42 (7), 76-79.
Wattanawaha, N. (1986). A study of equity in niˇthematics teaching and
learning in lower secondary school in Thailand. Unpublished
doctoral dissertation, University of Illinois, Urbana-Champaign.
Table 1: Variable Names, Descriptions, Means and Standard aviatioa
Variable Description N Mean S.D.
XROT Pretest equated scale score 4014 8.59 7.75
YROT Post-test equated scale score 3809 11.89 9.18
XSEX Sex (female = 1, male = 2) 4013 1.53 0.50
XAGE Age in months 3"93 171.13 9.16
XELDEST Birth order (1 = firstborn, 0 = else) 3967 0.22 0.41
YFOCCI Father's occupational status 3749 2.39 0.92
YMEDUC Mother's educational attainment 3762 1.95 0.79
YHLANG Language of instruction spoken at home 3777 0.52 0.50
(1 = yes, 0 = nio)
TXTBK Textbooks used often in class by teacher 3990 0.64 0.48
(1 = yes, 0 = no)
TEDMATH Post-secondary semesters of mathenatics 3167 5.04 6.95
completed by teacher
TYESCOR Opportunity to learn (0-40 items) 3635 29.98 7.42
THPYZAR Hours of mathematics instruction per year 3820 106.42 52.65
THWRKL Hours of homework assigned by teacher last week 3665 4.32 7.68
YMHWKL Hours of homework completed by student last week 3745 3.43 3.92
Table 2: Pretest Scores and Equating Coefficients
Core Form Rotated Form Equating
RGtated Formula Scores Formula Scores Coefficients
Form Mean S.D. Mean S.D. a b
A 8.82 7.74 5.08 5.80 1.335 2.039
B 8.35 7.61 5.56 5.61 1.355 0.825
C 8.73 7.98 7.97 6.45 1.237 -1.135
D 8.49 7.67 9.91 6.52 1.176 -3.169
Table 3: Student Achievement As Determinede By Prior Achievement,
Background Characteristics and Classroom Use of Textbooks
Alternativre Specifications
Indep. Vbls (1)a (2)D (3)C
XROT .82*** .80*** .80***
(59.04) (54.31) (54.63)
XSEX .29 .23
(1.31) (1.02)
XAGE -e07*** -.07***
(5.47) (5.47)
XELDEST -.07 -.17
(0.27) (0.63)
YFOCCI .08 .12
(0.64) (0.95)
YMEDUC .31* .29
(2.07) (1.88)
YHLANG .16 .05
(0.72) (0.22)
TXTBK .53*
(2.29)
Constant 4.71 15.81 15.56
(29.01) (6.76) (6.60)
r2 .48 .48 .49
Note. Dependent variable is mathematics post-test score. Numbers are
parameter estimates, with t-statistics in parentheses.
a/ N = 3801
b/ N = 3577
c/ N = 3547
*** p < .001
* p < .05
Table 4: Student Achievement As Determined By Prior Achievement,
Background Characteristics, Textbooks and Teacher Education
Students Students
All Studentsa With Without
Indep. Vbls (1) (2) Textbooksb TextbooksC
XROT .80*** .79*** .81*** .78***
(48.12) (48.08) (35.29) (32.85)
XSEX .10 .15 -.02 .46
(0.40) (0.59) (0.05) (1.21)
XAGE -.08*** -.08*** -.05** - 11***
(5.23) (5.29) (2.61) (5.15)
XELDEST -.24 -.26 -.43 .01
(0.79) (0.84) (1.04) (0.03)
YFOCCI .12 .12 .28 -.14
(0.83) (0.83) (1.47) (0.66)
YMEDUC .26 .23 .30 14
(1.47) (1.33) (1.27) (0.58)
YHLANG .05 .17 -.09 .61
(0.18) (0.66) (0.27) (1.58)
TXTBK .57* 1.30***
(2.21) (4.17)
TEDMATH .05** .11*** -.04 11***
(3.01) (4.88) (1.23) (5.33)
ZEDMATH - .15***
(4.*07)
Constant 16.55 16.30 13.23 21.60
(6.27) (6.19) (3.62) (5.77)
r2 .49 .49 .47 .52
Note. Dependent variable is mathematics post-test score. Numbers are
parameter estimates, with t-statistics in parentheses.
a/ N = 2796
b/ N = 1580
cl N = 1215
*** p < .001
** p < .01
* p < .05
Table 5: Student Achievement As Determined By Prior Achievement,
Background Characteristics, Textbooks, and Possible Mediating Factors
Alternative Specifications
Indep. Vbls (1)a (2)D (3)c (4)dI
XROT .81*** .80*** ,,79,** .80***
(52.08) (52.72) (51.45) (53.79)
XSEX .28 .14 .08 .27
(1.19) (0.60) (0.35) (1.20)
XAGE -.07*** -.08*** -.08*** -.G7***
(4.93) (5.84) (5.84) (5.36)
XELDEST -.14 -.29 -.33 -.18
(0.51) (1.06) (1.17) (0.68)
XFOCCI .16 .13 .11 .18
(1.21) (1.00) (0.80) (1.37)
YMEDUC .36* .27 .21 .24
(2.21) (1.71) (1.31) (1.60)
YHLANG .05 .14 .01 .08
(0.22) (0.59) (0.34) (0.33)
TXTBK .34 .42 .62** .55*
(1.37) (1.79) (2.58) (2.39)
TYESCOR .03*
(1.98)
THPYEAR -.00 1
(0.93)
THWRKL -.04**
(3.04)
YMHWRKL .13***
(4.38)
Constant 13.97 17.42 17.90 14.89
(5.46) (7.07) (7.23) (6.27)
r2 .50 .49 .49 .49
Note. Dependent variable is mathematics post-test score. Numbers are
parameter estimates, with t-statistics in parentheses.
a/ N = 3199; b/ N = 3349; c/ N = 3213; d/ N = 3478
*** p < .001; ** p < .01; * p < .05
Table 6: Estlmated Effects on Mathematics Achievement of
Adding Textbooks to Non-Textbook Classes
Pretest Posttest Gaina
Non-textbook 8.92 11.78 2.86
Textbook 8.45 11.91 3.46
Non-textbook with textbooks added 8.45 12.55b 4.10
a/ Gain = posttest minus pretest.
b/ Annex D provides computational details for this estimate.
Annex A
Basic Test Statistics
wtd N Mean S.D. min max
Pretest Core Right 4029.4 14.7 6.2 1.0 37.0
Wrong 4029.4 24.5 6.3 2.0 38.0
Form A Right 1010.2 10.8 4.6 2.0 29.0
Wrong 1010.2 22.9 5.1 6.0 33.0
Form B Right 1026.5 11.1 4.5 1.0 29.0
Wrong 1026.5 22.5 5.0 6.0 34.0
Form C Right 995.5 13A1 5.2 3.0 32.0
Wrong 995.5 20.7 5.4 2.0 32.0
Form D Right 1000.1 14.6 5.2 1.0 31.0
Wrong 1000.1 19.0 5.5 4.0 33.0
Posttest Core Right 3822.5 18.1 7.5 0.0 40.0
Wrong 3822.5 21.6 7.5 0.0 36.0
Form A Right 934.4 i3.1 5.7 0.0 299.0
Wrong 934.4 21.4 5.8 3.0 34.0
Form B Right 938.6 12.7 5.3 1.0 30.0
Wrong 938.6 21.8 5.3 4.0 34.0
Form C Right 967.4 16.1 6.0 2.0 34.0
Wrong 967.4 18.5 6.0 1.0 33.0
Form D Right 969.6 16.7 5.8 1.0 32.0
Wrong 969.6 17.7 5.8 3.0 34.0
Note: Number wrong does not include omits.
Annex B
Teacher Opportunity-To-Learn Questionnaire
What percentage of the students from the target class
do you estimate will get the item correct without guessing?
1 = virtually none
2 = 6-40%
3 = 41-60%
4 = 61-94%
5 = virtually all
9 = no response
During this school. year did you teach or review the
mathematics needed to answer the item correctly?
I = no
2 = yes
9 = no respense
If in this school year you did not review the
mathematics needed to answer this item correctly,
was it mainly because
1 = it had been taught prior to this school year
2 = it will be taught later (this year or later)
3 = it Is not in the school curriculum at all
4 = for other reasons
9 = no response
Note: These questions were repeated for each of the 180
cognitive items.
Annex C
Assumptions for Computing Effectiveness-Cost Ratios
Public expenditures on tertiary education, 1982a 3,563,430,000 baht
Students enrolled in tertiary education, 1982b 1,056,809
Per-student expenditure, 1982 3,372 baht
Public expenditures on teacher training, 1975a 194,384,000 baht
Students enrolled in tertiary education, 1975b 46,248
Per-student expenditure, 1975a 4,203 baht
Semester courses per year (per student) 8
Per-student per-course expenditure, 1982 tertiary 422 baht
Per-student per-course expenditure, 1975 teacher Lraining 525 baht
Average years teachingc 7.25
Per-student per-course expenditure, 1982 tertiary, per
teaching year 58 baht'
Per-student per-course expenditure, 1975 teacher
training, per teaching year 70 baht
Students taught per year (per teacher)c 160
Average teacher education expenditure per student,
1982 tertiary .36 baht
Aiverage teacher education expenditure per student,
1975 teacher training .43 baht
a/ Source: 1985 Statistical Yearbook, Paris: Unesco, Table 4.3.
b/ Source: 1985 Statistical Yearbook, Paris: Unesco, Table 3.11.
c/ Source: Unpublished data, SIPIS Thailand.
Annex D
Estimating the Effects of Adding Textbooks to Nlon-Textbook Classes
Mean Value Mean Value Param. Est.
Variable Textbook Gp. Non-Textbook Gp. Textbook Gp.
Name (1) (2) (3) (2) x (3)
INTERCEPT 15.93505 15.98
XROT 8.45 8.92 0.76202 6.80
XSEX 1.52 154 -0.42897 -0.66
XAGE 171.00 17li-55 -0.07783 -13.34
XELDEST 0.21 0u22 -0.45282 -0.10
YFOCCI 2.37 2.45 0.37762 0.93
YMEDUC 1.9,4 1.97 -0.06219 -0.12
YHLANG 0.54 0.46 -0.33556 -0.15
TYESCOR 29.62 31.09 0.11041 3.43
THPYEAR 104.58 108.36 0.00104 0.11
THWRKL 4.43 4.18 -0.10430 -0.44
YMHWKL 3.38 3.46 0.06126 0.21
TEDMATH 4.62 5.66 -0.01776 -0.10
Total 12.55