COST-EFFECTIVE APPROACHES
TO IMPROVE GLOBAL
LEARNING
What does recent evidence tell us are “Smart Buys”
for improving learning in low- and middle-income countries?
Recommendations of the Global Education Evidence Advisory Panel
October 2020
2
Launched in July 2020, the Global Education
Evidence Advisory Panel is an independent,
cross-disciplinary body composed of leading
education experts from around the world. Its
mandate is to provide succinct, usable, and
policy-focused recommendations to support
policymakers’ decision-making on education
investments in low- and middle-income
countries. It is convened jointly by the UK’s
Foreign, Commonwealth & Development
Office (FCDO) and the World Bank, and is
hosted by Building Evidence in Education
Global Group (BE2).
Cost-effective Approaches to Improve Global Learning Levels
Panelists3
PA N E L I S T S
Abhijit Banerjee Tahir Andrabi Sally Grantham-McGregor Hirokazu Yoshikawa
Co-chair of the panel Inaugural Dean, LUMS School Emeritus Professor of Child Professor of Globalization and
Professor of Economics, MIT of Education, and Professor of Health and Nutrition, UCL GOS Education, NYU Steinhardt
Nobel prize-winning economist Economics, Pomona College Institute of Global Health Community and developmental
celebrated for experimental approach Researcher on education and Pioneer in the rigorous study of Early psychologist; has done extensive
to alleviating global poverty. advisor to Government in Childhood Development in developing research across the US, LICs and
Pakistan. Co-founder of CERP. countries with a focus on parental MICs, with a particular focus on
engagement. Recently made an OBE. early childhood and inequality.
Jaime Saavedra Rukmini Banerji Kwame Akyeampong Susan Dynarski
Global Director, Education, CEO, Pratham Education Professor of International Professor of Public Policy,
The World Bank Foundation Education and Development, Education and Economics,
Former Minister of Education of Peru, Innovator in new pedagogical The Open University University of Michigan
currently head of Education global approaches and assessment, leader Expert in education systems in (Joining Harvard in 2021)
practice at the World Bank; also has of large movement to transform Africa, including on political economy Researcher at the forefront of
extensive expertise in poverty reduction. education in India and beyond. of reform, teacher training and understanding and reducing
complementary basic education. inequalities in education, including
for college access, financial aid
design, labour market outcomes,
and high school reforms.
Rachel Glennerster Karthik Muralidharan Sylvia Schmelkes Benjamin Piper
Chief Economist, Foreign, Professor of Economics, Co-chair of the panel Senior Director, Africa
Commonwealth, and University of California Provost of Universidad Education, RTI International
Development Office (FCDO) San Diego Iberoamericana, Mexico City Education expert who has done
Expert on assessing cost-effectiveness Global co-chair of education at JPAL. Sociologist and education researcher transformative work on the Tusome
of alternative interventions to reduce Lead PI in India for the Research on who headed Mexico’s National Institute national-scale literacy program in Kenya
poverty, including in education. Improving Systems of Education, RISE. for the Evaluation of Education; also and the PRIMR Initiative, which tested low-
Researcher and policy advisor. has expertise in intercultural bilingual cost, scalable approaches to improving
education, values, and adult learning. reading and math outcomes in Kenya.
Cost-effective Approaches to Improve Global Learning Levels
4Acknowledgements
Acknowledgements
The panel is grateful to staff of the UK Foreign, Commonwealth and Development Office,
the World Bank Group, and the Building Evidence in Education (BE2) Group for serving
as the GEEAP secretariat. In that capacity, a core team consisting of Deon Filmer, Rachel
Hinton, Alaka Holla, Reema Nayar, Halsey Rogers, and Anna Rudge helped establish
the panel and draft this note, with substantial contributions from Noam Angrist, Omar
Arias, Maria Brindlmayer, Kate Ross, and Laura Savage and with direction from Rachel
Glennerster and Jaime Saavedra. Kate Jefferies, Minna Mattero, Karolina Ordon,
Salim Salama, Kristyn Schrader-King, and Joseph Wales led the communications and
contributed to the design of the launch. Tania Fragnaud, Nimco Musa, and Ivy-Maria
Tompkins provided administrative support. The panel is grateful to the partner institutions
in BE2, with special thanks to UNESCO, UNICEF, and USAID, for providing support for this
initiative and feedback on the Smart Buys findings. The note was copy-edited by Janet
Hulstrand and designed by Aletz Em/Sonideas.
Cost-effective Approaches to Improve Global Learning Levels
Cost-effective Approaches to Improve Global Learning Levels
6 What does recent evidence tell us are “Smart Buys”
What does recent evidence
tell us are “Smart Buys”
for improving learning
in low- and middle-
income countries?
This discussion note was produced by the Global Education Evidence Advisory Panel, with
the support of its secretariat, which includes researchers at the U.K. Foreign Commonwealth
and Development Office, and the World Bank. The categorizations are based on the evidence
and on the deliberations of the panel. The judgments are the panel’s own, drawing on their
reading of the available research and evidence; their conclusions do not necessarily reflect
the policy positions of the panelists’ institutions, or of the convening and hosting institutions.
This high rate of Learning Poverty is just one indicator of the wide learning gaps that prevent
education systems in these countries from providing the kinds of opportunities to their
children that they should be able to. We need to understand not just what is effective at
getting more children into school, but also how to improve learning outcomes once they
are there. And given the scale of the challenge, resources within each country need to be
directed to the most cost-effective approaches possible.
Investment over the past decade in research on cost-effective ways to improve learning
gives us an opportunity to increase the value for money of education programs. In this note,
we classify interventions based on their cost-effectiveness at improving learning outcomes,
especially in low- and middle-income countries (LICs and MICs). We also provide guidance
on the contexts in which a specific intervention is likely to be useful in improving learning,
recognizing that even the best interventions will not be effective if they address a problem
that is not present in a given context, or if they are implemented poorly.
Despite the rapid growth of the evidence base, there are also many important interventions
for which rigorous, actionable evidence is still in short supply. These interventions are
discussed below in the section titled “Areas where governments nevertheless need to make
decisions or take action, but evidence on how to do it effectively is low.” This discussion,
and the “promising but low-evidence” category, should help in setting future priorities for
research.
Cost-effective Approaches to Improve Global Learning Levels
What does recent evidence tell us are “Smart Buys”7
Motivation
More than half of all children in low- and middle-income countries do not learn to read with comprehension by age
10, despite the ambitions of Sustainable Development Goal 4 for “inclusive and equitable quality education and
lifelong opportunities for all.”
We group educational interventions and categories of interventions into the following tiers, reflecting their cost-
effectiveness at improving learning and the strength of the evidence:
GREAT BUYS GOOD BUYS PROMISING BUT BAD BUYS
LOW-EVIDENCE
These interventions There is good For these approaches, Strong, repeated
are highly cost- evidence there are some small evidence shows
effective and are that these but rigorous studies that these programs
supported by a strong interventions are that show high levels of have not worked
evidence base. cost-effective. cost-effectiveness, but in the past in many
overall the evidence situations or are not
base is more limited. cost-effective.
Cost-effective Approaches to Improve Global Learning Levels
8 What does recent evidence tell us are “Smart Buys”
Approach to Classification
To classify interventions, the panel reviewed a range of recent, rigorous evaluative research on education and learning,
primarily in low- and middle-income countries, that, where possible, included some measure of cost-effectiveness.1 The
interventions discussed in this note were chosen because they have been rigorously tested using methodologies that
can distinguish the causal effect of an intervention. They are also backed up by a body of other evidence, including
evidence that the problem that the intervention was designed to address is widespread in low- and middle-income
countries.
The panel’s approach to classification is summarized below, and elaborated on in Appendix C:
• Outcome variable: This synthesis focuses on identifying the interventions that are most cost-effective in improving
learning in basic education, measured in terms of core cognitive skills (typically, literacy and numeracy).
• Learning and equity: Contrary to what is sometimes assumed under a dichotomous “access vs. learning” view, a
focus on learning as the outcome variable in this context is a tool for improving equity, inclusion, and opportunity.
• Evidence base: The evidence in this note includes the many high-quality systematic reviews and meta-analyses
published over the past decade (see References), as well as newer work that was produced to feed into the panel’s
deliberations.2
• Cost-effectiveness vs. effectiveness: Wherever possible, the panel prioritized evidence of cost-effectiveness in
making its determinations; education systems face budget constraints, and they need to allocate scarce resources
toward whatever interventions will deliver the most learning gains for the most children and youth on a given budget.
However, because many evaluations lack cost data, we also draw on studies focused solely on effectiveness to
identify which interventions tend to produce the largest learning gains, independent of cost.
• Scale: The panel has more heavily weighted the interventions that have been proven effective at a larger scale—
whether systemwide or, at a minimum, in hundreds of schools. The areas that appear promising but have not yet been
assessed at scale are included in the “promising but low-evidence” category.
• Duration of impacts: In making its assessments, the panel gives more credit to interventions that have been shown
to have long-term positive impacts.
• Interpretation of evidence: Classifications are not a counting exercise; rather they reflect principles drawn from the
evaluation results, combined with other knowledge about learning and behavior drawn from research in the fields of
Education, Psychology, and Development Economics.
1 Research conducted within the past 15 years.
2 One new addition to the literature on which this note draws is Angrist, Evans, Filmer, Glennerster, Rogers, and Sabarwal (2020), which estimates cost-effectiveness in terms of the
Learning-Adjusted Years of Schooling (LAYS) achieved by specific interventions. The LAYS indicator, the education measure incorporated into the Human Capital Index, provides a
common metric that improves on the standard-deviation measure used in much past comparative research. In Angrist et al (2020), the preferred metric for cost-effectiveness is LAYS
per $100; this measures how many years of high-quality schooling can be bought for an additional $100 spent on each intervention. (See Appendix D for details.)
Cost-effective Approaches to Improve Global Learning Levels
How to Use This Note 9
How to Use This Note
Audience
This note is intended to be helpful for technical staff in Ministries of Education, donor
agencies, local education groups, and non-profit organizations in thinking through
appropriate interventions. This global evidence it presents should be used alongside
context-specific analyses and system diagnostics. The classification and descriptions
in this note aim to offer a greater sense of prioritization and clarity than has been
possible in the past.
Context
Context is often crucial in determining whether an intervention will be successful
or unsuccessful. Contexts vary by the stage of development of a country, but they
also differ considerably within countries. To be used effectively, therefore, this
guidance should be combined with an assessment of context-specific needs and
implementation constraints, including context-specific mechanisms, the quality of
implementation, and political-economy constraints.
Applicability during the COVID-19 pandemic
As this note was being finalized (in October 2020), education systems around the
world were consumed with trying to keep children engaged in school and learning
during the COVID-19 pandemic. As schools begin to reopen in the middle of a global
economic downturn, there will be a premium placed on finding the most cost-
effective interventions for immediate learning recovery. Some of the interventions
described below might be even more relevant than they were previously; for
example, targeted instruction approaches might be even more apt if students have
fallen further behind grade-level expectations during school closures. Beyond that,
countries will need to think about how to “recover better” so that they don’t simply
replicate the failings of the pre-COVID status quo. This note can provide useful
guidance for those decisions.
The importance of providing good information on
effectiveness and cost-effectiveness
There can be substantial gains in children’s learning when systems shift from
less effective to more effective education programs. Most education spending in
developing countries is by the governments themselves. Therefore, advising partner
governments and other donors to invest their financing toward Great Buys or Good
Buys, or toward the system reforms described below, could be a very cost-effective
use of aid.
See Appendix A for more detail.
Cost-effective Approaches to Improve Global Learning Levels
10 The Importance of Systemic Reform
The Importance of
Systemic Reform
Although this note focuses on the impacts of various interventions aimed at improving
learning, the interventions are not the only thing that matters. To drive systemwide
improvements in learning and make them sustainable over the long term, systemic
reform is likely to be extremely important. Ensuring learning for all children and
youth requires an education system that is coherent and aligned toward learning.
While which reforms should be prioritized will depend on the specific context,
this alignment toward learning should encompass the key system actors, policies,
incentives, pedagogy, and capacity. This in turn requires political commitment to
help systems escape low-learning traps—and it requires commitment not just from
education ministers, but also from the heads of government. Good examples of
sustained systemwide reform are rare, but it can happen with strong and consistent
political leadership. The Brazilian state of Ceará, for example, has made remarkable
recent gains in learning during more than a decade of reforms, rising to become one
of the country’s top-performing states despite also being one of the poorest.
Yet even without systemic reform, interventions like the Great and Good Buys described
below can still substantially improve outcomes for millions of children and youth. Indeed,
they have already been shown to improve learning at scale, typically in systems that are
not yet well-aligned toward learning. To maximize the chances for sustained success,
policymakers should take several factors into account when they implement interventions,
such as complementarities across interventions, dynamic complementarities, and the role
of interventions in advancing or inhibiting systemic reform.
One major element of systemic reform, so comprehensive that it is hard to evaluate
rigorously, is realigning the curriculum, assessment, and examinations—and the
overall orientation of the system—away from elite students, and toward the actual
skill distribution in the entire student population. Education systems in many low-
and middle-income countries focus on schooling for the elite, at the expense of most
students. This tendency may be compounded by international benchmarking (from aid
consultants, as well as governments) and by aspirational standards. Overly ambitious
curricula, textbooks, and exams are the practical result of this focus on the elite. Many
of the most effective interventions, including some that are included in the Great and
Good Buys lists below, attempt to solve this curricular problem by going around it—for
example, by providing catch-up classes, or allowing teachers to go off the curriculum for
part of the day. However, if there is political appetite for systemic change, addressing
the curriculum and learning standards head-on could be highly cost-effective. It is not
possible to cost out this type of change in the way that other interventions are costed,
and it does require new materials and retraining of teachers, which could involve
considerable outlays; but given that the impacts are felt by all students in the system,
the cost per student is likely to be low. Some of the interventions described below,
such as those that focus on teaching at the right level for students, show ways to make
progress toward this goal without thorough systemic change.
Cost-effective Approaches to Improve Global Learning Levels
Great Buys11
Great Buys
The interventions in this category are likely to be highly cost-effective, either because of their large benefits, or
because of their low costs. The cost-effectiveness of these interventions can be an order of magnitude greater than
for interventions in even the Good Buys category.
Intervention Providing information to parents and children on the income-earning benefits of
education (where these are not known or not prominent in people’s minds); on
Giving information sources of funding available; and on the quality of local schools has increased
on the benefits, attendance and learning at low cost. This information can be shared through text
costs, and quality messages or videos (Chile, Peru), parents’ meetings (Madagascar, Chile, and the
of education Dominican Republic), or school report cards (Pakistan). In Mexico, information on the
income benefits of education improved learning outcomes, but not dropout rates,
with larger impacts for girls. (An extension of this idea is providing information on
student learning to educators; this proved very cost-effective in Argentina, though
not in India, and not by itself in Liberia.) These interventions have been tested at
large scale, with a low cost per child when delivered at scale. Note that this is about
providing specific and context-relevant information that shifts people’s beliefs about
the benefits of education or the quality of schooling, not general encouragement to
consider education positively.
Context
This can be effective where specific, locally relevant information of decent quality
from a trusted source is available. The delivery method of the information (for
example, text messages or meetings) must be tailored to the country’s specific needs.
Also, recipients must have the means to act on the information; for example, there
must be schools nearby so that families who are inspired to keep their girls in school
are able to do so safely; and communities that receive the information need to have
enough access to decision-making structures that they can spur action.
Cost-effective Approaches to Improve Global Learning Levels
12 Good Buys
Good Buys
There is good evidence that the interventions in this category can be highly cost-effective across a variety of contexts.
Intervention The most effective interventions change how teachers teach. Where primary school
teaching focuses on rote learning, and teacher knowledge is low step-by-step
Structured lesson lesson guides as part of multifaceted instructional programs can help improve
plans with linked pedagogy. Materials, ongoing training, and monitoring are required in order to
materials and enable teachers to use the plans effectively. A key benefit is that this approach can
ongoing teacher work even with weak teachers. In such contexts, well-designed interventions like
monitoring and this can support teacher professionalism by reinforcing good content and pedagogy
training and by freeing teachers to provide their students with more socioemotional support
and personalized learning. In a randomized controlled trial (RCT) across 169 rural
villages in the Gambia, scripted lesson plans, after-school supplementary classes,
and frequent monitoring and teacher coaching dramatically improved learning
outcomes. To be effective, the pedagogy needs to be evidence-based and applied
at the right level for the students. It is best delivered as system reform, with high-
level buy-in about what is being taught. For example, implementation of the Tusome
program in Kenya that combined these elements (building on evidence from an RCT)
was associated with a 30 percentage-point increase (from roughly 35 percent to
65 percent) in children reaching national benchmarks in both English and Kiswahili
(summary). Structured teachers’ guides can improve learning outcomes; however,
overly scripted (word-for-word) teachers’ guides are less effective than more
simplified guides. Interventions to target teaching instruction by learning level, not
grade, which are discussed below, also include structured teachers’ guides, close
monitoring, and linked materials. Interactive radio instruction could be another
potentially effective pedagogical intervention; although less rigorously tested, it
works on the same principles as other programs that do work effectively.
Context
This intervention can be effective where improvements in pedagogy, including moving
away from reliance on rote learning, are needed. This approach is most useful where
there are important gaps in teachers’ knowledge of content and pedagogy; less
structure is needed when teachers have strong content and pedagogical knowledge.
There is also a need for a curriculum that is well-designed and pitched at the right level.
In low-capacity settings, this approach may not be too challenging politically, because
teachers welcome the chance to focus on classroom teaching; however, in other
contexts teachers may resist these programs as infringements on their professional
autonomy, so it is important to get their buy-in first (for example, by sharing evidence of
effectiveness and including teachers in the implementation process).
Cost-effective Approaches to Improve Global Learning Levels
Good Buys13
Interventions Implementation approaches include providing targeted help for students who are
falling behind, and grouping children for all or part of the day based on their learning
Target teaching level, not on their age. This can be done with government teachers, volunteers, or
instruction by teaching assistants and implemented during school, make-up classes after school, or
learning level, during holidays. A very specific and structured approach to doing this has been tested
not grade (in or in Ghana, India, and Zambia. (See here for an overview of what works best where and
out of school) a discussion of scalability.) A less structured approach is to introduce tracking, where
children are grouped by their initial level of learning. This was highly cost-effective
in Kenya, but it often meets with resistance. Although not implemented through the
government, another cost-effective approach in Botswana has used mobile phones to
send targeted messages based on children’s levels, focusing on remediation for the
students who are farthest behind.
Context
These interventions are effective where there is a wide variety of learning levels
within a class and student learning levels are below grade-level curriculum
expectations (as in many LICs and MICs).
Interventions Where access to education is low, improving access to school increases children’s
schooling and can also improve their learning. This is often incorrectly interpreted as
Reduce travel a mandate for constructing new schools. School construction can pay off in settings
times to schools where there are no schools nearby (for example, Indonesia’s program increased both
access to schooling and long-run labor-market outcomes), but it is often an ineffective
and inefficient way of achieving the goal of increased access because it is expensive
and is not always well targeted to the neediest areas. Fortunately, there are other,
more cost-effective ways to reduce travel time and increase access to schooling.
Where many children live far away from a school, setting up community schools in
existing community buildings or houses increases school participation and learning,
and it does so at less cost than building new schools. Establishing new village-
run schools with community teachers in rural northwestern Afghanistan increased
enrollment and test scores among all children, but particularly among girls. (However,
it is important to note that the sustainability of community schools can be an issue,
so there need to be effective mechanisms for supporting them over the long term.)
But improving access it is not just about establishing schools. Reducing travel time to
schools in underserved areas can also have major impacts, for example through the
provision of bicycles to adolescent girls in India. Interventions like these may be more
sustainable.
Context
This can be effective where marginalized, hard-to-reach, or conflict-affected children
(especially girls) live far from school, and school participation is low, but there are
safe ways to reduce travel times.
Cost-effective Approaches to Improve Global Learning Levels
14 Good Buys
Interventions Need-based aid (for example, through conditional cash transfers) can be crucial for getting
children to continue in school—especially at the secondary level, where families still incur
Giving costs (including opportunity costs), even for public schools. But need-based aid alone
merit-based can fail to lead to learning gains, as was found in Cambodia. Merit-based scholarships,
scholarships to cash payments, or prizes targeted at disadvantaged children and youth can act as a
disadvantaged complementary incentive to improve attendance and student effort, resulting in higher
children and learning outcomes within the mainstream school system. In Kenya, scholarships were
provided to girls who performed well in their 6th grade exams, and 10 high-quality studies
youth
in 7 other countries further support the finding of the effectiveness of such incentives.
Long-term impacts on learning were also found for merit-based programs in Cambodia.
Context
This approach can be helpful where it’s possible to design scholarships so they do not
end up mainly going to students who are already advantaged (for example, where
inequality is low; or where it is possible to target merit only within a group of students
who are from poor households). On the other hand, where the school system as a
whole is failing the typical child, this approach will not be the best way to tackle the
problem. Note that this intervention does not include voucher programs that move
students from public to private schools, which can exacerbate inequality.
Interventions Computer hardware is often a very poor investment from a cost-effectiveness
standpoint: it is more expensive than other inputs, and school systems often
Using software struggle to integrate the technology into their teaching and learning. (See below
that adapts to under “Bad Buys.”) However, using adaptive or self-paced software that targets
the learning learning to the level of an individual child can be highly cost-effective, especially if
level of the child computers that can use the new software are already in place and can be maintained.
(where hardware Whether implemented in school or after school, this approach can increase learning
substantially, according to studies from India and other advanced countries. Uruguay
is already in
implemented this approach at scale, with suggestive evidence of positive impacts
schools) that were larger for students from disadvantaged backgrounds (linked reports in
Spanish). Note that this intervention is built around the idea of teaching at the right
level; however, it is categorized separately because of the high priority that many
policymakers place on finding ways to use technology to promote learning.
Context
An important caveat is that this approach is relevant only where electricity, internet
connection, teacher training, and widespread availability of hardware—including lower-tech
devices in the home—make this doable at low cost and in a way that is inclusive, and where
the software has been shown to be well-designed for learning. This intervention has more
evidence from high-income contexts and needs more evidence related to its use in LICs.
There is less evidence at scale for this intervention than for others in this category, but it is
included here because it is a very promising mechanism for implementing a well-supported
Good Buy (teaching at the right level); also, there is now an explosion of innovation in this
area that should soon yield more evidence, helping to assess its effectiveness.
Cost-effective Approaches to Improve Global Learning Levels
Good Buys15
Interventions There is substantial evidence that millions of poor children show lower levels of
language and cognitive development than their better-off peers over the first 5 years
Pre-primary of life. The deficit becomes quite large by the time these children enter primary school.
education This is likely to affect these children’s ability to benefit from standard schooling. Many
(ages 3-5) scholars have emphasized the importance of learning in the early years, and many
countries are currently expanding coverage of pre-primary education. A rigorous
literature shows that pre-primary interventions can have important long-term economic
benefits in high- and middle-income countries, provided that children attend regularly,
and that the classroom experience offered is better than what children already
experience in the home in terms of early stimulation and social-emotional support.
There is evidence from the U.S. of the dynamic complementarity of such programs,
with the effects of pre-primary education on adult earnings and educational attainment
compounded when followed by strong primary education systems, and vice-versa. The
evidence on the long-term impacts of preprimary education for low-income countries
is less extensive, but there is now substantial and relatively consistent evidence from
many different contexts that pre-primary education has positive impacts on learning
and on cognitive development for children ages 3-5. Several of these studies have
tested low-cost models at large scale (for example, India and Kenya) and models using
national systems (Uruguay), and have found positive impacts on learning, suggesting
that these interventions can be implemented effectively at scale. Some studies have
found short-term but not long-term impacts (India, Brazil, and Indonesia), while
other studies have found both short- and long-term effects such as higher levels of
enrollment in future schooling (Uruguay, U.S.) and more rapid cognitive development
and learning in primary school (Argentina, Ghana, and Mozambique). This suggests
that the dynamic complementarity between pre-primary and primary education found
in the U.S. may be important in low- and middle-income countries as well.
Together, these studies suggest that pre-primary education typically improves
learning for children ages 3-5 and that it can be done cost-effectively. Additional
evidence on how to make such gains persist beyond the pre-primary years would be
a useful area for future research, although it is worth noting that there is not always
evidence of long-term impacts for the other interventions in this note either.
Context
While gains made in pre-primary education may fade over time if children then
transition to low-quality primary education—making pre-primary investments
seemingly less cost-effective—those are the very settings in which expanded
preprimary education may be most needed in order to improve learning outcomes.
Since poor children tend to arrive at school with very lower levels of cognitive and
language development, creating challenges in the classroom from the earliest grades
of primary school, improvements to both preschool and primary education would
ideally go together. Moreover, given the dynamic complementarity observed in both
high- and low-income settings, further testing of pre-primary interventions in the
context of weak primary education would be useful in fully understanding this link.
The quality of pre-primary education itself also matters a lot: interventions will be
cost-effective only where they can provide substantially more stimulation and care
than the status quo (whether that is home-based care or private preschools).
Cost-effective Approaches to Improve Global Learning Levels
16 Promising but low-evidence
Promising but low-evidence
For these interventions, the evidence is limited, but the available findings suggest that these approaches can be
highly cost-effective. More testing to develop scalable models is recommended.
Intervention There is promising evidence that early childhood stimulation programs for parents
can generate benefits that last into adulthood, but the evidence of the long-term
Early childhood effects and scalability is more limited in low-income countries. Reviews have
stimulation found consistently strong evidence of short-term impacts on children’s cognitive,
programs (for language, and sometimes motor development among disadvantaged populations
ages 0 to 2), (Aboud and Yousafzai 2015, Engle et al 2011, Baker-Henningham 2010). A few
targeting parents studies have evaluated the scaling of these early parenting stimulation programs.
They have generally shown initial benefits in child development (Bangladesh study
1, Bangladesh study 2, China, Colombia, India, and Pakistan). Only two studies
did follow-ups two years after intervention: in Colombia, there were no remaining
benefits, but in Pakistan, the benefits to executive function, IQ, pro-social behavior,
and pre-academic skills persisted. And a study in Jamaica found gains in educational
attainment, IQ, mental health, and earnings in adulthood of a home visiting program
of early stimulation; however, more work is needed on how to replicate this kind of
success at scale and cost-effectively.
Context
There is plenty of evidence of the efficacy of early stimulation programs, but more
evidence on effectiveness, scalability, and the persistence of impacts in low-income
contexts is needed.
Cost-effective Approaches to Improve Global Learning Levels
Promising but low-evidence17
Intervention Low levels of teacher attendance and low levels of effort (based on what is
observable) are pervasive in LICs and MICs, compared to what is observed in high-
Teacher performing systems. This is due to failings in the support and motivation provided by
accountability the education system, rather than the failings of individual teachers, but it is costly
and incentive to student learning nonetheless. Test scores increase with more teacher presence
reforms (even when teacher quality is very weak). For this reason, there have been many
attempts to improve the accountability of teachers. Paying teachers based on student
performance or attendance has worked in some settings (for example, in India and
Mexico), but such incentives often fail, as they are hard to design well (Pakistan),
can be gamed (Kenya), or work best along with other inputs (Tanzania, India). Also,
they are hard to introduce because of resistance from both teachers and unions.
Instituting a probationary or contract period for new teachers worked well in Kenya,
but this approach can meet with political resistance. More research and development
in this area is needed.
Context
This approach is likely to be most effective where teachers’ effort is low, so that
there is a substantial margin for improvement (however, note that these are also the
contexts in which these reforms are difficult to implement effectively). These reforms
are politically very challenging to implement, even when well designed, and they can
be reversed if they are not embraced by teachers. Therefore, working with teachers
to design the reforms is important for sustainability.
Intervention Providing feedback to schools through community involvement (as has been done
in India, Indonesia, and Gambia) or gathering better data on teachers and students
Community (Indonesia) has often had little impact. Where involving community members in
involvement school management has worked, however, (as in Indonesia’s alternative approach,
in school Uganda, and Kenya), it is very cost-effective. One feature of successful interventions,
management as in Indonesia and Kenya, has been explicitly linking school committees that involve
community members that have high levels of authority. More work in testing various
designs is needed to understand when and why this works, including a study of the
composition, government structures, and complementary mechanisms, all of which
appear to be important for effectiveness.
Context
This approach may be most promising where power asymmetries between
school authorities and parents is not too great, and where there are potential
complementary sources of accountability for schools (such as well-functioning local
governments to which community members have good access); unfortunately, these
are also the settings where the need for these interventions might be less acute.
Cost-effective Approaches to Improve Global Learning Levels
18 Bad Buys
Bad Buys
Any of the approaches listed above, if implemented poorly or in inappropriate contexts, could be classified as “bad
buys.” But there are also other interventions where the evidence has repeatedly shown that the approaches—as
typically implemented—are either not effective or not cost-effective. While it may not be politically or practically
realistic to cut spending on these “bad buys,” school systems should strongly consider prioritizing the much more
cost-effective interventions discussed above when they are investing any additional budget that may become
available each year. Requesting funds for more cost-effective programs may also bolster the case for education
spending within the overall government budget, to the extent that finance ministries take cost-effectiveness into
account when they assess budget requests.
Intervention One mistake that many systems make is to assume that simply investing more in
inputs on the margin, without improving how they are used or for whom, will improve
Additional inputs learning. This approach can be tempting if the intention is to show that something
alone, when is being done about education, because new materials and infrastructure are more
other issues are visible than some of the cost-effective approaches to pedagogy and classroom
not addressed, organization listed as Good Buys above. It is also tempting, because of course
including: schools must have textbooks, other learning materials, teachers, and buildings in
order to operate.
• textbooks
• additional However, studies in many different settings have found that additional inputs alone,
teachers to used in support of “business as usual,” without improving how they are being used,
reduce class are not effective. Examples include textbooks (Kenya, Sierra Leone), teachers (Kenya,
size India), flip charts (Kenya), flexible grants to schools (Gambia, Indonesia, Tanzania),
• school buildings salaries (Indonesia), and libraries (India).1 In many education systems a combination
• grants of rote learning, teaching to the top of the class, and an overly ambitious curriculum
mean that providing additional inputs has no impact on learning, unless those inputs
• salary
are accompanied by fundamental change in how teachers teach.
• libraries
This does not mean that inputs are unimportant. Most of the Good Buys discussed
above involve providing new inputs as part of a strategy to change pedagogy. Good
materials, including appropriate-level textbooks and instruction at the right level,
provided alongside pedagogical improvements, can make a big difference in learning
outcomes, as discussed above. This is especially true in systems that lack even
minimal levels of resources, such as those with badly overcrowded classrooms.
1 In a few cases, providing additional inputs such as textbooks alone has been shown to have significant impacts on learning in rigorous
studies, as in a very early randomized controlled trial from Nicaragua. However, even that case emphasized the importance of
complementary interventions like programs to improve teaching. That study noted that a radio-lessons intervention evaluated in parallel had
much larger effects, probably because of the “inconsistent application of the textbook treatment (in the hands of teachers with relatively low
levels of education).”
Cost-effective Approaches to Improve Global Learning Levels
Bad Buys19
Intervention Computers and other educational technology are just another type of input,
but they deserve special mention, because they are often especially enticing to
Investments policymakers and other stakeholders (and are expensive). As with other inputs,
in laptops, investing in hardware alone is a bad buy. When not accompanied by well-thought-out
tablets, and complementary measures—including personalized adaptive software and teacher
other computer training teachers on how to use the software—adding computers has no impact at
hardware alone all (Peru and Colombia). This is also true in the U.S. and other advanced countries,
even though the level of computer literacy is higher there. Implementation issues are
a recurring challenge in hardware programs; an example is the lack of a coordinated
approach in the One Laptop per Child scheme in Brazil. It can be more cost-effective
to improve learning using technology that is already available, such as mobile phones
(see examples in Niger and Botswana).
Intervention Cash transfer programs are not a cost-effective tool for improving learning. This
may be unsurprising, since promoting education is not the primary objective of the
Cash transfers transfers, but the literature has often discussed their impacts on education outcomes.
(as a tool for Cash transfers have consistently been found to have beneficial effects on school
improving participation (both enrollment and dropout rates) where participation is low, but
learning) relatively few have found statistically significant impacts on learning. While this could
be partly because few evaluations had samples large enough to pick up learning
impacts in a setting where most children were already attending school, the high cost
also reduces cost-effectiveness even where there are learning impacts. Cash transfer
programs are an expensive way to improve learning, because they aim to increase
incomes substantially, and because targeting is costly. This suggests they are poor
value for money as an education intervention—and indeed analysis of their impact
on Learning-Adjusted Years of Schooling (LAYS) suggests that they are not as cost-
effective as other interventions—although they are effective for other objectives, such
as social protection. (See examples from Malawi, Mexico, Mexico2.)
Cost-effective Approaches to Improve Global Learning Levels
20 Areas where governments nevertheless need to make decisions or take action
Areas where governments
nevertheless need to make
decisions or take action,
but evidence on how to
do it effectively is low
Cost-effective Approaches to Improve Global Learning Levels
Areas where governments nevertheless need to make decisions or take action21
Beyond the interventions categorized above, there are many areas in which
governments consistently invest, but where unfortunately there is little evidence
on how to do it well, at least as the interventions are typically framed. For example,
governments must train, select, and allocate teachers, support girls’ education, make
education inclusive for students with disabilities, and ensure student safety. These
are aspects of schooling and learning where the government is obliged to act, but
there is relatively little robust evidence of ways to do so successfully. This may be
because too little evaluation has been done, as with interventions to improve access
for children with disabilities; or because the evidence that is available is inconclusive,
or even discouraging, as in the case of in-service teacher training on general skills.
There is an urgent need for more research and careful evaluation in these areas, to
find the most cost-effective approaches.
Some illustrative examples are:
General-skills teacher training (in-service)
In-service teacher training as typically provided is generalized, overly theoretical,
off-site training that does not respond to demonstrated teacher needs—and thus is
usually not a good investment. Of course, professional development for teachers
is essential, and specific, practical professional development (for example, through
in-school mentoring and induction programs) that support specific well-evidenced
changes in pedagogy can be highly effective. In fact, it is an integral part of most of
the Good Buy interventions, such as structured lesson plans. However, there is little
evidence showing that the typical stand-alone general-skills in-service training is
cost-effective. Indeed, much of the rigorous evidence that is available suggests that
it does not improve student learning outcomes, because the typical training rarely
incorporates the characteristics that make some professional development programs
effective. For example, a large-scale randomized controlled trial of a national teacher
professional development program in China showed no impacts from 15 days of
training at a centralized location, even when the training was reinforced with follow-
up reminders or evaluation. And in Costa Rica, a program to train teachers in active
learning techniques in math at the secondary school level actually resulted in lower
rates of learning.
Context
In-service professional development can be highly effective when it is an integral
part of a well-evidenced specific pedagogical reform (see Good Buys for a list of
such programs); and when it includes practical training, classroom practice, and
reinforcement over time. It is most likely to be effective where targeting teachers for
training based on pedagogical gaps is feasible, and where the environment allows
a focus on practical training. It is unlikely to be effective where there is strong
institutional inertia favoring the delivery of low-quality, overly theoretical training to
all teachers.
Cost-effective Approaches to Improve Global Learning Levels
22 Areas where governments nevertheless need to make decisions or take action
Selection and allocation of teachers
All countries must select and allocate teachers, and high-performing systems appear
to do it well based on objective factors. But there is a shortage of good evidence on
how to do this effectively, making this an important area for further experimentation
and research. Patronage-based recruitment of teachers likely undermines system
credibility and learning, and some work on the effects of a new meritocratic hiring
system for teachers in Mexico found that even though the test wasn’t good at
predicting who would be a good teacher, just having the test weeds out a lot of bad
candidates. (The test’s lack of predictive power is consistent with findings of earlier
work in Ecuador.) Given that the allocation of teachers across schools can be quite
uneven, and driven by favoritism, information about how to improve this process
would also be helpful. However, there is little evidence yet on how such reforms
affect learning.
Context
Reforms to prioritize merit, and objective standards are most likely to be effective
where policymakers are willing to forgo politically beneficial selection and
deployment, where it is technically feasible to set up meritocratic processes, and
where there are enough qualified candidates to allow merit-based selection.
Differentiating support by gender
While some effective access programs target only girls, most of the interventions
referenced in this note have impacts for both girls and boys, with some general
access interventions proving particularly beneficial for girls (for example, in
Afghanistan). Most evaluations of programs that did not specifically target girls
have found bigger impacts on girls’ access than on boys’ (where girls were the
disadvantaged gender), and general interventions deliver gains for girls in access and
learning that are comparable to gains from girl-targeted interventions. But given that
girls’ participation rates remain lower than those of boys in 53 developing countries,
with particularly large disparities in West Asia and Sub-Saharan Africa, more research
is needed on the effectiveness of programs that specifically target girls, especially
in areas where girls are far behind boys. This is especially true for adolescent girls:
gender disparities are highest at that age, and keeping girls in school has major
benefits (including for health outcomes). A study in Ghana found that secondary-
school scholarships had the biggest impact on girls’ attendance. Once in school, girls
appear to learn at rates similar to or higher than boys in most contexts.
Context
Differentiating support by gender is most likely to be effective where one gender is at
a strong disadvantage in terms of school participation.
Cost-effective Approaches to Improve Global Learning Levels
Areas where governments nevertheless need to make decisions or take action23
Targeted support for children living with disabilities
Over half of the 65 million children with disabilities in LICs and MICs are not in school.
Some interventions (such as inclusive teacher training, training for parents, and some
computer-based interventions) might be effective in improving the foundational
skills of primary-school-aged children with disabilities, but the quality of evidence is
generally low, in part because small sample sizes make these programs difficult to
evaluate.
Interventions to safeguard students from violence
Despite legal prohibitions, corporal punishment and other violence inflicted on and
perpetrated by students are highly prevalent in many countries. The violence is not
only damaging in itself, but it has other long-term impacts because it discourages
access to schooling and hinders learning. Promoting student safety is paramount;
and if students feel safe, it is likely to have knock-on effects on schooling persistence
and learning. However, there is not yet a strong evidence base about how to tackle
violence in the education sector, although there are some encouraging studies. For
example, a socioemotional skills program in El Salvador showed improved behavior
and grades, partly by reducing students’ propensity toward violence; and efficacy
trials in Jamaica show that teacher violence can be reduced with intervention in
preschools and primary schools, using targeted teacher training and coaching.
There are many other examples of “necessary but limited-evidence” intervention
areas that deserve more research. As more evidence becomes available on these
topics, the panel will evaluate them in future meetings.
Cost-effective Approaches to Improve Global Learning Levels
24Appendices
Appendices
Cost-effective Approaches to Improve Global Learning Levels
Appendix A25
APPENDIX A
ore Detail on How
M
to Use This Note
Audience
This note is intended to be helpful for technical staff in Ministries of Education, donor
agencies, local education groups, and nonprofit organizations in thinking through
appropriate interventions. It should be used along with context-specific analysis and
system diagnostics. The classification and descriptions are not definitive; they simply
aim to offer a greater sense of prioritization and clarity than has been possible in
the past, in part due to the paucity of data concerning costs until recently. It will be
especially useful in thinking through where to invest additional marginal resources—
for example, is it better to invest new resources in general teacher training ,or in
training focused on the use of structured lesson plans? But it can also help in more
closely examining where large parts of the education budget are being spent, and
exploring whether that money could be used more cost-effectively.
Context
Context is often crucial in determining whether an intervention will be successful
or unsuccessful. Contexts vary by the stage of development of a country, but also
differ considerably within countries. To be used effectively, therefore, this guidance
should be combined with an assessment of context-specific educational needs and
implementation constraints. This starts with prioritizing objectives, given that in some
countries the key challenge is still increasing access to education, while in others,
children are in school but are not learning; and in still others, learning has improved
on average, but disadvantaged children do much more poorly than the average. With
these objectives set, as an aid, the tables above have described the types of contexts
in which each intervention is most likely to be cost-effective. For example, information
interventions can be incredibly cost-effective, because they are inexpensively
delivered; however, exactly what kind of information is useful and relevant to
parents and children in shaping their decisions about education will differ in different
contexts. One key element of context is political economy. The categorization in the
table does not reflect the political feasibility of each intervention, but only its cost-
effectiveness at improving outcomes. But because political economy is central, the
descriptions do note which interventions are likely to be more politically challenging
to deliver. The impacts on learning also depend on the quality of implementation.
For example, although “teaching at the right level” interventions have typically
been very cost-effective, just calling a program “teaching at the right level” will not
automatically make it a Good Buy; it needs to be effectively implemented in order to
have an impact.
Cost-effective Approaches to Improve Global Learning Levels
26 Appendix A
How to use during the COVID-19 pandemic
As this note was being finalized (October 2020), education systems around the
world were consumed with trying to keep children engaged in school and learning
during the pandemic. As schools begin to reopen in the middle of a global economic
downturn, there will be a premium placed on finding the most cost-effective
interventions for immediate recovery of learning. Some interventions might be even
more relevant than before; for example, targeted instructional approaches might be
even more apt if students have fallen farther behind grade-level expectations during
school closures. Beyond that, countries will need to think about how to “recover
better” so that they don’t simply replicate the failings of the pre-COVID status quo. In
that regard, this note can provide useful guidance for these purposes.2
The importance of providing good information on
effectiveness and cost-effectiveness
Moving from less to more effective education programs delivers substantial gains in
learning, and most education spending in developing countries is by the governments
themselves. Therefore, advising partner governments and other donors to invest their
financing toward Great Buys or Good Buys, or toward the system reforms described
below, could be a very cost-effective use of aid. Some recent experimental work
indicates that policymakers value good evidence, and act on it when they receive
it. However, providing such advice requires substantial advisor time, very high-
quality embedded technical assistance, or research teams, and the advice needs
to be closely tied to the evidence or it can be counterproductive. That said, where
education spending is inefficient but there is the will and capacity to improve, the
provision of good evidence-based information on the relative cost-effectiveness of
various approaches and interventions could in itself be highly cost-effective.
This note is not intended to provide a comprehensive view on all possible
interventions in the education sector. As governments and teams design programs
and portfolios, they should be drawing on context-specific diagnostic work and
evidence from a range of sources.
Comparable evidence does not yet cover the universe of all possible
interventions in this sector. Despite all the progress made in recent years, the
evidence base in education remains small and fragmented (compared to the evidence
base in the health sector, for example); and only a small number of evaluation studies
collect data on costs. Some interventions have had too few rigorous evaluations to
be assessed well. Even where there is evidence, many of the evaluations looked
at single interventions rather than packages of interventions, even though a
growing literature shows that greater impact is often achieved when several good
interventions are combined. In addition, for some interventions that might be Great
or Good Buys, we still know too little about the politics of effective implementation
at scale. While many of the interventions discussed above have been delivered in
challenging contexts, the capacity and accountability of the education system is
2 During the pandemic, there has been substantial innovation and testing in educational technology, parental engagement, and remote
learning. While in most cases it is too early to draw lessons learned for inclusion in this note, as a result of these experiences, such innovative
strategies may lead to new cost-effective interventions that could be integrated into the education system beyond the pandemic.
Cost-effective Approaches to Improve Global Learning Levels
Appendix A27
likely to have major effects on the sustainability and scalability of impacts. Finally,
some very large-scale interventions have not been evaluated rigorously at all. For
all of these reasons, expanding this evidence base should be a priority, with more
investment in building the data and the evaluation capacity of low- and middle-
income countries.
Beyond the well-evaluated and well-researched interventions summarized in this
note, there are other policies and programs that may be appropriate in specific
contexts. However, policymakers should strongly consider whether any of the well-
researched, highly cost-effective approaches (Great and Good Buys) are relevant in
their context, especially as many of those interventions address problems that are
very common. If there is a strong case for an intervention that has not been well
researched, especially in secondary schooling, policymakers should include a careful
monitoring plan and consider whether a robust evaluation of impact is possible
(recognizing that a poor evaluation will not add to the knowledge base). There should
be a higher burden of justification for investing in Bad Buys: the business case for an
intervention identified as rarely cost-effective should have a well-developed and well-
supported theory of change, including consideration of the opportunity cost of the
intervention, and a plan for monitoring its effectiveness.
Future “Smart Buys,” and further work. The panel will convene periodically to
review additional categories of interventions, and to review the evolving evidence
base for past categories. Examples of additional categories of interventions that are
important, but were not reviewed in this first review, include school leadership and
instruction in the mother tongue. Another category of interventions could include
research emerging from the COVID-19 pandemic.
Cost-effective Approaches to Improve Global Learning Levels
28 Appendix B
APPENDIX B
ore Detail on
M
the Importance of
Systemic Reform
Although this note focuses on the impacts of various interventions on learning,
interventions are not all that matters. To drive systemwide improvements in
learning and make them sustainable over the long term, systemic reform is likely
to be extremely important. Ensuring learning for all children and youth requires an
education system that is coherent and aligned toward learning. While which reforms
should be prioritized in a particular setting will depend on the context, alignment
toward learning should encompass the key system actors, policies, incentives,
pedagogy, and capacity. This in turn requires political commitment, to help systems
escape low-learning traps—and the commitment needs to be not just from the
education minister, but from the head of government as well. This includes a durable
commitment to consistent implementation and regular review of what is working.
Good examples of sustained systemwide reform are rare, but it can happen with
strong and consistent political leadership. The Brazilian state of Ceará has made
Cost-effective Approaches to Improve Global Learning Levels
Appendix B29
remarkable recent gains in learning during more than a decade of reforms, rising to
become one of the country’s top-performing states despite being among the poorest.
Yet even without systemic reform, interventions like the Great and Good Buys
discussed in this note can still substantially improve outcomes for millions of
children and youth. Indeed, they have already been shown to improve learning at
scale, typically in systems that are not yet well-aligned toward learning. To maximize
the chances for sustained success, policymakers should take several factors into
account when they implement interventions like those discussed below:
They should look for horizontal complementarities across interventions. A deep
understanding of the specific context is essential in order to design programs that
have a chance of success; and part of this context is how a given program interacts
with other interventions. In the example of Ceará, Brazil, numerous interventions
that supported each other were combined—the setting of clear foundational learning
goals for all children; regular assessment to inform teaching; practical teacher
trainings; and financial incentives provided to municipalities. Another example of
complementarities is pairing the right curricula and the right kind of coaching in
kindergarten, as has been done in the U.S.
They should also recognize the importance of dynamic complementarities.
Complementarities in interventions over time also matter. For example, in the United
States higher-quality preschool delivers larger long-term gains in educational and life
outcomes when it is followed by higher-quality primary schooling, and vice versa.
They should think about how the interventions will advance (or inhibit) systemic
reform. Systemic reform takes at least several years to show substantial impacts on
a range of outcomes. In the meantime, policymakers should focus on interventions
that will not inhibit that reform. Some implementation challenges involve behavioral
change needed from key stakeholders and may require more political and systemic
reform; others are technical challenges with implementing the reforms with fidelity at
scale. Noting these differences is important for systemic reform. Some interventions
may make complementary interventions more effective or easier to introduce
later. For example, if a country already has a system of practical, classroom-based
professional development for its teachers, introducing an improved curriculum is
more likely to be implementable and to have the desired effects in the classroom.
Cost-effective Approaches to Improve Global Learning Levels
30 Appendix C
APPENDIX C
ore Detail on
M
Classification Parameters
Detailed considerations made in assessing the evidence for inclusion within this report
include:
Outcome variable
This synthesis focuses on identifying the interventions that are most cost-effective in
improving learning in basic education, measured in terms of core cognitive skills (typically
literacy and numeracy). These skills are relevant everywhere around the world: they improve
employment, income, health, civic participation, and a host of other development goals.
Consequently, children who do not acquire them will be at a disadvantage throughout their
lives. Educational interventions also have other important impacts, such as reduced crime,
improved employment prospects, and better health status, that are not always mediated
by improvements in learning; future meetings of the panel will consider interventions that
advance those goals directly. Moreover, because improving learning has proved far more
challenging than expanding access to education, this note has focused on that goal. In
cases where impacts on cognitive skills are often not measured, such as in early childhood
development, the panel has relied on proxies, such as the effects on school readiness.
Learning and equity
Contrary to what is sometimes assumed under a dichotomous “access vs. learning”
view, a focus on learning as the outcome variable in this context is a tool for improving
equity, inclusion, and opportunity. There is a reason that Sustainable Development Goal
4 highlights foundational literacy and numeracy skills as a key indicator to be tracked:
because the children and youth most harmed by the learning crisis are those who fail
to acquire those skills during basic education. The panel’s goal is to highlight those
interventions that will advance learning for those students. The evaluations it draws
on generally either focus on disadvantaged schools and children, or have been tested
systemwide; in the latter case, the panel recommends only those interventions that are
shown to be effective for less advantaged students.
Evidence base
The evidence reflected here includes the many high-quality systematic reviews and meta-
analyses published over the past decade, as well as newer work that was produced to
feed into the panel’s deliberations (see References).3 Of course, this rigorous evaluation
3 One new addition to the literature on which this note draws is Angrist, Evans, Filmer, Glennerster, Rogers, and Sabarwal (2020), which estimates
cost-effectiveness in terms of the Learning-Adjusted Years of Schooling (LAYS) achieved by specific interventions. The LAYS indicator, which is the
education measure incorporated in the Human Capital Index, provides a common metric that improves on the standard-deviation measure used in
much past comparative research. In Angrist et al (2020), the preferred metric for cost-effectiveness is LAYS per $100, which measures how many
years of high-quality schooling can be bought for an additional $100 spent on each intervention. (See Appendix D for details.)
Cost-effective Approaches to Improve Global Learning Levels
Appendix C31
literature, although it has greatly expanded over the past 20 years, still has numerous
gaps; so the panel has supplemented it with other types of evidence where necessary.
While the primary evidence base is from low- and middle-income countries, the panel has
also drawn on evidence from high-income countries, where relevant and necessary, to fill
out the evidence base.
Cost-effectiveness vs. effectiveness
Wherever possible, the panel prioritized evidence of cost-effectiveness in making its
determinations; education systems face budget constraints, and they need to allocate
scarce resources toward whatever interventions will deliver the most learning gains for the
most children and youth on a given budget. However, because many evaluations lack cost
data, we have also drawn on data on effectiveness—evidence on which interventions tend
to produce the largest learning gains, independent of cost. Because the estimated impacts
from this larger group of effectiveness interventions appears to share the same range as
the cost-effectiveness group, drawing on those findings is a reasonable strategy.4
Scale
Many interventions that succeed as smaller pilots fail to achieve results when scaled
up, whether because in scaling up implementation quality declines, or because political
resistance to the intervention increases. Therefore, while it has considered a range of
evidence, the panel has weighted more heavily the interventions that have been proven
effective at a larger scale— whether systemwide or, at a minimum, in hundreds of schools.
The areas that appear promising but have not been assessed at scale are included in the
“promising but limited-evidence category.”
Duration of impacts
In making its assessments, the panel gives more credit to interventions that have been
shown to have long-term positive impacts. However, this criterion cannot be applied
blindly. Interventions are often evaluated over only a couple of years, so there is not
always evidence of whether the impacts persist or fade over time. Furthermore, in many
cases the initial intervention may need to be repeated or complemented by other policies
or programs to have its full impact. Therefore, the panel has considered duration of
impacts only as one of multiple factors. Unlike most of the other categories, preschool
and early stimulation interventions do have long-term evidence, which has influenced the
categorization of those two sets of interventions.
Interpretation of evidence
Synthesizing these lessons is not just a counting exercise, both because there are gaps
in the evidence and because the context of the implementation matters. Therefore, the
classifications also reflect principles drawn from the evaluation results, combined with
other knowledge about learning and behavior from educational research, psychology, and
development economics.
4 Angrist, Evans, Filmer, Glennerster, Rogers, and Sabarwal (2020).
Cost-effective Approaches to Improve Global Learning Levels
32 Appendix D
APPENDIX D
Effectiveness and Cost-
Effectiveness Measured
in Terms of LAYS
The key to making judgments about relative cost-effectiveness (and
effectiveness in general) is to have good data and a common metric. Past studies
(such as Kremer, Brannen, and Glennerster 2013) have made these comparisons
by measuring effectiveness in standard-deviation improvements on learning
assessments and then dividing by cost. The new paper that informed this note
(Angrist, Evans, Filmer, Glennerster, Rogers, and Sabarwal 2020) assesses cost-
effectiveness in terms of Learning-Adjusted Years of Schooling (LAYS), a measure
of education that was introduced in the 2018 World Development Report and that
forms a core component of the World Bank’s Human Capital Index (HCI). (See also
the background paper introducing LAYS; published version here.) LAYS combines
the quantity and quality of schooling into a single metric of progress. It is calculated
by multiplying a country’s average number of years of schooling by its average test
score performance relative to a high-performance benchmark. For example, if this
high-performance benchmark is Singapore’s performance, this procedure produces a
measure of learning-adjusted years of schooling expressed in Singapore-equivalent
years. LAYS was initially developed for country-level comparisons; it has since been
expanded to compare specific interventions and policies evaluated in 150 studies
across 46 countries (Angrist, Evans, Filmer, Glennerster, Rogers, and Sabarwal 2020).
The LAYS conversion uses globally comparable learning outcomes produced by the
World Bank for the HCI (Angrist, Djankov, Goldberg, and Patrinos 2019).
LAYS offers two advantages over previous metrics for making such comparisons.
First, because it expresses intervention impacts in terms of additional years of high-
quality schooling delivered, it allows direct comparison of the interventions that affect
both the quantity and the quality of schooling. By contrast, previous analyses have
typically looked at either one or the other. Second, the notion of “additional years of
high-quality schooling” or “additional years of schooling, adjusted for quality” is easier
for a non-specialist audience to understand than “standard deviations of learning.”
The figures below, reproduced from Angrist et al (2020), show cost-effectiveness
(Figure 1) and effectiveness (Figure 2), for various interventions and categories
of interventions. Given this note’s focus on cost-effectiveness, Figure 1 provides
the most relevant guidance for policy. Figure 2 is included as background, to show
that the effect sizes from the Figure 1 subset of interventions (those for which data
on cost-effectiveness is available) are reasonably representative of the effect sizes
from the larger set of well-evaluated interventions. As noted earlier, this study was
not the only evidence that the categorization in this note relies upon, but it was one
important input to the panel’s judgments.
Cost-effective Approaches to Improve Global Learning Levels
Appendix D33
Figure 1: Learning-Adjusted Years of School (LAYS) Gained Per $100, by Category
Giving information on education benefits, costs and quality (N=2)
Interventions to target teaching instruction by learning
level not grade (in of out of school) (TaRL) (N=9)
Structural lesson plans w/ linked materials and ongoing teacher
monitoring & training (N=4)
Community involvement in school management (N=8)
Teacher accountability and incentive reforms (N=3)
Targeted interventions to reduce travel time to schools (N=6)
Giving merit-based scholarships to disadvantaed children and youth (N=3)
Early childhood development (ECD) (N=8)
Cash transfers (as a tool to improve learning) (N=6)
Additional inputs alone (textbooks, class size, laptops/tablets, grants,
libraries, etc) (N=16)
General-skills teacher training (in-service) (N=3)
0 3 10 20 50 150
LAYS gained per $100 (log Scale)
Source: Adapted from Angrist, Evans, Filmer, Glennerster, Rogers, and Sabarwal (2020)
Note: Each category of education intervention shows the learning-adjusted years of schooling (LAYS) gained from a given intervention or policy across more than 150 impact estimates in 46
countries. Study categories are ranked by mean impact, from highest to lowest. The shaded boxplot delineates the 25th and 75th percentiles. The y-axis is reported on a natural log scale.
Figure 2: Learning-Adjusted Years of School (LAYS) Gained, by Intervention Category
Giving information on education benefits, costs and quality
Interventions to target teaching instruction by learning
level not grade (in of out of school) (TaRL)
Structural lesson plans w/ linked materials and ongoing teacher
monitoring & training
Community involvement in school management
Teacher accountability and incentive reforms
Targeted interventions to reduce travel time to schools
Giving merit-based scholarships to disadvantaed children and youth
Early childhood development (ECD)
Cash transfers (as a tool to improve learning)
Additional inputs alone (textbooks, class size, laptops/tablets, grants,
libraries, etc)
General-skills teacher training (in-service)
-0.2 0 0.2 0.4 0.6 0.8 1
LAYS gained
Studies with cost e ectiveness data All studies
Source: Adapted from Angrist, Evans, Filmer, Glennerster, Rogers, and Sabarwal (2020)
Note: Each category of education intervention shows the learning-adjusted years of school (LAYS) from a given intervention or policy across more than 150 impact estimates in 46 countries. The
boxplot is ordered from largest to smallest mean cost-effectiveness (see Figure 1), and the shaded boxplot delineates the 25th and 75th percentile.
Cost-effective Approaches to Improve Global Learning Levels
34References
References
Systematic Reviews and Meta-Analyses
Reviews: Published Articles
Conn, Katharine M. 2017. “Identifying Effective Education Interventions in Sub-Saharan Africa: A Meta-Analysis of
Impact Evaluations.” Review of Educational Research 87 (5): 863-898.
Evans, David K., and Anna Popova. 2016. “What Really Works to Improve Learning in Developing Countries? An
Analysis of Divergent Findings in Systematic Reviews.” World Bank Research Observer 31 (2): 242-270.
Ganimian, Alejandro J., and Richard J. Murnane. 2016. “Improving Education in Developing Countries: Lessons from
Rigorous Impact Evaluations.” Review of Educational Research 86 (3): 719-755.
Kremer, Michael, Conner Brannen, and Rachel Glennerster. 2013. “The Challenge of Education and Learning in the
Developing World.” Science 340 (6130): 297-300.
McEwan, Patrick J. 2015. “Improving Learning in Primary Schools of Developing Countries: A Meta-Analysis of
Randomized Experiments.” Review of Educational Research 85 (3): 353-394.
Reviews: Books/Chapters/Reports
Bruns, Barbara, Deon Filmer, and Harry P. Patrinos. 2011. Making Schools Work: New Evidence on Accountability
Reforms. Washington, DC: World Bank.
Evans, David K., and Fei Yuan. 2019. “What We Learn About Girls’ Education from Interventions that Do Not
Focus on Girls.” World Bank Policy Research Working Paper No. 8944. https://elibrary.worldbank.org/doi/
abs/10.1596/1813-9450-8944
Glewwe, P. W., E. Hanushek, S.D. Humpage, and R. Ravina. 2014. “School Resources and Educational Outcomes
in Developing Countries: A Review of the Literature from 1990 to 2010,” in Education Policy in Developing
Countries, ed. P. W. Glewwe. Chicago: University of Chicago Press.
Krishnaratne, Shari and Howard White. 2013. Quality Education for All Children? What Works in Education in
Developing Countries. Working Paper 20. New Delhi: International Initiative for Impact Evaluation (3ie).
Snilstveit, Birte, Jennifer Stevenson, Daniel Phillips, Martina Vojtkova, Emma Gallagher, Tanja Schmidt, Hannah Jobse,
Maisie Geelen, Maria Grazia Pastorello, and John Eyers. 2015. Interventions for Improving Learning Outcomes
and Access to Education in Low and Middle- Income Countries: A Systematic Review. 3ie Systematic Review
24. London: International Initiative for Impact Evaluation (3ie). https://www.3ieimpact.org/evidence-hub/
publications/systematic-reviews/interventions-improving-learning-outcomes-and-access
World Bank. “Learning to Realize Education’s Promise. World Development Report 2018.” A World Bank Group
Flagship Report. https://www.worldbank.org/en/publication/wdr2018
Individual Evaluations Cited
Abdul Latif Jameel Poverty Action Lab (JPAL). 2017. “Roll Call: Getting Children Into School.” Policy Bulletin. https://
www.povertyactionlab.org/sites/default/files/publication/roll-call-getting-children-into-school.pdf
Aboud, Frances E., and Aisha K. Yousafzai. 2015. “Global Health and Development in Early Childhood.” Annual Review
of Psychology 66: 433-457. https://pdfs.semanticscholar.org/39ca/5b70154859fa8e34855f6789605cb9fbd5af.
pdf
Allende, Claudia, Francisco Gallego, and Christopher Nielson. 2019. “Approximating the Equilibrium Effects of
Informed School Choice.” Unpublished manuscript, Princeton University. https://christopherneilson.github.io/
work/documents/SchoolChoiceInfoExp.pdf
Aker, Jenny C., Christopher Ksoll, and Travis J. Lybbert. 2012. “Can Mobile Phones Improve Learning? Evidence from
a Field Experiment in Niger.” American Economic Journal: Applied Economics 4 (4): 94-120. https://sites.tufts.
edu/jennyaker/files/2010/02/ABC_apr20122.pdf
Andrabi, Tahir, Jishnu Das, and Asim Ijaz Khwaja. 2017. “Report Cards: The Impact of Providing School and Child
Test Scores on Educational Markets.” American Economic Review 107 (6): 1535-63. https://www.aeaweb.org/
articles?id=10.1257/aer.20140774
Angrist, Noam, Peter Bergman, Caton Brewster, and Moitshepi Matsheng. 2020. “Stemming Learning Loss During the
Pandemic: A Rapid Randomized Trial of a Low-Tech Intervention in Botswana.” Available at SSRN 3663098.
https://www.csae.ox.ac.uk/materials/papers/csae-wps-2020-13.pdf
Andrew, Alison, Orazio Attanasio, Britta Augsburg, Monimalika Day, Sally Grantham‐McGregor, Costas Meghir, Fardina
Mehrin, Smriti Pahwa, and Marta Rubio‐Codina. 2020. “Effects of a Scalable Home‐Visiting Intervention on
Child Development in Slums of Urban India: Evidence from a Randomised Controlled Trial.” Journal of Child
Psychology and Psychiatry 61 (6): 644-652. https://acamh.onlinelibrary.wiley.com/doi/full/10.1111/jcpp.13171
Cost-effective Approaches to Improve Global Learning Levels
References35
Andrew, A, O. Attanasio, E. Fitzsimons, S. Grantham-McGregor, C. Meghir, and M. Rubio-Codina. 2018. “Impacts Two
Years After a Scalable Early Childhood Development Intervention to Increase Psychosocial Stimulation in the
Home: A Follow-Up of a Cluster Randomised Controlled Trial in Colombia.” PLoS Med 15 (4): e1002556. https://
doi.org/10.1371/journal.pmed.1002556
Attanasio, Orazio P., Camila Fernández, Emla O. A. Fitzsimons, Sally M. Grantham-McGregor, Costas Meghir, and Marta
Rubio-Codina. 2014. “Using the Infrastructure of a Conditional Cash Transfer Program to Deliver a Scalable
Integrated Early Child Development Program in Colombia: Cluster Randomized Controlled Trial.” BMJ 349:
g5785. https://doi.org/10.1136/bmj.g5785
Attanasio, Orazio, Ricardo Paes de Barro, Pedro Carneiro, David Evans, Lycia Lima, Pedro Olinto, and Norbert Schady.
2017. “Impact of Free Availability of Public Child Care in Brazil.” https://developmentevidence.3ieimpact.org/
search-result-details/impact-evaluation-repository/impact-of-free-availability-of-public-childcare-on-labour-
supply-and-child-development-in-brazil/4581
Avitabile, Ciro, and Rafael De Hoyos. 2018. “The Heterogeneous Effect of Information on Student Performance:
Evidence from a Randomized Control Trial in Mexico.” Journal of Development Economics 135: 318-348. https://
www.sciencedirect.com/science/article/pii/S0304387818304565?via%3Dihub
Baird, Sarah, Craig McIntosh, and Berk Özler. 2019. “When the Money Runs Out: Do Cash Transfers Have Sustained
Effects on Human Capital Accumulation?” Journal of Development Economics 140: 169-185. https://
escholarship.org/content/qt2rd3f9jv/qt2rd3f9jv.pdf
Baker-Henningham, Helen, and Florencia López Bóo. 2010. “Early Childhood Stimulation Interventions in Developing
Countries: A Comprehensive Literature Review.” IDB Working Paper Series, No. IDB-WP-213. Washington, DC:
Inter-American Development Bank (IDB). https://www.econstor.eu/bitstream/10419/89041/1/IDB-WP-213.pdf
Baker-Henningham, Helen, Yakeisha Scott, Marsha Bowers, and Taja Francis. 2019. “Evaluation of a Violence-
Prevention Programme with Jamaican Primary School Teachers: A Cluster Randomised Trial.” International
Journal of Environmental Research and Public Health 16 (15): 2797. https://www.ncbi.nlm.nih.gov/pmc/articles/
PMC6696405/
Banerjee, A. V., R. Banerji, E. Duflo, R. Glennerster, and S. Khemani. 2010. “Pitfalls of Participatory Programs: Evidence
from a Randomized Evaluation in Education in India.” American Economic Journal: Economic Policy 2 (1): 1-30.
https://www.nber.org/papers/w14311
Banerjee, Abhijit, Rukmini Banerji, James Berry, Esther Duflo, Harini Kannan, Shobhini Mukerji, Marc Shotland,
and Michael Walton. 2017. “From Proof of Concept to Scalable Policies: Challenges and Solutions, With an
Application.” Journal of Economic Perspectives 31 (4): 73-102. https://www.nber.org/papers/w22746
Banerjee, Abhijit, Shawn Cole, Esther Duflo, and Leigh Linden. 2007. “Remedying Education: Evidence from Two
Randomized Experiments in India.” The Quarterly Journal of Economics 122 (3): 1235-64. https://www.nber.
org/papers/w11904
Barr, A., F. Mugisha, P. Serneels, and A. Zeitlin. 2012. “Information and Collective Action in the Community Monitoring
of Schools: Field and Lab Experimental Evidence from Uganda.” Unpublished manuscript, University of
Nottingham. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.372.6834&rep=rep1&type=pdf
Barrera-Osorio, Felipe, Andreas De Barros, and Deon P. Filmer. 2018. “Long-Term Impacts of Alternative Approaches
to Increase Schooling: Evidence from a Scholarship Program in Cambodia.” Policy Research Working
Paper No. WPS 8566. Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/
en/838871535033752683/Long-term-impacts-of-alternative-approaches-to-increase-schooling-evidence-from-
a-scholarship-program-in-Cambodia
Barrera-Osorio, F., and D. Raju. 2015. “Teacher Performance Pay: Experimental Evidence from Pakistan.” Washington,
DC: The World Bank. http://documents1.worldbank.org/curated/en/973061468189533912/pdf/WPS7307.pdf
Barrera-Osorio, Felipe, and Leigh L. Linden. 2009. “The Use and Misuse of Computers in Education: Evidence From a
Randomized Controlled Trial of a Language Arts Program.” Cambridge, MA: Abdul Latif Jameel Poverty Action
Lab (JPAL). www. leighlinden. com/Barrera-Linden 20. https://www.semanticscholar.org/paper/The-Use-and-
Misuse-of-Computers-in-Education-%3A-from-Barrera-Osorio-Linden/04fbe09878cd7698d341cb2663e23d28
e836ad44?p2df
Behrman, Jere R., Susan W. Parker, and Petra E. Todd. 2011. “Do Conditional Cash Transfers for Schooling Generate
Lasting Benefits? A Five-Year Followup of PROGRESA/Oportunidades.” Journal of Human Resources 46 (1): 93-
122. https://www.jstor.org/stable/25764805?seq=1
Berlinski, S., S. Galiani, and P. Gertler. 2009. “The Effect of Pre-Primary Education on Primary School Performance.”
Journal of Public Economics 93 (1-2): 219-234. https://www.sciencedirect.com/science/article/abs/pii/
S0047272708001308
Berlinski, Samuel, Sebastian Galiani, and Marco Manacorda. 2008. “Giving Children a Better Start: Preschool
Attendance and School-Age Profiles.” Journal of Public Economics 92 (5-6): 1416-1440. http://personal.lse.
ac.uk/manacorm/preschool.pdf
Berlinski, Samuel, and Matias Busso. 2017. “Challenges in Educational Reform: An Experiment on Active Learning
in Mathematics.” Economics Letters 156: 172-75. https://www.sciencedirect.com/science/article/pii/
S0165176517301854
Bettinger, E., M. Kremer, and J. E. Saavedra. “Are Educational Vouchers Only Redistributive?” Prepared for the
CESifo/PEPG joint conference, “Economic Incentives: Do They Work in Education? Insights and Findings from
Behavioral Research,” CESifo Conference Center, May 16-17, 2008, Munich.
Cost-effective Approaches to Improve Global Learning Levels
36References
Beuermann, Diether W., Julian Cristia, Santiago Cueto, Ofer Malamud, and Yyannu Cruz-Aguayo. 2015.
“One Laptop Per Child at Home: Short-Term Impacts from a Randomized Experiment in Peru.”
American Economic Journal: Applied Economics 7 (2): 53-80. https://pdfs.semanticscholar.org/
cbe1/6a88e600b91bb7188584f0d45a8eb32001a2.pdf
Black, Maureen M., Susan P. Walker, Lia C. H. Fernald, Christopher T. Andersen, Ann M. DiGirolamo, Chunling Lu, Dana
C. McCoy et al. 2017. “Early Childhood Development Coming of Age: Science through the Life Course.” The
Lancet 389 (10064): 77-90. https://www.sciencedirect.com/science/article/abs/pii/S0140673616313897
Blimpo, M., D. Evans, and N. Lahire. 2015. “Parental Human Capital and Effective School Management:
Evidence from the Gambia.” Washington D.C.: World Bank. http://documents1.worldbank.org/curated/
en/923441468191341801/pdf/WPS7238.pdf
Borkum, Evan, Fang He, and Leigh L. Linden. 2012. “The Effects of School Libraries on Language Skills: Evidence from
a Randomized Controlled Trial in India.” No. w18183. National Bureau of Economic Research. https://www.nber.
org/papers/w18183
Burde, D., and L. Linden. 2013. “Bringing Education to Afghan Girls: A Randomized Controlled Trial of Village-Based
Schools.” American Economic Journal: Applied Economics 5 (3): 27–40. doi:10.1257/app.5.3.27 http://www.
leighlinden.com/Afghanistan_Girls_Ed.pdf
Chinen, Marjorie, and Johannes M. Bos. 2016. “Impact Evaluation of the Save the Children Early Childhood Stimulation
Program in Bangladesh: Final Report.” American Institutes for Research. http://pubdocs.worldbank.org/
en/163331484753270396/SIEF-Bangladesh-Endline-Report-Nov2016FINAL.pdf
Cruz-Aguayo, Yyannú, Pablo Ibarrarán, and Norbert Schady. 2017. “Do Tests Applied to Teachers Predict Their
Effectiveness?” Economics Letters 159: 108-111. https://doi.org/10.1016/j.econlet.2017.06.035
de Hoyos, Rafael, Alejandro Ganimian, and Peter Holland. 2019. “Teaching With the Test: Experimental Evidence on
Diagnostic Feedback and Capacity-Building for Schools in Argentina.” World Bank Economic Review. https://
doi.org/10.1093/wber/lhz026
de Hoyos Navarro, E. Rafael, Vincente A. Garcia Moreno, and Harry Anthony Patrinos. 2015. “The Impact of an
Accountability Intervention With Diagnostic Feedback: Evidence from Mexico (English).” Policy Research
Working Paper No. WPS 7393 Washington, D.C.: World Bank Group. http://documents.worldbank.org/curated/
en/297561468188928817/The-impact-of-an-accountability-intervention-with-diagnostic-feedback-evidence-
from-Mexico
de Ree, Joppe, Karthik Muralidharan, Menno Pradhan, and Halsey Rogers. 2018. “Double for Nothing? Experimental
Evidence on an Unconditional Teacher Salary Increase in Indonesia.” The Quarterly Journal of Economics 133
(2): 993–1039. https://doi.org/10.1093/qje/qjx040
Dillon, Moira R, Harina Kannan, Joshua T. Dean, Elizabeth S. Spelke, and Esther Duflo, 2017. “Cognitive Science in the
Field: A Preschool Intervention Durably Enhances Intuitive But Not Formal Mathematics.” Science 357 (6346):
47-55. https://science.sciencemag.org/content/357/6346/47/tab-figures-data
Doss, C., E. M. Fahle, S. Loeb, and B. N. York. 2017. “Supporting Parenting Through Differentiated and Personalized
Text-Messaging: Testing Effects on Learning During Kindergarten.” CEPA Working Paper No. 16-18. Stanford
Center for Education Policy Analysis. https://cepa.stanford.edu/content/supporting-parenting-through-
differentiated-and-personalized-text-messaging-testing-effects-learning-during-kindergarten
Dubois, Pierre, Alain De Janvry, and Elisabeth Sadoulet. 2012. “Effects on School Enrollment and Performance of a
Conditional Cash Transfer Program in Mexico.” Journal of Labor Economics 30 (3): 555-589. http://www.jstor.
org/stable/10.1086/664928?seq=1
Duflo, Esther. 2000. “Schooling and Labor Market Consequences of School Construction in Indonesia: Evidence from
an Unusual Policy Experiment.” National Bureau of Economic Research (NBER) Working Paper No. w7860,
https://economics.mit.edu/files/726
Duflo, E., P. Dupas, and M. Kremer. 2015. “Education, HIV, and Early Fertility: Experimental Evidence from Kenya.” The
American Economic Review. 105 (9): 2757–97. https://doi.org/10.1257/aer.20121607
Duflo, E., and L. Breierova. 2004. “The Impact of Education on Fertility and Child Mortality: Do Fathers Really Matter
Less Than Mothers?” NBER Working Paper No. 10513. https://www.nber.org/papers/w10513
Duflo, E., P. Dupas, and M. Kremer. 2011. “Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from
a Randomized Evaluation in Kenya.” American Economic Review 101 (5): 1739-1774. https://www.aeaweb.org/
articles/pdf/doi/10.1257/aer.101.5.1739
Duflo, Esther, Rema Hanna, and Stephen P. Ryan. 2012. “Incentives Work: Getting Teachers to Come to School.”
American Economic Review 102 (4): 1241-78. https://www.aeaweb.org/articles?id=10.1257/aer.102.4.1241
Duflo, Esther, Pascaline Dupas, and Michael Kremer. 2019. “The Impact of Free Secondary Education: Experimental
Evidence from Ghana.” Working Paper. https://web.stanford.edu/~pdupas/DDK_GhanaScholarships.pdf
Duflo, Esther, Pascaline Dupas, and Michael Kremer. 2015. “School Governance, Teacher Incentives, and Pupil-Teacher
Ratios: Experimental Evidence from Kenyan Primary Schools.” Journal of Public Economics 123 (March 2015):
92-110. https://scholar.harvard.edu/files/kremer/files/contract_teacher_jpe_123_2015.pdf
Eble, Alex, Chris Frost, Alpha Camara, Baboucarr Bouy, Momodou Bah, Maitri Sivaraman, Jenny Hsieh et al. 2020.
“How Much Can We Remedy Very Low Learning Levels in Rural Parts of Low-Income Countries? Impact and
Generalizability of a Multi-Pronged Para-Teacher Intervention from a Cluster-Randomized Trial in The Gambia.”
Cost-effective Approaches to Improve Global Learning Levels
References37
Journal of Development Economics . https://static1.squarespace.com/static/55c143d9e4b0cb07521c6d17/t/5dc
972ce2d9cf416c1804dec/1573483263511/Eble+et+al.+-+Para+teachers+for+WP+2019.11.11.pdf
Engle, Patrice L., Lia C. H. Fernald, Harold Alderman, Jere Behrman, Chloe O’Gara, Aisha Yousafzai, and
Meena Cabral de Mello, et al. 2011. “Strategies for Reducing Inequalities and Improving Developmental
Outcomes for Young Children in Low-Income and Middle-Income Countries.” The Lancet 378 (9799):
1339-53. https://digitalcommons.calpoly.edu/cgi/viewcontent.cgi?referer=https://scholar.google.
com/&httpsredir=1&article=1049&context=psycd_fac
Escueta, Maya, Vincent Quan, Andre Joshua Nickow, and Philip Oreopoulos. 2017. “Education Technology: An
Evidence-Based Review.” NBER Working Paper No. 23744. http://www.nber.org/papers/w23744.pdf
Fairlie, Robert W., and Jonathan Robinson. 2013. “Experimental Evidence on the Effects of Home Computers on
Academic Achievement Among Schoolchildren.” American Economic Journal: Applied Economics 5 (3): 211-40.
https://ssrn.com/abstract=2266806
Filmer, D. and N. Schady. 2009. “School Enrollment, Selection and Test Scores.” Policy Research Working Paper No.
WPS 4998. Washington, DC: World Bank. http://documents1.worldbank.org/curated/en/495041468224995463/
pdf/WPS4998.pdf
Gallego, Francisco, Christopher Neilson, and Oswaldo Molina. 2016. “The Impact of Information Provision on Human
Capital Accumulation and Child Labor in Peru.” https://www.poverty-action.org/printpdf/21321
Gertler, P., J. Heckman, R. Pinto, et al. “Labor Market Returns to an Early Childhood Stimulation Intervention in
Jamaica.” 2014. Science 344 (6187): 998-1001. doi:10.1126/science.1251178
Glewwe, Paul, Michael Kremer, and Sylvie Moulin. 2009. “Many Children Left Behind? Textbooks and Test Scores in
Kenya.” American Economic Journal: Applied Economics 1 (1): 112-35. https://www.nber.org/papers/w13300
Glewwe, Paul, Michael Kremer, Sylvie Moulin, and Eric Zitzewitz. 2004. “Retrospective vs. Prospective Analyses of
School Inputs: The Case of Flip Charts in Kenya.” Journal of Development Economics 74 (1): 251-268. https://
www.sciencedirect.com/science/article/abs/pii/S030438780300186X
Gormley, W. T. Jr., D. Phillips, and S. Anderson. 2018. “The Effects of Tulsa’s Pre‐K Program on Middle School Student
Performance.” Journal of Policy Analysis and Management 37 (1): 63-87. https://doi.org/10.1002/pam.22023
Hamadani, Jena D., Syeda F. Mehrin, Fahmida Tofail, Mohammad I. Hasan, Syed N. Huda, Helen Baker-Henningham,
Deborah Ridout, and Sally Grantham-McGregor. 2019. “Integrating an Early Childhood Development
Programme into Bangladeshi Primary Health-Care Services: An Open-Label, Cluster-Randomised Controlled
Trial.” The Lancet Global Health 7 (3): e366-e375. https://doi.org/10.1016/S2214-109X(18)30535-7
Hasan, Amer, Haeil Jung, Angela Kinnell, Amelia Maika, Nozomi Nakajima, and Menno Pradhan. 2019. “Contrasting
Experiences: Understanding the Longer-Term Impact of Improving Access to Preschool Education in Rural
Indonesia.” http://documents1.worldbank.org/curated/en/689351574170836739/pdf/Contrasting-Experiences-
Understanding-the-Longer-Term-Impact-of-Improving-Access-to-Preschool-Education-in-Rural-Indonesia.pdf
Heckman, James J., Bei Liu, Mai Lu, and Jin Zhou. 2020. “Treatment Effects and the Measurement of Skills in a
Prototypical Home Visiting Program.” Institute of Labor Economics (IZA). Working Paper No. 13346. http://ftp.
iza.org/dp13346.pdf
Hjort, Jonas, Diana Moreira, Gautam Rao, and Juan Francisco Santini. 2019. “How Research Affects Policy:
Experimental Evidence from 2,150 Brazilian Municipalities.” National Bureau of Economic Research No.
W25941. https://gautam-rao.com/pdf/HMRS.pdf
Ho, J., H. Thukral, and M. Laflin. 2009. “Tuned In to Student Success: Assessing the Impact of Interactive Radio
Instruction for the Hardest-to-Reach.” Journal of Education for International Development 4 (2): 34-51. http://
citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.507.3541&rep=rep1&type=pdf
Ganimian, Alejandro, Karthik Muralidharan, and Christopher R. Walters. 2020. “Improving Early-Childhood
Human Development: Experimental Evidence from India.” Society for Research on Educational
Effectiveness Conference. https://www.sree.org/assets/conferences/2020s/virtual/
InvitedSymposiumGlobal_P2_slides.pdf
Gertler, Paul, James Heckman, Rodrigo Pinto, Arianna Zanolini, Christel Vermeersch, Susan Walker, Susan M. Chang,
and Sally Grantham-McGregor. 2014. “Labor Market Returns to an Early Childhood Stimulation Intervention in
Jamaica.” Science 344 (6187): 998-1001. https://pubmed.ncbi.nlm.nih.gov/24876490/
Jamison, Dean T., Barbara Searle, Klaus Galda, and Stephen P. Heyneman. 1981. “Improving Elementary Mathematics
Education in Nicaragua: An Experimental Study of the Impact of Textbooks and Radio on Achievement.” Journal
of Educational Psychology 73 (4): 556.
Jensen, Robert. 2010. “The (Perceived) Returns to Education and the Demand for Schooling.” The Quarterly Journal of
Economics 125 (2): 515-548. https://doi.org/10.1162/qjec.2010.125.2.515
Johnson, R. C., and C. K. Jackson. 2019. “Reducing Inequality Through Dynamic Complementarity: Evidence from
Head Start and Public School Spending.” American Economic Journal: Economic Policy 11 (4): 310-49. https://
www.aeaweb.org/articles?id=10.1257/pol.20180510
Kremer, Michael, Conner Brannen, and Rachel Glennerster. 2013. “The Challenge of Education and Learning in the
Developing World.” Science 340 (6130): 297-300. https://science.sciencemag.org/content/340/6130/297.
full?ijkey=tq1ax.4Tmcjac&keytype=ref&siteid=sci
Kremer, Michael, Edward Miguel, and Rebecca Thornton. 2009. “Incentives to Learn.” The Review of Economics and
Statistics 91 (3): 437-456. https://www.nber.org/papers/w10971
Cost-effective Approaches to Improve Global Learning Levels
38References
Kuper, Hannah, Ashrita Saran, and Howard White. 2018. Rapid Evidence Assessment (REA) of What Works to
Improve Educational Outcomes for People with Disabilities in Low and Middle-Income Countries. Campbell
Collaboration and International Centre for Evidence in Disability. https://www.gov.uk/research-for-
development-outputs/improving-educational-outcomes-for-people-with-disabilities-in-low-and-middle-income-
countries-why-does-it-matter-and-what-works
Loyalka, P., X. Huang, L. Zhang, J. Wei, H. Yi, Y. Song, Y. Shi, and J. Chu. 2016. “The Impact of Vocational Schooling on
Human Capital Development in Developing Countries: Evidence from China.” World Bank Economic Review 30
(1): 143-170. https://openknowledge.worldbank.org/handle/10986/22651
Loyalka, Prashant, Anna Popova, Guirong Li, and Zhaolei Shi. 2019. “Does Teacher Training Actually Work? Evidence
from a Large-Scale Randomized Evaluation of a National Teacher Training Program.” American Economic
Journal: Applied Economics 11 (3): 128-54. https://www.aeaweb.org/articles?id=10.1257/app.20170226
Martinez, Sebastian, Sophie Naudeau, and Vitor Pereira. 2017. “Preschool and Child Development Under Extreme
Poverty: Evidence from a Randomized Experiment in Rural Mozambique.” Washington, DC: World Bank. http://
documents1.worldbank.org/curated/en/756171513961080112/pdf/WPS8290.pdf
Mbiti, Isaac, Karthik Muralidharan, Mauricio Romero, Youdi Schipper, Constantine Manda, and Rakesh Rajani. 2019.
“Inputs, Incentives, and Complementarities in Education: Experimental Evidence from Tanzania.” The Quarterly
Journal of Economics 134 (3): 1627-73. https://www.nber.org/papers/w24876
Moses W. Ngware, Njora Hungi, Patricia Wekulo, Maurice Mutisya, Joan Njagi, Nelson Muhia, Elvis Wambiya, Hermann
Donfouet, Grace Gathoni, and Shem Mambe. 2018. “Impact Evaluation of Tayari School Readiness Program
in Kenya.” African Population and Health Research Center, RTI International, and Children’s Investment Fund
Foundation. https://aphrc.org/wp-content/uploads/2019/07/Impact_Evaluation_ECDE_Tayari-long-report.pdf
Muralidharan, Karthik, and Nishith Prakash, 2017. “Cycling to School: Increasing Secondary School Enrollment for Girls
in India.” American Economic Journal: Applied Economics 9 (3): 321-350. http://www.nber.org/papers/w19305
Muralidharan, Karthik, Abhijeet Singh, and Alejandro J. Ganimian. 2019. “Disrupting Education? Experimental Evidence
on Technology-Aided Instruction in India.” American Economic Review 109 (4): 1426-60.
Muralidharan, Karthik, and Venkatesh Sundararaman. 2011. “Teacher Performance Pay: Experimental Evidence from
India.” Journal of Political Economy 119 (1): 39–77. JSTOR, www.jstor.org/stable/10.1086/659655.
Nguyen, Trang. 2013. “Information, Role Models, and Perceived Returns to Education: Experimental Evidence from
Madagascar.” Washington, DC: World Bank.
Ngware, Moses W., Njora Hungi, Patricia Wekulo, Maurice Mutisya, Joan Njagi, Nelson Muhia, Elvis Wambiya,
Hermann Donfouet, Grace Gathoni, and Shem Mambe. 2018. Impact Evaluation of Tayari School Readiness
Program in Kenya. Endline Version. African Population and Health Research Center, RTI International, and
Children’s Investment Fund Foundation. https://aphrc.org/wp-content/uploads/2019/07/Impact_Evaluation_
ECDE_Tayari-long-report.pdf
Perera, Marcelo, and Diego Aboal. 2017. “Evaluación del Impacto de la Plataforma Adaptativa de Matemática en los
Resultados de los Aprendizajes.” Centro de Investigaciones Económicas. https://www.ceibal.edu.uy/storage/
app/media/documentos/CINVE-Informe_PAM_03102017.pdf
Perera, Marcelo, and Diego Aboal. 2017. “Diferencias por Género y Contexto Socioeconómico del Impacto de la
Plataforma Adaptativa de Matemática.” CINVE. https://www.ceibal.edu.uy/storage/app/media/documentos/
CINVE-extension_Informe_PAM_NSEyGENERO.pdf
Piper, B., J. Destefano, and E. M. Kinyanjui, et al. 2018. “Scaling Up Successfully: Lessons from Kenya’s Tusome
National Literacy Program.” Journal of Educational Change 19: 293–321. https://doi.org/10.1007/s10833-018-
9325-4
Piper, B., and M. Korda Poole. 2010. “Early Grade Reading Assessment (EGRA) Plus: Liberia.” Program evaluation
report. RTI International. Prepared for USAID/Liberia. https://www.eddataglobal.org/documents/index.
cfm?fuseaction=pubDetail&ID=283
Piper, B., Y. Sitabkhan, J. Mejia, and K. Betts. 2018. “Effectiveness of Teachers’ Guides in the Global South: Scripting,
Learning Outcomes, and Classroom Utilization.” Research Triangle Park, North Carolina: RTI Press. RTI Press
Publication No. OP-0053-1805 https://doi.org/10.3768/rtipress.2018.op.0053.1805
Piper, Benjamin, Stephanie Simmons Zuilkowski, Margaret Dubeck, Evelyn Jepkemei, and Simon J. King. 2018.
“Identifying the Essential Ingredients to Literacy and Numeracy Improvement: Teacher Professional
Development and Coaching, Student Textbooks, and Structured Teachers’ Guides.” World Development. 106
(C): 324-336. https://doi.org/10.1016/j.worlddev.2018.01.018
Popova, Anna, David K. Evans, Mary E. Breeding, and Violeta Arancibia. 2018. “Teacher Professional Development
Around the World: The Gap Between Evidence and Practice.” Policy Research Working Paper No. WPS8572.
Washington, DC: World Bank. https://documents.worldbank.org/en/publication/documents-reports/
documentdetail/349051535637296801/teacher-professional-development-around-the-world-the-gap-
between-evidence-and-practice
Pradhan, Menno, Daniel Suryadarma, Amanda Beatty, Maisy Wong, Arya Gaduh, Armida Alisjahbana, and Rima Prama
Artha. 2014. “Improving Educational Quality Through Enhancing Community Participation: Results From a
Randomized Field Experiment in Indonesia.” American Economic Journal: Applied Economics 6 (2): 105-126.
https://openknowledge.worldbank.org/handle/10986/3559
Cost-effective Approaches to Improve Global Learning Levels
References39
Sabarwal, S., D. Evans, and A. Marshak. 2014. “The Permanent Textbook Hypothesis: School Inputs and Student
Outcomes in Sierra Leone.” Washington, DC: World Bank. http://documents1.worldbank.org/curated/
en/806291468299200683/pdf/WPS7021.pdf
Visaria, Sujata, Rajeev Dehejia, Melody M. Chao, and Anirban Mukhopadhyay. 2016. “Unintended Consequences
of Rewards for Student Attendance: Results From a Field Experiment in Indian Classrooms.” Economics of
Education Review 54: 173-184. https://www.sciencedirect.com/science/article/pii/S0272775716304071
Walker, Susan P., Susan M. Chang, Christine A. Powell, and Sally M. Grantham-McGregor. 2005. “Effects of Early
Childhood Psychosocial Stimulation and Nutritional Supplementation on Cognition and Education in Growth-
Stunted Jamaican Children: Prospective Cohort Study.” The Lancet 366 (9499): 1804-1807. https://www.
sciencedirect.com/science/article/abs/pii/S0140673605675745
Woolf, Sharon, J. Lawrence Aber, and Jere R. Berhman. “The Impacts of Teacher Training and Parental Engagement on
Kindergarten Quality in Ghana.” Abdul Latif Jameel Poverty Action Lab (JPAL). https://www.poverty-action.org/
study/improving-kindergarten-quality-ghana
Yoshikawa, Hirokazu, Diana Leyva, Catherine E. Snow, Ernesto Treviño, M. Barata, Christina Weiland, and Celia
J. Gomez et al. 2015. “Experimental Impacts of a Teacher Professional Development Program in Chile on
Preschool Classroom Quality and Child Outcomes.” Developmental Psychology 51 (3): 309. https://pubmed.
ncbi.nlm.nih.gov/25706589/
Yousafzai, Aisha K., Jelena Obradović, Muneera A. Rasheed, Arjumand Rizvi, Ximena A. Portilla, Nicole Tirado-Strayer,
Saima Siyal, and Uzma Memon. 2016. “Effects of Responsive Stimulation and Nutrition Interventions on
Children’s Development and Growth at Age 4 Years in a Disadvantaged Population in Pakistan: A Longitudinal
Follow-Up of a Cluster-Randomised Factorial Effectiveness Trial.” The Lancet Global Health 4 (8): e548-e558.
https://doi.org/10.1016/S2214-109X(16)30100-0
References: Other Works Cited
Angrist, N., S. Djankov, P. K. Goldberg, and H. A. Patrinos. 2019. Measuring Human Capital. Policy Research
Working Paper No. 8742. Washington, DC: World Bank. http://documents1.worldbank.org/curated/
en/540801550153933986/pdf/Measuring-Human-Capital.pdf
Angrist, N., D. K. Evans, D. Filmer, R. Glennerster, H. Rogers, and S. Sabarwal. 2020. How to Improve Education
Outcomes Most Efficiently? A Comparison of 150 Interventions Using the New Learning-Adjusted Years of
Schooling Metric. Washington, DC: World Bank.
Evans, David. 2018. “The Power of a Label: Merit Scholarships vs Needs-Based Scholarships?” World Bank Blog http://
blogs.worldbank.org/developmenttalk/power-label-merit-scholarships-vs-needs-based-scholarships
Filmer, Deon, Halsey Rogers, Noam Angrist, and Shwetlena Sabarwal. 2020. “Learning-Adjusted Years of Schooling
(LAYS): Defining a New Macro Measure of Education.” Economics of Education Review. https://www.
sciencedirect.com/science/article/pii/S0272775719300263
Kraay, Aart C. 2019. “The World Bank Human Capital Index: A Guide.” The World Bank Research Observer 34 (1): 1-33
https://academic.oup.com/wbro/article/34/1/1/5492444
Loureiro, Andre, Louisee Cruz, Ildo Lautharte, and David K. Evans. 2020. “The State of Ceará in Brazil is a Role
Model for Reducing Learning Poverty.” Washington, DC: World Bank. https://openknowledge.worldbank.org/
handle/10986/34156
Pritchett, Lant. 2015. “Creating Education Systems Coherent for Learning Outcomes: Making the Transition from
Schooling to Learning.” Research on Improving Systems of Education (RISE). https://riseprogramme.org/sites/
default/files/publications/RISE_WP-005_Pritchett_0.pdf
World Bank. 2019. Ending Learning Poverty: What Will It Take? Washington, DC: World Bank. https://openknowledge.
worldbank.org/bitstream/handle/10986/32553/142659.pdf?sequence=7&isAllowed=y
Cost-effective Approaches to Improve Global Learning Levels