MAY 2023   1 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING What does recent evidence tell us are “Smart Buys” for improving learning in low- and middle-income countries? Recommendations of the Global Education Evidence Advisory Panel (GEEAP) The Panel is convened by the Foreign, Commonwealth & Development Office (FCDO), the World Bank, the United Nations International Children’s Emergency Fund (UNICEF), and the United States Agency for International Development (USAID) 2 Advisory Panel Advisory Panel 3 Advisory Panel Susan Dynarski Rachel Glennerster Professor of Education, Associate Professor, Division Harvard Graduate of the Social Sciences and School of Education the College, University Researcher at the forefront of of Chicago; former Chief understanding and reducing Economist at FCDO inequalities in education, including Expert on assessing cost-effectiveness for college access, financial aid of alternative interventions to reduce design, labour market outcomes, poverty, including in education. Launched in July 2020, the Global Education Evidence and high school reforms. Researcher and policy advisor. Advisory Panel is an independent, cross-disciplinary body Sally Grantham-McGregor Karthik Muralidharan composed of leading education experts from around the Emeritus Professor of Child Health and Nutrition, UCL GOS Professor of Economics, University of California world. Its mandate is to provide succinct, usable, and Institute of Global Health Pioneer in the rigorous study of San Diego Global co-chair of education at policy-focused recommendations to support policymakers’ Early Childhood Development in JPAL. Lead PI in India for the developing countries with a focus Research on Improving Systems on parental engagement. An of Education (RISE) program. decision-making on education investments in low- and Officer of the Most Excellent Order of the British Empire (OBE). middle-income countries. The Panel is convened by the Foreign, Commonwealth & Development Office (FCDO), the Benjamin Piper Sara Ruto Director, Global Education, Bill Former Chief & Melinda Gates Foundation Administrative Secretary World Bank, the United Nations International Children’s Education expert who supports grantees that work to improve foundational literacy for Education, Kenya Experienced researcher; served as Emergency Fund (UNICEF), and the United States Agency and numeracy outcomes in low-income countries. Previously supported large- scale education programs across Sub- director/CEO of the PAL Network since its inception in 2015; former Regional Manager of Uwezo East for International Development (USAID). Saharan Africa, the Middle East, and Asia. Africa using evidence to draw public attention to children’s learning. Jaime Saavedra Sylvia Schmelkes Kwame Akyeampong Tahir Andrabi Former Minister of Education Professor and Researcher, Co-chair of the panel Inaugural Dean, LUMS School of Peru, currently head Research Institute for of Education, and Professor of of the Education global the Development of Professor of International Economics, Pomona College practice at the World Bank Education, Universidad Education and Development, Researcher and policymaker with Iberoamericana, Mexico City. The Open University Researcher on education and advisor to Government in Pakistan. Co- extensive expertise on education, Sociologist and education researcher Expert in education systems in founder of the Centre for Economic inequality and poverty reduction. who headed Mexico’s National Institute Africa, including on political economy Research in Pakistan CERP. for the Evaluation of Education; also of reform, teacher training and has expertise in intercultural bilingual complementary basic education. education, values, and adult learning. Hirokazu Yoshikawa Abhijit Banerjee Rukmini Banerji Professor of Globalization and Co-chair of the panel CEO, Pratham Education Education, NYU Steinhardt Foundation. Award winner Community and developmental Professor of Economics, MIT psychologist; has done extensive Nobel prize-winning economist of the Yidan Prize for research across the US, LICs and celebrated for experimental approach Educational Development MICs, with a particular focus on to alleviating global poverty. Innovator in new pedagogical early childhood and inequality. approaches and assessment, leader of large movement to transform education in India and beyond. 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 4 Advisory Panel Acknowledgements 5 Acknowledgements This report was produced by the Global Education Evidence Advisory The Global Education Evidence Advisory Panel (GEEAP) is grateful to the U.K. Foreign Panel, convened by FCDO, the World Bank, UNICEF and USAID, with Commonwealth and Development Office (FCDO), the World Bank, the United Nations International Children’s Emergency Fund (UNICEF), and the United States Agency for International the support of the GEEAP secretariat. The judgments are the Panel’s Development (USAID) for convening the Panel and contributing to the GEEAP secretariat, which own, drawing on their reading of the available research and evidence; supported the Panel as they prepared this report. The Panel gives special thanks to the Douglas their conclusions do not necessarily reflect the policy positions of the B. Marshall Jr. Family Foundation for co-funding the research for this report. Panelists’ institutions, or of the GEEAP convening institutions. The Panel thanks the University of Chicago, especially the research support staff at the This publication should be cited as: 2023 Cost-Effective Approaches to Improve Global Learning Development Innovation Lab, for their exceptional support to ensure a rigorous and systematic - What does recent evidence tell us are “Smart Buys” for improving learning in low- and research approach underlying this report. In particular, they would like to thank Emily Cupito for middle-income countries? [K. Akyeampong, T. Andrabi, A. Banerjee, R. Banerji, S. Dynarski, managing the research team, and Santiago Velasquez, Shashank Patil, and Sharad Hotha from R. Glennerster, S. Grantham-McGregor, K. Muralidharan, B. Piper, S. Ruto, J. Saavedra, S. the University of Chicago, as well as Abelardo De Anda Casas from the University of Oxford for Schmelkes, H. Yoshikawa]. London, Washington D.C., New York. FCDO, the World Bank, UNICEF, their research activities and analysis. Additional research contributors included representatives and USAID. from University of Chicago, The Abdul Latif Jameel Poverty Action Lab (J-PAL), Youth Impact, and MannionDaniels: Margaret Andersen, Noam Angrist, Madeline Brancel, Elisa Chen, Cynthia Ji, Anthony Khaiat, Jora Li, Jiya Nair, and Emily Trinder. Alaka Holla from the World Bank contributed to the research and cost analysis design. The GEEAP secretariat accompanied the Panel throughout the creation of the report by providing research input, drafting suggestions, and offering comments in response to Panel requests, as well as facilitating Panel meetings. The Panel conveners represented on the GEEAP secretariat included: staff of the FCDO; in particular Rachel Hinton and Kate Ross, the World Bank Group; in particular Halsey Rogers and Deon Filmer, UNICEF Innocenti – Global Office of Research and Foresight; in particular Cristina Ramos, Jessica Bergmann and UNICEF Programme Group (in particular Manuel Cardoso), and USAID; in particular Hetal Thukral. The secretariat team also included staff and consultants from MannionDaniels, led by Maria Brindlmayer and Rolla Khadduri. The report was designed by Josh Trinder. Selam Asmelash, Tania Fragnaud, Jiya Nair, and Mary Pietrusko provided administrative support. Sara AlHattab, Louise Boothe, Adam Cathro, Charlie Covington, Stefano De Cupis, Anja Nielsen, Emma Pencheon, Becky Phillips, Kristyn Schrader-King, and Helen Wylie led the communications. The Panel thanks the following people for their contributions on an array of topics covered in the report during different stages of the consultation process: • Frances Aboud, Joshua Jeong, Dana McCoy, Abbie Raikes, Sharon Wolf, and Aisha Yousafzai were consulted on early childhood-related evidence. • Harold Alderman and Pascaline Dupas provided valuable input on health-related topics. • Amy Bellinger, Nicole Haberland, Meredith Kozak, and Caine Rolleston provided insights on evidence in gender and other sections of the report. • Anupama Dathan, Guthrie Gray-Lobe, Anne Healy, Victoria Tan, Caitlin Tulloch, Samuel Wolf, and The Ed Tech Hub, including Jonny D’Rozario, Joel Mitchell, Caitlin Moss, Jamie Proctor, and Sangay Thinley, were consulted on a range of different topics. • The GEEAP conveners consulted with many of their education specialists, including USAID’s Joanie Cohen-Mitchel, Laura Conrad, Melissa Donaher, Ashley Henderson, Meghan Hussey, Julia Johna, Beth Johnson, Josh Josa, Rebeca Martinez, Cassandra Scarpino, Anna Spector, and Elena Walls, and UNICEF’s Matt Brossard, Renaud Comba, and Nicolas Reuge. 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 6 Executive Summary Executive Summary 7 • Teaching socio-emotional and life skills Executive Summary • Involving communities in school management • Targeting interventions towards girls Effective but Relatively Expensive: This report provides a guide for governments and other stakeholders in low- There is good evidence that these interventions are effective, but they are a relatively expensive way to and middle-income countries about what works in improving learning and deliver learning outcomes. They might be appropriate for school systems with larger budgets or to achieve education outcomes. Produced by the Global Education Evidence Advisory Panel non-education objectives. (GEEAP), an independent interdisciplinary panel of global experts in international • Transferring cash (as a tool for improving learning) educational evidence and implementation, the current report updates a previous • Feeding in primary schools version of the Smart Buys. Bad Buys: With this report, the Panel aims to summarize the best, most up-to-date evidence on cost-effectiveness at scale in a user-friendly way for policymakers. It groups many different types of education policies and Strong, repeated evidence shows that these interventions have not worked in the past in many situations or programs into categories from Great Buys to Bad Buys, along with categories where more knowledge is are not cost-effective. needed. • Investing in hardware like laptops, tablets and computers alone • Providing additional inputs alone, when other issues are not addressed, including: Great Buys: • Textbooks • Additional teachers to reduce class size These interventions are highly cost-effective and are supported by a strong body of evidence. • School buildings • Providing information on the benefits, costs, and quality of education • Grants • Supporting teachers with structured pedagogy (a package that includes structured lesson plans, learning materials, and ongoing teacher support) • Salary • Targeting teaching instruction by learning level, not grade (in or out of school) • Libraries Good Buys: For this update, the Panel carried out a systematic search of over 13,000 additional studies, through which it identified over 400 additional high-quality evaluation studies. The added studies bring the total number There is good evidence that these interventions are cost-effective. of evaluations referenced in the Smart Buys categorizations to over 550, and they expand the categories of interventions evaluated into new areas such as health, nutrition, and socioemotional development. These • Providing parent-directed early childhood stimulation programs (for ages 0 to 36 months) evaluations were reviewed based on various criteria—most notably, how cost-effective they were at improving • Providing quality pre-primary education (for ages 3 to 5) learning and other outcomes, and whether they had been successfully implemented at scale. On the basis on this • Reducing travel times to schools evidence, the Panel was able to classify many interventions into the above categories: • Giving merit-based scholarships to disadvantaged children and youth Context, political economy, and implementation details are all critical factors in the effectiveness of each of • Administering school-based mass deworming where worm-load is high these interventions. Under each sub-section, we discuss the specific educational, developmental, and political economy contexts in which the interventions are most likely to be cost-effective based on the evidence and Promising but Limited Evidence: implementation challenges. We encourage policymakers to review both the contextual factors and the evidence on For these interventions, there are some rigorous studies that show high levels of effectiveness, but cost-effectiveness when contemplating scaling these interventions. evidence on cost-effectiveness or examples of implementation at scale are lacking. • Using software that allows personalized learning and adapts to the learning level of the child (where hardware is already in schools) • Augmenting teaching teams with community-hired staff • Providing mass school-based treatment of specific health conditions • Leveraging mobile phones to support learning • Safeguarding students from violence 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 8 Smart Buys Smart Buys 9 What does recent evidence tell us are “Smart Buys” for improving learning in low- and middle-income countries? Introduction This report was produced by the Global Education Evidence Advisory Panel (GEEAP), an independent interdisciplinary panel of global experts in education research and implementation. The Panel received support from its secretariat, which includes researchers at the U.K. Foreign Commonwealth and Development Office (FCDO), UNICEF, USAID, and the World Bank. The categorizations are based on available evidence and on the outcome of deliberations of the Panel members. This updated report relies heavily on a systematic search of the literature on interventions that have been shown to affect educational outcomes. The judgments are the Panel’s own, drawing on their reading and assessment of the available research and evidence; their conclusions do not necessarily reflect the policy positions of the Panelists’ institutions, or of the convening and hosting institutions. About two-thirds of all children in low- and middle-income countries (LICs and MICs) do not learn to read with comprehension by age 10, despite the ambitions of Sustainable Development Goal 4 for “inclusive and equitable quality education and lifelong opportunities for all.” This high level of learning poverty in LICs and MICs is an indication of the inability of education systems to provide equitable and appropriate learning opportunities for all children. We need to understand not just what is effective at getting more children into school, but also, more importantly, how to ensure that, once in school, every child has an equal opportunity to maximize their learning. Given the scale of the challenge and the resource constraints facing nearly all countries, resources need to be focused on the most cost-effective ways to improve learning. Research in the past decade on cost-effective ways to improve learning gives us an opportunity to increase the value for money of education programs and reforms. In this note, we classify interventions based on their cost- effectiveness at improving learning outcomes, especially in LICs and MICs. We also provide guidance on the contexts in which each specific intervention is likely to be useful in improving learning, recognizing that even the best interventions will not be effective if they are not based on a sound contextual understanding of the problem, or if they are implemented poorly. Despite the rapid growth of available evidence in literature, there are many important interventions for which consistent, actionable evidence is still lacking. We discuss these under “Areas where governments nevertheless need to make decisions or take action, but evidence on how to do it effectively is low”. This category and the “Promising but limited evidence” category should help in setting future priorities for research and exploration. 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 10 Smart Buys Smart Buys 11 How this report improves on previous “Smart Buys” reports Approach to Classification This new report reflects a major expansion of the evidence base reviewed by the Panel. This new report reflects a major expansion of the evidence base reviewed by the Panel. The first “Smart Buys” report, To classify interventions, the Panel reviewed a range of rigorous evaluative research on education and published in 2020, was based on 150 interventions and policies whose evaluations met the Panel’s criteria for learning, primarily in LICs and MICs, giving greater weight to those with some measure of cost-effectiveness. quality. For this updated report, a systematic search of over 13,000 studies was conducted. Based on this search, The interventions discussed in this note were chosen because they have been rigorously tested using over 400 additional high-quality evaluation studies were identified and reviewed. Out of the total 550 studies, 120 methodologies that can distinguish the causal effect of an intervention. They are also backed up by a body of studies include cost data, allowing cost-effectiveness comparisons (see Appendix A). other evidence, including evidence that the problem targeted by the intervention is commonly found in LICs and MICs. The Panel made several other advances. First, we expanded the categories considered in this update to cover additional important interventions categories: we surveyed the literature on health and nutrition interventions The panel’s approach to classification is summarized below, and elaborated on in Appendix C: delivered through schools that could impact education outcomes, and the literature on socio-emotional interventions. Second, rather than focusing only on learning as an outcome, we expanded our search to include • Outcome variable: This synthesis focuses on identifying interventions that are most cost-effective in credible proxies for learning; this change allowed us to include many more studies of early childhood interventions. improving learning in basic education, measured in terms of learning core cognitive skills (typically, literacy Third, we proposed an additional classification of smart buys: “Effective but relatively expensive.” Finally, we placed and numeracy in school-aged children, and cognitive proxies in early childhood). In addition, the Panel even greater emphasis on scalability, giving higher weight to interventions that had been studied or implemented at analyzed the impact of interventions on enrollment, attendance, and drop out. larger scale and had been successfully adopted within government systems. • Learning and equity: Contrary to what is sometimes assumed under a dichotomous “access vs. learning” view, a focus on learning as the outcome variable in this context is a tool for improving equity, inclusion, and opportunity. Cost-effective Approaches to Improve Global Learning Levels • Evidence base: The evidence in this note focuses on rigorous causal studies. This includes randomized What does recent evidence tell us are “Smart Buys”? controlled trials as well as quasi-experimental research (see Appendix A for more detail). In addition, the panel reviewed high-quality systematic reviews, meta-analyses, and work that was produced to feed into the Panel’s deliberations.1 We group educational interventions and categories of interventions into the following tiers, reflecting their cost- effectiveness at improving learning and the strength of the evidence: • Cost-effectiveness vs. effectiveness: Wherever possible, the Panel prioritized evidence of cost- effectiveness in making its determinations. Education systems face budget constraints, and they need to GREAT BUYS These interventions are highly cost-effective and are supported by a strong body of evidence. allocate scarce resources toward whatever interventions will deliver the most learning gains for the most GOOD BUYS There is good evidence that these interventions are cost-effective. children and youth on a given budget. However, because many evaluations lack cost data, we also draw on studies focused solely on effectiveness. In these cases, the assessments of intervention costs have to be EFFECTIVE BUT RELATIVELY EXPENSIVE There is good evidence that these interventions are effective, but they more subjective, drawing in part on costs from other studies of similar interventions. are relatively expensive to deliver. They may be appropriate for systems with large budgets or to achieve non- education objectives. • Scale: The Panel has more heavily weighted the interventions that have been proven effective at a larger scale—whether systemwide or, at a minimum, in hundreds of schools. The areas that appear promising but PROMISING BUT LIMITED EVIDENCE For these interventions, there are some rigorous studies that show high have not yet been assessed at scale are included in the “promising but limited evidence” category. effectiveness or cost-effectiveness, but either evidence on cost-effectiveness or evidence at scale is limited. • Duration of impacts: In making its assessments, the Panel gives more credit to interventions that have BAD BUYS Strong, repeated evidence shows that these interventions have not worked in the past in many demonstrated long-term positive impacts. situations or are not cost-effective. • Interpretation of evidence: Classifications are not a counting exercise; rather, they reflect principles drawn from the evaluation results, combined with other knowledge about learning and behavior drawn PROMISING EFFECTIVE BUT from research in the fields of Education, Psychology, and Development Economics. G R E AT B U Y S GOOD BUYS BUT LIMITED R E L AT I V E LY BAD BUYS EVIDENCE EXPENSIVE These There is good For these There is good Strong, repeated interventions evidence interventions, there evidence evidence shows are highly that these are some rigorous that these that these 1 One input into the Panel report was Angrist, Evans, Filmer, Glennerster, Rogers, and Sabarwal (2020), which estimates cost-effectiveness in terms of the Learning-Adjusted Years of interventions are studies that show interventions are interventions have Schooling (LAYS) achieved by specific interventions. The LAYS indicator, the education measure incorporated into the Human Capital Index, provides a common metric that improves are supported by high levels of not worked in on the standard-deviation measure used in much past comparative research. In Angrist et al (2020), the preferred metric for cost-effectiveness is LAYS per $100; this measures how a strong body of are a relatively the past in many many years of high-quality schooling can be gained for an additional $100 spent on each intervention (see Appendix D for details). However, this report also includes many additional evidence. but evidence on expensive way to situations or are studies based on the systematic review as described earlier. deliver learning or examples of outcomes. implementation at scale are lacking. 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 12 Smart Buys Smart Buys 13 How to Use This Note The importance of providing good information on effectiveness and cost- effectiveness There can be substantial gains in children’s learning when systems shift from less effective to more effective education programs. Most education spending in developing countries is by the governments themselves. Audience Therefore, a cost-effective use of aid could be advising partner governments and other donors to invest their This note is intended to be helpful for technical staff in Ministries of Education, donor agencies, financing toward Great Buys or Good Buys, or toward the system reforms described below, and to design and local education groups, implementers, and non-profit organizations in thinking through appropriate implement those interventions and reforms in a context-specific way. (see Appendix A for more detail.) interventions. The global evidence it presents should be used alongside analyses of education challenges locally. The classification and descriptions in this note aim to offer a greater sense of prioritization and clarity than has been possible in the past. EDUCATION SYSTEM Context G R E AT B U Y S Context is often crucial in determining whether an intervention will be successful or unsuccessful. These interventions are highly cost-e ective and Contexts vary by the stage of development of a country, but they also differ considerably within are supported by a strong body of evidence. countries. To be used effectively, therefore, this guidance should be combined with assessing whether this intervention addresses a need in this particular context and whether the intervention can be implemented well at scale. GOOD BUYS There is good evidence that these interventions are cost-e ective. Implementation In addition to overall effectiveness and cost-effectiveness, there is sizable variation in effectiveness of interventions within categories. For example, providing information on the returns to education PROMISING BUT can be highly cost-effective, but has high variance, whereas structured pedagogy can also be highly LIMITED EVIDENCE For these approaches, there are cost-effective, but has less variance. An intervention’s placement in a category does not mean a new some rigorous studies that show high intervention in a new country will have the same impacts as the evidence described in this report. levels of e ectiveness, but evidence Context-appropriate design is crucial, and we therefore encourage readers to use the evidence as a on cost-e ectiveness or examples of implementation at scale are lacking. starting point for national and local discussions, using insights that can only be gleaned from deep knowledge of particular contexts. EFFECTIVE BUT R E L AT I V E LY E X P E N S I V E Implementation fidelity is a critical element of program impacts. In some settings and under some There is good evidence that these interventions are e ective, but they OTHER delivery models, cost-effectiveness could be high, while in other settings it might be much lower. are a relatively expensive way to INTERVENTION Recommendations in this report should not be read as universally applicable. Rather, they should be deliver learning outcomes. AREAS used as a guide to understand the existing evidence and where interventions have worked well, and Where governments should then be adapted to local settings, and implemented carefully, with a focus on feasibility and need to make decisions, but scalability within a given government system. evidence on how to do it e ectively is Interpreting zero-size effects limited Effects statistically not different from zero must be interpreted with caution. Sometimes, they are mistakenly understood as conclusive evidence that something would not work. But we may find zero- results for many reasons, including imperfect implementation, stakeholder opposition, low or poor BAD BUYS compliance, or other binding constraints. A zero-result could even emerge just because the study Strong, repeated evidence shows that does not have sufficient statistical power to detect small changes. The Panel’s judgments in this report these programs have not worked in reflect this (for example by putting less weight on less precise studies), but readers are also advised to the past in many situations or are not cost-e ective. keep this in mind when they contextualize these findings and suggestions. 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 14 Smart Buys Great Buys 15 The importance of systemic reform Great Buys Although this note focuses on the impacts of specific interventions aimed at improving learning, interventions are not all that matters. To drive system-wide The interventions in this category are likely to be highly improvements in learning and make them sustainable over the long term, systemic reform is likely to be extremely important. Ensuring learning for all children and youth cost-effective, either because of their large benefits, or requires an education system that is coherent and aligned toward learning. Which because of their low costs. The cost-effectiveness of these reforms should be prioritized will depend on the specific context, but this alignment interventions can be an order of magnitude greater than toward learning should encompass the key actors, policies, incentives, pedagogy, and for interventions in even the Good Buys category. capacity. This in turn requires political commitment to help systems escape low-learning traps—and it requires commitment not just from education ministers, but also from the heads of government. The Brazilian state of Ceará, for example, of schooling, not general encouragement to consider has made remarkable recent gains in learning during more than a decade of reforms, rising to become one of the Providing information on education positively. country’s top-performing states, despite also being one of the poorest. the benefits, costs, and quality of educatio n Context Yet even without systemic reform, interventions like the Great and Good Buys described below can still substantially improve outcomes for millions of children and youth. They have improved learning at scale, often in systems that This can be effective where specific, locally relevant are not yet well-aligned toward learning. To maximize the chances for sustained success, policymakers should take information of decent quality from a trusted source is several factors into account when they implement interventions, such as complementarities across interventions, available. The delivery method of the information (for dynamic complementarities, and the role of interventions in advancing or inhibiting systemic reform. example, text messages or meetings) must be tailored to the country’s specific needs. Also, recipients must One major element of systemic reform, so comprehensive that it is hard to evaluate rigorously, is realigning the have the means to act on the information; for example, curriculum, assessment, and examinations—and the overall orientation of the system—away from elite students, there must be schools nearby so that families who are and toward the actual skill distribution in the entire student population. Education systems in many LICs and Providing information on the benefits, inspired to keep their girls in school are able to do so MICs focus on schooling for the elite, at the expense of most students. This tendency may be compounded by costs, and quality of education safely; and communities that receive the information international benchmarking against elite schools in other countries and by aspirational standards. Overly ambitious need to have enough access to decision-making curricula, textbooks, and exams are the practical result of this focus on the elite. Many of the most effective structures to spur action. Providing information to parents and children on the interventions, including some that are included in the Great and Good Buys lists below, attempt to solve this income-earning benefits of education (where these curricula problem by going around it—for example, by providing catch-up classes, or encouraging teachers to are not known or not prominent in people’s minds), use accurate assessments of student learning to reorganize the lessons. However, if there is political appetite for on sources of funding available, and on the quality of systemic change, addressing the curriculum and learning standards head-on could be highly cost-effective. It is local schools has increased attendance and learning at not possible to cost out this type of change in the way that other interventions are costed, and it does require new low cost. This information can be shared through text materials and retraining of teachers, which could involve considerable outlays; but given the impacts are felt by all messages or videos (Chile and Peru), parents’ meetings students in the system, the cost per student is likely to be low. (Madagascar, Chile, and the Dominican Republic), or school report cards (Pakistan). In Mexico, information Governments should also consider investing in independent and good-quality measurement of student learning on the income benefits of education improved learning outcomes. Better data on learning outcomes can create scope for improved diagnosis and specific remediation outcomes, but not dropout rates, with larger impacts for through increased visibility of systemic shortcomings and even effectiveness of interventions. girls. An extension of this idea is providing information on student learning to educators; this proved very cost- effective in Argentina, though not in India, and not by itself in Liberia. These interventions have been tested at large scale, with a low cost per child when delivered at scale. Note that this is about providing specific and context-relevant information that shifts people’s beliefs about the benefits of education or the quality 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 16 Great Buys Great Buys 17 Context Supporting teachers with structured pedagogy (a Structured Pedagogy Implementation Details package that includes Countries seeking to improve pedagogy may want structured lesson plans, to use structured support of teaching to achieve Structured pedagogy is a packaged intervention typically based on gradual, step-by-step introduction of skills in learning materials, and a transformation in how foundational learning is an order consistent with how children learn. It incorporates practical activities designed to build specific skills, ongoing teacher support ) delivered, and strengthen teacher professionalism and teacher training, mentoring, and materials. Typically, teachers are provided lesson plans and/or teaching guides skills. A version with less structure could be utilized that can illustrate step-by-step learning activities, important lesson routines, concepts, and sometimes common when teachers have strong content knowledge and mistakes students make, and ways to remedy them. Many structured pedagogy programs include support high-quality pedagogical practice. A key benefit of for continuous assessment so that teachers are able to revise their instruction based on whether children are structured pedagogy programs is that this approach learning at the expected pace, as well as respond to the range of skills in their classrooms. The teachers’ guides can be effectively implemented even by teachers with range from providing specific lesson instructions to more general scaffolding for teachers, depending on the limited training or poor outcomes, although structured program, but all advise teachers on how to sequentially and incrementally build on students’ existing skills Supporting teachers with structured approaches have also been effectively used in high- through teacher demonstration, teacher and students practicing together, and independent practice activity. pedagogy income countries with well-trained teachers (U.S.). The integration of the lesson plans and student learning materials (typically including textbooks and sometimes Lesson plans and materials should be designed to including workbooks) is an important part of the approach. Designing context-appropriate teaching material is Structured pedagogy refers to interventions that suit the specific context, pitched at students’ learning a long and iterative process involving experts, teachers, and ministry officials, with extensive piloting that takes include a set of integrated elements that focus on levels, and ensure that teachers are well prepared to into account appropriate pacing and sequencing, and provides guidance on reteaching if children are struggling improving classroom instruction. While a wide variety teach the range of student skills in their classrooms, as and/or differentiate within classrooms. Programs are not one-size-fits-all and instead are designed based on of interventions call themselves structured pedagogy well as to respond to students who are struggling or local realities, reflecting the existing knowledge base of teachers and students, pedagogical content knowledge (see this article), we define structured pedagogy find the content too easy. Teachers will often welcome of teachers, gaps in current teaching, and the extent of learning diversity within and across classrooms. The interventions as those with a coherent package the chance to focus on classroom teaching rather than level of specificity in scripting in teacher guides can be reduced over time or with level of teacher proficiency. of investments that include lesson plans, learning preparation; however, in other contexts, teachers may materials, skills-based ongoing teacher training, and resist these programs, so it is important to get their The design of the teacher support elements differs, but typically includes skills-based teacher training and teacher mentoring that are carefully coordinated to inputs in design and buy-in first (for example, including some type of teacher support structures. Teachers are provided ongoing teacher training that emphasizes how reinforce each other. This approach has increased teachers in the development of the program, and to practically implement lessons in the classroom rather than discuss abstract concepts. As many teachers will learning at relatively low cost in Kenya-1, Liberia and providing clear communication on how this intervention adapt lesson plans to students’ needs as well as their own, they are trained in how to do this well. Instead of South Africa-1, especially improving foundational literacy can improve teachers’ pedagogical knowledge and by introducing many new elements at once, teachers spend time practicing the pedagogical techniques. During and numeracy. Most of the effective interventions providing teachers additional support). training, these structured pedagogy programs should be presented as an aid to teaching which frees teachers provide support and clear guidance to teachers about to focus on instruction, student needs, and creativity. what to cover in what order, but do not involve fully scripted lessons (Kenya-1, Liberia and South Africa), In addition to training, teachers are supported with frequent feedback by trained coaches based on in-class although there are cases where detailed scripts and observation and, in some cases, communities of practice to engage with other teachers in the process. daily lesson plans proved effective (Kenya-2). (For more Teachers are given an opportunity to reflect on their experiences and troubleshoot their challenges. This detail see the Implementation Box). The support in improves teachers’ confidence in applying skills. It is important that the intervention has sufficient buy-in at all teaching is meant to ensure that classroom instruction levels of the government and there is no conflict between what is covered in the program and other parts of systematically builds on previous lessons with clarity, the education system (e.g. curriculum, exams, teaching time etc). minimizing the risk that students misunderstand. Each component of the structured pedagogy package is designed to complement the others, and it is this interaction between package components that is essential to program impact (Kenya-3, South Africa). The approach has been introduced at large scale (nationwide in Kenya-4, large scale in Uganda and in several provinces in South Africa-1, 2, 3, 4). 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 18 Great Buys Great Buys 19 School closures due to the COVID-19 pandemic have Targeting teaching instruction by learning increased the variation in learning levels within many classrooms, making targeting instruction by learning Targeted instruction – Implementation Details for One level, not grade (in or out level even more important in many contexts. This Approach of school) approach has been shown to be effective in primary grades, especially grades 3-5. Most interventions target For any intervention to be effective, it must be implemented well. While many versions of targeted the mastery of foundational reading and math skills. instruction have proven to be effective, this box highlights the implementation details for Teaching at the Right Level (TaRL ), one type of targeted instruction program that has been shown to have positive impacts TM Targeted instruction programs like Teaching at the Right Level (TaRL) have been implemented in a variety of at scale. settings and evaluated rigorously (India-1,2,3,4,5). These include conducting TaRL programs in school during Developed by the Indian NGO Pratham, TaRL focuses on ensuring that all children learn foundational school hours, in-school in additional working hours reading and math skills. For a variety of reasons, many children get “left behind” and are not able to Targeting teaching instruction by during summer vacations, and in community settings. cope adequately with grade level curriculum. In the TaRL approach, grade level curriculum is put aside learning level, not grade (in or out of Implementation has been done by teachers, supported for some time each day, so that instruction can start at the level of the child. This enables acceleration school) by their superiors, and by community volunteers (paid in learning and assists in enabling the child to “catch up”. TaRL focuses on basic literacy and math goals and unpaid). There are models which have an in-school for children aged 7-8 years and above. Depending on class size, TaRL programs group children by their Implementation approaches include providing targeted component as well as a community component. current learning levels either across grades or within the same class. In India this is usually done across help for students who are falling behind, and grouping grades 3, 4 and 5, but if class sizes are large, this grouping can be done within the existing class. children for all or part of the day based on their learning level rather than their age. This can be done Teachers conduct simple one-on-one oral assessments with every student and then group them by their with government teachers, volunteers, or teaching current learning level. The assessment is aligned to instructional practice, and helps to organize instruction assistants (see section on “Augmenting teaching and track students’ learning progress. These assessments are easy to implement and interpret, and can teams with community-hired staff”), and implemented be done periodically to regroup children as they progress (the ASER tool is often used in these programs). during school, through after-school make-up classes, Data generated through this process are easy to aggregate, display and discuss, and is easily understood or during holidays. Recent evaluations show that by instructors, teachers and parents. targeted teaching instruction has been increasingly implemented by government teachers. A version of this TaRL instruction consists of simple and engaging daily learning activities appropriate for each learning approach that includes an interactive pedagogy has level/group. TaRL incorporates level-appropriate, low-cost material for teaching (e.g., story or word cards been developed and rigorously tested at large scale in India and then tested in thousands of schools in or letter charts), which can be used by different- level children/groups in different ways. Ghana and implemented at scale in Cote d’Ivoire and Zambia (see here for an overview of aspects of the most Teachers implementing TaRL are trained by “leaders of practice” who themselves have conducted effective, scalable model). A less-intensive approach practice. Ideally, these trainers also provide on-going, in-class academic and practical support, and thus is to introduce tracking, where children are grouped also serve as mentors and monitors. Learning data from assessments are reviewed frequently at the by their initial level of learning without any change in sub-district level to inform wider program delivery and course corrections, and the findings on children’s pedagogy. This was highly cost-effective in Kenya, but progress are also discussed frequently with the teachers. As with other learning improvement programs, met with resistance in India-1. Although conventional alignment with other elements in the system (training, monitoring, support from others higher up in the targeted teaching instruction interventions do not use hierarchy) provides better support to classroom practice and helps to prioritize the program in the school technology, we discuss below how technology can be system. an effective medium for delivering personalized content at students’ learning levels (see sections on software that allows personalized learning and use of mobile phones). Context These interventions are effective in contexts where there is a wide variety of learning levels within a class, and student learning levels are below grade-level curriculum expectations (as in many LICs and MICs). 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 20 Good Buys Good Buys 21 community level, rather than contexts that would rely on from many different contexts that pre-primary education Good Buys home visits. When implementing programs with home visits, leveraging existing community health workers positively impacts learning and cognitive development for children ages 3-5 (Meta analysis-1, 2). Several of could decrease costs. these studies have tested low-cost models at a large scale by leveraging existing infrastructure and improving There is good evidence that the interventions in this quality by implementing multi-component interventions, Providing quality category can be highly cost-effective across a variety of pre-primary education (for including teacher training or hiring (India-1, Kenya- 1) or expanding the current infrastructure (Uruguay, which contexts. ages 3 to 5) utilizes national systems), and have found positive impacts on learning, suggesting that these interventions can be implemented effectively at scale. Another model programs that target parents can have at least medium- to increase access at scale is the construction of new Providing parent-directed term impacts; although there were no remaining benefits pre- primary school facilities. While newly constructed early childhood stimulation programs (for ages 0 to 36 in Colombia-2 two years after an intervention, benefits facilities led to a sizable increase in school participation months) persisted two and four years later (Pakistan, China-3, in Argentina-1 and higher cognitive skills in Mozambique, Jamaica-4) and into adulthood (Jamaica-5). it reduced child cognition in Cambodia, where children switched from attending primary to the new pre-primary Context Providing quality pre-primary education schools. (ages 3 to 5) Unlike primary and secondary education, most countries Some studies have found short-term but not long-term do not have established infrastructure or dedicated Providing children with high-quality pre-primary impacts (Brazil, Kenya-2, Indonesia, and India-2,), while resources for large-scale programming directed at education has the potential to improve their learning other studies have found both short- and long-term parents. However, some have leveraged existing outcomes and provide long-term economic benefits. effects, such as higher levels of enrollment in future Providing parent-directed early childhood platforms that serve large populations of disadvantaged There is substantial evidence that millions of schooling (Denmark, Uruguay, U.S.-2), and more rapid stimulation programs (for ages 0 to 36 families, like social protection-based platforms (e.g., poor children show lower language and cognitive cognitive development and learning in primary school months) Colombia-1, Rwanda). Healthcare delivery platforms, development levels than their better-off peers over the (Argentina-2, and Mozambique). Though increasing pre- prominent in the first 1,000 days, are another promising first five years of life, increasing the risk of inequality primary school attendance does not necessarily improve Early childhood stimulation programs that train parents approach for at-scale delivery (as has been done in to grow (Report-1, 2). The deficit becomes quite large cognition or school readiness (China, Indonesia), it to provide more stimulation to their children can Bangladesh). Still, whether these programs must be by the time these children enter primary school, suggests that dynamic complementarity between generate benefits that last into adulthood. Reviews provided universally or targeted to specific populations which is likely to affect their ability to benefit from pre-primary and primary education in the U.S. may be have consistently found strong evidence of short-term (e.g., more disadvantaged, malnourished) is unclear. standard schooling. Many scholars have emphasized relevant for LICs and MICs. impacts on children’s cognition and language among There is some indication from a parallel randomized the importance of learning in the early years, and disadvantaged populations (Report, Literature review, controlled trial in 3 countries: India, Pakistan, and many countries are currently expanding coverage of In addition to studies examining the effects of increasing Meta analysis-1, 2, 3). Previous studies found gains in Zambia, suggesting that children from low-resource pre-primary education. Rigorous literature shows that access to pre-primary education, there is growing educational attainment, IQ, mental health, and earnings families benefit in cognition from a home-visiting pre-primary interventions can have important long- evidence on approaches to improving quality. Classroom in adulthood of a home visiting program focused on parenting intervention, but those from high-resource term economic benefits in high- and middle-income quality in kindergarten is associated with later learning early stimulation (Jamaica-1, 2). Moreover, home visiting families do not, thus reducing the wealth gap in child countries, provided that children attend regularly and improvements and educational attainment (Ecuador, stimulation programs have generally shown initial development. that the classroom experience is better than what U.S.-3). There is some evidence that efforts to improve benefits in child development (Bangladesh, China-1, children already experience in the home concerning preschool quality through teacher training programs Colombia-1, India-1, and Pakistan) and have been While there is enormous potential for countries to early stimulation and social-emotional support. There is (Colombia, Jamaica, Ghana, Peru) and/or changes in effective at scale (Peru). Additional models that leverage invest in and cost-effectively improve early cognition evidence from the US-1 of the dynamic complementarity pedagogy (India-2) can improve learning and cognitive community members to deliver home visits may be and language through early childhood stimulation of such programs, with the effects of pre-primary outcomes, as well as bring socio-emotional benefits, particularly scalable (Colombia-1 and China-2). programs that target parents, it is essential they do so education on adult earnings and educational attainment though adding an extra teacher has led to mixed results with monitoring and evaluation systems to understand compounding when followed by quality primary (India-3, Thailand, Colombia). Together, these studies Furthermore, there may be ways to reduce delivery costs. how to target the program best. Furthermore, early education systems and vice-versa. The evidence on suggest that pre-primary education typically improves For example, group sessions have been shown to be at childhood stimulation programs that target parents may the long-term impacts of pre-primary education for learning for children of ages 3-5, and it can be done least as effective as hybrid or home visit-only models be particularly cost-effective in contexts with cultural low-income countries is less extensive. However, there cost-effectively. Additional evidence on making such (Kenya, India-2). Additionally, early childhood stimulation support and infrastructure for group delivery at the is now substantial and relatively consistent evidence 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 22 Good Buys Good Buys 23 gains persist beyond pre-primary years would be a key increased both access to schooling and long-run labor- Giving merit-based Administering area for future research. market outcomes), but it is often an ineffective and scholarships to school-based mass inefficient way of achieving the goal of increased access disadvantaged children deworming where Context because it is expensive and is not always well targeted and youth worm-load is high to the neediest areas. Fortunately, there are other, more Gains made in pre-primary education may fade cost-effective ways to reduce travel time and increase over time if children transition to low- quality or low access to schooling. In areas where many children conceptually and programmatically integrated primary live far from a school, setting up community schools education programs—making pre-primary investments in existing community buildings or houses increases seeming less cost-effective. However, those are the school participation and learning, and does so at lower very settings in which expanded pre-primary education cost than building new schools. Establishing new may be most needed in improving learning outcomes. village-run schools with community teachers in rural Since poor children tend to arrive at school with lower northwestern Afghanistan increased enrollment and Giving merit-based scholarships to Administering school-based mass cognitive and language development levels, creating test scores among all children, particularly among girls. disadvantaged children and youth deworming where worm-load is high challenges in the classroom from the earliest grades of (However, it is important to note that the sustainability of primary school, improvements to preschool and primary community schools can be an issue, so there needs to Need-based aid (for example, through conditional cash Mass treatment for parasitic worms has been shown education would ideally go together. Moreover, given be effective mechanisms for supporting them over the transfers) can be useful for getting children to continue to increase child attendance at school in contexts with the dynamic complementarity observed in both high- long term.) Improving education access is not just about in school—especially at the secondary level, where high worm-load. Deworming has been implemented and low-income settings, further testing of pre-primary setting up new schools. In many cases, access can be families still incur costs (including opportunity costs), nationwide in many countries and studied at scale interventions in the context of weak primary education improved by providing transport. For example, studies in even for public schools. But need-based aid alone can (Nigeria). In contexts with high worm-loads, school- would be helpful in fully understanding this link. The India and Zambia have found that providing bicycles to fail to lead to learning gains, as was found in Cambodia. based mass deworming improved attendance and quality of pre-primary education also matters a lot: adolescent girls led to significant improvements in both Merit-based scholarships, cash payments, or prizes labor market outcomes (Kenya, early 20th century interventions will be cost-effective only where they can school participation and learning outcomes. It is also targeted at disadvantaged children and youth can act USA, Nigeria, Cambodia). It has been shown to be provide more stimulation and care than the status quo cheaper than building a large number of small schools. as a complementary incentive to improve attendance cost-effective for improving attendance at schools. In (whether home-based care, private preschools, or early and student effort, resulting in higher learning outcomes contexts where the prevalence of worms was lower, attendance at primary schools). Context within the mainstream school system. In Kenya, deworming had no effect on attendance (China and scholarships were provided to girls who performed Sri Lanka). In a high-prevalence context, a preschool This can be effective where marginalized, hard-to-reach, well in their 6th grade exams. Ten high-quality studies deworming program had no measurable impact on Reducing travel times to or conflict-affected children (especially girls) live far from in 7 other countries further support the finding of the enrollment or test scores 7 to 12 years after treatment, schools school, and school participation is low. The appropriate effectiveness of such incentives. Long-term impacts on but the study was not designed to detect small effects response depends on safety considerations (e.g., the learning were also found for merit-based programs in (Uganda). The question of precisely how high the worm response in Afghanistan involved providing schools in Cambodia. burden needs to be for deworming to measurably existing buildings, while in very local contexts, bicycles improve attendance, and the impact of deworming on were used). Context test scores, is not settled. This approach can be helpful in areas where it is Context possible to design scholarships so that they do not only go to already highly advantaged students (for example, Worm-load, not just prevalence, is a key determinant of where inequality is low; or where scholarships go to the intervention impact. The impact of delivering deworming Reducing travel times to schools best-performing students from a disadvantaged group). drugs through means other than schools has not been On the other hand, where the school system as a whole studied and, therefore, its effect and cost-effectiveness Where access to education is low, improving access is failing the typical child, this approach may not be the are unknown. The drugs used in the studies have been to schools increases children’s schooling and can also best way to tackle this wider problem. Note that this Albendazole or Mebendazole or Praziquantel, with improve their learning. The approaches summarized intervention does not include voucher programs that dosage ranging from 400 mg to 600 mg. The frequency here and which make this intervention cost-effective move students from public to private schools. of the delivery has varied across interventions, but did not involve constructing new schools. School has been at least twice a year. Delivery is particularly construction can pay off in settings where there are low cost and administratively straight forward where no schools nearby (for example, Indonesia’s program deworming can be layered on school feeding. 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 24 Promising but Limited Evidence Promising but Limited Evidence 25 Note that these computer assisted learning programs Promising but Limited Evidence Augmenting vary in content—some teach to the level of the child, teaching teams with others remediate but within the curriculum, while others are based on class-level teaching. As there is very limited evidence at scale or with governments for this For these interventions, the evidence is limited, either intervention, it is categorized as “Promising but limited because there are no examples showing the approach can evidence.” be implemented at scale, or because evidence on cost- Context effectiveness is missing. More testing to develop scalable models is recommended. This approach is relevant only where there is electricity, internet connection, teacher training, and widespread availability of hardware—including lower-tech devices Augmenting teaching teams with in the home—to make this doable at low-cost and in a community-hired staff way that is inclusive, and where the software has been A note on ed-tech shown to be well-designed for learning. This would be To improve access to education, LICs and MICs very costly in settings without pre-existing hardware governments (with constrained budgets) may consider Investing in education technology hardware alone is almost always a bad use of funds. What matters is infrastructure, and governments in these settings augmenting the teaching team to include not just how the technology is used by teachers, students, parents, and administrators. Purchasing new hardware should prioritize investing in other programs highlighted permanent regular teachers, but also community like tablets or computers is expensive and does not improve learning outcomes on its own (see section on throughout this report to improve learning outcomes in a teaching assistants, apprentices, or volunteers. Indeed, “Investing in hardware like laptops, tablets and computers alone”). However, existing technology devices can more cost-effective way. Furthermore, this intervention an expanded learning team is common in countries at be used as a medium to effectively deliver quality content and improve learning outcomes when implemented has more evidence from high-income contexts and many levels of development. Delivering primary-level carefully. Education technology should complement and enhance teachers’ engagement and not seek limited evidence at scale or with government partners. learning by expanding the team with community- to replace it (see here and here). Throughout this report we discuss evidence related to specific uses of Testing these programs at scale, with government, and level hires has been cost-effective in many settings in LICs should be a priority for future research. New (India-1, 2, Kenya-1), but implementing this approach at research should seek to evaluate programs that replace scale requires careful design and customization with targets learning at the level of an individual child can regular instruction with computer-assisted learning. attention paid to political considerations. Students be cost-effective if computers that can use the new taught by community teaching assistants had similar Using software that allows software are already in place and can be maintained. (India-1) or higher test scores (Kenya-1) than students personalized learning and adapts to the learning Most evidence comes from out-of-school programs taught by regular teachers. Hiring additional local level of the child (where which avoid crowding out other learning. Studies education facilitators on part-time contracts was also hardware is already in from Cambodia, China (1, 2, 3, 4, 5, 6), India, and found to increase average instructional time and schools) some advanced countries using different levels of improve learning outcomes at a very low cost in Indian personalisation and adaptiveness have shown these Early Child Development programs (India-3). Often, programs were effective when implemented after those hired at the community level have lower formal school. The evidence of using computer-assisted qualifications or less experience and thus lower pay learning during school is less robust. While Uruguay than permanent in- service teachers. But, a study in implemented this approach at scale, with suggestive India found that these teachers might make up for lower evidence of positive impacts that were larger for formal training with higher effort (India-1). Moreover, in Using software that allows students from disadvantaged backgrounds, a program Kenya and India, these teaching assistants were present personalized learning and adapts to implemented by the government in China had no in school more frequently than permanent in-service the learning level of the child (where impacts on learning outcomes. teachers. In the Indian state of Tamil Nadu, an at-scale hardware is already in schools) initiative hired over 200,000 volunteers to deliver While any degree of personalization (see here) can remedial learning after school hours to over 3 million Computer hardware is not only expensive but school be helpful, higher degrees of personalization and children as a part of a systemic initiative to address systems often struggle to integrate the technology adaptiveness had larger impacts (India, Cambodia, Meta COVID- 19-induced learning losses. The program was into their teaching and learning (see “Bad Buys”). analysis). One study in China found that it was possible both effective and cost-effective, and it also improved However, using adaptive or self-paced software that to reduce costs by using tablets with peers in pairs. equity. The improvements in equity were driven by 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 26 Promising but Limited Evidence Promising but Limited Evidence 27 children from less-advantaged backgrounds being more challenge. Multi-micronutrient supplementation appears Providing mass school-based Leveraging mobile phones likely to participate in the program (India-2). more effective than providing one or two micronutrients treatment of specific health to support learning conditions on their own (there were positive impacts in China- 7 Context and India-1 but no impact in Sri Lanka-1, Sri Lanka-2, India-2, Ethiopia, and Tanzania). Expanding the teaching team to supplement regular classroom teachers, though cost-effective, may face Mass preventative malaria treatment where incidence political resistance. The acceptability of the approach is high has improved attendance and test scores (Sri may well depend on whether there is a strong tradition Lanka), cognitive skills (Kenya and Uganda), and some of community assistant teachers locally, whether measures of concentration (Mali). Screening and permanent teachers see them as helping them or treatment programs are more expensive and have been replacing them, and what the pathway is to becoming less effective: a researcher-implemented program did a permanent teacher. There is considerable variation Providing mass school-based not improve numeracy or cognition (Kenya). Asking Leveraging mobile phones to in qualifications, remuneration, pathways of hiring, and treatment of specific health teachers to screen children for malaria and then support learning duration and nature of contracts in countries where conditions administer treatment was also not effective in reducing expanding the learning team with community-level malaria or improving education outcomes (Malawi). Leveraging technology that is already available in many hiring exists (see discussion from Sub-Saharan Africa). Children with untreated health conditions may miss Mass treatment in highly endemic areas is thus more households, like mobile phones, can be a cost-effective In Kenya, the government came under pressure to school and/or learn less well in school. While many cost-effective, but evidence at scale is lacking (the tool to reach students and their families and improve make contract teaching assistants hired by communities conditions require individualized diagnosis and largest positive study had 1,300 students). learning outcomes. Sending short text messages via permanent, not least after they had been unpaid treatment and are best provided through the health mobile phones to nudge engagement with content has for many months. Thus, scoping what alternative system, a few can cost-effectively be delivered through School feeding is often thought of as a health reduced dropout rates (Brazil) and improved learning pathways are possible in a given context and proper schools or preschools while also improving attendance, intervention, but most evidence is about its impact (Costa Rica). In addition, informing parents about their implementation is crucial. learning, or both. There are multiple studies on the on attracting children to school. We therefore do not pupil’s performance in class via text message (Chile) has cost-effectiveness of mass screening and provision include it under health (see “Effective but relatively improved learning outcomes. However, in Botswana, Professionalization of teachers is likely to be crucial of eyeglasses, multi-micronutrient supplementation, expensive”). While some studies do report a positive using only text messages to deliver numeracy questions in recruiting and motivating permanent teachers. The and preventative treatment for malaria, though only impact on learning when malnutrition is common, multi- was not effective in improving test scores unless it was evidence reviewed here focuses on community-level eyeglasses have been tested at scale (China-1). micronutrient supplementation has a stronger cost- accompanied by live tutoring calls. hiring as a supplement rather than as an alternative to effectiveness evidence base than school feeding. the existing teacher workforce, and does not assess Mass testing and distribution of eyeglasses to students Reaching students at home through tutoring phone how these community approaches do or do not alter with refraction errors improved test scores in two Context calls, in addition to text message approaches, is also who becomes a permanent teacher. separate studies in four different locations in China cost-effective. For example, personalized phone calls (China-1, 2), although positive impacts were not Mass treatment is an effective approach when and instruction delivered by teachers or volunteers found in all counties. The benefits were greater for prevalence in the area is high, treatment is extremely improved learning in multiple settings (Botswana, underperforming students (China-1). Poor quality of safe, and/or individual diagnosis is more expensive or Bangladesh-1, and Nepal). In some contexts, this low- schools might explain why glasses did not improve complex than treatment. As with many interventions that tech approach was more effective for children with learning everywhere (China-2). At least one study was improve attendance, the impact on test scores is likely lower initial test scores, less educated parents, and for large-scale (2,500 schools). In these studies, eyeglasses to vary with the quality of teaching. A key contextual low-income families (Bangladesh-1). Implementation have been shown to be highly cost-effective at factor for eyeglasses is the cost of the program and fidelity has been shown to improve with repeated increasing learning. Providing nutrient deficient children whether children are willing to wear free glasses (40 to experimentation in delivering foundational literacy with multi-micronutrient supplementation daily at 50% of students refused to wear glasses in studies). For and numeracy over mobile phones when implemented school/preschool improved test scores, early cognition, malaria, programs that attempted to include diagnosis by NGOs as well as by government teachers (see and later-life literacy at low cost (China- 4, China-5, were less effective, but mass treatment for malaria here). Mobile phones can be combined with targeted China-6, Bangladesh, and Guatemala). Supplements without diagnosis is most appropriate in contexts and instruction (see section on targeting teaching instruction were sourced locally, always included multivitamins, and seasons when malaria is highly prevalent. by learning level, not grade) to facilitate cost-effective often included iron, zinc and iodine. While cost-effective, delivery. When instruction is not targeted, phone evidence on implementation at scale is missing: one calls alone might not work (Sierra Leone). More study with 2,400 participants had no effect on cognition automated phone call approaches, such as audio-based (Tanzania). As supplementation has to be daily, this is a 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 28 Promising but Limited Evidence Promising but Limited Evidence 29 interactive voice response have also been effective in In Jamaica, an intervention that taught teachers Teaching girls life skills like negotiation, leadership, important for their success, as evidenced by the study in Bangladesh-2, although not in Ghana. ways to develop positive relationships with high-risk conflict resolution, and critical thinking has been shown China. preschoolers using praise and rewards led to a higher to improve their enrollment rates and labor market Context attendance, language, and math scores. In addition, a participation (Zambia, Uganda, India). These skills have government-run program in Peru that raised awareness been taught through structured in-school curriculum Targeting interventions Mobile phones do not work by themselves; they are about bullying and encouraged reporting bullying led (for skills like negotiation, problem-solving) as well towards girls a tool that can connect educators with students and to a decrease in dropout rate but did not affect test as through activities in after-school clubs (for skills deliver targeted instruction, and can be particularly scores. A school-based, gender-related, violence like leadership and conflict resolution). Additionally, useful in remote areas or during school disruptions. More mitigation program in Uganda, aimed at retention games that taught self-regulation in Kenya improved evidence is needed on evaluation and at-scale adoption in primary grades, did improve student retention. children’s inhibitory control task scores, but had no by governments in public schools. Given many of these Evidence is needed on which of the violence mitigation impact on most other executive functions. Most of these interventions were tested as substitutes for regular interventions are most effective and cost-effective, as interventions did not improve test scores; however, schooling during the COVID-19 pandemic, evidence is well as the best way to incorporate effective approaches encouraging competitive behavior (grit) among students also needed on effectiveness when they are implemented into existing, pre- or in-service teacher training or after- in Turkey through curriculum and activities did improve as complements to schooling. Lastly, low-tech devices school clubs. performance in language and math. like phones are well placed to deliver simple activities, and are unlikely to be effective delivering complicated or Context In China, a program that used scripted counseling Targeting interventions towards girls high-competence activities. to reduce anxiety by teaching skills like goal-setting, While all children deserve to be safe, the main threat of emotion management, and self-awareness also reduced Girls’ participation rates remain lower than those of violence is likely to differ by country, age, and gender. dropout rates. However, there was a long-term effect boys in 53 developing countries, with particularly Safeguarding students Different responses will therefore be needed, for only for students already at risk of dropping out, large disparities in West Asia and Sub-Saharan Africa. from violence example, for preventing bullying vs. corporal punishment, especially because most other students lost interest in This is especially true for adolescent girls: gender or for supporting children who have suffered from war- or taking the counseling. disparities are highest at that age, and enabling girls to gang-related violence. stay in school has major benefits (including for health Supportive and predictable environments through outcomes). In other contexts, including much of Latin healing classrooms are designed to help students America, boys participation rates lag those of girls. Many Teaching socio-emotional who lived through trauma or conflict to cope with non-targeted interventions (such as reducing the cost and life skills social and emotional hurdles to learning. However, of school) disproportionately benefit the gender that is the evidence on their effectiveness is scarce and has falling behind (Meta analysis), but in some contexts, girls shown no effect on literacy skills and little to no effect continue to face girl-specific barriers (see this review). on numeracy skills (Sierra Leone, Congo, Lebanon). Safeguarding students from violence While such interventions may affect non-cognitive Some targeted interventions to increase girls’ outcomes for children, like school perception or enrollment have been successful, including programs Despite legal prohibitions, corporal punishment and mental health, more evidence is needed about their teaching girls to negotiate with their parents, enrollment other violence inflicted on and perpetrated by students effectiveness on education. drives that spread awareness about benefits of girls’ are prevalent in many countries. Children deserve education, female-only toilets, take-home rations on to be safe from violence whether or not it improves More evidence is needed on how to implement attending school, and constructing girl-friendly schools, educational outcomes. However, violence may also Teaching socio-emotional and life socio-emotional and life skills development with the highest effect seen for building girl-friendly have educational impacts because it discourages skills programs well and on the cost-effectiveness of schools (India, Zambia, and Uganda-1, 2). However, access to schooling and hinders learning. If students these interventions at scale in both formal and only studies on take-home rations and girl-friendly feel safe, there are likely to be knock-on effects on Evidence on the effects of teaching life skills like non-formal settings, especially as governments are schools examine the adaptability and scaling-up of the enrolement and learning. However, while there is negotiation, conflict resolution, or leadership on increasingly incorporating them into their curricula. interventions. As some girl-targeted interventions (e.g., promising research on effective ways to mitigate students’ academic and later-life outcomes in LICs and construction of schools and washrooms) are quite costly, school violence, evidence in LICs and MICs is nascent. MICs is limited. Similarly, there is limited evidence on Context more evidence is needed on which interventions can In El Salvador, after-school clubs that taught students the educational impact of programs designed to reduce bridge the gender gap cost-effectively. how to be less impulsive and resolve conflict reduced anxiety or support students coping with trauma or After-school social clubs can be most cost-effective absenteeism, improved social behavior, and math conflict. The largest body of evidence is on teaching life when leveraging existing infrastructure. Maintaining and science test scores, but had no effect on reading. skills to girls. students’ interest in these interventions might be 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 30 Promising but Limited Evidence Effective but Relatively Expensive 31 Context Context It is important to understand gender gaps in each This approach may be most promising where power Effective but Relatively Expensive context, because they vary both within and between asymmetries between school authorities and parents countries, and by age. Gender gaps also vary between is not too great, and where there are potential participation and learning: girls may lag in participation complementary sources of accountability for schools There is good evidence that these interventions are but once in school their learning rates may be high. (such as well-functioning local governments to which effective, but they are a relatively expensive way to deliver Understanding the drivers of gender gaps helps identify community members have good access); unfortunately, learning outcomes. appropriate policies: for example, general education these are also the settings where the need for these programs that make school cheap and convenient are interventions might be less acute. useful when girls’ participation is not prioritized but also is not opposed by families, while more gender-targeted Transferring cash (as a tool primary Schools Feeding in Primary schools policies may be needed in other cases. for improving learning) Involving communities in school managemen t Transferring Cash (as a tool for Feeding in Primary Schools improving education outcomes) While school feeding provides a range of benefits, from Involving communities in school Cash transfers have consistently been found to an education perspective, it mainly improves enrollment management have beneficial effects on school participation (both and attendance, likely by reducing the cost of schooling. enrollment and dropout rates) where participation Cost-effectiveness evidence is lacking. Though relatively Providing feedback to schools through community is low, and have been adopted and tested at scale. expensive and logistically challenging to implement, involvement (as has been done in India, Indonesia, Relatively few studies have found statistically significant school feeding has been implemented successfully and Gambia) or gathering better data on teachers and impacts on learning. This may reflect the fact that at scale (Cambodia, India-1) and has been shown to students (Indonesia) has often had little impact. Where the impacts on attendance are relatively small, partly increase school enrollment (Jamaica). A combination of involving community members in school management because many of the studies were done in countries school feeding for all students and take-home rations has worked, however, (as in Indonesia’s alternative where most children were already attending school. for girls increased attendance in both the morning and approach, Uganda, and Kenya), it is very cost- Cash transfers are also expensive (the cost of targeting the afternoon (Uganda). Further, though their cost is effective. One feature of successful interventions, as in cash to the most needy is often costly), further reducing similar, on-site cooked meals were more effective in Indonesia and Kenya, has been explicitly linking school their cost-effectiveness. Cash transfer programs improving attendance than packaged food. In India, committees that involve community members that have are therefore not a cost-effective tool for improving shifting from packaged food to on-site cooked meals high levels of authority. More work in testing various learning. This may be unsurprising, since they aim to increased attendance (India-2). If school feeding is designs is needed to understand when and why this increase incomes substantially and promoting education taking place, it is highly cost-effective to add deworming works, including a study of the composition, government is not the primary objective of the transfers, despite the where parasitic worms are an issue (Cambodia). structures, and complementary mechanisms, all of large literature on their impacts on education outcomes. which appear to be important for effectiveness. Analysis of cash transfers’ impact on Learning-Adjusted School feeding can also improve cognition, mainly Years of Schooling (LAYS) suggests that they are not as where malnutrition is common (Burkina Faso, Ghana, cost-effective as other interventions—although they are Argentina, Jamaica, and India). Providing meals of low effective for other objectives, such as social protection. nutritional value in schools in China (eggs) and Kenya (see examples from Malawi, Mexico, Mexico). (beans and cornmeal) had no effect on test scores 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 32 Effective but Relatively Expensive Bad Buys 33 while meals that had meat or milk in them in Kenya nutrition in otherwise reasonably well-funded and well- improved test scores. However, multi-micronutrient supplementation has a stronger cost-effectiveness functioning education systems are thus good targets for this policy. In many contexts where the cost and Bad Buys evidence base than school feeding (see ‘Promising but logistics of school feeding are easier but enrollment Limited Evidence’). and attendance is already high, school feeding might be seen as a social support (much like cash transfers) There is good evidence that these interventions are Context rather than an education program. The impact of this effective, but they are a relatively expensive way to deliver intervention has been largely tested in primary school learning outcomes. Any of the approaches listed above, Though extensively adopted at scale, school feeding grades. The effects are larger on students in lower if implemented poorly or in inappropriate contexts, could requires a daily logistical effort and cost, which is grades and for girls (India-1, Uganda). challenging in the poorest contexts. Yet it is contexts be classified as “Bad Buys.” But there are also other with low attendance and high malnutrition where interventions where the evidence has repeatedly shown school feeding has the biggest impact. Areas with poor that the approaches—as typically implemented—are either not effective or not cost-effective. While it may not be politically or practically realistic to cut spending on these “Bad Buys,” school systems should strongly consider prioritizing the much more cost-effective interventions. Investing in hardware Rica). This is also true in the U.S. and other advanced like laptops, tablets and computers alone countries, even though the level of computer literacy is higher there. Implementation issues are a recurring challenge in hardware programs; an example is the lack of a coordinated approach in the One Laptop per Child scheme in Brazil. While it is ineffective to invest in hardware alone to improve learning outcomes, it can be cost-effective to improve learning using technology that is already available, such as mobile phones (see section on leveraging mobile phones to support learning). Investing in hardware like laptops, tablets and computers alone Computers and other educational technology are often especially enticing to policymakers and other stakeholders (and are expensive). As discussed below, adding inputs alone has not been shown to be effective at improving learning outcomes. As with other inputs, investing in hardware alone is a bad buy. When not accompanied by well-thought-out complementary measures—including personalized adaptive software and teacher training on how to use the software–adding computers has had no impact at all in Peru, Colombia, and many other countries, or negative impacts on learning outcomes (Israel, Costa 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 34 Bad Buys  35 that inputs are unimportant. Most of the “Good Buys” Providing additional inputs alone, when other issues are discussed above involve providing new inputs as part not addressed, including: of a strategy to change pedagogy. Good materials, Textbooks, Additional including appropriate-level textbooks and instruction teachers to reduce class at the right level, provided alongside pedagogical size, School buildings, improvements, can make a big difference in learning Grants, Salary, Libraries outcomes, as discussed above. This is especially true in systems that lack even minimal levels of resources, such as those with badly overcrowded classrooms. Providing additional inputs alone, when other issues are not addressed, including: textbooks, additional teachers to reduce class size, school buildings, grants, salary, libraries One mistake that many systems make is to assume that simply investing more in inputs on the margin, without improving how they are used or for whom, will improve learning. This approach can be tempting if the intention is to show that something is being done about education, because new materials and infrastructure are more visible than some of the cost- effective approaches to pedagogy and classroom organization listed as “Good Buys” above. It is also tempting, because of course schools must have textbooks, other learning materials, teachers, and buildings in order to operate. However, studies in many different settings have found that additional inputs alone, used in support of “business as usual,” without improving how they are being used, are not effective. Examples include textbooks (Kenya-1, Sierra Leone), teachers (Kenya-2, India), flip charts (Kenya-3), flexible grants to schools (Gambia, Indonesia, Tanzania), salaries (Indonesia), and libraries (India).² In many education systems, a 22 1 112eddffvg v combination of rote learning, teaching to the top of the class, and an overly ambitious curriculum means that providing additional inputs has no impact on learning, unless those inputs are accompanied by fundamental changes in how teachers teach. This does not mean 1 2 In a few cases, providing additional inputs such as textbooks alone has been shown to have significant impacts on learning in rigorous studies, as in a very early randomized controlled trial from Nicaragua. However, even that case emphasized the importance of complementary interventions like programs to improve teaching. That study noted that a radio-lessons intervention evaluated in parallel had much larger effects, probably because of the “inconsistent application of the textbook treatment (in the hands of teachers with relatively low levels of education).” 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING w 36 Bad Buys Other Intervention Areas 37 Illustrative examples of areas where governments consistently invest: Areas where governments nevertheless need to make Reforming teacher accountability and incentives decisions or take action, Low levels of teacher attendance and low levels of effort (based on what is observable) are pervasive in LICs and MICs, compared to what is observed in high-performing but evidence on how to do systems. A considerable part of this is due to failings in the support and motivation provided by the it effectively is limited. education system, rather than the failings of individual teachers, but it is costly to student learning nonetheless. Test scores increase with more teacher presence (even when teacher quality is very weak). For this reason, there have been many attempts to improve the accountability of teachers. However, linking teachers’ pay to student performance can lead to gaming the system (Kenya), is extremely hard to implement appropriately (Pakistan), may work only with other inputs (Tanzania), and thus often fails to improve student learning outcomes (in Kenya, Pakistan, Tanzania, U.S.). Moreover, data on student performance to benchmark such incentives against in public schools can be of poor quality (India-1). The examples where this approach has been successful (India-2, Rwanda and Mexico) underline the importance of teacher accountability even if linking pay to performance is extremely hard to do in practice, as discussed above. Incentives based on average performance were seen to improve learning only in high performing schools or for high-performing students (Tanzania, China). Pay-for-performance programs are also hard to introduce because of resistance from both teachers and unions. Training in-service teachers with focus on general skills In-service teacher training as typically provided often is generalized, overly theoretical, off-site, does not respond to demonstrated teacher needs, and thus is usually not a good investment. Of course, professional development for teachers is essential, and specific, practical professional development (for example, through in-school mentoring and induction programs) that supports specific, well-evidenced changes in pedagogy can be highly effective. In fact, it is an integral part of most of the “Great Buy” and “Good Buy” interventions, such as structured lesson plans. However, there is little evidence showing that the typical stand-alone, general-skills in-service training is cost-effective. Indeed, much of the rigorous evidence that is available suggests that it does not improve student learning outcomes, because the typical training rarely incorporates the characteristics that make some professional development programs effective. For example, a large-scale randomized controlled trial of a national teacher professional development program in China showed no impacts from 15 days of training at a centralized location, even when the training was reinforced with follow up reminders or evaluation. And in Beyond the interventions categorized above, there are many areas in which Costa Rica, a program to train teachers in active techniques (such as promoting discussion and project work) governments consistently invest, but where, unfortunately, there is little evidence in math at the secondary school level actually resulted in lower rates of learning. However, in Uganda, an on how to do it well, at least as the interventions are typically framed. For example, 11 day-long intervention with training and regular mentoring that taught teachers to help students to pose governments must train, select, and allocate teachers, make education inclusive for sharp questions, frame hypotheses, and check them using information gathered from everyday life over rote students with disabilities, and ensure student safety. These are aspects of schooling memorisation and assimilating information found large effects (24 pp over a 51% baseline) on passing rate and learning where the government is obliged to act, but there is relatively little in the primary leaving exam and students’ test scores. robust evidence of ways to do so successfully. This may be because too little evaluation has been done, as with interventions to improve access for children with disabilities; or because the evidence that is available is inconclusive, or even discouraging, as in the case of in-service teacher training on general skills. There is an urgent need for more research and careful evaluation in these areas, to find the most cost-effective approaches. 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 38 Other Intervention Areas Bad Buys 39 Context In-service professional development can be effective when it is an integral part of a well-evidenced, specific Appendices pedagogical reform (see “Good Buys” for a list of such programs); and when it includes practical training, classroom practice, and reinforcement over time. It is most likely to be effective where targeting teachers for training based on pedagogical gaps is feasible, and where the environment allows a focus on practical training. It is unlikely to be effective where there is strong institutional inertia favoring the delivery of low- quality, overly theoretical training to all teachers. Selecting and allocating teachers All countries must select and allocate teachers, and high-performing systems appear to do it well based on objective factors. But there is a shortage of good evidence on how to do this effectively, making this an important area for further experimentation and research. Patronage-based recruitment of teachers likely undermines system credibility and learning, and some work on the effects of a new meritocratic hiring system for teachers in Mexico found that even though the test wasn’t good at predicting who would be a good teacher, simply having the test weeds out a lot of bad candidates. (The test’s lack of predictive power is consistent with findings of earlier work in Ecuador.) Given that the allocation of teachers across schools can be quite uneven, and driven by favoritism, information about how to improve this process would also be helpful. However, there is little evidence yet on how such reforms affect learning. Targeting support for children with special needs Over half of the 65 million children with special-needs in LICs and MICs are not in school. Some interventions, such as special schools with trained teachers and special need-specific learning material, training for parents, and some computer-based interventions, might be effective in improving the foundational skills of primary-school-aged children with special needs (Kenya, Report). However, the quality of evidence is generally limited, in part because small sample sizes make these programs difficult to evaluate. More research on the best way to provide need-specific support to these learners is important, given the gap in learning due to disabilities in LICs and MICs. 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 40 Appendix A Appendix B 41 APPENDIX A APPENDIX B Research methods M  ore Detail on How for this update to Use This Note Audience This update to the Smart Buys report was based on a systematic search of the literature for interventions that have educational outcomes (attendance, This note is intended to be helpful for technical staff in Ministries of enrollment, dropout, test scores, or cognitive proxies). A more detailed Education, donor agencies, local education groups, and nonprofit description of the research methods used to update the report (including a Executive Summary organizations in thinking through appropriate interventions. It should be detailed cost-effectiveness analysis) is available in the Online Appendix. used along with context-specific analyses and system diagnostics. The classification and descriptions are not definitive; they simply aim to offer a greater sense of prioritization To conduct this search, the research team downloaded papers from 18 and clarity than has been possible in the past, in part due to the paucity of data databases. The team looked for papers that focused on education or learning and were concerning costs until recently. It will be especially useful in thinking through where to evaluated to find out their impact. The research team shortlisted studies that were conducted invest additional marginal resources—for example, is it better to invest new resources in in LICs and MICs, were studied using a rigorous causal design, and were either a peer-reviewed general teacher training, or in training focused on the use of structured lesson plans? But article or an academic working paper. The team also included the results of meta-analyses and it can also help in more closely examining where large parts of the education budget are used the results of systematic reviews to ensure the list of studies was complete. being spent, and exploring whether that money could be used more cost-effectively. Using these criteria, the research team analyzed the titles and abstracts of 13,262 new papers and shortlisted 725 papers for further review. The Context full list of studies is here. Looking at the most prevalent categories from this list, the Panel decided to focus on five subcategories of topics: Early Childhood Development, Health, Context is often crucial in determining whether an intervention will be Teachers and Pedagogy, Ed-tech and Inputs, and Socio-Emotional and Gender. Of the papers in successful or unsuccessful. Contexts vary by the stage of development these topics, approximately 320 papers passed a review for research rigor. These 320 papers of a country, but also differ considerably within countries. To be used were not originally included in the 2020 Smart Buys. In this review, the team ensured that basic effectively, therefore, this guidance should be combined with an assessment of context- rules of causal studies applied. For example, if the study was a randomized controlled trial, the specific educational needs and implementation constraints. This starts with prioritizing checks included ensuring that the outcomes were collected in the same way for the treatment objectives, given that in some countries the key challenge is still increasing access to and control groups, that the study did not have large differential attrition, that the researcher education, while in others, children are in school but are not learning; and in still others, clustered the standard errors at the level of randomization, and that there were at least 20 learning has improved on average, but disadvantaged children do much more poorly units of randomization. For quasi-experimental methods such as regression discontinuity than the average. With these objectives set, as an aid, the tables above have described design, difference-in-difference, and propensity scoring matching, the team ensured the the types of contexts in which each intervention is most likely to be cost-effective. For research methods were strong such as including appropriate controls, mapping pre-trends, and example, information interventions can be incredibly cost-effective, because they are reviewing robustness checks. For all studies, the team ensured the researchers looked at the inexpensively delivered; however, exactly what kind of information is useful and relevant precision of the estimates, such as by including standard errors or confidence intervals. to parents and children in shaping their decisions about education will differ in different contexts. One key element of context is political economy. The categorization in the table The Panel members discussed the evidence for interventions in each of the does not reflect the political feasibility of each intervention, but only its cost effectiveness above five topics as small groups. They decided how to categorize each intervention, at improving outcomes. But because political economy is central, the descriptions do note including discussing whether to move existing interventions to a new category, whether to which interventions are likely to be more politically challenging to deliver. The impacts on create new subsections, and how to categorize these new subsections. Where available, the learning also depend on the quality of implementation. For example, although “teaching at team analyzed the cost data reported in the studies and updated the earlier cost-effectiveness the right level” interventions have typically been very cost-effective, just calling a program analysis (see Appendix E for detailed cost-effectiveness analysis). The Panel was consulted on “teaching at the right level” will not automatically make it a Great Buy; it needs to be the resulting list and added additional papers based on their expertise. effectively implemented in order to have an impact. 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 42 Appendix B Appendix B 43 Interpreting zero-size effects of the interventions discussed above have been delivered in challenging contexts, the capacity and accountability of the education system is likely to have major effects on the Effects statistically not different from zero must be interpreted with sustainability and scalability of impacts. Finally, some very large-scale interventions have caution. Sometimes, they are mistakenly understood as conclusive not been evaluated rigorously at all. For all of these reasons, expanding this evidence base evidence that something would not work. But, we may find null results if should be a priority, with more investment in building the data and the evaluation capacity implementation is imperfect or contextual implications were such that the effects of low- and middle income countries. were nullified through some other simultaneous counter-effect emerging due to the intervention. Sometimes, compliance with the intervention could be poor or some crucial Beyond the well-evaluated and well-researched interventions elements for the intervention to be successful are lacking and hence the potential summarized in this note, there are other policies and programs that effectiveness arising out of complementarity of inputs is not realized. A zero-result may be appropriate in specific contexts. However, policymakers should could even emerge just because the study does not have sufficient statistical power to strongly consider whether any of the well-researched, highly cost-effective approaches detect small changes. When interpreting null effects we have tried to be careful in noting (Great and Good Buys) are relevant in their context, especially as many of those which pieces of the intervention did not work and to provide careful reasoning about interventions address problems that are very common. If there is a strong case for an mechanisms to nuance our findings and such discretion is advisable for the readers too. intervention that has not been well researched, especially in secondary schooling, policymakers should include a careful monitoring plan and consider whether a robust The importance of providing good information on evaluation of impact is possible (recognizing that a poor evaluation will not add to the effectiveness and cost-effectiveness knowledge base). There should be a higher burden of justification for investing in Bad Buys: the business case for an intervention identified as rarely cost-effective should Moving from less to more effective education programs delivers substantial gains in have a well-developed and well-supported theory of change, including consideration of learning, and most education spending in developing countries is by the governments the opportunity cost of the intervention, and a plan for monitoring its effectiveness. themselves. Future “Smart Buys”: The Panel will convene periodically to review additional Therefore, advising partner governments and other donors to invest categories of interventions, and to review the evolving evidence base for past their financing toward Great Buys or Good Buys, or toward the system categories. reforms described below, could be a very cost-effective use of aid. Some recent experimental work indicates that policymakers value good evidence, and act on it when they receive it. However, providing such advice requires substantial advisor time, very high quality embedded technical assistance, or research teams, and the advice needs to be closely tied to the evidence or it can be counterproductive. That said, where education spending is inefficient but there is the will and capacity to improve, the provision of good evidence-based information on the relative cost-effectiveness of various approaches and interventions could in itself be highly cost-effective. This note is not intended to provide a comprehensive view on all possible interventions in the education sector. As governments and teams design programs and portfolios, they should be drawing on context-specific diagnostic work and evidence from a range of sources. Comparable evidence does not yet cover the universe of all possible interventions in this sector. Despite all the progress made in recent years, the evidence base in education remains small and fragmented (compared to the evidence base in the health sector, for example); and only a small number of evaluation studies collect data on costs. Some interventions have had too few rigorous evaluations to be assessed well. Even where there is evidence, many of the evaluations looked at single interventions rather than packages of interventions, even though a growing literature shows that greater impact is often achieved when several good interventions are combined. In addition, for some interventions that might be Great or Good Buys, we still know too little about the politics of effective implementation at scale. While many 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 44 Appendix C Appendix C 45 Yet even without systemic reform, interventions like the Great and Good Buys discussed in this note can still substantially improve outcomes for millions of children and youth. Indeed, they have already been shown to improve learning at scale, typically in systems that are not yet well- aligned toward learning. To maximize the chances for sustained success, policymakers should take several factors into account when they implement interventions like those discussed below: They should look for horizontal complementarities across interventions. A deep understanding of the specific context is essential in order to design programs that have a chance of success; and part of this context is how a given program interacts with other interventions. In the example of Ceará, Brazil, numerous interventions that supported each other were combined—the setting of clear foundational learning goals for all children; regular assessment to inform teaching; practical teacher training; and financial incentives provided to municipalities. Another example of complementarities is pairing the right curricula and the right kind of coaching in kindergarten, as has been done in the U.S. They should also recognize the importance of dynamic APPENDIX C complementarities. Complementarities in interventions over time also More Detail on matter. For example, in the U.S., higher-quality preschool delivers larger long-term gains in educational and life outcomes when it is followed by higher-quality primary schooling, and vice versa. the Importance of They should think about how the interventions will advance (or inhibit) systemic reform. Systemic reform takes at least several years to Systemic Reform show substantial impacts on a range of outcomes. In the meantime, policymakers should focus on interventions that will not inhibit that reform. Some implementation challenges involve behavioral change needed from Although this note focuses on the impacts of various interventions on key stakeholders and may require more political and systemic reform; others are learning, interventions are not all that matters. To drive system-wide technical challenges with implementing the reforms with fidelity at scale. Noting improvements in learning and make them sustainable over the long these differences is important for systemic reform. Some interventions may make term, systemic reform is likely to be extremely important. Ensuring learning complementary interventions more effective or easier to introduce later. For example, if for all children and youth requires an education system that is coherent and aligned a country already has a system of practical, classroom-based professional development toward learning. While which reforms should be prioritized in a particular setting will for its teachers, introducing an improved curriculum is more likely to be implementable depend on the context, alignment toward learning should encompass the key system and to have the desired effects in the classroom. actors, policies, incentives, pedagogy, and capacity. This in turn requires political commitment, to help systems escape low-learning traps—and the commitment needs Governments should also consider investing in independent and good- to be not just from the education minister, but from the head of government as well. quality measurement of student learning outcomes. Bureaucracies This includes a durable commitment to consistent implementation and regular review and political systems can respond better to specific issues in teaching of what is working. Good examples of sustained systemwide reform are rare, but it can and learning when deficits are measured accurately. If data generated happen with strong and consistent political leadership. The Brazilian state of Ceará has within the schooling-system are not representative of student learning, it is reduced made remarkable recent gains in learning during more than a decade of reforms, rising to burdensome paper trails without being practically useful. Better data on learning to become one of the country’s top-performing states despite being among the poorest. outcomes can create scope for improved diagnosis and specific remediation through increased visibility of systemic shortcomings as well as effectiveness of interventions. 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 46 Appendix D Appendix D 47 APPENDIX D feed into the Panel’s deliberations (see References).³ Of course, this rigorous evaluation  ore Detail on M 1 literature, although it has greatly expanded over the past 20 years, still has numerous gaps; so the Panel has supplemented it with other types of evidence where necessary. Classification While the primary evidence base is from low- and middle-income countries, the Panel has also drawn on evidence from high-income countries, where relevant and necessary, to fill out the evidence base. Parameters Cost-effectiveness vs. effectiveness Wherever possible, the Panel prioritized evidence of cost-effectiveness in making its determinations; education systems face budget constraints, and they need to allocate scarce resources toward whatever interventions will deliver the most learning gains for the Detailed considerations made in assessing the evidence for inclusion most children and youth on a given budget. However, because many evaluations lack cost within this report include: data, we have also drawn on data on effectiveness—evidence on which interventions tend to produce the largest learning gains, independent of cost. Because the estimated impacts Outcome variable from this larger group of effectiveness interventions appears to share the same range as This synthesis focuses on identifying the interventions that are most cost-effective in the cost-effectiveness group, drawing on those findings is a reasonable strategy.⁴ 2 improving learning in basic education, measured in terms of core cognitive skills (typically literacy and numeracy). These skills are relevant everywhere around the world: they Scale improve employment, income, health, civic participation, and a host of other development Many interventions that succeed as smaller pilots fail to achieve results when scaled goals. up, whether because in scaling up implementation quality declines, or because political resistance to the intervention increases. Therefore, while it has considered a range of Consequently, children who do not acquire them will be at a disadvantage throughout evidence, the Panel has weighted more heavily the interventions that have been proven their lives. Educational interventions also have other important impacts, such as reduced effective at a larger scale—whether systemwide or, at a minimum, in hundreds of schools. crime, improved employment prospects, and better health status, that are not always The areas that appear promising but have not been assessed at scale are included in the mediated by improvements in learning; future meetings of the Panel will consider “promising but limited-evidence” category. interventions that advance those goals directly. Moreover, because improving learning has proved far more challenging than expanding access to education, this note has Duration of impacts focused on that goal. In cases where impacts on cognitive skills are often not measured, In making its assessments, the Panel gives more credit to interventions that have been such as in early childhood development, the Panel has relied on proxies, such as the shown to have long-term positive impacts. However, this criterion cannot be applied effects on school readiness. blindly. Interventions are often evaluated over only a couple of years, so there is not always evidence of whether the impacts persist or fade over time. Furthermore, in many Learning and equity cases the initial intervention may need to be repeated or complemented by other policies Contrary to what is sometimes assumed under a dichotomous “access vs. learning” or programs to have its full impact. Therefore, the Panel has considered the duration of view, a focus on learning as the outcome variable in this context is a tool for improving impacts only as one of multiple factors. Unlike most of the other categories, preschool equity, inclusion, and opportunity. There is a reason that Sustainable Development Goal and early stimulation interventions do have long-term evidence, which has influenced the 4 highlights foundational literacy and numeracy skills as a key indicator to be tracked: categorization of those two sets of interventions. because the children and youth most harmed by the learning crisis are those who fail to acquire those skills during basic education. The Panel’s goal is to highlight those Interpretation of evidence interventions that will advance learning for those students. The evaluations it draws Synthesizing these lessons is not just a counting exercise, both because there are gaps on generally either focus on disadvantaged schools and children, or have been tested in the evidence and because the context of the implementation matters. Therefore, the system-wide; in the latter case, the Panel recommends only those interventions that are classifications also reflect principles drawn from the evaluation results, combined with shown to be effective for less advantaged students. other knowledge about learning and behavior from educational research, psychology, and development economics. Evidence base The evidence reflected here includes the many high-quality systematic reviews and meta- 3 One new addition to the literature on which this note draws is Angrist, Evans, Filmer, Glennerster, Rogers, and Sabarwal (2020), which estimates analyses published over the past decade, as well as newer work that was produced to 1 21 cost-effectiveness in terms of the Learning-Adjusted Years of Schooling. (LAYS) achieved by specific interventions. The LAYS indicator, which is the education measure incorporated in the Human Capital Index, provides a common metric that improves on the standard-deviation measure used in much past comparative research. In Angrist et al (2020), the preferred metric for cost-effectiveness is LAYS per $100, which measures how many years of high-quality schooling can be bought for an additional $100 spent on each intervention. (see Appendix D for details. 2 4 Angrist, Evans, Filmer, Glennerster, Rogers, and Sabarwal (2020). 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 48 Appendix E Appendix E 49 APPENDIX E most relevant guidance for policy. Figure 2 shows effectiveness, and is included as E  ffectiveness and Cost- background to show that the effect sizes from interventions in Figure 1 are reasonably representative of the effect sizes from the larger set of well-evaluated interventions. Effectiveness Measured It is important to note that interventions that report impacts on cognitive proxies rather than learning or access measures were not included in this analysis. Additionally, no in Terms of LAYS assumptions were made to weight interventions based on their number of beneficiaries. However, as noted earlier, this analysis was not the only evidence that the categorization in this note relies upon, but it was one important input to the Panel’s judgment. The Panel recognizes the importance of these caveats and encourages the production of The key to making judgments about relative cost-effectiveness (and effectiveness in research that addresses them, while commiting to continue updating this analysis as the general) is to have good data and a common metric. Past studies (such as Kremer, evidence base grows. Overall, the purpose of the analysis was not to provide a precise Brannen, and Glennerster 2013) have made these comparisons by measuring assessment and ranking of the cost-effectiveness of different interventions. Instead, its effectiveness in standard-deviation improvements on learning assessments and then purpose was to inform the discussion presented in the note by considering differences dividing by cost. The paper that informed this and our previous note (Angrist, Evans, in orders of magnitude between the interventions in terms of their cost-effectiveness. Filmer, Glennerster, Rogers, and Sabarwal 2020) assesses cost effectiveness in terms of Learning-Adjusted Years of Schooling (LAYS), a measure of education that was introduced in the 2018 World Development Report and that forms a core component of the World Bank’s Human Capital Index (HCI) (see also the background paper introducing LAYS; published version here.) LAYS combines the quantity and quality of schooling into a single metric of progress. It is calculated by multiplying a country’s average number of years of schooling by its average test score performance relative to a high-performance benchmark. For example, if this high-performance benchmark is Singapore’s performance, this procedure produces a measure of learning-adjusted years of schooling expressed in Singapore-equivalent years. LAYS was initially developed for country-level comparisons; it has since been expanded to compare specific interventions and policies evaluated in over 250 studies across 46 countries (see Angrist, Evans, Filmer, Glennerster, Rogers, and Sabarwal 2020). The LAYS conversion uses globally comparable learning outcomes produced by the World Bank for the HCI (Angrist, Djankov, Goldberg, and Patrinos 2019). LAYS offers two advantages over previous metrics for making such comparisons. First, because it expresses intervention impacts in terms of additional years of high quality schooling delivered, it allows direct comparison of the interventions that affect both the quantity and the quality of schooling. By contrast, previous analyses have typically looked at either one or the other. Second, the notion of “additional years of high- quality schooling” or “additional years of schooling, adjusted for quality” is easier for a non-specialist audience to understand than “standard deviations of learning.” In total, this note used over 400 sources, with more than 100 including cost data. The original report included 150 studies, 70 of which have cost data. Out of the 250 new studies used to update this note, 58 included data on the cost of the intervention. When available, we took the cost effectiveness measures reported by the authors of the papers. When these measures were not explicitly reported, we calculated Figure 1: Learning Adjusted Years of Schooling (LAYS) the intervention’s cost-effectiveness as standard deviations per $100 2015 USD, before transforming those estimates to LAYS per $100 2015 USD. For this analysis we per $100 USD, by Intervention Type  focused on learning impacts. This new data was combined with the data from Angrist et al. (2020), which includes effects on both learning and access. The comparison Source: Data adapted from Angrist et al (2020) shown in blue. is reproduced in Figures 1 and 2. Figure 1 shows cost-effectiveness, and is the Notes: Subsections are ranked by median impact. The boxplot delineates the 25th and 75th percentiles. The x-axis is reported on a natural log scale. New data includes 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 50 Appendix E References 51 learning outcomes. Data from Angrist et al (2020) includes both learning and access outcomes. No data was included for School Feeding; Latrines, Water, and Sanitation Awareness; Safeguarding from Violence, Socio-Emotional and Life Skills Development; References and Maternal Mental Health Counseling, due to lack of availability of cost estimates. Aber, J., Tubbs, C., Torrente, C., Halpin, P., Johnston, B., Starkey, L., Wolf, S. (2017). Promoting children’s learning and development in conflict-affected countries: Testing change process in the Democratic Republic of the Congo. Development and Psychopathology, 29(1), 53-67. hiips://www.cambridge.org/core/journals/develop ment-and- psychopathology/article/abs/promoting- childrens-learning-and-development-in-conflictaffected-countries-testing- change-process-in-the- democratic-republic-of-the-congo/60172D9E82B56CD90063C3176FF18AEB Aboud, F. E., & Yousafzai, A. K. (2015). Global health and development in early childhood. Annual review of psychology, 66, 433-457. hiips://www semanticscholar org/paper/Global -health-and-development-in- early- childhood.-Aboud-Yousafzai/702bc5c1cc4326f26ec72d5de8a56141dac16e27?p2df Aboud, F. E., Bougma, K., Lemma, T., & Marquis, G. S. (2017). Evaluation of the effects of iodized salt on the mental development of preschool aged children: a cluster randomized trial in northern Ethiopia Maternal & child nutrition, 13(2), e12322. hiips://onlinelibrary.wiley.com/doi/pdf/10.1111/mcn.12322?casa_token=y8kC3J8UxhQAAAAA:donyG7Do mHMDL47WxMy7Bx_RYZxfZZ1xlDudaD2fbIirrA_j7KapyiYp_iGNQ8OjrtaG_MO1YF49bA Adrogue, C., & Orlicki, M. E. (2013). Do In-school feeding programs have an impact on academic performance?: the case of public schools in Argentina. hiips://ri.conicet.gov.ar/bitstream/handle/11336/27459/CONICET_Digital_ Nro.9a3c9e0b -4e00-4989- 9a43-f47721862c5d_A.pdf?sequence=2 Adukia, A. (2017). Sanitation and education. American Economic Journal: Applied Economics, 9(2), 23-59. hiips://www. jstor.org/stable/pdf/26156194.pdf Afoakwah, E., Carballo, F., Caro, A., D’Cunha, S., Dobrowolski, S., & Fallon, A. (2021). Dialling up Learning: Testing the Impact of Delivering Educational Content via Interactive Voice Response to Students and Teachers in Ghana. EdTech Hub. hiips://docs.edtechhub.org/lib/P22BR78B/download/TD6Z8KUC/Afoakwah%20et%20al.%20 - %202021%20-%20Dialling%20up%20Learning%20Testing%20the%20Impact%20of%20Delive.pdf Afridi, F., Barooah, B., & Somanathan, R. (2016). The mixture as before? Student responses to the changing content of school meals in India. Student Responses to the Changing Content of School Meals in India. hiips://www. econstor.eu/bitstream/10419/142363/1/dp9924.pdf Figure 2: Learning Adjusted Years of Schooling (LAYS) Alan, S., Boneva, T., & Ertac, S. (2019). Ever failed, try again, succeed better: Results from a randomized educational Gained, by Intervention Category intervention on grit. The Quarterly Journal of Economics, 134(3), 1121-1162. hiips://academic.oup.com/qje/article-abs tract/134/3/1121/5342089?redirectedFrom=fulltext Source: Additional data adapted from Angrist et al (2020). Alderman, H., Gilligan, D. O., & Lehrer, K. (2012). The impact of food for education programs on school participation in northern Uganda. Economic Development and Cultural Change, 61(1), 187-218. hiips://www.journals.uchicago. Notes: Categories are ranked by median impact. The boxplot delineates the 25th edu/doi/abs/10.1086/666949?journalCode=edcc and 75th percentiles. Data points in blue represent studies from Angrist et al (2020) that do Allende, C., Gallego, F., & Neilson, C. (2019). Approximating the equilibrium effects of informed school choice (No. not include cost effectiveness information. 2019-16). hiips://christopherneilson.github.io/work/documents/SchoolChoiceInfoExp.pdf Andrabi, T., Das, J., & Khwaja, A. I. (2017). Report cards: The impact of providing school and child test scores on educational markets. American Economic Review, 107(6), 1535-1563. hiips://www.aeaweb.org/articles?id=10.1257/ aer.20140774 Andrew, A., Attanasio, O., Augsburg, B., Day, M., Grantham McGregor, S., Meghir, C., & Rubio Codina, M. (2020). Effects of a scalable home visiting intervention on child development in slums of urban India: evidence from a randomised controlled trial. Journal of child psychology and psychiatry, 61(6), 644-652. hiips:// acamh. onlinelibrary.wiley.com/doi/full/10.1111/jcpp.13171 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 52 References References 53 Andrew, A., Attanasio, O., Bernal, R., Sosa, L. C., Krutikova, S., & Rubio-Codina, M. (2019). Preschool quality and child Atukunda, P., Muhoozi, G. K., Westerberg, A. C., & Iversen, P. O. (2019). Nutrition, hygiene and stimulation education development (No. w26191). National Bureau of Economic Research. hiips://www.nber.org/papers/w26191 for impoverished mothers in rural Uganda: effect on maternal depression symptoms and their associations to child Andrew, A., Attanasio, O., Fitzsimons, E., Grantham-McGregor, S., Meghir, C., & Rubio-Codina, M. (2018). Impacts 2 development outcomes. Nutrients, 11(7), 1561. hiips://www.mdpi.com/2072-6643/11/7/1561/pdf years after a scalable early childhood development intervention to increase psychosocial stimulation in the home: Aurino, E., Gelli, A., Adamba, C., Osei-Akoto, I., & Alderman, H. (2018). Food for thought? Experimental evidence on A follow-up of a cluster randomised controlled trial in Colombia. PLoS medicine, 15(4), e1002556 hiips://journals. the learning impacts of a large-scale school feeding program in Ghana (Vol. 1782). Intl Food Policy Res Inst. hiips:// plos.org/plosmedicine/article?id=10 .1371/journal.pmed.1002556 www.ifpri.org/publication/food-thought-experimental-evidence-learning-impacts- large-scale-school-feeding- Angrist, J., & Lavy, V. (2002). New evidence on classroom computers and pupil learning. The Economic Journal, program 112(482), 735-765. Baird, S., Hicks, J. H., Kremer, M., & Miguel, E. (2016). Worms at work: Long-run impacts of a child health investment. hiips://economics.mit.edu/sites/default/files/publications/New%20Evidence%20on%20Classroom%20Co The quarterly journal of economics, 131(4), 1637-1680. hiips://academic.oup.com/qje/article-abstract/131/4/1637/246 mputers%20and%20Pupil%20Lear.pdf 8871?redirectedFrom=fulltext Angrist, N., & Meager, R. (2022). The role of implementation in generalisability: A synthesis of evidence on targeted Baird, S., McIntosh, C., & Özler, B. (2019). When the money runs out: Do cash transfers have sustained effects on educational instruction and a new randomised trial CEDIL Syntheses Paper 4. hiips://policycommons.net/ human capital accumulation?. Journal of Development Economics, 140, 169-185. hiips://www sciencedirect com/ artifacts/3455193/cedil -syntheses-wp-4- generalizable_evidence_targeted_instruction/4255508/ science/article/pii/S0304387 818312732 Angrist, N., Bergman, P., & Matsheng, M. (2022). Experimental evidence on learning using low-tech when school is Baker-Henningham, H., & López Bóo, F. (2010). Early childhood stimulation interventions in developing countries: a out. Nature human behaviour, 6(7), 941-950. hiips://www.nature.com/articles/s41562 -022- 01381-z comprehensive literature review. hiips://www.econstor.eu/bitstream/10419/89041/1/IDB -WP- 213.pdf Angrist, N., Cullen, C., Ainumugisha, M., Bathena, S. P., Bergman, P., Crossley, C., Letsomo, T., Matsheng, M., Panti, R. Baker-Henningham, H., Scott, Y., Francis, T., & Walker, S. P. (2021). Effects of a teacher-training violence prevention M., Sabarwala, S., & Sullivan, T. (2023). Learning curve: Progress in the replication crisis. Retrieved April 17, 2023, program in Jamaican preschools on child behavior, academic achievement, and school attendance in grade from one of primary school: follow up of a cluster randomized trial. Frontiers in psychology, 12, 652050. hiips://www hiips://static1.squarespace.com/static /60d0c05ace34212ef5a1131b/t/64079e5eb1e95800153fa654/1678 frontiersin org/articles/10.3389/fpsyg.2021.652050/ful l 220895546/Angrist_Cullen_Bergman_et-al_Learning-Curve-Progress-in-the-Replication-Crisis_2023.pdf Bandiera, O., Buehren, N., Burgess, R., Goldstein, M., Gulesci, S., Rasul, I., & Sulaiman, M. (2020). Women’s Angrist, N., Evans, D. K., Filmer, D., Glennerster, R., Rogers, F. H., & Sabarwal, S. (2020). How to improve education empowerment in action: evidence from a randomized control trial in Africa. American Economic Journal: Applied outcomes most efficiently? A Comparison of 150 interventions using the new Learning-Adjusted Years of Economics, 12(1), 210-259. hiips://www.jstor.org/stable/pdf/26866466.pdf Schooling metric. hiips://documents1.worldbank.org/curated/en/801901603314530125/pdf/How -to-Improve- Banerjee, A. V., Banerji, R., Duflo, E., Glennerster, R., & Khemani, S. (2010). Pitfalls of participatory programs: Education- Outcomes-Most-Efficiently-A-Comparison-of-150-Interventions-Using-the-New-Learning-Adjusted-Years- Evidence from a randomized evaluation in education in India. American Economic Journal: Economic Policy, 2(1), of-Schooling-Metric.pdf 1-30. hiips://www nber org/papers/w14311 Angrist, N., Jukes, M. C., Clarke, S., Chico, R. M., Opondo, C., Bundy, D., & Cohee, L. M. (2023). School- based malaria Banerjee, A. V., Cole, S., Duflo, E., & Linden, L. (2007). Remedying education: Evidence from two randomized chemoprevention as a cost-effective approach to improve cognitive and educational outcomes: a meta-analysis. experiments in India. The Quarterly Journal of Economics, 122(3), 1235-1264. hiips://www.nber.org/papers/w11904 arXiv preprint arXiv:2303.10684. hiips://arxiv.org/abs/2303.10684 Banerjee, A., Banerji, R., Berry, J., Duflo, E., Kannan, H., Mukerji, S., & Walton, M. (2017). From proof of concept to Araujo, M. C., Dormal, M., Grantham-McGregor, S., Lazarte, F., Rubio-Codina, M., & Schady, N. (2021). Home visiting scalable policies: Challenges and solutions, with an application. Journal of Economic Perspectives, 31(4), 73-102. at scale and child development. Journal of Public Economics Plus, 2, 100003. hiips://www.sciencedirect.com/ hiips://www.aeaweb.org/articles?id=10.1257/jep.31.4.73 science/article/pii/S2666551421000012 Banerjee, Abhijit, Rukmini Banerji, James Berry, Esther Duflo, Harini Kannan, Shobhini Mukherji, Marc Shotland, Ardington, C. (2023). (publication). Funda Wande Limpopo Midline Evaluation. Funda Wande. Retrieved April 11, and Michael Walton. 2016. “Mainstreaming an effective intervention: Evidence from randomized evaluations of 2023, from hiips://fundawande.org/img/cms/news/Limpopo%20Second%20Midline%20Report%202023%20V03. ‘Teaching at the Right Level’ in India.” NBER Working Paper No. 22746. hiips://www.nber.org/papers/w22746 pdf Bann, C. M., Wallander, J. L., Do, B., Thorsten, V., Pasha, O., Biasini, F. J., & Carlo, W. A. (2016). Home- based Ashraf, N., Banerjee, A., & Nourani, V. (2021). Learning to teach by learning to learn. en. In: Working paper, 115. early intervention and the influence of family resources on cognitive development. Pediatrics, 137(4). hiips:// hiips://cdep.sipa.columbia.edu/sites/default/files/cdep/LLLT Draft_Paper_November.pdf publications.aap.org/pediatrics/article/137/4/e20153766/81457/Home -Based-Early- Intervention-and-the-Influence- Ashraf, N., Bau, N., Low, C., & McGinn, K. (2020). Negotiating a better future: How interpersonal skills facilitate of intergenerational investment. The Quarterly Journal of Economics, 135(2), 1095-1151. hiips://academic.oup.com/ Barr, A., Mugisha, F., Serneels, P., & Zeitlin, A. (2012). Information and collective action in community- based qje/article/135/2/1095/5698825 monitoring of schools: Field and lab experimental evidence from Uganda. Unpubl. Pap. hiips://citeseerx.ist.psu. Attanasio, O. P., Fernández, C., Fitzsimons, E. O., Grantham-McGregor, S. M., Meghir, C., & Rubio- Codina, M. edu/viewdoc/download?doi=10.1.1.372.6834&rep=rep1&type=pdf (2014). Using the infrastructure of a conditional cash transfer program to deliver a scalable integrated early Barrera-Osorio, F., & Linden, L.L. (2009). The Use and Misuse of Computers in Education : Evidence from a child development program in Colombia: cluster randomized controlled trial. Bmj, 349. hiips://www.bmj.com/ Randomized Controlled Trial of a Language Arts Program 1 March 2009. hiips://www.semanticscholar.org/paper/ content/349/bmj.g5785 The -Use-and-Misuse-of-Computers-in-Education-%3A-from- Barrera-Osorio-Linden/04fbe09878cd7698d341cb266 3e23d28e836ad44?p2df 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 54 References References 55 Barrera-Osorio, F., & Raju, D. (2017). Teacher performance pay: Experimental evidence from Pakistan. Journal of Blimpo, M. P., Blimpo, M., Evans, D., & Lahire, N. (2015). Parental human capital and effective school management: Public Economics, 148, 75-91. hiips://documents1.worldbank.org/curated/en/973061468189533 912/pdf/WPS7307. evidence from The Gambia. World Bank Policy Research Working Paper, (7238). hiips://documents.worldbank.org/ pdf en/publication/documents - reports/documentdetail/923441468191341801/parental-human-capital-and-effective- Barrera-Osorio,Felipe; De Barros,Andreas; Filmer,Deon P.. Long-term impacts of alternative approaches to increase school-management- evidence-from-the-gambia schooling: evidence from a scholarship program in Cambodia (. Policy Research working paper,no. WPS 8566, Bobonis, G. J., Miguel, E., & Puri-Sharma, C. (2006). Anemia and school participation. Journal of Human resources, hiip://documents.worldbank.org/curated/en/8 38871535033752683/Long-term- impacts-of-alternative-approaches- 41(4), 692-721. hiips://www.jstor.org/stable/pdf/40057287.pdf to-increase-schooling-evidence-from-a-scholarship-program-in- Cambodia Boivin, M. J., Sikorskii, A., Familiar-Lopez, I., Ruiseñor-Escudero, H., Muhindo, M., Kapisi, J. & Dorsey, Beg, S., Fitzpatrick , A., Lucas, A., Tsinigo, E., &  Atimone, H. (2020). Strengthening teacher accountability G. (2016). Malaria illness mediated by anaemia lessens cognitive development in younger Ugandan children. to reach all students: Milestone 4: Impact analysis report. https://documents1.worldbank.org/curated/ Malaria journal, 15(1), 1-12. hiips://pubmed.ncbi.nlm.nih.gov/27076184/ en/184851585721598666/pdf/Strengthening-Teacher-Accountability-to-Reach-All-Students-STARS-Milestone-4- Bold, T., Kimenyi, M., Mwabu, G., & Sandefur, J. (2018). Experimental evidence on scaling up education reforms Impact-Analysis-Report.pdf in Kenya. Journal of Public Economics, 168, 1-20. hiips://www.sciencedirect.com/science/article/abs/pii/ Beg, S., Halim, W., Lucas, A. M., & Saif, U. (2022). Engaging teachers with technology increased achievement, S0047272718301518?via%3Dihub bypassing teachers did not. American Economic Journal: Economic Policy, 14(2), 61-90. hiips://www.aeaweb.org/ Borkum, E., He, F., & Linden, L. L. (2012). The effects of school libraries on language skills: Evidence from a articles?id=10.1257/pol.20200713 randomized controlled trial in India (No. w18183). National Bureau of Economic Research. hiips://www.nber.org/ Behrman, J. R., Parker, S. W., & Todd, P. E. (2011). Do Conditional Cash Transfers for Schooling Generate Lasting papers/w18183 Benefits? A Five-Year Followup of PROGRESA/Oportunidades. The Journal of Human Resources, 46(1), 93–122. Boswell, M., & Rozelle, S. (2016). Evaluating the effectiveness of computers as tutors in China. hiips://www.3ieimpact. hiip://www jstor org/stable/25764805 org/evidence -hub/publications/impact-evaluations/evaluating-effectiveness- computers-tutors-china Berlinski, S., & Busso, M. (2017). Challenges in educational reform: An experiment on active learning in mathematics. Bouguen, A., Filmer, D., Macours, K., & Naudeau, S. (2019). Preschool and Parental Response in a Second Best World. Economics Letters, 156, 172-175. hiips://www.sciencedirect.com/science/article/pii/S0165176517301854 hiips://openknowledge.w orldbank.org/bitstream/handle/10986/37066/2018_Bouguen%20Filmer%20Maco urs%20 Berlinski, S., & Galiani, S. (2007). The effect of a large expansion of pre-primary school facilities on preschool Naudeau_JHR_AAM.pdf attendance and maternal employment. Labour Economics, 14(3), 665-680. hiips://www.sciencedirect.com/ Bowen, A., Ma, H., Ou, J., Billhimer, W., Long, T., Mintz, E., & Luby, S. (2007). A cluster-randomized controlled trial science/article /abs/pii/S092753710700005X evaluating the effect of a handwashing-promotion program in Chinese primary schools. The American journal of Berlinski, S., Busso, M., Dinkelman, T., & Martínez, C. (2021). Reducing parent-school information gaps and tropical medicine and hygiene, 76(6), 1166-1173. hiips://www.edu- links.org/sites/default/files/media/file/Bowen%20 improving education outcomes: Evidence from high-frequency text messages (No. w28581). National Bureau of et%20al%202007.pdf Economic Research. hiips://www.nber.org/papers/w28581 Brinkman, S. A., Hasan, A., Jung, H., Kinnell, A., & Pradhan, M. (2017). The impact of expanding access to early Berlinski, S., Galiani, S., & Gertler, P. (2009). The effect of pre-primary education on primary school performance. childhood education services in rural Indonesia. Journal of Labor Economics, 35(S1), S305-S335. hiips://www. Journal of public Economics, 93(1-2), 219-234. hiips://www.sciencedirect.com/science/article/abs/pii/ journals.uchicago.edu/doi/abs/10.1086/691278?journalCode=jole S0047272708001308 Britto, P. R., Lye, S. J., Proulx, K., Yousafzai, A. K., Matthews, S. G., Vaivada, T., & Lancet Early Childhood Berlinski, S., Galiani, S., & Manacorda, M. (2008). Giving children a better start: Preschool attendance and school-age Development Series Steering Committee. (2017). https://www.sciencedirect.com/science/article/pii/ profiles. Journal of Public Economics, 92(5-6), 1416-1440. hiips://personal.lse.ac.uk/manacorm/preschool.pdf S0140673616313903 Betancourt, T. S., McBain, R., Newnham, E. A., Akinsulure-Smith, A. M., Brennan, R. T., Weisz, J. R., & Hansen, N. B. Britto, P. R., Lye, S. J., Proulx, K., Yousafzai, A. K., Matthews, S. G., Vaivada, T., & Lancet Early Childhood (2014). A behavioral intervention for war-affected youth in Sierra Leone: a randomized controlled trial. Journal Development Series Steering Committee. (2017). Nurturing care: promoting early childhood development. The of the American Academy of Child & Adolescent Psychiatry, 53(12), 1288-1297. hiips://www.ncbi.nlm.nih.gov/pmc/ Lancet, 389(10064), 91-102. hiips://www.sciencedirect.com/science/article/abs/pii/S0140673616313903 articles/PMC5022359/ Brunette, T., Piper, B., Jordan, R., King, S., & Nabacwa, R. (2019). The impact of mother tongue reading instruction Beuermann, D.W., Cristia, J., Cueto, S., Malamud, O., & Cruz-Aguayo, Y. (2015). One Laptop per Child at Home: Short- in twelve ugandan languages and the role of language complexity, socioeconomic factors, and program Term Impacts from a Randomized Experiment in Peru†. American Economic Journal: Applied Economics, 7, 53-80. implementation. Comparative Education Review, 63(4), 591–612. hiips://doi.org/10.1086/705426 hiips://www.semanticscholar.org/paper/One -Laptop-per-Child-at- Home%3A-Short-Term-Impacts-a-Beuermann- Burde, D., & Linden, L. L. (2013). Bringing education to Afghan girls: A randomized controlled trial of village-based Cristia/36bb83e48529703b39e27e7b50547621a30d600e?p2df schools. American Economic Journal: Applied Economics, 5(3), 27–40. hiips://doi.org/10.1257/app.5.3.27 Black, M. M., Walker, S. P., Fernald, L. C., Andersen, C. T., DiGirolamo, A. M., Lu, C., & Grantham- McGregor, S. (2017). Carneiro, P., Cruz-Aguayo, Y., Pachon, R. H., & Schady, N. (2022). Dynamic Complementarity in Elementary Schools: Early childhood development coming of age: science through the life course. The Lancet, 389(10064), 77-90. Experimental Estimates from Ecuador. Inter-American Development Bank. https://www.ucl.ac.uk/~uctppca/ hiips://www.sciencedirect.com/science/article/abs/pii/S0140673616313897 CarneiroCruzPachonSchady_030122.pdf  Bleakley, H. (2007). Disease and development: evidence from hookworm eradication in the American Chakraborty, T., & Jayaraman, R. (2019). School feeding and learning achievement: evidence from India’s midday South. The quarterly journal of economics, 122(1), 73-117. hiips://academic.oup.com/qje/article - meal program. Journal of Development Economics, 139, 249-265. hiips://www.sciencedirect.com/science/article/ abstract/122/1/73/1924773?redirectedFrom=fulltext abs/pii/S0304387818305145 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 56 References References 57 Chandler, A. M. K., Walker, S. P., Connolly, K., & Grantham-McGregor, S. M. (1995). School breakfast improves verbal De Hoyos, R., Ganimian, A. J., & Holland, P. A. (2021). Teaching with the test: experimental evidence on diagnostic fluency in undernourished Jamaican children. The Journal of Nutrition, 125(4), 894-900. hiips://pubmed.ncbi.nlm. feedback and capacity building for public schools in Argentina. The World Bank Economic Review, 35(2), 499-520. nih.gov/7722692/ hiips://academic.oup.com/wber/article - abstract/35/2/499/5611142?redirectedFrom=fulltext Chetty, Raj, John N. Friedman, Nathaniel Hilger, Emmanuel Saez, Diane Whitmore Schanzenbach, and Danny Yagan. De Ree, J., Muralidharan, K., Pradhan, M., & Rogers, H. (2018). Double for nothing? Experimental evidence on an How does your kindergarten classroom affect your earnings? Evidence from Project STAR. The Quarterly journal of unconditional teacher salary increase in Indonesia. The Quarterly Journal of Economics, 133(2), 993-1039. hiips:// economics 126, no. 4 (2011): 1593-1660. https://academic.oup.com/qje/article/126/4/1593/1923939 academic.oup.com/qje/article/133/2/993/4622956 Cheung, M., & Berlin, M. P. (2015). The Impact of a Food for Education Program on Schooling in Cambodia. Asia & the Dembélé, M., Chudgar, A., & Ndow, I. (2016, December). The use of contract teachers in sub-Saharan Africa: A Pacific Policy Studies, 2(1), 44-57. hiips://onlinelibrary.wiley.com/doi/pdfdirect/10.1002/app5.21 review of the situation in 24 countries. In 9th International Policy Dialogue Forum, Secretariat of the International Chujan, W., & Kilenthong, W. T. (2021). Short-term impact of an early childhood education intervention Task Force on Teachers for Education (Vol. 2030, pp. 3-7). in rural Thailand. Journal of Human Capital, 15(2), 269-290. hiips://www.journals.uchicago.edu/doi/ Dembélé, M., Chudgar, A., & Ndow, I. (2016, December). The use of contract teachers in sub-Saharan Africa: A abs/10.1086/712727?journalCode=jhc review of the situation in 24 countries. In 9th International Policy Dialogue Forum, Secretariat of the International Cilliers, J., Fleisch, B., Prinsloo, C., & Taylor, S. (2020). “How to Improve Teaching Practice?: An Experimental Task Force on Teachers for Education (Vol. 2030, pp. 3-7). hiips://teachertaskforce.org/sites/default/files/2020 -07/ Comparison of Centralized Training and In-Classroom Coaching,” Journal of Human Resources, University of Contract%20teachers%20in%20sub- Saharan%20Africa_EN.pdf Wisconsin Press, vol. 55(3), pages 926-962. hiips://jhr.uwpress.org/content/early/2019/02/04/jhr.55.3.0618 Dillon, M. R., Kannan, H., Dean, J. T., Spelke, E. S., & Duflo, E. (2017). Cognitive science in the field: A preschool -9538R1 intervention durably enhances intuitive but not formal mathematics. Science, 357(6346), 47-55. hiips://www. Cilliers, J., Fleisch, B., Kotze, J., Mohohlwane, M., & Taylor, S. (2022). The challenge of sustaining effective teaching: science.org/doi/10.1126/science.aal4724 Spillovers, fade-out, and the cost-effectiveness of teacher development programs. Economics of Education Dinarte, L., & Egana-delSol, P. (2019). Preventing Violence in the Most Violent Contexts. hiips://ope nknowledge. Review, 87, 102215. hiips://doi.org/10.1016/j.econedurev.2021.102215 worldbank.org/server/api/core/bitstreams/2a0ddac6-5547-5eaa-8097- 465874906215/content DOI: 10.1257/ Clarke, S. E., Jukes, M. C., Njagi, J. K., Khasakhala, L., Cundill, B., Otido, J. & Brooker, S. (2008). Effect of intermittent aer.20191972 preventive treatment of malaria on health and education in schoolchildren: a cluster- randomised, double-blind, Dubois, P., de Janvry, A., & Sadoulet, E. (2012). Effects on School Enrollment and Performance of a Conditional Cash placebo-controlled trial. The Lancet, 372(9633), 127-138. hiips://www.ncbi.nlm.nih.gov/pmc/articles/PM C2495044/ Transfer Program in Mexico. Journal of Labor Economics, 30(3), 555–589. hiips://doi.org/10.1086/664928 Clarke, S. E., Rouhani, S., Diarra, S., Saye, R., Bamadio, M., Jones, R. & Sacko, M. (2017). Impact of a malaria Duflo, A., Kiessel, J., &  Lucas, A. (2020). Experimental Evidence on Alternative Policies to Increase Learning at intervention package in schools on Plasmodium infection, anaemia and cognitive function in schoolchildren in Scale, NBER Working Papers 27298, National Bureau of Economic Research, Inc.https://www.nber.org/system/ Mali: a pragmatic cluster-randomised trial. BMJ global health, 2(2), e000182. hiips://gh.bmj.com/content/2/2/ files/working_papers/w27298/w27298.pdf e000182 Duflo, E. (2001). Schooling and labor market consequences of school construction in Indonesia: Evidence from an Crawfurd, L., Evans, D., Hares, S., & Sandefur, J. (2021). Live Tutoring Calls Did Not Improve Learning during the unusual policy experiment. American Economic Review, 91(4), 795–813. hiips://doi.org/10.1257/aer.91.4.795 COVID-19 Pandemic in Sierra Leone. CGD Working Paper 591, Center for Global Development, Washington, DC. Duflo, E., Dupas, P., & Kremer, M. (2011). Peer effects, teacher incentives, and the impact of tracking: Evidence from hiips://www.cgdev.org/publication/live -tutoring-calls-did-not-improve- learning-during-covid-19-pandemic-sierra- a randomized evaluation in Kenya. American economic review, 101(5), 1739-1774. hiip://cega.berkeley.edu/assets/ leone cega_research_projects/14/Peer_Effects_and_the_Impact_of_Tracking.p df Croke, K., & Atun, R. (2019). The long run impact of early childhood deworming on numeracy and literacy: Duflo, E., Dupas, P., & Kremer, M. (2015). School governance, teacher incentives, and pupil–teacher ratios: Evidence from Uganda. PLOS Neglected Tropical Diseases, 13(1), e0007085. https://journals.plos.org/plosntds/ Experimental evidence from Kenyan primary schools. Journal of public Economics, 123, 92-110. hiip://cega. article?id=10.1371/journal.pntd.0007085 berkeley .edu/assets/cega_research_projects/14/School_Governance_and_Pupil- Teacher_Ratios.pdf Cruz-Aguayo, Y., Ibarrarán, P., & Schady, N. (2017). Do tests applied to teachers predict their effectiveness?. Durlak, J. A., Weissberg, R. P., Dymnicki, A. B., Taylor, R. D., & Schellinger, K. B. (2011). The impact of enhancing Economics Letters, 159, 108-111. hiips://www.sciencedirect.com/science/article/abs/pii/S0165176517302604 students’ social and emotional learning: A meta‐analysis of school‐based universal interventions. Child Dalton, A., Wolmarans, P., Witthuhn, R. C., van Stuijvenberg, M. E., Swanevelder, S. A., & Smuts, C. M. (2009). A development, 82(1), 405-432. hiips://srcd.onlinelibrary.wiley.com/doi/10.1111/ j.1467- 8624.2010.01564.x randomised control trial in schoolchildren showed improvement in cognitive function after consuming a bread Ebenezer, R., Gunawardena, K., Kumarendran, B., Pathmeswaran, A., Jukes, M. C., Drake, L. J., & de Silva, N. (2013). spread, containing fish flour from a marine source. Prostaglandins, leukotrienes and essential fatty acids, 80(2-3), Cluster-randomised trial of the impact of school-based deworming and iron supplementation on the cognitive 143-149. hiips://www.sciencedirect.com/science/article/abs/pii/S0952327808001944?casa_token=oI_FlS55_8cAA abilities of schoolchildren in Sri Lanka’s plantation sector. Tropical medicine & international health : TM & IH, 18(8), AAA:Tl1bWly_VXc4bhj6lzjAaYg0bL-Oey0a5BSOxD4j80kef_vQZpi0KV1JNPDAXXgXkfJRg-o 942–951. hiips://doi.org/10.1111/tmi.12128 De Barros, A. (2022). Explaining the productivity paradox: Experimental evidence from Education Technology. Edmonds, E., Feigenberg, B., & Leight, J. (2021). Advancing the agency of adolescent girls. The Review of Economics Retrieved April 17, 2023, from hiips://de-barros.com/publication/de-barros-explaining- 2022/de%20Barros%20 and Statistics, 1-46. hiips://direct.mit.edu/rest/article/doi/10.1162/rest_a_01074/102825/Advancing-the-Agency-of- -%202022%20-%20Explaining%20the%20Productivity%20Paradox.pdf Adolescent- Girls De Hoyos Navarro, R. E., Garcia-Moreno, V. A., & Patrinos, H. A. (2015). The impact of an accountability intervention Engle, P. L., Fernald, L. C., Alderman, H., Behrman, J., O’Gara, C., Yousafzai, A., ... & Iltus, S. (2011). Strategies for with diagnostic feedback: Evidence from Mexico. World Bank Policy Research Working Paper, (7393). hiips:// reducing inequalities and improving developmental outcomes for young children in low- income and middle- documents1.worldbank.org/curated/en/297561468188928817/pd f/WPS7393.pdf 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 58 References References 59 income countries. The lancet, 378(9799), 1339-1353. https://digitalcommons.calpoly.edu/cgi/viewcontent. Freeman, M. C., Greene, L. E., Dreibelbis, R., Saboori, S., Muga, R., Brumback, B., & Rheingans, R. (2012). Assessing cgi?referer=hiips://scholar.google.com/&httpsredir=1& article=1049&context=psycd_fac the impact of a school-based water treatment, hygiene and sanitation programme on pupil absence in Nyanza Escueta, M., Quan, V., Nickow, A. J., & Oreopoulos, P. (2017). Education technology: An evidence-based review. Province, Kenya: a cluster-randomized trial. Tropical medicine & international health, 17(3), 380-391. hiips:// hiips://www.nber.org/papers/w23744 onlinelibrary.wiley.com/doi/pdfdirect/10.1111/j.1365 -3156.2011.02927.x Estrada, R. (2017). Rules vs. Discretion in public service: Teacher hiring in Mexico. hiips://scioteca.caf.com/ Gallego, F., Näslund-Hadley, E., & Alfonso, M. (2018). Tailoring instruction to improve mathematics skills in bitstream/handle/123456789/1083/Est rada%20%282017%29%20Rules%20vs.% 20Discretion%20%28003%29. preschools: A randomized evaluation. hiip://repositorio.minedu.gob.pe/handle/20.500.12799/5781 pdf?sequence=1&isAllowed=y Ganimian, A. J., Muralidharan, K., & Walters, C. R. (2021). Augmenting state capacity for child development: Evans, D. (2018). The Power of a Label: Merit Scholarships vs needs-based scholarships? World Bank Blogs. Experimental evidence from India (No. w28780). National Bureau of Economic Research. hiips://www.nber.org/ Retrieved April 17, 2023, from hiips://blogs.worldbank.org/developmenttalk/power -label-merit- scholarships-vs- system/files/working_papers/w28780/w28780.pdf needs-based-scholarships Ganimian, A., Muralidharan, K., & Walters, C. R. (2020). Improving Early-Childhood Human Development: Evans, D. K., & Yuan, F. (2022). What we learn about girls’ education from interventions that do not focus on girls. The Experimental Evidence from India. In Society for Research on Educational Effectiveness Conference. hiips://www. World Bank Economic Review, 36(1), 244-267. https://academic.oup.com/wber/article/36/1/244/6278419?login=false sree.org/assets/conferences/2020s/virtual/InvitedSymposiumGlobal_P2_slides.pdf Evans, D., Lima, L., Olinto, P., & Schady, N. (2017). Impact of free availability of public childcare on labour supply and Garn, J. V., Greene, L. E., Dreibelbis, R., Saboori, S., Rheingans, R. D., & Freeman, M. C. (2013). A cluster-randomized child development in Brazil. hiips://developmentevidence.3ieimpact.org/searc h-result- details/impact-evaluation- trial assessing the impact of school water, sanitation and hygiene improvements on pupil enrolment and gender repository/impact-of-free-availability-of-public-childcare-on-labour-supply-and- child-development-in-brazil/4581 parity in enrolment. Journal of water, sanitation and hygiene for development, 3(4), 592-601. hiips://iwaponline. Fafchamps, M., & Mo, D. (2018). Peer effects in computer assisted learning: evidence from a randomized experiment. com/washdev/article -abstract/3/4/592/30334/A-cluster-randomized-trial- assessing-the-impact-of Experimental Economics, 21, 355-382. hiips://www.nber.org/system/files/working_papers/w23195/w23195.pdf Gertler, P., Heckman, J. J., Pinto, R., Chang, S. M., Grantham-McGregor, S., Vermeersch, C., & Wright, Fairlie, R. W., & Robinson, J. (2013). Experimental evidence on the effects of home computers on academic A. (2021). Effect of the jamaica early childhood stimulation intervention on labor market outcomes at age 31 (No. achievement among schoolchildren. American Economic Journal: Applied Economics, 5(3), 211-240. hiips://papers. w29292). National Bureau of Economic Research. hiips://www.nber.org/system/files/working_papers/w29292/ ssrn.com/sol3/papers.cfm?abstract_id=2266806 w29292.pdf Fernando, D., De Silva, D., Carter, R., Mendis, K. N., & Wickremasinghe, R. (2006). A randomized, double-blind, Gertler, P., Heckman, J., Pinto, R., Zanolini, A., Vermeersch, C., Walker, S., & Grantham-McGregor, S. (2014). Labor placebo-controlled, clinical trial of the impact of malaria prevention on the educational attainment of school market returns to an early childhood stimulation intervention in Jamaica. Science, 344(6187), 998-1001. hiips:// children. The American journal of tropical medicine and hygiene, 74(3), 386-393. hiips://www.researchgate.net/ www.science.org/doi/full/10.1126/science.1251178 profile/Deepika -Fernando/publication/7253446_A_randomized_double- blind_placebo- controlled_clinical_trial_of_ Gertler, P., Heckman, J., Pinto, R., Zanolini, A., Vermeersch, C., Walker, S., & Grantham-McGregor, S. (2014). Labor the_impact_of_Malaria_prevention_on_the_educational_attainment_of_school market returns to an early childhood stimulation intervention in Jamaica. Science, 344(6187), 998-1001. hiips:// _children/links/53d45a830cf2a7fbb2ea0a46/A-randomized-double-blind-placebo-controlled-clinical-trial- of-the- www science org/doi/full/10.1126/science.1251178 impact-of-Malaria-prevention-on-the-educational-attainment-of-school-children.pdf Glewwe, P., Ilias, N., & Kremer, M. (2010). Teacher incentives. American Economic Journal: Applied Economics, 2(3), Fernando, S. D., Gunawardene, D. M., Bandara, M. R., De Silva, D., Carter, R., Mendis, K. N., & Wickremasinghe, 205-227. hiips://papers.ssrn.com/sol3/papers.cfm?abstract_id=1701522 A. R. (2003). The impact of repeated malaria attacks on the school performance of children. hiips:// Glewwe, P., Kremer, M., & Moulin, S. (2009). Many children left behind? Textbooks and test scores in Kenya. www.ajtmh.org/configurable/content/journals$002ftpmd$002f69$002f6$002farticle - p582. American Economic Journal: Applied Economics, 1(1), 112-135. hiips://www.nber.org/papers/w13300 xml?t:ac=journals%24002ftpmd%24002f69%24002f6%24002farticle-p582.xml Glewwe, P., Park, A., & Zhao, M. (2016). A better vision for development: Eyeglasses and academic performance in Fiala, N., Garcia-Hernandez, A., Narula, K., & Prakash, N. (2022, February 10). Wheels of Change: Transforming rural primary schools in China. Journal of Development Economics, 122, 170-182. hiips://www.sciencedirect.com/ Girls’ lives with bicycles. EconPapers. Retrieved April 17, 2023, from hiips://econpapers.repec.org/paper/ science/article/abs/pii/S0304387816300402 uctuconnp/2022 -04.htm Gormley Jr, W. T., Phillips, D., & Anderson, S. (2018). The effects of Tulsa’s Pre‐K program on middle school student Filmer, D., & Schady, N. (2009). School enrollment, selection and test scores - documents1.worldbank.org. Retrieved performance. Journal of Policy Analysis and Management, 37(1), 63-87. hiips://onlinelibrary.wiley.com/doi/ April 17, 2023, from hiips://documents1.worldbank.org/curated/en/495041468224995463/pdf/WPS4998.pdf full/10.1002/pam.22023 Filmer, D., Habyarimana, J., & Sabarwal, S. (2020). Teacher performance-based incentives and learning inequality. Grantham-McGregor, S., Adya, A., Attanasio, O., Augsburg, B., Behrman, J., Caeyers, B., ... & Vats, K. (2020). Group hiips://elibrary.worldbank.org/doi/abs/10.1596/1813 -9450-9382 sessions or home visits for early childhood development in India: a cluster RCT. Pediatrics, 146(6). Filmer, D., Rogers, H., Angrist, N., & Sabarwal, S. (2020). Learning-adjusted years of schooling (LAYS): Defining a new hiips://publications.aap.org/pediatrics/article/146/6/e2020002725/77094?casa_token=Wv2Wiutq8K4AAA macro measure of education. Economics of Education Review, 77, 101971. hiips://openknowledge.worldbank.org/ AA:FRNeWBrHW3jLfLAM68etf5bHr9hibi- JjCOflmvhxdwXsc1Rrroxw7QLLlC8ZMglkwAY8lE?autologincheck=redirected server/api/core/bitstreams/aa3a096f -333a-55e7-84ab- 7e59795c11ff/content Gray-Lobe, G., Keats, A., Kremer, M., Mbiti, I., & Ozier, O. W. (2022). Can Education be Standardized? Evidence from Fleisch, B., Schöer, V., Roberts, G., & Thornton, A. (2016). System-wide improvement of early-grade mathematics: Kenya. Evidence from Kenya (June 5, 2022). The University of Chicago, Becker Friedman Institute for Economics New evidence from the Gauteng Primary Language and Mathematics Strategy. Working Paper, (2022-68). hiips://papers.ssrn.com/sol3/papers.cfm?abstract_id=4129184 International Journal of Educational Development, 49, 157-174. hiips://www.sciencedirect.com/science/article/abs/pii/ S0738059316300220?via%3Dihub 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 60 References References 61 Gutierrez, I. A., Molina, O., & Ñopo, H. (2018). Stand against bullying: An experimental school intervention. hiips:// Jayaraman, R., & Simroth, D. (2015). The impact of school lunches on primary school enrollment: evidence from repositorio.minedu.gob.pe/bitstream/handle/20.500.12799/6429/Stand%20Against%20Bullying%2 0An%20 India’s midday meal scheme. The Scandinavian Journal of Economics, 117(4), 1176-1203. hiips://onlinelibrary.wiley. Experimental%20School%20Intervention.pdf?sequence=1&isAllowed=y com/doi/full/10.1111/sjoe.12116 Halliday, K. E., Okello, G., Turner, E. L., Njagi, K., Mcharo, C., Kengo, J., & Brooker, S. J. (2014). Impact of intermittent Jensen, R. (2010). The (perceived) returns to education and the demand for schooling. The screening and treatment for malaria among school children in Kenya: a cluster randomised trial. PLoS medicine, 11(1), Quarterly Journal of Economics, 125(2), 515-548. hiips://academic.oup.com/qje/article- e1001594. abstract/125/2/515/1882172?redirectedFrom=fulltext hiips://journals.plos.org/plosmedicine/article?id=10.1371/jou rnal.pmed.1001594 Jensen, S. K., Placencio-Castro, M., Murray, S. M., Brennan, R. T., Goshev, S., Farrar, J., ... & Betancourt, T. S. (2021). Hamadani, J. D., Mehrin, S. F., Tofail, F., Hasan, M. I., Huda, S. N., Baker-Henningham, H., ... & Grantham-McGregor, Effect of a home-visiting parenting program to promote early childhood development and prevent violence: a S. (2019). Integrating an early childhood development programme into Bangladeshi primary health-care services: cluster-randomized trial in Rwanda. BMJ global health, 6(1), e003508. hiips://gh.bmj.com/content/6/1/e003508. an open-label, cluster-randomised controlled trial. The Lancet Global Health, 7(3), e366-e375. hiips://www abstract sciencedirect com/science/article/pi i/S2214109X18305357 Jeong, J., Franchett, E. E., Ramos de Oliveira, C. V., Rehmani, K., & Yousafzai, A. K. (2021). Parenting interventions to Hamory, J., Miguel, E., Walker, M., Kremer, M., & Baird, S. (2021). Twenty-year economic impacts of deworming. promote early child development in the first three years of life: A global systematic review and meta-analysis. PLoS Proceedings of the National Academy of Sciences, 118(14), e2023185118. hiips://www.pnas.org/doi/full/10.1073/ medicine, 18(5), e1003602. hiips://journals.plos.org/plosmedicine/article?id=10.1371/journal.pmed.1003602 pnas.2023185118 Johnson, R. C., & Jackson, C. K. (2019). Reducing inequality through dynamic complementarity: Evidence from Hasan, A. (2019). Contrasting Experiences understanding the Longer-Term Impact of Improving access to preschool Head Start and public school spending. American Economic Journal: Economic Policy, 11(4), 310-349. hiips://www. Education in Rural Indonesia. World Bank Policy Research Working Paper, (9060). hiips://documents1.worldbank. aeaweb.org/articles?id=10.1257/pol.20180510 org/curated/en/689351574170836739/pdf/Contrasting -Experiences- Understanding-the-Longer-Term-Impact-of- Kachingwe, M., Chikowe, I., Van der Haar, L., & Dzabala, N. (2021). Assessing the impact of an intervention project Improving-Access-to-Preschool-Education-in-Rural- Indonesia.pdf by the young women’s christian association of Malawi on psychosocial well-being of adolescent mothers Hassan, H., Islam, A., Siddique, A., & Wang, L. C. (2021). Munich Papers in Political Economy. hiips://www.monash.edu/ and their children in Malawi. Frontiers in public health, 9, 585517. hiips://www.frontiersin.org/articles/10.3389/ data/assets/pdf_file/0007/265141 6/WP2021n03_V1.pdf fpubh.2021.585517/full Heckman, J. J., Liu, B., Lu, M., & Zhou, J. (2020). Treatment effects and the measurement of skills in a prototypical Kazianga, H., De Walque, D., & Alderman, H. (2012). Educational and child labour impacts of two food- for-education home visiting program. hiips://docs.iza.org/dp13346.pdf schemes: Evidence from a randomised trial in rural Burkina Faso. Journal of African Economies, 21(5), 723-760. Hernandez-Agramonte, J. M., Namen, O., & Naslund-Hadley, E. (2022). Supporting Early Childhood Development hiips://academic.oup.com/jae/article - abstract/21/5/723/784400?redirectedFrom=fulltext Remotely: Experimental Evidence from SMS Messages. hiips://papers.ssrn.com/sol3/papers.cfm?abstract_id=4099125 Kleiman-Weiner, M., Luo, R., Zhang, L., Shi, Y., Medina, A., & Rozelle, S. (2013). Eggs versus chewable vitamins: Which Hjort, J., Moreira, D., Rao, G., & Santini, J. F. (2021). How research affects policy: Experimental evidence from 2,150 intervention can increase nutrition and test scores in rural China?. China Economic Review, 24, 165-176. brazilian municipalities. American Economic Review, 111(5), 1442-80. hiips://www.nber.org/papers/w25941 hiips://www.sciencedirect.com/science/article/abs/pii/S1043951X120012 53?casa_token=IFno- Holla, A., Bendini, M., Dinarte, L., & Trako, I. (2021). Is Investment in Preprimary Education Too Low? Lessons from 0PJ5LAAAAAA:expRl6UdT2d14Jh7vg3pc25JNTvcROESm_K0adZXLCqIccUzXp6WcX_QcCtBF4fAWuE TDqM (Quasi) Experimental Evidence across Countries. hiips://openknowledge.worldbank.org/handle/10986/35894 Krämer, M., Kumar, S., & Vollmer, S. (2021). Improving child health and cognition: Evidence from a school-based Stockard, J., Wood, T. W., Coughlin, C., & Khoury, C. R. (2018). The Effectiveness of Direct Instruction Curricula: A nutrition intervention in India. The Review of Economics and Statistics, 103(5), 818-834. hiips://direct.mit.edu/rest/ Meta-Analysis of a Half Century of Research. Review of Educational Research, 88(4), 479–507. hiip://www.jstor.org/ article -pdf/103/5/818/1975675/rest_a_00950.pdf stable/45277261 Kremer, M., Brannen, C., & Glennerster, R. (2013). The challenge of education and learning in the developing world. Hulett, J. L., Weiss, R. E., Bwibo, N. O., Galal, O. M., Drorbaugh, N., & Neumann, C. G. (2014). Animal source foods have Science, 340(6130), 297-300. hiips://www.science.org/doi/full/10.1126/science.1235350?casa_token=Gstp808_ a positive impact on the primary school test scores of Kenyan schoolchildren in a cluster-randomised, controlled JHMAAAAA:ITQmbogw 0kD1njCkmFiefKaDnv8CTLFx-j_QqLbPAMAtZ4jjGkDCwLccg0K04B60jx7WU9X4ZPTXMA feeding intervention trial. British Journal of Nutrition, 111(5), 875-886. hiips://pubmed.ncbi.nlm.nih.gov/24168874/ Kuper, H., Saran, A., White, H., Kumar, S. T., Tolin, L., Muthuvel, T., & Wapling, L. (2018). Rapid evidence assessment Husain, N., Kiran, T., Fatima, B., Chaudhry, I. B., Husain, M., Shah, S., & Chaudhry, N. (2021). An integrated parenting (REA) of what works to improve educational outcomes for people with disabilities in low-and middle-income intervention for maternal depression and child development in a low-resource setting: Cluster randomized controlled countries. Campbell Collaboration: New Delhi, and International Centre for Evidence on Disability: London. hiips:// trial. Depression and anxiety, 38(9), 925-939. hiips://onlinelibrary.wiley.com/doi/abs/10.1002/da.23169 www gov uk/research -for-development-outputs/improving- educational-outcomes-for-people-with-disabilities-in- Islam, A., Wang, L. C., & Hassan, H. (2022). Delivering Remote Learning Using a Low-Tech Solution: Evidence from low-and-middle-income-countries-why-does-it-matter- and-what-works an RCT during the Covid-19 Pandemic (No. 43). EdTech Hub. hiips://docs.edtechhub.org/lib/FE3VBQQW/download/ Lai, F., Luo, R., Zhang, L., Huang, X., & Rozelle, S. (2015). Does computer-assisted learning improve learning FWEPGNS3/Islam%20et%20al.%20 -%202022%20-%20Delivering%20Remote%20Learning%20Using%20a%20Low- outcomes? Evidence from a randomized experiment in migrant schools in Beijing. Economics of Education Review, Tech%20Soluti.pdf 47, 34-48. hiips://www.sciencedirect.com/science/article/abs/pii/S027277571500045X Ito, H., Kasai, K., Nishiuchi, H., & Nakamuro, M. (2021). Does Computer-Aided Instruction Improve Children’s Cognitive Lai, F., Zhang, L., Hu, X., Qu, Q., Shi, Y., Qiao, Y., ... & Rozelle, S. (2013). Computer assisted learning as extracurricular and Noncognitive Skills?. Asian Development Review, 38(1), 98-118. hiips://direct.mit.edu/adev/article/38/1/98/98308/ tutor? Evidence from a randomised experiment in rural boarding schools in Shaanxi. Does -Computer-Aided-Instruction-Improve-Children-s Journal of Development Effectiveness, 5(2), 208-231. hiips://www.tandfonline.com/doi/ abs/10.1080/19439342.2013.780089?journalCode=rjde20 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 62 References References 63 Lai, F., Zhang, L., Qu, Q., Hu, X., Shi, Y., Boswell, M., & Rozelle, S. (2015). Teaching the language of wider Mahawithanage, S. T. C., Kannangara, K. K. N. P., Wickremasinghe, R., Chandrika, U. G., Jansz, E. R., Karunaweera, communication, minority students, and overall educational performance: Evidence from a randomized experiment N. D., & Wickremasinghe, A. R. (2007). Impact of vitamin A supplementation on health status and absenteeism of in Qinghai Province, China. Economic Development and Cultural Change, 63(4), 753-776. hiips://www.journals. school children in Sri Lanka. Asia Pacific journal of clinical nutrition, 16(1), 94-102. hiips://apjcn.nhri.org.tw/server/ uchicago.edu/doi/10.1086/681233 apjcn/16/1/94.pdf Lavinas, L., & Veiga, A. (2013). Brazil’s one laptop per child program: impact evaluation and Major, L., Francis, G. A., & Tsapali, M. (2021). The effectiveness of technology-supported personalised learning in implementation assessment. Cadernos de pesquisa, 43, 542-569. hiips://www.scielo.br/j/cp/a/ low-and middle-income countries: A meta-analysis. British Journal of Educational Technology, 52(5), 1935-1964. svTdLfvdMD89B5tKL3YXyzH/?format=pdf&lang=en hiips://bera-journals.onlinelibrary.wiley.com/doi/epdf/10.1111/bjet.13116 Leaver, Clare, Owen Ozier, Pieter Serneels, and Andrew Zeitlin. (2021). “Recruitment, Effort, and Retention Effects Makamu, F., Azam, M., & Kazianga, H. (2018). Returns to controlling a neglected tropical disease: schistosomiasis of Performance Contracts for Civil Servants: Experimental Evidence from Rwandan Primary Schools.” American control programme and education outcomes in Nigeria. Journal of African Economies, 27(5), 538-557. hiips:// Economic Review, 111 (7): 2213-46.https://www.aeaweb.org/articles?id=10.1257/aer.20191972 academic.oup.c om/jae/article-abstract/27/5/538/4967902?redirectedFrom=fulltext Lichand, G., & Christen, J. (2021). Behavioral nudges prevent student dropouts in the pandemic (No. 363). Working Maluccio, J. A., Hoddinott, J., Behrman, J. R., Martorell, R., Quisumbing, A. R., & Stein, A. D. (2009). The Impact of Paper. https://www.ccwd.uzh.ch/assets/pdf/preventing_students_dropout_in_the_pandemic.pdf Improving Nutrition During Early Childhood on Education among Guatemalan Adults. The Economic Journal, Liu, C., Lu, L., Zhang, L., Luo, R., Sylvia, S., Medina, A., ... & Zhu, T. (2017). Effect of deworming on indices of health, 119(537), 734–763. hiips://doi.org/10.1111/j.1468-0297.2009.02220.x cognition, and education among schoolchildren in rural China: a cluster-randomized controlled trial. The American Martinez, S., Naudeau, S., & Pereira, V. A. (2017). Preschool and child development under extreme poverty: evidence journal of tropical medicine and hygiene, 96(6), 1478. hiips://www.ncbi.nlm.nih.gov/pmc/articles/PMC5462590/ from a randomized experiment in rural Mozambique. World Bank Policy Research Working Paper, (8290). hiips:// Locks, L. M., Manji, K. P., McDonald, C. M., Kupka, R., Kisenge, R., Aboud, S., & Duggan, C. P. (2017). documents1.worldbank.org/curated/en/756171513961080112/pdf/WPS8290.pdf The effect of daily zinc and/or multivitamin supplements on early childhood development in Mbiti, I., Muralidharan, K., Romero, M., Schipper, Y., Manda, C., & Rajani, R. (2019). Inputs, incentives, and Tanzania: results from a randomized controlled trial. Maternal & child nutrition, 13(2), e12306. hiips:// complementarities in education: Experimental evidence from Tanzania. The Quarterly Journal of Economics, onlinelibrary.wiley.com/doi/pdfdirect/10.1111/mcn.12306?casa_token=jftKF5PAt1MAAA AA:ElcSza 134(3), 1627-1673. hiips://econweb.ucsd.ed u/~kamurali/papers/Working%20Papers/IIC%20(Current%20WP).pdf X5kEGNICH7SOxIl0NsctRUnW9ncuL0U26RPW5nAQm0wOYEaRXTO4rAajq3lpv7Evqk28cxcw Meiring, T. (2021). Impact evaluation of Funda Wande in-service teacher coaching intervention. hiips://sa-tied.wider. Lopez Garcia, I., Saya, U. Y., & Luoto, J. E. (2021). Cost-effectiveness and economic returns of group- based parenting unu.edu/sites/default/files/SA-TIED-WP156_1.pdf interventions to promote early childhood development: Results from a randomized controlled trial in rural Kenya. Menendez, A., Hoadley, U., & Solovyeva, A. (2021). READ Liberia Impact Evaluation Endline Report. hiips://pdf.usaid. PLoS Medicine, 18(9), e1003746. gov/pdf_docs/PA00Z92Q.pdf hiips://journals.plos.org/plosmed icine/article?id=10.1371/journal.pmed.1003746 Miguel, E., & Kremer, M. (2004). Worms: identifying impacts on education and health in the presence of treatment Loureiro, A., Cruz, L., Lautharte, I., & Evans, D. K. (2020). The State of Ceara in Brazil is a role model for externalities. Econometrica, 72(1), 159-217. hiips://onlinelibrary.wiley.com/doi/abs/10.1111/j.1468 -0262.2004.00481.x reducing learning poverty. World Bank, Washington DC hiips://openknowledge.worldbank.org/entities/ Mo, D., Bai, Y., Shi, Y., Abbey, C., Zhang, L., Rozelle, S., & Loyalka, P. (2020). Institutions, implementation, publication/487d228e -6395-5f31-9bfa-1eb3801ad889 and program effectiveness: Evidence from a randomized evaluation of computer- assisted learning in rural Loyalka, P., Popova, A., Li, G., Liu, C., & Shi, H. (2017). Unpacking teacher professional development. REAP Working China. Journal of Development Economics, 146, 102487. hiips://www.sciencedirect.com/science/article/abs/pii/ Paper 314. S0304387820300626?via%3Dihub hiips://kingcenter.stanford.edu/sites/g/files/ sbiybj16611/files/media/file/1009wp_0.pdf Moscoviz, L., & Evans, D. K. (2022). Learning loss and student dropouts during the covid-19 pandemic: A review of Loyalka, P., Sylvia, S., Liu, C., Chu, J., & Shi, Y. (2019). Pay by design: Teacher performance pay design and the the evidence two years after schools shut down (pp. 1-24). Center for Global Development. hiips://www.ungei.org/ distribution of student achievement. Journal of Labor Economics, 37(3), 621-662. hiips://www.journals.uchicago. sites/default/files/2022 -04/learning-loss-and-student-dropouts-during-covid-19- pandemic-review-evidence-two- edu/doi/epdf/10.1086/702625 years.pdf Luo, R., Emmers, D., Warrinnier, N., Rozelle, S., & Sylvia, S. (2019). Using community health workers to deliver a Muralidharan, K., & Prakash, N. (2013, August 16). Cycling to school: Increasing secondary school enrollment for girls scalable integrated parenting program in rural China: A cluster-randomized controlled trial. Social Science & in India. NBER. Retrieved April 17, 2023, from hiips://www nber org/papers/w19305 Medicine, 239, 112545. hiips://www.sciencedirect.com/science/article/abs/pii/S0277953619305398 Muralidharan, K., & Sundararaman, V. (2010). The impact of diagnostic feedback to teachers on student learning: Luo, R., Shi, Y., Zhang, L., Liu, C., Rozelle, S., Sharbono, B., ... & Martorell, R. (2012). Nutrition and educational Experimental evidence from India. The Economic Journal, 120(546), F187-F203. hiips://econweb.ucsd.edu/~kamur performance in rural China’s elementary schools: Results of a randomized control trial in Shaanxi Province. ali/papers/Published%20Articles/Publisher%20Formatted%20Versions/ Muralidharan,%20Sundararaman%20-%20 Economic development and cultural change, 60(4), 735-772. hiips://www.jstor.org/stable/10.1086/665606 Diagnostic%20Feedback%20-%20Published%20Version%20(EJ).pdf Ma, X., Wang, H., Shi, Y., Sylvia, S., Wang, L., Qian, Y., & Rozelle, S. (2021). Improving learning by improving vision: Muralidharan, K., & Sundararaman, V. (2011). Teacher performance pay: Experimental evidence from India. Journal of evidence from two randomized controlled trials of providing vision care in China. Journal of Development political Economy, 119(1), 39-77. hiips://www journals uchicago.edu/doi/10.1086/659655 Effectiveness, 13(1), 1-26. hiips://www.tandfonline.com/doi/abs/10.1080/19439342.2021.1876139?journalCode=rjde20 Muralidharan, K., & Sundararaman, V. (2013). Contract teachers: Experimental evidence from India (No. w19440). Ma, Y., Fairlie, R. W., Loyalka, P., & Rozelle, S. (2020). Isolating the “Tech” from Edtech: experimental evidence on National Bureau of Economic Research. hiips://www nber org/papers/w19440 computer assisted learning in China (No. w26953). National Bureau of Economic Research. hiips://www.nber.org/ papers/w 26953 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 64 References References 65 Muralidharan, K., Singh, A., & Ganimian, A. J. (2019). Disrupting education? Experimental evidence on s10833-018-9325-4?utm_medium=affiliate&utm_source=commission_junction&utm_campaign=3_nsn6445_ technology-aided instruction in India. American Economic Review, 109(4), 1426-1460. hiips://www.aeaweb.org/ brand_PID4003003&utm_content=de_textlink articles?id=10.1257/aer.20171112 Popova, A., Evans, D. K., Breeding, M. E., & Arancibia, V. (2022). Teacher professional development around the Neilson, C., Castro, F., Molina, O. (2019). The Impact of Information Provision on Human Capital Accumulation and world: The gap between evidence and practice. The World Bank Research Observer, 37(1), 107-136. hiips:// Child Labor in Peru. hiips://poverty-action.org/study/information-about-returns-post- primary-education-peru documents.worldbank.org/en/publication/documents - reports/documentdetail/349051535637296801/teacher- Nguyen, T. (2013). Information, role models and perceived returns to education experimental professional-development-around-the-world-the- gap-between-evidence-and-practice evidence from Madagascar. hiips://documents1.worldbank.org/curated/en/655701468053368779/ Powell, C. A., Walker, S. P., Chang, S. M., & Grantham-McGregor, S. M. (1998). Nutrition and education: a randomized pdf/813810BRI0Info00Box379814B 00PUBLIC0.pdf trial of the effects of breakfast in rural primary school children. The American journal of clinical nutrition, 68(4), Ngware, M. W., Hungi, N., Wekulo, P., Mutisya, M., Njagi, J., Muhia, N., & Mambe, S. (2018). Impact Evaluation of 873-879. hiips://academic.oup.com/ajcn/article/68/4/873/4648663 Tayari School Readiness Program in Kenya. Endline Version. African Population and Health Research Center, RTI Pradhan, M., Suryadarma, D., Beatty, A., Wong, M., Gaduh, A., Alisjahbana, A., & Artha, R. P. (2014). Improving International, and Children’s Investment Fund Foundation. Consulté le May, 6, 2021. hiips://aphrc.org/wp- content/ educational quality through enhancing community participation: Results from a randomized field experiment in uploads/2019/07/Kenya_ECDE_ImpactEvalution_TayariFINAL_20022019.pdf Indonesia. American Economic Journal: Applied Economics, 6(2), 105-126. hiips://openknowledge.worldbank.org/ Ngware, M., Hungi, N., Wekulo, P., Mutisya, M., Njagi, J., Muhia, N., & Mambe, S. (2018). Impact evaluation of Tayari entities/publication/cf63d5f9 -2007-5e25-bd07-5cd1e69830ca school readiness program in Kenya. hiips://aphrc.org/wp- content/uploads/2019/07/Impact_Evaluation_ECDE_ Pritchett, L. (2015). Creating education systems coherent for learning outcomes: Making the transition from schooling Tayari-long-report.pdf to learning. Research on Improving Systems of Education (RISE). hiips://riseprogramme.org/sites/default/files/ Nyqvist, M. B., & Guariso, A. (2022). Supporting learning in and out of school: Experimental evidence from India. publications/RISE_WP -005_Pritchett_0.pdf Working Paper, Retrieved April 17, 2023, from hiips://www.povertyactionlab.org/sites/default/files/rese arch-paper/ Radhakrishnan, K., Sabarwal, S., Sharma, U., Cullen, C., Crossley, C., Letsomo, T., & Angrist, N. (2021). WP3731_Learning-in-India_Nyqvist-et- all-June2022.pdf Remote Learning: Evidence from Nepal during COVID-19. World Bank, Washington, DC. hiip://hdl. handle. Perera, M., & Aboal, D. (2020, May 12). The impact of a mathematics computer‐assisted learning platform on net/10986/36031. hiips://www.researchgate.net/profile/Uttam -Sharma- 4/publication/353549026_Remote_ students’ mathematics test scores. CINVE. Retrieved April 17, 2023, from hiips://cinve.org.uy/the-impact-of-a- Learning_Evidence_from_Nepal_during_COVID- 19/links/610279e91e95fe241a96945c/Remote-Learning-Evidence- mathematics-computer%e2%80%90assisted-learning-platform-on- students-mathematics-test-scores/ from-Nepal-during-COVID-19.pdf Pia R Britto, Stephen J Lye, Kerrie Proulx, Aisha K Yousafzai, Stephen G Matthews, Tyler Vaivada, Rafael Perez- Rodriguez-Segura, D. (2022). EdTech in developing countries: A review of the evidence. The Escamilla, Nirmala Rao, Patrick Ip, Lia C H Fernald, Harriet MacMillan, Mark Hanson, Theodore D Wachs, Haogen World Bank Research Observer, 37(2), 171-203. hiips://academic.oup.com/wbro/article - Yao, Hirokazu Yoshikawa, Adrian Cerezo, James F Leckman, Zulfiqar A Bhutta, Nurturing care: promoting early abstract/37/2/171/6333790?redirectedFrom=fulltext childhood development, The Lancet, Volume 389, Issue 10064, 2017, Pages 91-102, ISSN 0140-6736,hiips://doi. Rose, Pauline, & Yorke, Louise. (2019). General versus Girl-Targeted Interventions: A False Dichotomy? A response to org/10.1016/S0140-6736(16)31390-3 Evans and Yuan (2019). Zenodo. hiips://doi.org/10.5281/zenodo.3249882 Piper, B., & Korda Poole, M. (2010). Early Grade Reading Assessment (EGRA) Plus: Liberia. Program evaluation report. Rossin-Slater, M., & Wüst, M. (2020). What is the added value of preschool for poor children? long-term and RTI International. hiips://www.eddataglobal.org/documents/index.cfm?fuseaction=pubDetail&ID=283 intergenerational impacts and interactions with an infant health intervention. American Economic Journal: Applied Piper, B., Destefano, J., Kinyanjui, E. M., & Ong’ele, S. (2018). Scaling up successfully: Lessons from Kenya’s Tusome Economics, 12(3), 255-286. hiips://www.aeaweb.org/articles?id=10.1257/app.20180698 national literacy program. Journal of Educational Change, 19, 293-321. hiips://link.springer.com/article/10.1007/ Sabarwal, S., Evans, D., & Marshak, A. (2014). The permanent input hypothesis: the case of textbooks and (no) s10833 -018-9325- 4?utm_medium=affiliate&utm_source=commission_junction&utm_campaign=3_nsn6445_brand_ student learning in Sierra Leone. World Bank Policy Research Working Paper, (7021). hiips://documents1. PID4003 003&utm_content=de_textlink worldbank.org/curated/en/806291468299200683/pdf/WP S7021.pdf Piper, B., Sitabkhan, Y., & Nderu, E. (2018). Mathematics from the beginning: Evaluating the Tayari Preprimary Singh, A. (2020). Myths of official measurement: Auditing and improving administrative data in developing countries. Program’s impact on early mathematics skills. Global Education Review, 5(3), 57-81. hiips://ger.mercy.edu/index. Research on Improving Systems of Education (RISE) Working Paper, 42. hiips://riseprogramme.org/sites/default/ php/ger/article/view/434 files/2020 -11/RISE_WP-042-Singh.pdf Piper, B., Zuilkowski, S. S., & Mugenda, A. (2014). Improving reading outcomes in Kenya: First-year effects of the Singh, A., Romero, M., & Muralidharan, K. (2022). COVID-19 Learning loss and recovery: Panel data evidence from PRIMR Initiative. International Journal of Educational Development, 37, 11-21. hiips://pdf.usaid.gov/pdf_docs/ India (No. w30552). National Bureau of Economic Research. hiips://www.nber.org/papers/w30552 PA00K27S.pdf Singla, D. R., Shafique, S., Zlotkin, S. H., & Aboud, F. E. (2014). A 22-element micronutrient powder benefits language Piper, B., Zuilkowski, S. S., Dubeck, M., Jepkemei, E., & King, S. J. (2018). Identifying the essential ingredients to but not cognition in Bangladeshi full-term low-birth-weight children. The Journal of nutrition, 144(11), 1803-1810. literacy and numeracy improvement: Teacher professional development and coaching,student textbooks, and hiips://academic.oup.com/jn/article/144/11/1803/4644509 structured teachers’ guides. World Development, 106, 324-336. https://www.sciencedirect.com/science/article/pii/ Smart, A. (2021). Teachers’ guides: Isn’t that what they should be? . IARTEM e-journal Vol 13. No. 1 hiips://doi. S0305750X18300287 org/10.21344/iartem.v13i1.971 Piper, B., Destefano, J., Kinyanjui, E. M., & Ong’ele, S. (2018). Scaling up successfully: Lessons from Kenya’s Tusome Snilstveit, B, Stevenson, J, Phillips, D, Vojtkova, M, Gallagher, E, Schmidt, T, Jobse, H, Geelen, M, Pastorello, M, and national literacy program. Journal of Educational Change, 19, 293-321. https://link.springer.com/article/10.1007/ Eyers, J, 2015. Interventions for improving learning outcomes and access to education in low- and middle- income 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 66 References References 67 countries: a systematic review, 3ie Systematic Review 24. London: International Initiative for Impact Evaluation Weiland, C., & Yoshikawa, H. (2013). Impacts of a prekindergarten program on children’s mathematics, language, (3ie). https://www.proquest.com/docview/2568048380/fulltextPDF/A4F15208B9A347D8PQ/1?accountid=14657 literacy, executive function, and emotional skills. Child development, 84(6), 2112-2130. hiips://srcd.onlinelibrary. Spaull. N. (2023) 2023 Background Report for the 2030 Reading Panel. Cape Town hiips://www.readingpanel.co.za/_ wiley.com/doi/pdf/10.1111/cdev.12099 files/ugd/b385b7_7476724ee8a74ba8be8320a3be46b5cc.pdf Willoughby, M. T., Camerota, M., King, K. M., Nduku, T., & Piper, B. (2023). Leveraging item-level accuracy and Ssenkande, G. W., Mugyenyi, P., & Achola, D. (2022). Improving Literacy in Uganda: Why Pedagogical Reforms and reaction time to address ceiling effects in the measurement of inhibitory control in preschool-aged children. Intervention Programs are Underperforming. International Journal of Educational Reform, 10567879221085208. Frontiers in Psychology, 14. hiips://www .frontiersin.org/articles/10.3389/fpsyg.2023.861441/full hiips://www.norc.org/PDFs/LARA/LARA%20P%20and%20IE_Midterm%20IE%20and%20Final%20PE%2 0Report.pdf Wolf, S., Aber, J. L., & Behrman, J. (2018). The impacts of teacher training and parental awareness on kindergarten Stockard, J., Wood, T. W., Coughlin, C., & Khoury, C. R. (2018). The Effectiveness of Direct Instruction Curricula: A quality in Ghana. Preliminary Results Brief., Innovation for Poverty Action Washington, DC: World Bank Group. Meta-Analysis of a Half Century of Research. Review of Educational Research, 88(4), 479–507. hiip://www.jstor.org/ hiips://documents.worldbank.org/pt/publication/documents - reports/documentdetail/513981520948916408/the- stable/45277261 impacts-of-teacher-training-and-parental-awareness- on-kindergarten-quality-in-ghana Sylvia, S., Luo, R., Zhang, L., Shi, Y., Medina, A., & Rozelle, S. (2013). Do you get what you pay for with school-based Wong, H. L., Luo, R., Zhang, L., & Rozelle, S. (2013). The impact of vouchers on preschool attendance and elementary health programs? Evidence from a child nutrition experiment in rural China. Economics of Education Review, 37, school readiness: A randomized controlled trial in rural China. Economics of Education Review, 35, 53-65. hiips:// 1-12. hiips://www sciencedirect com/science/article/pii/S0272775713000952 www sciencedirect com/science/article/abs/pii/S0272775713000472 Talaat, M., Afifi, S., Dueger, E., El-Ashry, N., Marfin, A., Kandeel, A., ... & El-Sayed, N. (2011). Effects of hand hygiene World Bank. (2017). How to escape low-learning traps. hiips://openknowledge.worldbank.org/server/api/core/ campaigns on incidence of laboratory-confirmed influenza and absenteeism in schoolchildren, Cairo, Egypt. bitstreams/6ffcc8ef -2098-5a5a-831f- 6572c06dd855/content Emerging infectious diseases, 17(4), 619. hiips://www.ncbi.nlm.nih.gov/pmc/articles/PMC3377412/ World Bank. (2017). Unhealthy politics drives misalignments. hiips://openknowledge.worldbank.org/server/api/core/ Taylor, S., & Spaull, N. (2022). What works and what scales? Returning to a tradition of evidence-based system-wide bitstreams/bf9c370 6-4ca9-5938-ab20- 5ce95333273a/content programmes. In Interventions: Early Grade Reading and Mathematics Interventions in South Africa (pp. 300–316). World Bank. (2017). World Development Indicators 2017. World Bank. essay, Oxford University Press. https://resourcehub.oxford.co.za/higher- education/resources-higher-education/ hiips://openkn owledge.worldbank.org/bitstream/handle/10986/28340/9781464810961.pdf early-grade-interventions World Bank. (2017). World development report 2018: Learning to realize education’s promise. The World Bank. The Education Commission. Transforming the education workforce: Learning Teams for a Learning Generation. hiips://openknowledge.worldbank.org/bitstream/handle/10986/28340/9781464810961.pdf?sequence=68& Retrieved April 17, 2023, from hiips://educationcommission.org/transformingtheeducationworkforce/ isAllowed=y Tubbs Dolan, C., Kim, H. Y., Brown, L., Gjicali, K., Borsani, S., Houchaimi, S. E., & Aber, J. L. (2022). Yoshikawa, H., & Kabay, S. (2015). Education for All 2000-2015: achievements and challenges: The Evidence Base Supporting Syrian Refugee children’s academic and social-emotional learning in national education on Early Childhood Care and Education in Global Contexts. UNESCO. hiips://unesdoc.unesco.org/ark:/48223/pf00 systems: A cluster randomized controlled trial of non-formal remedial support and mindfulness programs 00232456 in Lebanon. American Educational Research Journal, 59(3), 419-460. hiips://journals.sagepub.com/doi/ Yousafzai, A. K., Obradović, J., Rasheed, M. A., Rizvi, A., Portilla, X. A., Tirado-Strayer, N., & Memon, U. (2016). Effects abs/10.3102/00028312211062911?journalCode=aera of responsive stimulation and nutrition interventions on children’s development and growth at age 4 years in a Walker, S. P., Chang, S. M., Powell, C. A., & Grantham-McGregor, S. M. Effects of early childhood psychosocial disadvantaged population in Pakistan: a longitudinal follow-up of a cluster-randomised factorial effectiveness trial. stimulation and nutritional supplementation on cognition and education in growth-retarded Jamaican children: The Lancet Global Health, 4(8), e548-e558. hiips://www.sciencedirect.com/science/article/pii/S2214109X16301000 prospective cohort study. hiips://www.sciencedirect.com/science/article/abs/pii/S0140673605675745 Yousafzai, A. K., Rasheed, M. A., Rizvi, A., Armstrong, R., & Bhutta, Z. A. (2014). Effect of integrated responsive Walker, S. P., Chang, S. M., Younger, N., & Grantham-McGregor, S. (2010). The effect of psychosocial stimulation stimulation and nutrition interventions in the Lady Health Worker programme in Pakistan on child development, on cognition and behaviour at 6 years in a cohort of term, low‐birthweight Jamaican children. Developmental growth, and health outcomes: a cluster-randomised factorial effectiveness trial. The Lancet, 384(9950), 1282- Medicine & Child Neurology, 52(7), e148-e154. hiips://onlinelibrary.wiley.com/doi/full/10.1111/j.1469 1293. hiips://www thelancet com/journals/lancet/article/PIIS0140 - 6736(14)60455-4/fulltext -8749.2010.03637.x Wang, H., Chu, J., Loyalka, P., Tao, X., Shi, Y., Qu, Q., & Rozelle, S. (2014). Can school counseling reduce school dropout in developing countries?. Rural Education Action Program Working Paper No, 275. hiips://fsi -live.s3.us- The full list of studies identified through the systematic search can be accessed here: west-1.amazonaws.com/s3fs-public/275- can_school_counseling_reduce_school_dropout_in_developing_countries. hiips://docs.google.com/file/d/1wGMn2g5jUDtkYUKydxqlcEklIGT5wlk4/edit?usp=docslist_api&fi letype=msexcel pdf Wang, L., Qian, Y., Warrinnier, N., Attanasio, O. P., Rozelle, S., & Sylvia, S. (2021). Parental investment, school choice, and the persistent benefits of intervention in early childhood (No. 931). Working Paper. hiips://www.econstor.eu/ bitstream/10419/260501/1/wp931.pdf Wang, X., Hui, Z., Dai, X., Terry, P. D., Zhang, Y., Ma, M., ... & Zhang, B. (2017). Micronutrient-fortified milk and academic performance among Chinese middle school students: a cluster-randomized controlled trial. Nutrients, Photos in this report were sourced from Unsplash, 2023 9(3), 226. hiips://pubmed.ncbi.nlm.nih.gov/28257107 / 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING 2023 COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING This third Global Education Evidence Advisory Panel (GEEAP) report provides a guide for governments and other stakeholders in low- and middle-income countries about what works in improving learning and education outcomes. The current report, based on a review of over 13,000 additional studies, updates a previous version of the ‘Smart Buys’. With this report, the Panel summarizes the best, most up-to-date evidence on cost-effectiveness at scale in a user-friendly way for policymakers. It groups many different types of education policies and programs into categories from Great Buys to Bad Buys, along with categories where more knowledge is needed. Public Disclosure Authorized 1 Public Disclosure Authorized Public Disclosure Authorized COST-EFFECTIVE APPROACHES PRIORITIZING LEARNING TO IMPROVE GLOBAL DURING COVID-19 LEARNING Public Disclosure Authorized Public Disclosure Authorised The Most Effective Ways to Keep Children Learning What does recent evidence tell us are “Smart Buys” During and Post-Pandemic for improving learning in low- and middle-income countries? Recommendations of the Global Education Evidence Advisory Panel Recommendations of the Global Education Evidence Advisory Panel October 2020 January 2022 January 2022 Prioritizing Learning During Covid-19 Prioritizing Learning During Covid-19: The In this first GEEAP report on Cost-effective Most Effective Ways to Keep Children Learning Approaches To Improve Global Learning: What During and Post-Pandemic. This second Global does recent evidence tell us are “Smart Buys” Education Evidence Advisory Panel (GEEAP) for improving learning in low- and middle- report draws on insights from the latest income countries? the Panel provides evidence- research to document the impacts of and based recommendations to improve learning responses to Covid-19. It offers guidance on outcomes. Even pre-pandemic, more than half of how education systems in LICs and MICs can all children in low- and middle-income countries respond to the damage caused by the pandemic did not learn to read with comprehension by and ensure that the learning needs, especially age 10, despite the ambitions of Sustainable of marginalized and disadvantaged groups, Development Goal 4 for “inclusive and equitable are addressed. The report provides practical, quality education and lifelong opportunities for focused advice for policymakers—advice that all.” represents the consensus recommendations of an independent, interdisciplinary panel of global Link: https://bit.ly/3nHwzEu experts, based on the best evidence available during a rapidly changing crisis. Link: http://bitly.ws/Dv4o