REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Public Disclosure Authorized REMOTE LEARNING DURING COVID-19: Public Disclosure Authorized Lessons from Today, Principles for Tomorrow Public Disclosure Authorized Public Disclosure Authorized 1 REMOTE LEARNING DURING COVID-19: Lessons from Today, Principles for Tomorrow Alberto Muñoz-Najar Alison Gilberto Amer Hasan Cristóbal Cobo João Pedro Azevedo Maryam Akmal © 2021 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW, Washington, DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org License: Creative Commons Attribution CC BY 3.0 IGO This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. This report was also supported with funding from the Global Partnership for Education. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Please cite the work as follows: Muñoz-Najar, Alberto; Gilberto, Alison; Hasan, Amer; Cobo, Cristóbal; Azevedo, João Pedro; Akmal, Maryam. 2021. Remote Learning during COVID-19: Lessons from Today, Principles for Tomorrow. Washington, D.C.: World Bank Group. Any queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@ worldbank.org Cover design: Danielle Willis. Report design: Scaff Design. REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW 3 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW EXECUTIVE SUMMARY School closures during COVID-19 led to an person schooling prior to the pandemic. Poorer unprecedented global experiment in the delivery countries lag far behind richer ones in the scale and of remote learning. This report seeks to assess scope of their remote learning measures. Thus, once what lessons can be drawn from experiences of can expect that the results in developing country remote learning during COVID-19 in K-12 education, settings is likely to reveal an even starker picture, with a focus on low- and middle-income countries. It given that many have delivered very little remote focuses on the period from March 2020 to October instruction despite full or partial school closures of 2021 and addresses the following key questions: one year or more. For remote learning to be effective it requires 1. Was remote learning during COVID-19 three complementary, critical components: taken up and if so, was it effective? That effective teachers, suitable technology, and is, did children learn as much as they did engaged learners. These components are critical during pre-pandemic, in-person learning? for take-up, which is a necessary first step for effectiveness. A teacher with high subject content 2. What lessons can governments derive knowledge, technical skills in using technology and from this wide-spread experience? supporting resources, and appropriate pedagogical techniques is likely to be more effective at remote 3. How might policymakers use these learning than a teacher lacking some or all of these lessons to reimagine learning as schools attributes. Availability of technology is a necessary begin to reopen? but not sufficient condition for effective remote learning, as it needs to be suited to the context in This report is part of a larger effort led by the World which it is deployed. Last but not least, effective Bank to provide guidance and technical assistance remote learning requires an engaged learner, whose to optimize country effectiveness in the design and engagement depends on intrinsic motivation, teacher execution of remote learning strategies. It has been and technology effectiveness, and contextual factors developed in conjunction with Remote Learning such as the home environment. During the Global School Lockdown: Multi-Country Lessons, a qualitative study conducted between May Governments deployed remote learning in a and November 2020 to understand the perceived variety of ways. Multimodal responses were common effectiveness of remote and remedial learning but there were differences between regions and solutions implemented across 17 countries. country income levels. Some countries took advantage of available preexisting education technology infrastructure to deploy remote learning strategies Key Findings ranging from paper-based take-home packages to radio, TV, phone, and internet-based solutions. Emerging evidence as to the effectiveness of Others did not. For instance, as of June 2021 nearly remote learning during COVID-19 is mixed at 40 percent of the countries in Sub-Saharan Africa did best. When compared to in-person learning prior not undertake any remote learning strategy despite to the pandemic learning outcomes have been having full and partial school closures of about one generally worse with remote learning. The empirical year, on average, resulting in millions of children not evidence from low-, middle-, and high-income receiving any instruction during that period. Most countries reveals different and nuanced results. In of the countries in the different regions chose to low-income countries, remote learning was not as implement multimodal remote learning. Others, such widely taken-up as in middle-income countries. as Mali and Lebanon, relied on unimodal solutions. While take-up was not as much of an issue in high- Regardless of whether countries chose unimodal or income settings, evidence suggests that remote multimodal approaches, their strategies varied in learning did not produce as much learning as in- terms of design, use, and contextual features. 4 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Governments tried to facilitate take-up of remote This report proposes five principles for reimagining learning in multiple ways. Some governments learning as countries strive to build back more partnered with the private sector and/or delivered effective, equitable and resilient education targeted aid directly to households to facilitate systems, drawing on ongoing lessons from remote children’s access to remote learning (including learning during COVID-19 school closures: tutoring). A number of countries adjusted curricula to accommodate the reduction in school days. Adopt Suitable Technology Some countries supported teachers with remote teacher training and other forms of assistance. 1. Ensure remote learning is fit-for-purpose. When At the same time, other countries have sought to deciding on modes of remote learning, countries help disadvantaged groups during the pandemic by must account for access and utilization of improving access to remote learning infrastructure, technology among both teachers and students, designing learning materials in minority languages including digital skills when online solutions are or gearing them toward children with disabilities, involved. They must also ensure that teachers and offering flexible and self-paced platforms. are supported to acquire and use the technical and pedagogical competencies needed for the Yet many countries struggled to ensure take- chosen mode of remote learning. up and some even found themselves in a remote learning paradox. That is, too often Prioritize Effective Teachers countries chose a distance learning approach that was unsuited to the possibilities and needs of the 2. Use technology to enhance the effectiveness majority of students. For example, in some countries, of teachers. Teacher professional development governments provided online (digital) learning must target the development of the skills needed solutions, although a majority of students could not to be an effective teacher in a remote setting. access these due to infrastructure (lack of devices) or connectivity constraints, thus resulting in uneven 3. Establish meaningful two-way interactions. Using take-up and amplified inequalities. Several factors the most appropriate technology for the local lead to low student take-up, such as inadequate context, it is imperative to enable opportunities home environments, children’s attention spans, low for students and teachers to interact with each digital literacy of students, teachers and/or parents, other, with suitable adaptations to the delivery of and the accessibility of devices. In other countries, the curriculum. teachers lacked the skills to effectively utilize the remote learning technologies that were deployed to Ensure Learners are Engaged deliver remote instruction. 4. Engage and support parents and students as What once was a digital divide for some is now partners in the teaching and learning process. a digital chasm for many. Pre-pandemic access to Given the isolation and disconnection caused technology, as well as access to resources and skills by school closures, it is imperative that parents to utilize technology effectively, differs widely within and families more broadly are engaged and and across countries. Parental engagement and supported to help students access remote support, which is critical to facilitate remote learning, learning opportunities, and to ensure their socio- varies by education and socioeconomic background, emotional well-being. and household income losses during the pandemic only deepened the divide. Marginalized children 5. Rally all actors to cooperate for learning. and vulnerable groups, such as girls, students with Cooperation across governmental departments disabilities and ethnic minorities, are likely to be as well as partnerships between the public disproportionately affected and at greater risk of and private sector, and even between groups falling further behind. of teachers and school principals is vital to the effectiveness of remote learning and to ensure that the system continues to adapt, learn and improve in an ever-changing remote learning landscape. 5 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Conclusion Despite the challenges that this report notes, in the course of just 18 months, a variety of experiences have been documented, capacities created, and new forms of education delivery adopted at a scale. This presents a number of possibilities for reimagining how education can be offered and enriched in the years to come. This is especially pertinent as governments around the world continue to implement school closures and plan to offer multiple remote learning strategies in the face of a lingering pandemic. In some countries schools remain fully or partially closed. In others, schools are open in some regions for some grades some of the time. Remote learning remains a key means of ensuring continuity of learning as school systems return to increased in-person learning. When its three complementary components–teachers, students and technology–are well-aligned, remote learning has the potential to address the inequality in learning that has held back global progress on learning poverty for far too long. 6 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW 1. INTRODUCTION Aleksey Oryshchenko 7 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW 1. INTRODUCTION Education systems worldwide reacted to COVID-19 32 percent of countries worldwide either fully (14 by closing schools and rolling out remote learning countries1) or partially (50 countries) closed schools. options for their students as an emergency response. The longest closures have been in South Asia and School closures compounded an ongoing learning Latin America and the Caribbean, with an average crisis (Box 1.1). At the peak of the pandemic, in of 429 and 387 days, respectively, in which schools April 2020, about 1.6 billion K-12 learners in over were fully or partially closed, in both cases equivalent 190 countries were deprived of in-person schooling to more than the 75 percent of their total instruction (UNESCO 2020a; Figure 1.1). As of October 2021, time since the pandemic began. Figure 1.1 School Closure Status from February 2020 to October 2021 210 180 150 Number of countries 120 90 60 30 0 p- 1 un 20 un 21 16 pr- 0 14 ul-2 0 12 ct-2 0 11 pr- 1 1 O 021 0 u 20 1 ep 20 0 0 a 20 a 21 1 0 1 1 -N 02 2 -A 02 -J 02 -O 02 02 -A 02 -J 02 Au 2 02 -J 02 02 -M 02 -M 02 8- -20 0 9- l-20 0 -M 20 -M 20 0 0 2 13 g-2 15 y-2 10 y-2 16 r-2 15 -2 13 -2 11 r-2 10 -2 12 v-2 11 c-2 -2 17 b-2 10 n-2 12 b-2 g ct u a a Se o e a e e -S -D -A -J -J -F -F 8- 16 Fully closed Fully closed Partially closed Partially closed Fully Fullyopen open Academic break Academic break Source: World Bank. Calculations based on UNESCO global monitoring of school closures caused by COVID-19. Note: The analysis covers schools from pre-primary to upper secondary level. See Annex figure A.1 for the evolution of school closures by region. 8 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW COVID-19 has led to an unprecedented experiment The report is organized as follows: Section 2 in the deployment of remote learning as a means describes the conceptual framework, underscores to counter school closures and to deliver education what matters for remote learning take-up and effectively and at scale.2 This is particularly relevant effectiveness, and explains the types of learning as the pandemic continues to linger and undermines experiences considered. Section 3 assesses how the traditional relationship between teachers and countries responded to school closures induced students. Looking ahead to other similar system- by COVID-19. Section 4 discusses what we are wide shocks, it is clear that for learning to be effective learning from country responses. Section 5 presents for everyone, everywhere, hybrid education, which key principles that school systems can draw on to combines both in-person and remote learning, is reimagine learning going forward. here to stay (Saavedra et al. 2020). This report seeks to assess what lessons can be drawn from the ongoing global experiences of remote learning during COVID-19. It examines the period March 2020 to October 2021 and addresses the following key questions: 1. Was remote learning during COVID-19 taken up and if so, was it effective? That is, did children learn as much as they did during pre-pandemic, in-person learning? 2. What lessons can governments derive from this wide-spread experience? 3. How might policymakers use these lessons to reimagine learning as schools begin to reopen? To answer these questions, the report draws on a variety of sources, including qualitative country case studies early in the pandemic developed using key informant interviews, a global survey of country responses jointly fielded and analyzed by the World Bank, UNICEF, the United Nations Educational, Scientific and Cultural Organization (UNESCO), and the Organisation for Economic Co-operation and Development (OECD), emerging quantitative evidence from research studies conducted during this period, a global tracker on education recovery facilitated by a partnership between the World Bank, Johns Hopkins University and UNICEF, and high-frequency household surveys fielded by the World Bank. 9 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Box 1.1 School Closures Exacerbate the Learning Crisis and Widen its Impact Economic and health shocks stemming from COVID-19 have pushed children to drop out, forcing some to leave school permanently. Millions of children are being deprived of daily school meals while schools are closed. Around 369 million students worldwide went without school meals at the peak of the pandemic in April 2020. By October 2021, that number remained stubbornly-high: 187 million children (World Food Programme 2021). This can lead disadvantaged children to dropout or be at higher risk of doing so (UNESCO, UNICEF, World Bank, and OECD 2021b). For instance, as of September 2020 over 300,000 students in Peru – about 15 percent of the student population – had dropped out since school closures began. This high drop-out rate has been attributed to economic instability, in addition to difficulties in accessing remote learning programs (Perez 2020). In the case of Punjab in Pakistan, 2.5 percent of girls enrolled in schools pre-pandemic might not return to classes (Hasan et al. 2021). Additionally, children whose families have experienced COVID-19 are at higher risk of dropout (World Bank 2020e). It is estimated that between March and October 2020 around 10.7 million students (a majority between 12 and 17 years old) were at high risk of dropping out from primary and secondary education due to income shocks (Azevedo et al. 2019). School closures are harming children’s mental health and their motivation to learn. • A longitudinal study in the United States before and during the pandemic, showed that students using remote learning had lower levels of emotional, social, and academic wellbeing compared to classmates that attended in-person schooling (Duckworth et al. 2021). • Another study analyzed 14-18 year-old students in Ecuador during COVD-19 and determined that 16 percent presented symptoms indicating depression (Asanov et al. 2020). This was an increase compared to pre-pandemic figures. One of the main factors behind this was the lack of socio- emotional support schools could offer to students and the social isolation caused by the remote learning experience. • In Brazil, the Lemann Foundation, Itaú Social, Imaginable Futures, and Datafolha, conducted a series of telephone surveys of more than 1,000 caregivers, selected randomly. The study found that: students’ lack of motivation for learning activities at home increased from 46 percent in May to 54 percent in September 2020; the perception that learning had not improved increased from 46 percent to 50 percent; over 70 percent of students felt anxious about remote education, and 31 percent of respondents had worries regarding dropping out. Marginalized children and at-risk groups, such as low-income girls, children with disabilities, and ethnic minorities, are disproportionately affected and risk falling further behind. School closures have caused girls to become more vulnerable to child marriage, gender-based violence and early pregnancy, leaving 5.2 million girls worldwide in primary and secondary school at risk of dropping out from school permanently (UNESCO 2020b). Notably: • Given a scenario of eight months of school closures in Sub-Saharan Africa, teenage pregnancy could increase as much as 65 percent , equivalent to 1 million girls that might not return to classes when schools reopen (World Vision International 2020). • Similarly, girls in Punjab have been spending more time on household chores compared to boys during the pandemic (Hasan et al. 2021). Finally, indigenous children have lower primary enrolment levels compared to national averages in their countries, are more vulnerable to leave school due to household economic shocks, and often face instruction in a language other than their native language, even when they are able to access remote learning (Azevedo et al. (2021).3 Source: World Bank, UNESCO, UNICEF (forthcoming). 10 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Notes 1. List of countries fully closed as of October 2021: The Bahamas, Barbados, Belize, Brunei Darussalam, Cuba, Dominica, Fiji, Grenada, Jamaica, Myanmar, Philippines, Sri Lanka, Sao Tome and Principe, and Uganda. 2. Technology here is meant to run the gamut from paper and pencil to internet-connected electronic devices. 3. As reported in a survey of education ministries conducted by UNESCO, UNICEF, World Bank and OECD, henceforth referred to as the joint survey. UNESCO, UNICEF, the World Bank, and OECD (2021b). References Asanov, I., F. Flores, D. McKenzie, M. Mensmann, and M. Schulte. 2020. Remote-learning, Time-Use, and Mental Health of Ecuadorian High-School Students during the COVID-19 Quarantine. Policy Research Working Paper, No. 9252. Washington, DC: World Bank.: Retrieved from https://openknowledge. worldbank.org/handle/10986/33799 Azevedo, J. P., D. Goldenberg, S. Montoya, R. Nayar, H. Rogers, J. Saavedra., and B. W. Stacy. 2021. Will Every Child Be Able to Read by 2030? Defining Learning Poverty and Mapping the Dimensions of the Challenge. Policy Research Working Paper 9588. Washington, DC: World Bank. Retrieved from http://gaml.uis.unesco.org/wp-content/uploads/sites/2/2021/03/Azevedo-et-al-2021_Will-Every- Child-Be-Able-to-Read-by-2030.pdf Duckworth, A. L., T. Kautz, A. Defnet, E. Satlof-Bedrick, S.N. Talamas, B.L. Luttges, and L. Steinberg. 2021. “Students Attending School Remotely Suffer Socially, Emotionally, and Academically.” Educational Researcher 50(7): 479-482. https://doi.org/10.31234/osf.io/rpz7h Hasan, A., Geven, K., and A. Tahir, 2021. SMS Girl Data Insights : How has COVID-19 Affected Support for Girls’ Education in Punjab, Pakistan? (English). World Bank, South Asia Gender Innovation Lab Policy Brief, Washington, DC: World Bank. Retrieved from http://documents.worldbank.org/curated/ en/464681618460372445/SMS-Girl-Data-Insights-How-has-COVID-19-Affected-Support-for-Girls- Education-in-Punjab-Pakistan Pérez, A. 2020. “300.000 Estudiantes Peruanos Desertan en Medio de la Pandemia.” Retrieved from https:// iep.org.pe/noticias/analisis-300-000-estudiantes-peruanos-desertan-en-medio-de-la-pandemia/ Saavedra, J., C. Aedo, O. Arias, A. Pushparatnam, M. Gutierrez, and F. Rogers. 2020. Realizing the Future of Learning: From Learning Poverty to Learning for Everyone, Everywhere (English). Washington, DC: World Bank. United Nations Educational, Scientific and Cultural Organization (UNESCO). 2020a. “From COVID-19 Learning Disruption to Recovery: A Snapshot of UNESCO’s Work in Education in 2020.” Retrieved from https://en.unesco.org/news/covid-19-learning-disruption-recovery-snapshot-unescos-work- education-2020 United Nations Educational, Scientific and Cultural Organization (UNESCO). 2020b. “UNESCO COVID-19 Education Response: How Many Students are at Risk of Not Returning to School?” Retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000373992 11 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW United Nations Educational, Scientific and Cultural Organization (UNESCO), UNICEF, World Bank and Organization for Economic Co-operation and Development (OECD). 2021b. What’s Next? Lessons on Education Recovery: Findings from a Survey of Ministries of Education amid the COVID-19 Pandemic. UNESCO, Paris, UNICEF, New York, World Bank, Washington DC, OECD, Paris. Retrieved from https:// openknowledge.worldbank.org/handle/10986/36393 World Bank, UNESCO, UNICEF. (forthcoming). The State of the Crisis in Global Education. World Food Programme. 2021. “Global Monitoring of School Meals During COVID-19 School Closures.” Retrieved from https://cdn.wfp.org/2020/school-feeding-map/ World Vision International. 2020. COVID-19 Aftershocks: Access Denied. World Vision International. Teenage Pregnancy Threatens to Block a Million Girls Across Sub-Saharan Africa from Returning to School. Monrovia, California: World Vision International. https://www.wvi.org/sites/default/files/2020-08/ Covid19%20Aftershocks_Access%20Denied_small.pdf 12 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW 2. CONCEPTUAL FRAMEWORK carl-jorgensen-leyUrzdwurc 13 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW 2. CONCEPTUAL FRAMEWORK What matters for remote learning • Third, are learners engaged in remote learning? take-up and effectiveness? » The development of cognitive and socio- Three key features of remote learning guide this emotional skills has always been the result of an report’s analytical framework (Figure 2.1): interplay between a wide range of characteristics of the children, their school, their teachers, • First, is the teacher knowledgeable and does she and their households. Children need regular have the skills needed to teach remotely? interactions facilitated by teachers and parents that encourage engagement with the learning » Consider two polar examples: on the high end, content, ignite motivation, and allow for regular a teacher has in-depth content knowledge feedback. For children to be engaged, they and possesses not only the skills to use the not only need access to the remote learning digital platforms and resources, but she is also technology being used by their school system capable of adjusting her teaching practice in (such as paper packets, phones, laptops, or order to secure the engagement of students internet connections) but they also need access when teaching remotely. Such a teacher is to engaging content as well as regular feedback likely to be more effective than her counterpart and motivation from teachers. This report seeks on the low end of the spectrum: a teacher to assess whether the conditions that students with little to no content knowledge or who has have found themselves facing during COVID-19 limited knowledge on how to effectively use school closures allows them to be engaged technology when teaching. These challenges learners. impose barriers to teaching remotely but also to create the needed learning environment Figure 2.1 provides a simplified depiction of how between students and teachers. The reality these three features may combine with each other. is that school systems likely have teachers all In practice, a continuum of combinations is possible. along such a continuum. The combinations below could just as easily describe differences across countries as they could • Second, is technology for remote teaching— the disparate experiences of groups within a given ranging from paper take-home packets to country: richer or poorer households, families living internet-connected electronic devices—suitable? in urban or rural areas, and students connected to high-speed internet, or those without connectivity. » This report considers a technology suitable if Three stylized situations are worth highlighting it is both widely available and appropriate for based on experiences of remote learning during the remote learning needs of the context in COVID-19: which it is deployed. It must also help teachers be effective in a remote learning environment. Thus, deploying remote learning via the internet in settings where internet penetration and bandwidth are low are likely to be examples of technology being unavailable and inappropriate as students who live in this context would not be able to access these learning solutions. Also, such technology deployment undermines teacher effectiveness as even the most knowledgeable teachers would be unable to reach the majority of their students. Globally, school systems are operating with technology resources that range from low to high suitability. 14 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Figure 2.1 Conceptual Framework for Remote Learning During COVID-19 Learners Missed Teacher are engaged Left behind opportunities Left behind is effective Leaders Missed Missed opportunities opportunities Left behind Technology is suitable Source: World Bank. • ‘Left behind’ countries or groups • ‘Missed opportunities’ can occur • ‘Leaders’ are able to deliver can be characterized by education when either technology is unavailable contextually-appropriate remote delivered by a teacher who has or inappropriate or when teachers are learning opportunities taught little to no experience or training not knowledgeable or not effective. by a teacher with high content in delivering education in a remote In such a situation, education is knowledge who is experienced or learning modality. It could also being delivered by knowledgeable trained in remote learning. Students be one where a teacher with low teachers with little to no experience in these settings have easy access content knowledge is tasked with or training in delivering remote to the chosen remote education delivering online remote education learning. Alternatively, in this technology which could range from in a country where student access to scenario, high content knowledge is paper take-home packets to online technology or connectivity are low. In paired with a technology that is not courses. Remote learning take-up is this scenario, take-up of any remote widely accessible or where—despite consequently expected to be high and learning being deployed is likely to be availability—bandwidth issues are likely to maximize a two-way student- low, as will be its effectiveness. limiting. This situation could also teacher interaction. The resulting arise when appropriately-designed effectiveness might also be high. and accessible remote education Such experiences, while limited, are content is delivered by a teacher with characteristic of countries that could low content knowledge. Take-up of reasonably be considered remote opportunities for remote learning learning leaders during the pandemic. and their effectiveness are likely to be sub-optimal and lead to missed opportunities for success. 15 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Types of learning experiences one-to-many (teacher to students), one-to-one (one teacher to one student, or student-to-student), or In this section, we define the different types of many-to-many (many students interacting among learning experiences considered in this report (see themselves and with multiple teachers). Box 2.1 for a summary). Synchronous approaches Remote learning These remote learning approaches allow real-time Remote learning refers to synchronous or interactions between students and teachers while asynchronous instruction provided in a place outside lessons and instructional content are being shared. the classroom. Synchronous learning means that students are connected to learning experiences Asynchronous approaches where a teachers’ immediate feedback is possible. These remote learning approaches allow interactions Asynchronous or self-directed learning means that between students and teachers to take place before or students can learn at their own pace and chosen after delivery of the lessons and instructional content. time. Remote learning takes an array of forms ranging from paper-based take-home packages to online platforms. Remote learning is also possible through a variety of different channels, such as mobile phones, television, radio, and tutors. In-person learning In-person learning refers to instruction at school in real time, with teachers and students interacting in person. Learning is typically a result of interactions between teachers, learners and their peers, and supported by parents, caregivers, learning resources, schools, and school leaders (Saavedra et al. 2020). The learning experience depends on the ability of the teacher to teach and on the ability and motivation of the learner to learn. In addition, parental socioeconomic background, features of the learning space, the language of instruction, and peer interactions are among many factors that may influence learning outcomes. In particular, peer interaction is a critical factor influencing language skills and socioemotional development. It is also a considerable factor in shaping student motivation to succeed at school. Hybrid learning Hybrid learning combines in-person learning with remote learning. It is sometimes also referred to as blended learning. Multimodal approaches Multimodal approaches refer to settings where school systems use multiple modes of delivery for learning. These are also sometimes referred to as multi-channel approaches. These can be either synchronous or asynchronous. Multimodal approaches can use analog and/or digital channels. They can include different forms of interaction: 16 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Box 2.1 Remote Learning Encompasses a Variety of Modalities and How These are Combined Matters Remote learning modalities can run the gamut from paper-based take-home packages to radio or television broadcasts, to mobile phones, to tutors, and to online platforms (UNESCO, UNICEF, World Bank, and OECD 2020a). For some of these modalities, access to electronic devices with appropriate software and connectivity are necessary. For others, no such access is needed. These modalities can be used in a variety of ways and combined to expand take-up and also effectiveness: Students lacking access to any digital connectivity and device can be reached using paper- • based take-home packages or tutoring for remote learning. Students with radio spectrum and radio access can listen to pre-recorded radio instruction • (Anzalone and Bosch 2005). Students with TV signal and TV access can access remote learning through live or pre-recorded • lessons, and edutainment programs transmitted over free-to-air TV, cable TV, or paid satellite TV, among others (World Bank 2020b). Students with telephone access can make or receive audio calls with teachers, tutors and peers. • Students lacking broadband network connectivity but with mobile phone and mobile network • access can also engage in remote learning using text messages (SMS) and audio calls. Students with limited broadband network connectivity and device access can still use digital • solutions. In such environments, desktop computers, laptops, tablets and smartphones can allow access to offline apps, pre-loaded e-content, or preloaded content on a variety of storage media (such as CDs and USB drives). Students with broadband network connectivity and device access can make use of online • platforms through desktop computers, laptops, tablets, smartphones, or digital TV. Online platforms can include web portals, mobile apps, and podcasts, to name a few. Children with such access can use a broader set of digital solutions for remote learning. They can attend classes live through video conferencing platforms (such as Zoom or Microsoft Teams for instance), watch and revisit pre- recorded lessons, play educational games, download audio and video content as well as e-textbooks, and participate in online forums. They can also benefit from an online learning management system, or cloud-based collaborative file editing (World Bank 2020a), or listen to educational radio programs broadcast online. While radio and TV primarily allow one-way interactions, it is possible to combine these with • other technologies and to use them for two-way interactions. Two-way interactions are possible if technologies are used to complement each other. For example, • the use of TV and radio can be complemented with messaging apps to generate feedback from students, calls from teachers to assess student progress or help solve questions. Hotlines and online helpdesks—with toll-free numbers, SMS, email and social media (World Bank 2020c)— can provide technical assistance and pedagogical and socio-emotional support for students, caregivers, and teachers (McBurnie 2020; World Bank 2020b). SMS can also be used to inform students of broadcast schedules, to expand the take-up of TV and radio programs.1 Ultimately, how remote learning strategies are used and combined can make them more or less interactive, more or less synchronous, and this combination of attributes can influence its effectiveness. Source: World Bank. 17 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Note 1. Devices with broadband network connectivity (online) and mobile network or fixed-telephone lines (offline), such as computers or phones, allow two-way interactions. However, it is possible for these to be used for one-way interactions only. A one-way interaction may arise if these high-tech resources are underused. Internet and mobile phones could be used as merely information delivery devices. For example, this is the case with live online classes that do not leave time for questions and answers, or pre-recorded classes that do not provide opportunities for discussion. References Anzalone, S., and A. Bosch. 2005. Improving Educational Quality with Interactive Radio Instruction: A Toolkit for Policymakers and Planners. Africa Region Human Development Working Paper Series, 52. Wash- ington, DC: World Bank. Retrieved from https://documents.worldbank.org/en/publication/docu- ments-reports/documentdetail/288791468035958279/improving-educational-quality-with-inter- active-radio-instruction-a-toolkit-for-policymakers-and-planners McBurnie, C. 2020. “Using Interactive Radio Instruction to Mitigate the Educational Impact of COVID-19: A Curated Resource List.” EdTech Hub. doi:10.5281/zenodo.3762228. Retrieved from https://docs. edtechhub.org/lib/A3T2DQ4D Saavedra, J., C. Aedo, O. Arias, A. Pushparatnam, M. Gutierrez, and F. Rogers. 2020. Realizing the Future of Learning: From Learning Poverty to Learning for Everyone, Everywhere (English). Washington, DC: World Bank. United Nations Educational, Scientific and Cultural Organization (UNESCO), UNICEF, World Bank and Organization for Economic Co-operation and Development (OECD). 2021b. What’s Next? Lessons on Education Recovery: Findings from a Survey of Ministries of Education amid the COVID-19 Pandemic. UNESCO, Paris, UNICEF, New York, World Bank, Washington DC, OECD, Paris. Retrieved from https://openknowledge.worldbank.org/handle/10986/36393 World Bank. 2020a. “Mobile Distance and Hybrid Education Solutions. A Knowledge Pack.” Retrieved from https://pubdocs.worldbank.org/en/685691598013656403/WorldBank-EdTech-Team-Knowledge- Pack-MobileDistance-HybridEducationSolutions-version2.pdf World Bank. 2020b. “Television Education Knowledge Pack.” Retrieved from https://pubdocs.worldbank. org/en/267791593613610668/Education-TV-Knowledge-Pack-WorldBank-Edtech-Team.pdf World Bank. 2020c. “Remote Learning Response to COVID-19 Knowledge Pack.” Retrieved from https://pubdocs.worldbank.org/en/925611587160522864/KnoweldgePack-COVID19- RemoteLearning-LowResource-EdTech.pdf 18 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW 3. HOW DID COUNTRIES RESPOND TO EDUCATION DISRUPTIONS? giovanni-gagliardi 19 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW 3. HOW DID COUNTRIES RESPOND TO EDUCATION DISRUPTIONS? While multimodal responses were America and the Caribbean decided to implement multimodal remote learning programs. Others common, regional and country focused on  unimodal solutions, such as Mali and income differences abound Lebanon, though both chose different unimodal solutions.1 The vast majority of countries offered multiple modes of remote learning. According to the Regional differences in remote learning strategies Joint Survey, most countries delivered remote are substantial. According to the Global Education learning through online media (91 percent) and Recovery Tracker, as of September 2021, all regions TV (85 percent), followed by paper-based take- delivered remote learning strategies through the home materials (82 percent), and mobile phones internet or mobile phone, with a notably higher number (70 percent). Eighty percent of governments in of countries from Latin America and the Caribbean the Middle East and North Africa, 93 percent in (18) than elsewhere, while TV and radio were also Europe and Central Asia, and 97 percent in Latin used in most regions, as seen in Figure 3.1. Figure 3.1 Provision of Remote Learning Strategies by Region 20 18 16 14 Number of Countries 12 10 8 6 4 2 0 East Asia & Europe & Latin America & Middle East & South Asia Sub-Saharan North America Pacific Central Asia Caribbean North Africa Africa Internet Mobile Phone Radio TV Take-home Packages Tutoring Source: World Bank. Calculations based on Johns Hopkins University et al. (2021) as of September 2021. Note: Sample size of 74 countries. Data covers countries with remote learning strategies at any education level, including pre- primary, primary, lower secondary, and upper secondary education. Some countries offered one or more of remote learning strategies. 20 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW While high income countries could rely on income countries adopted a variety of two or more internet-based solutions, poorer countries often remote learning strategies. Nine percent of lower- combined a variety of remote learning modalities. middle-income countries chose to deliver remote Figure 3.2 shows how remote learning strategies learning exclusively through the internet while 86 differed among countries based on their income percent adopted a variety of two or more remote level. Fifty-three percent of high-income countries learning strategies. In low-income countries, 100 for which there is information – with high internet percent opted for providing remote learning through and mobile phone penetration - delivered remote a combination of three or more remote learning learning strategies exclusively through the internet. strategies (Johns Hopkins University, World Bank, In contrast, more than 85 percent of upper-middle- and UNICEF 2021; UNICEF 2021). Figure 3.2 Provision of Remote Learning Strategies by Income Level High income Upper middle income Lowe middle income Low income Combinations of 3 Remote All Remote Learning Internet Only TV and Radio Only Learning Strategies Only Strategies Take-home Package Combinations of 4 Remote TV Only and Tutoring Learning Strategies Only Internet and Mobile Internet and TV Only Combination of 5 Remote Phone Only Learning Strategies Only Source: World Bank. Calculations based on Johns Hopkins University et al. (2021) as of September 2021. Note: Sample size of 74 countries. Data covers countries with remote learning strategies at any education level, including pre- primary, primary, lower secondary, and upper secondary education. No country was found delivering only mobile phone, only radio, only take-home packages, only tutoring, or combinations of two remote learning strategies that does not include internet and mobile phones, so these categories were excluded from the graph. 21 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Figure 3.3 shows the diverse array of hybrid and nor adaptive. These involve radio lessons remote learning strategies that have been deployed followed by toll-free phone calls at the end in various countries since school closures began in of each radio segment to allow students to March 2020. The examples documented in this report call in with their questions. have been taken from over 130 semi-structured interviews conducted for the report “Remote 31. Contextual factors Learning During the Global School Lockdown: Multi- Lastly, each of the examples is heavily Country Lessons” between May and November 2020 influenced by prevailing contextual factors. with key informants such as Ministry of Education For instance, different modalities require policy makers, teacher union officers, and leaders different levels of parental involvement. of nongovernmental organizations (NGOs). These Those in Estonia explicitly require parents examples are organized according to the following: to perform specific tasks to facilitate remote learning. On the other end of the 1 1. Design features spectrum, and more commonly, household The remote learning strategies documented infrastructure will play a key role in the in this report vary in terms of their design success of any remote learning modality, features; whether they rely on visual or audio even if this is not explicit in the design. elements, the extent to which they encourage parental involvement, and the extent to The consideration of design and usage features is which they rely on school infrastructure. For particularly important when considering whether example, consider Haiti, where an online or not remote learning is reaching vulnerable platform was the only strategy pursued (a populations. For instance, children with disabilities unimodal approach). This reached only a are often less likely to attend school or dropout fraction of the student population. In Brazil, before completing primary school. Learners with states such as Minas Gerais delivered visual and hearing impairments, especially younger remote learning via TV and teachers used a students, lack accessibility to remote learning unless mobile app to follow up with their students they have instructions in their local sign language or (a multimodal approach). In Cambodia, the in Braille, considerations that are not always factored government provided paper-based learning into the delivery of remote learning. Parents might materials for the most vulnerable students lack literacy in the language of instruction, inhibiting and complemented these with SMS and them to help their children at home (McClain-Nhlapo Telegram messages for teacher-student et al. 2020). follow-up (also a multimodal approach). 2 1. Usage features The examples covered in this report also vary in terms of their usage features; whether they are asynchronous, interactive, or adaptive. Estonia is facilitating remote learning through online platforms that allows for an interactive and synchronous experience. Uruguay is using adaptive software on devices that have been provided to all students. In contrast, Kenya has primarily relied on one-way learning that is not synchronous, interactive, or adaptive, an approach that relies on radio, TV broadcasts, and pre-recorded online lessons. Other countries have used a variety of remote learning strategies with features that fall in between. For example, Sierra Leone is deploying strategies that are interactive but neither synchronous 22 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Figure 3.3 A Variety of Remote Learning Strategies Were Deployed During COVID-19 Examples of Remote Learning Strategies TV + TV + Adaptive Live online lessons Radio + Take-home Radio + TV + Online platforms pre-recorded In-class software + teaching- teacher teacher / learning lessons + teacher pre-recorded phone follow- packet + online involvement learning resources online lessons / mobile up lessons SMS only lessons follow-up Visual Audio Features Design Household Infrastructure School Infrastructure Synchronous Features Usage Interactive Adaptive Learner Contextual Engagement Factors Parental Engagement Uruguay Estonia Peru Brazil Sierra Leone Cambodia Haiti Kenya Examples Covered (Ceibal en (E-Schoolbag) (Aprendo en (Conexao (All Children (PR@CTIC) (KICD in this Report Casa) (Radio Casa) Escola) Education Learn) Remote Program) Learning) Note: For Kenya, data collected as of May 2020; for Brazil, Peru, and Sierra Leone, data collected as of July 2020; for Cambodia, Estonia, and Haiti, data collected as of November 2020; for Uruguay, data collected as of February 2021. are features that were observed during the data collection process, are inferred based on the interviews conducted, and for no data was available. 23 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Most countries are implementing For example, in Haiti, the government developed an education response plan for COVID-19 to support remote learning as a reaction to 350,000 children from disadvantaged communities the pandemic to continue learning through different remote strategies (Global Partnership for Education 2020a). While some countries have been able to Education experts in the country urged policymakers take advantage of preexisting technology to analyze the resources available to ensure infrastructure to deploy remote learning education continuity while schools were closed to strategies, the majority have been forced to react avoid exacerbating existing inequalities, especially to the pandemic without such infrastructure in for those students without access to connectivity place. Figure 3.4 shows countries that supplied (UNESCO 2020c). Yet in a country where less than remote learning through online platforms (UNESCO, 32 percent of the population has access to the UNICEF, World Bank, and OECD 2021b), taking internet (World Bank 2019b), Haiti’s government into account the country’s internet access and prioritized the development of PRACTIC, an of education performance measured in terms of online learning platform to support students from learning-adjusted years of schooling (LAYS).2 The the ninth and twelfth grades who have to sit for types of online learning platforms used by teachers national examinations, while excluding the majority and students while schools are closed can be open- of the student population (Ministère de l’Éducation source platforms such as Moodle and Canvas, Nationale et de la Formation Professionnelle domestic platforms managed by a local authority, 2020). The examples from Nigeria’s Edo state (Box opensource platforms such as Google Classroom, 3.2) and Haiti show that countries that were not and paid commercial platforms such as Blackboard. prepared in terms of infrastructure and internet At first glance, most of the countries providing online accessibility when the pandemic arrived but still learning are those with relatively high internet access deployed remote learning strategies relying heavily and relatively high LAYS, such as the Netherlands on digital technologies have created a situation and Singapore in the upper-right of the figure. Yet where student take-up has been unequal and yet countries in the lower-left of the figure also provide preventable. Even though the COVID-19 pandemic online learning. In fact, of the 155 countries shown in spread at a limited pace in Haiti and did not result the figure, 95 (or around 60 percent) provide remote in protracted school closures, this was a missed learning online. Box 3.1 presents Peru’s experience. opportunity to build an accessible remote learning strategy and use suitable technologies to ensure Some governments are falling education continuity. victim to the “remote learning This is true regardless of whether countries have paradox” high LAYS and high internet penetration or vice versa. Despite relatively high internet penetration Some countries are providing online remote in pre-pandemic times, Kuwait and Jordan in the learning even though they had limited technology bottom-right of the figure have relatively lower LAYS infrastructure pre-COVID-19. This is the case for and offer remote learning online. In such cases, countries on the left of the vertical bar in Figure deploying effective remote learning strategies for 3.4. These countries were not adequately prepared children most at risk of falling behind is even more in terms of infrastructure and internet accessibility critical to avoid students lagging further. Countries to supply remote learning online when the pandemic with low internet penetration and relatively forced schools to close. This has been termed the lower LAYS, such as Ghana in the bottom-left of “remote learning paradox”, where governments Figure 3.4, also chose to provide remote learning provide online remote learning solutions but a online. In such cases, remote learning may be majority of their students cannot access these hampered by households’ relatively lower internet solutions. This is creating a situation where take-up accessibility. Finally, countries with relatively high remains uneven due to lack of access to the devices LAYS and low internet access also face challenges or connectivity required for remote learning (Aedo in reaching and engaging children through online et al. 2020). instruction, such as Peru in the top-left of the figure. 24 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Box 3.1 Peru: Combining Low-tech and High-tech Remote Learning Interventions During the Pandemic and the Critical Role of Teachers Context Peru’s Ministry of Education (MINEDU) reacted quickly to the COVID-19 pandemic by planning, developing, and launching “Aprendo en Casa” (I learn at home), a comprehensive multimodal strategy to deliver remote learning at scale in less than two weeks. To implement a remote education program aimed at reaching all students, a team at MINEDU gathered to assess the current capacity and resources required for this program. The initiative considered the inputs needed to implement an effective remote education program: ensure multimodality of channels, create an inventory of existing content, make the program accessible to students from disadvantaged backgrounds and those with disabilities, and provide support to both teachers and parents. Furthermore, teachers received guidelines on observing learning sessions through the channel of their preference, communicating with students and parents, and subscribing to online learning courses through PeruEduca, the national online training platform. Technology appropriateness In Peru, access to devices and connectivity needed for remote learning varies across the country; about 80 percent of households possess a TV, 84 percent a radio, and only 24 percent have connection to the Internet at home.3 Thus, the authorities deployed a multimodal strategy that used all three channels to deliver remote learning solutions. It was felt that this could be scaled up to reach all students while schools were closed. As a result, Aprendo en Casa reached almost 85 percent of students. Take-up has been relatively high due to the appropriateness of technologies to the local context. With its radio learning program, Peru’s government partnered with over 1,100 local radios to reach students in remote areas and created content delivered in nine native languages. The Ministry of Education also curated external content for Aprendo en Casa’s TV learning program and created engaging content that was broadcast for free on the main TV channels. Moreover, telecommunication operators agreed to zero-rate Aprendo en Casa’s core digital site so that students, parents, and teachers could access all available educational resources, from any device, without paying for the bandwidth. This strategy was complemented with constant communication of weekly schedules for learning sessions, frequent teacher-student follow-up, and a strong monitoring system to understand the program’s coverage and engagement. Teacher effectiveness and learner engagement Regular teacher-student interaction has been key to ensure high take-up and engagement. MINEDU’s pedagogical team hypothesized that just providing engaging content through a one-way approach was not going to be as effective as ensuring regular interaction with the students. Teachers needed to communicate and provide feedback to students and their families through other means, including phone calls, text messages, and social media (Accinelli 2020). According to the Monitoring and Evaluation (M&E) Unit phone survey conducted in May 2021, 77 percent of students and parents have received support from teachers at least once in the past week and 89 percent of students and parents were satisfied with the communication. Moreover, 95.5 percent of parents said that at least one teacher requested students to complete and send homework in the past week. Students complete those activities and send them back to teachers mainly through WhatsApp. According to the survey, among teachers who contacted parents and requested students to complete the learning activities, 95.3 percent ended up providing feedback. Frequent teacher-student interaction and quality learning materials for remote learning allowed for moderate to high satisfaction rates: 66.8 percent of students were satisfied with the TV learning program, 47.7 percent with the radio program, and 79.6 percent with the content and learning materials accessed from the initiative’s website. Source: Information taken from interviews conducted by the World Bank EdTech team with Peruvian government officials, MINEDU’s monitoring and evaluation dashboard for 2020 and 2021, and the following secondary sources: Accinelli 2020 and Munoz-Najar 2020. 25 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Figure 3.4 Remote Learning Provision, Learning-Adjusted Years of Schooling (LAYS) and Households’ Internet Access 6 7 8 9 10 11 12 13 LAYS ( years) 5 4 3 2 1 0 0 10 20 30 40 50 60 70 80 90 100 Households with Internet Access (%) Countries offering Remote Learning through Countries not offering Remote Learning through Online Platforms. Online Platforms or with no information Linear Fit Linear Fit Source: World Bank. Calculations based on UNESCO, UNICEF, World Bank, and OECD (2021a) as of June 2021; LAYS database from WBopendata as of October 2021; and Technologies database extracted from International Telecommunication Union (2021). Note: Sample of 155 countries (95 countries providing remote learning through online platforms, and 60 countries either not providing remote learning through online platforms or without information). Governments responded to the survey question “S4 Q1. Which distance learning solutions were or are being offered in your country during the pandemic in 2020 and/or 2021?”. It considers at least one of the education levels (pre-primary, primary, lower secondary, and upper secondary). The y-axis reports learning-adjusted years of schooling (LAYS), understood as “average years of schooling measured in terms of productivity of the top performer” (Filmer et al. 2018). While the x-axis reports the estimated proportion of households with access to internet at home. The horizontal red line represents the LAYS country median (percentage) for all countries with latest available data: countries falling above the median are classified as having relatively higher LAYS, and countries falling below the median are classified as having relatively lower LAYS. The vertical red line represents the median percentage of households with internet access for all countries with available data. See Annex figure A.2 for a similar analysis using learning poverty instead of LAYS. 26 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Box 3.2 State of Edo in Nigeria: The Challenge of Scaling-up a Mobile Learning Solution Context The Edo Basic Education Sector Transformation (EdoBEST) program has demonstrated transformational learning outcomes for more than 250,000 children across over 800 public primary schools in Edo State in Nigeria. Schools that are part of the EdoBEST program score 6 percent higher on English and 7 percent higher on mathematics than those not in the program. During the COVID-19 pandemic, the state of Edo in Nigeria launched Edo-BEST@Home, a public-private partnership between Edo state, the World Bank, and Bridge International Academies. This initiative provides a fully online remote learning program that can be accessed through a computer or mobile phone and includes interactive audio lessons, digital self- study activity packets, digital storybooks, mobile interactive quizzes, and virtual classrooms. The program provides: 1. Engaging programming with content aligned to the curriculum 2. Constant support from teachers and parents to students 3. Automated formative assessment in the form of interactive quizzes 4. On-going support to teachers through virtual coaching Technology appropriateness Access to devices and connectivity needed for remote learning varies across Edo state; 46 percent of households possess a radio, 69 percent have a television, and 91 percent have a mobile phone (NPC/ Nigeria and ICF 2019).4 The Edo-BEST@Home program took these constraints into consideration by focusing on delivering content and learning activities through mobile phones (De Simone et al. 2020).4 By mid-July 2020, Edo’s remote learning program reached 930 out of 1,000 primary schools in the state and over 7,000 virtual classrooms were created to deliver remote education. However, only 29 percent of Edo’s primary school population had accessed the program’s interactive mobile-based platform. According to government officials, the main reason for the low take-up was that working parents have to use their mobile phones for their jobs and, if they have more than one child, it can be unfeasible to share one device among many children (Munoz-Najar and Oviawe 2020). Despite this, between October 2020 and February 2021, 122,760 users or 49 percent of total student enrollment accessed the state’s mobile interactive quizzes. It remains a challenge to better understand what constrains students from engaging with each of Edo-BEST@Home’s resources (interactive audio lessons, digital self-study activity packets, digital storybooks, mobile interactive quizzes, and virtual classrooms) as well as to identify the engagement level and total learning time while students are using these remote learning resources. Teacher effectiveness and learner engagement Before COVID-19, more than 11,000 teachers and school leaders had already gone through a pedagogical and digital training program to develop skills to support students to use digital technologies in the classroom. Moreover, an existing coaching program for teachers was strengthened and adapted for remote delivery. Coaches support teachers while they are using Edo-BEST@Home platform and the virtual classrooms. Teachers can answer students’ questions through the virtual classrooms, grade students’ homework and provide feedback, and communicate with both students and parents through phone calls, text messages, and WhatsApp. Source: Interviews conducted by the World Bank EdTech team to Edo’s State Universal Basic Education Board (SUBEB) government officials, World Bank task team leaders (Martin De Simone, Aisha Garba Mohammed, and Gloria Aitalohi Joseph- Raji), World Bank team members (Andrew Ragatz) as well as from the following secondary sources: De Simone et al. (2020); Munoz-Najar et al. (2020); and Obaseki (2021). 27 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Countries leveraged partnerships learning; and the point in the academic year when in a variety of ways schools were closed (ECLAC and UNESCO 2020). For example, Chile, which was at the beginning Governments partnered with the private sector of the school year when the pandemic arrived, or delivered targeted aid directly to households reorganized their school year by bringing forward to facilitate children’s access to remote the winter break (Schleicher 2020). Other countries learning. For example, among the 143 countries prioritized subjects or focused the curriculum on that participated in the Joint Survey,4 70 percent certain disciplines. India decided to reduce curricular of countries declared having plans for subsidizing content for upper-secondary students for the or providing internet access at zero cost in 2021 academic year 2020-2021 to reduce course load and beyond (UNESCO, UNICEF, World Bank, and (Nagari 2020). Similarly, the Punjab government OECD 2021b). However, there is great variability in Pakistan also opted to reduce the curriculum by across richer and poorer countries with 68 percent 40-50 percent (The News International 2020). In of high-income countries and only 25 percent of some cases, these curricular adaptations suffered low-income countries subsidizing internet access. from delayed ministerial guidelines, hindering class This strategy consisted of granting free access preparations. to national online platforms or providing funds for internet subscriptions, as was done by Chile, Most countries supported Colombia, Thailand, and Saudi Arabia. Governments teachers with remote teaching, also entered into agreements for zero-rated data tariffs with service providers. Governments have although the effectiveness of this also partnered with radio and TV broadcasters support is uncertain to reach students in remote areas. Sierra Leone partnered with 12 community radio stations and Teacher training and support with remote Peru collaborated with over 1,100 broadcasters to learning is key. According to the Joint Survey, 73 retransmit the Ministry’s radio learning program to percent of countries reported providing teachers reach students in remote areas. Other measures with special training, 89 percent reported providing included providing subsidized devices, as in Algeria, instructions on distance instruction, 80 percent Bhutan, and Iran; and supporting access to online reported providing teaching content for remote learning platforms through mobile phones, as in learning, 78 percent reported providing professional, Botswana, Mozambique, and Zambia, and through psychosocial and emotional support, and 59 percent landlines, as in Azerbaijan, Egypt and Poland. reported providing information communication Similarly, in Peru, the government partnered with technology (ICT) tools and connectivity. A 2021 Khan Academy to allow students to practice math national survey of school principals from public exercises with content that is aligned to the National schools from Brazil showed that 83 percent of Curriculum. As a result, total learning time on the teachers received assistance to work on digital platform and monthly active users have increased platforms, such as online support, courses or training, approximately 50 percent as compared to before and guidelines and structures for pre-recorded and the pandemic.4 Box 3.3 provides a brief overview live classes, among others. Additionally, in over 40 of how education systems partnered with the World percent of schools, all teachers were trained for Bank during COVID-19. remote teaching. In only 2 percent of the schools were no teachers trained (Secretaria de Educação Básica and Ministério du Educação 2021). Some countries adjusted curricula in response to the shortened Despite this strong focus on teacher support in school year remote instruction, there is no hard evidence on the effectiveness of these programs. A number of countries adjusted curricula to reflect the reduction in school days. Countries What evidence does exist suggests limited impacts, considered the following characteristics for particularly as programs faced implementation adjusting the curriculum: the national or subnational hurdles and teachers struggled with burnout and curricula characteristics; the country’s resources anxiety. and institutional capacities to deploy remote 28 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Box 3.3 World Bank Support to Education Systems During COVID-19 As of May 2021, the World Bank Group’s Education Global Practice had channeled US$1.6 billion to supporting countries in their responses to COVID-19. This includes but is not limited to activities focused on remote learning. This box provides highlights from this portfolio of operations delivering remote learning opportunities during the pandemic. In Egypt, the World Bank, together with the OECD, Harvard Global Education Innovation Initiative, and Hundred, have been working with the Ministry of Education and Technical Education on expanding a Knowledge Bank Study Portal.5 In Nicaragua, the Ministry of Education is implementing the National Strategy for Basic and Secondary Education in response to COVID-19. The World Bank has been supporting this strategy with a US$6.7- million project. Among other activities, the project considers reinforcing remote learning and blended learning in schools through mobile digital classrooms and adaptive learning, as well as assisting in the development of tools and capacity building activities. In Sierra Leone and Liberia, the World Bank, together with the OECD, Harvard Global Education Innovation Initiative, and Hundred, have been working with both countries in a project called Rising Academy Network on air. The nationwide project adapted their curriculum in response to school closures caused by COVID-19 and created a radio program of 20 weeks, free of charge, that uses using radio scripts and SMS content for parents. The project was implemented swiftly due to pre-existing relationships with the education ministries in both countries. In Sierra Leone, the World Bank has been working with the Ministry of Education, Science and Technology on the Free Education (FREE) Project, a US$50-million grant approved by the International Development Association (IDA) in June 2020 that aims to mitigate student learning losses during COVID-19 through: “Distance learning to 1.4 million children; sensitization campaigns, gender-responsiveness and back- to-school arrangements; school safety protocols and psychosocial support guidelines to all of the 11,000 primary and secondary schools, as well as training for 22,000 teachers; hygiene and safety products for all 11,000 schools; annual School Census 2020; and a legal and policy framework to support the inclusion of all children in Sierra Leone.” Turkey, in response to pandemic-induced school closures, introduced remote learning in March 2020 through its online digital education system (Eğitim Bilişim Ağı or EBA) and EBA TV. EBA virtual classrooms followed in April 2020. To support these efforts, the World Bank has been working with the Ministry of National Education in the Safe Schooling and Distance Learning Project, approved in June 2020 with funding of US$160 million. The project consists of three components: 1. Emergency connectivity and information technology (IT) infrastructure for education in emergencies, which supports the expansion of the EBA and the development and rollout of a new digital education system; 2. Digital content for safety and quality, which supports the distance education content, aiming at a gradual return to in-person schooling, and to strengthen hybrid teaching and learning (in-person and online); and 3. Institutional capacity for education technology resilience, supporting coordination, management, monitoring and evaluation during the project and after its conclusion (World Bank 2020g). As of October 2020, the EBA platform had reached 18 million students and over one million teachers. Source: El Zayat (2020); Lamba and Reimers (2020); World Bank (2021a); World Bank (2021b); and World Bank (2020f). 29 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Notes 1. As reported in a survey of education ministries conducted by UNESCO, UNICEF, the World Bank, and OECD, henceforth referred to as the Joint Survey. See UNESCO, UNICEF, the World Bank and OECD (2021b). 2. LAYS is understood as the average years of schooling measured in terms of productivity of the top performer, combining quantity (years) and quality (learning) of schooling into a single indicator. Filmer et al. (2018). 3. TV, radio and mobile penetration obtained from UNICEF Data and Internet penetration obtained from World Bank tcdata360 4. Based on discussions with the Khan Academy team in Peru. 5. Other World Bank educational projects in Egypt can be found in: https://documents1.worldbank. org/curated/ar/346091522415590465/pdf/PAD-03272018.pdf References Accinelli, A. 2020, June 11. “Going the Distance. Peru’s Two-week Remote Education Response to COVID-19.” UKFIET blog. Retrieved from https://www.ukfiet.org/2020/going-the-distance-perus- two-week-remote-education-response-to-covid-19/ Aedo, C., V. Nahata, and S. Sabarwal. 2020. “The Remote Learning Paradox: How Governments Can Truly Minimize COVID-related Learning Losses.” World Bank blog. https://blogs.worldbank.org/education/ remote-learning-paradox-how-governments-can-truly-minimize-covid-related-learning-losses De Simone, M., A. Garba, G. Joseph-Raji., J. O. Oviawe, A. Ragatz, and A. Twinomugisha. 2020. “Learning Despite the Crisis: The Case of Edo State in Nigeria. World Bank blog.” Retrieved from https://blogs. worldbank.org/education/learning-despite-crisis-case-edo-state-nigeria Economic Commission for Latin America and the Caribbean (ECLAC), and United Nations Educational, Scientific and Cultural Organization (UNESCO). 2020. “Education in the Time of COVID-19.” https:// caribbean.eclac.org/publications/education-time-covid-19 El Zayat, N. 2020. “Egypt (K-12 Egyptian Knowledge Bank Study Portal and New Form of Assessment).” World Bank, Global Education Initiative, OECD, HundrED. Retrieved from https://documents1.worldbank.org/ curated/en/191341599145436193/pdf/Egypt-K-12-Egyptian-Knowledge-Bank-Study-Portal-and- New-Form-of-Assessment.pdf Filmer, D., H. Rogers, N. Angrist, and S. Sabarwal. 2018. Learning-Adjusted Years of Schooling: Defining A New Macro Measure of Education. Policy Research Working Paper;No. 8591. Washington, DC: World Bank. Retrieved from https://openknowledge.worldbank.org/handle/10986/30464 License: CC BY 3.0 IGO International Telecommunications Union (ITU). 2021. “World Telecommunication/ICT Indicators Database, 24th Edition/December 2020.” Retrieved from https://www.itu.int/en/ITU-D/Statistics/Pages/ publications/wtid.aspx Lamba, K., and F. Reimers. 2020. “Sierra Leone and Liberia: Rising Academy Network on Air.” Retrieved from https://oecdedutoday.com/wp-content/uploads/2020/05/Sierra-Leone-Liberia-Rising-Academy- Network.pdf Johns Hopkins University, World Bank, and UNICEF. 2021. “COVID-19 Global Education Recovery Tracker.” Web site. https://www.covideducationrecovery.global 30 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW McClain-Nhlapo, C. V., R. Kulbir Singh, A. H. Martin, H. K. Alasuutari, N. Baboo, S.J. Cameron, . . . M. Tucker. 2020. “Pivoting to Inclusion: Leveraging Lessons from the COVID-19 Crisis for Learners with Disabilities (English).” Retrieved from http://documents.worldbank.org/curated/en/777641595915675088/ Pivoting-to-Inclusion-Leveraging-Lessons-from-the-COVID-19-Crisis-for-Learners-with-Disabilities Munoz-Najar, A. 2020. “Peru: Aprendo en Casa (I Learn at Home)”. World Bank, Global Education Initiative, OECD, HundrED. https://oecdedutoday.com/wp-content/uploads/2020/09/Peru-Aprendo-en- Casa.pdf Munoz-Najar, A., and J. O. Oviawe. 2020. Nigeria: Edo-BEST@Home. World Bank, Global Education Initiative, OECD, HundrED. https://oecdedutoday.com/wp-content/uploads/2020/11/Nigeria-Edo-BEST-at- Home.pdf Nagari, A. 2020, July 7. “HRD Minister Asks CBSE to Revise Class 9th and 12th Curriculum for 2020- 21 to Reduce Load on Students.” Hindustan Times. https://www.hindustantimes.com/education/ hrd-minister-asks-cbse-to-revise-class-9th-and-12th-curriculum-for-2020-21-to-reduce-load-on- students/story-amiaegemwyH5qCrgcyBGyM.html NPC/Nigeria, and ICF. 2019. Nigeria Demographic and Health Survey 2018. Abuja, Nigeria, and Rockville, Maryland, US: NPC and ICF. Obaseki, G. 2021, June 11. “How an Exodus of Young Nigerians Spurred a Rethink on Schools: The Governor of Edo State Describes a Radical Overhaul That is Attracting Attention Abroad.” Financial Times. Available at: https://www.ft.com/content/6f634094-686a-4123-85dc-3be43d35e1e4 Secretaria de Educação Básica, and Ministério da Educação. 2021. “Questionario dos Diretores Escolares.” Retrieved from Painel Educação Básica e o COVID-19: https://painelcovid-seb.mec.gov.br/ questionario/ The News International. 2020, August 10. “Punjab Govt Reduces Syllabus of All Classes for Academic Year 2021.” International The News International. https://www.thenews.com.pk/latest/698697-punjab- govt-reduces-syllabus-of-all-classes-for-academic-year-2021 UNICEF. 2021. “COVID-19 and School Closures: One year of Education Disruption.” Retrieved from https:// data.unicef.org/wp-content/uploads/2021/03/COVID19-and-school-closures.pdf United Nations Educational, Scientific and Cultural Organization (UNESCO). 2020a. “From COVID-19 Learning Disruption to Recovery: A Snapshot of UNESCO’s Work in Education in 2020.” Retrieved from https://en.unesco.org/news/covid-19-learning-disruption-recovery-snapshot-unescos-work- education-2020 United Nations Educational, Scientific and Cultural Organization (UNESCO), UNICEF, World Bank and Organization for Economic Co-operation and Development (OECD). 2021b. What’s Next? Lessons on Education Recovery: Findings from a Survey of Ministries of Education Amid the COVID-19 Pandemic. UNESCO, Paris, UNICEF, New York, World Bank, Washington DC, OECD, Paris. Retrieved from https:// openknowledge.worldbank.org/handle/10986/36393 World Bank (2020f). “Responding to Covid-19 and Looking Beyond: Turkey Invests in Safer Schooling and Distance Education with the World Bank Support.” Retrieved from: https://www.worldbank.org/en/ news/press-release/2020/06/25/responding-to-covid-19-and-looking-beyond-turkey-invests-in- safer-schooling-and-distance-education-with-the-world-bank-support World Bank (2020g). Project Appraisal Document on a Proposed Loan in the Amount of 143.8 million euros to the Republic of Turkey for a Safe School and Distance Education Project. Washington DC: World Bank. https://documents1.worldbank.org/curated/en/788991593396173808/pdf/Turkey-Safe- Schooling-and-Distance-Education-Project.pdf 31 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW World Bank (2021a). Protecting Children from Learning Loss: Sierra Leone’s Experience with Distance Learning and Keeping All Students Safe. Washington DC: World Bank. Retrieved from https://www. worldbank.org/en/results/2021/03/17/protecting-children-from-learning-loss-sierra-leone-s- experience-with-distance-learning-and-keeping-all-students-safe World Bank. 2021b. “The World Bank in Nicaragua.” Retrieved from https://www.worldbank.org/en/ country/nicaragua/overview#3 32 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW 4. WHAT ARE WE LEARNING FROM COUNTRY RESPONSES? ralston-smith 33 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW 4. WHAT ARE WE LEARNING FROM COUNTRY RESPONSES? This section presents a summary of ongoing lessons higher for children of more educated parents. While emerging from global experiences of remote learning more than 80 percent of students from households during COVID-19. Given that the pandemic is ongoing, with more educated adults (i.e., tertiary education) this list of lessons is necessarily a ‘live’ one. have remained engaged in remote learning during school closures, engagement is much lower among Pre-existing socioeconomic households where adults have lower education levels, as seen in countries such as Bolivia, Mexico, Peru, inequalities are hampering Ghana, and Mozambique. Children’s engagement take-up of remote learning with remote learning is generally low where parents or caregivers lack any type of education and, in several Take-up of remote learning during school countries, these children were three-to-four times closures is being hindered by pre-existing less likely to engage in a learning activity compared inequities prevalent in most systems. Data from to households where parents have tertiary education household surveys collected by the World Bank (Azevedo et al. 2022), as seen in the Philippines and between April and September 2020 suggest that the Peru (Figure 4.1). level of learning engagement during the pandemic is Figure 4.1 Children Engaging in Any Remote Learning Activity Since School Closures, Selected Countries by Highest Adult Educational Level in the Household Middle East East Asia Europe and Latin America and North Sub-Saharan Africa and Pacific Central Asia and Caribbean Africa Phillippines Mongolia Tajikistan Croatia Bolivia Colombia Peru Tunisia Sierra Leone South Sudan Ghana Mozambique Proportion of households with enrroled students 100 91 who declare they was engaged in learning (%) 74 96 96 96 90 92 91 83 89 67 85 87 85 85 80 83 66 79 78 78 67 61 70 72 61 72 71 67 67 68 65 64 60 64 61 58 57 58 50 48 49 30 40 36 44 42 40 40 38 30 33 34 20 31 20 25 13 23 18 17 10 16 16 14 11 10 9 0 No education Primary Secondary Tertiaty No education Primary Secondary Tertiaty No education Primary Secondary Tertiaty Primary Secondary Tertiaty No education Primary Secondary Tertiaty No education Primary Secondary Tertiaty No education Primary Secondary Tertiaty No education Primary Secondary Tertiaty No education Primary Secondary Tertiaty No education Primary Secondary Tertiaty No education Primary Secondary Tertiaty No education Primary Secondary Tertiaty Average Source: World Bank. Calculations based on High-Frequency Phone Survey (World Bank 2020d) as of March 16, 2021. Note: Survey respondents were at the household level. Household responses refer to the last 7 days before the date of interview. Latin American and the Caribbean responses from Wave 1 were removed to avoid a different reference window. Countries were selected from various regions and a subset of all countries is shown to facilitate presentation. Data from all countries suggests similar patterns. 34 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Remote learning materials produced during Income losses during the pandemic have only COVID-19 are often in the most commonly-spoken exacerbated these inequalities. Ability to afford national or international languages, leaving out new devices or to cover extra costs for remote around 40 percent of students worldwide who are learning connectivity varies by parental background. unable to access education in the language they For instance, parents with incomplete schooling speak or understand (UNESCO 2021b).1 Similarly, in South Africa need to allocate 10 percent of students in remote areas, learners with disabilities, their monthly expenditure to afford 1 gigabyte of and migrant and displaced students need tailored internet data, which is a substantial burden for support to recover learning losses. Some countries families already experiencing financial constraints have supported these groups during the pandemic in (World Bank 2020a). Therefore, while the most the form of improved access to infrastructure, design advantaged children may continue to access of learning materials in different minority languages learning opportunities as school systems transition and providing flexible and self-paced platforms to remote learning, children whose families cannot (UNESCO, UNICEF, and World Bank 2020). However, afford these additional expenses are likely falling these measures only partly mitigate the higher further behind. learning losses that these students face compared to students from more advantaged backgrounds. Take-up is also influenced by Not all parents have been able to support their constraints in both supply and children’s remote learning during COVID-19 in demand equal measure. A study from the Netherlands reports that 75 percent of the parents with a graduate degree Emerging data on take-up during remote felt capable of helping their children in secondary learning suggests a great deal of variation education with schoolwork, compared to 40 percent among countries. High-frequency phone surveys of the parents with lower levels of education (Bol to monitor the impact of the pandemic on remote 2020). Another study in the United Kingdom in 2020 learning take-up from April 2020 and June 2021 determined that economically better-off parents gathered data on remote learning activities such were spending on average 30 percent more time as completing teachers’ assignments, using mobile with their children’s home schooling compared to learning apps, watching or listening to educational economically disadvantaged parents (Andrew et al TV or radio programs, and tutoring sessions. 2020). This means that students from disadvantaged According to the data as of March 2021, take-up of backgrounds have less parental support and remote learning greatly varies by country, ranging reinforcement of positive attitudes towards learning, from as high as 100 percent of children engaging in which is affecting their learning experiences while remote learning since school closures in Bulgaria to schools are shut down (OECD 2020). Furthermore, 52 percent in Burkina Faso, to as low as 20 percent better-off parents can provide learning resources in Ethiopia and the Philippines (Figure 4.2). Some (such as learning pods, which consist of families countries , such as Lao P.D.R. and Ethiopia, that faced collectively paying for private tutors and other relatively higher levels of learning poverty prior to materials) that less well-off parents may not be able the pandemic or relatively lower quality of education to afford. expressed in learning-adjusted years of schooling also struggled with low student engagement with Home schooling is putting a higher burden on remote learning (see Annex Figure A.3). This poses women. A study in March 2020 in Catalonia, Spain, an urgent need to engage children already at high revealed that 79 percent of female parents or risk of experiencing learning losses during school caregivers reported supporting their children with closures. schoolwork. In contrast, only 43 percent of male adults reported doing so. Support also varied by the age of the student, as parents reported supporting children in primary education more often compared to those in secondary education (Bonal and Gonzales 2020). 35 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Figure 4.2 Remote Learning Take-Up was Lower in Countries that Already Had High Rates of Learning Poverty 100 90 80 70 Learning Poverty (%) 60 50 40 30 20 10 0 0 10 20 30 40 50 60 70 80 90 100 Households with children engaging in remote learning (%) Source: World Bank. Calculations based on High-Frequency Phone Survey (World Bank 2020d) as of March 16, 2021, and Learning Poverty database from Azevedo et al. (2019) as of August 2021. Note: Sample size of 27 countries. Uruguay, like Chile and Costa Rica, has managed to Geography also limits the availability of stable internet ensure high take-up of its remote learning program. or phone signals. Similarly, informational constraints In the specific case of Uruguay, prior experience can also hamper take-up: parents and children with with educational technologies has played a key role low awareness of recently created programs for remote in ensuring high usage of online learning platforms learning can be at a disadvantage. Even when households while schools were closed during the COVID-19 have device access, parental involvement in facilitating pandemic (Box 4.1). remote learning may vary depending on: These data point to a few patterns on both the supply • Digital skills required to maneuver the device (e.g., and demand sides. On the supply side, the chosen parents with little or no education may be less remote learning strategy might be unable to reach a likely to operate a laptop.) majority of students in some settings. In others, the technology used may be unreliable and the resulting • Complexity of the digital device (e.g., laptops irregular access could discourage take-up. In still might require higher parental involvement than a others the choice of content offered for remote TV or radio). learning may not be aligned with student needs. • Parental work status (e.g., working parents might On the demand side, parents who cannot afford internet have less time to be involved in the children’s or mobile subscriptions or devices could be left out. remote learning). 36 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW • Age of the children (e.g., younger children might are accessible to students as well as to provide need more frequent or sustained parental support solar-powered radios for vulnerable children with compared to older students). limited access to electricity (Global Partnership for Education 2020b). Nevertheless, according to For example, Malawi has leveraged an existing phone-based surveys and U-Reports,2 only 36 interactive radio instruction program adapted to percent of respondents claimed that school-age the COVID-19 context (Gondwe 2020). Malawi’s children have been accessing remote learning government worked with national radio broadcasting programs. See Box 4.1 for a similar disconnect corporations to ensure remote learning initiatives between supply and take-up in Kenya. Box 4.1 Kenya: A Case Study of Mismatch Between Supply and Take-Up Context In Kenya, the government partnered with private providers to increase the availability of an existing multimodal remote learning program that included radio, TV, and online learning programs, rather than developing new programs in response to the pandemic. During the first months of the pandem- ic, the Kenya Institute for Curriculum Development (KICD) worked to increase the availability of an ex- isting remote learning program. First, a partnership was developed with the main national radio station and 42 community stations to broadcast KICD’s radio learning program to reach students in remote areas and broadcast hours were also extended from four hours to eight hours per day. Second, all TV signal providers agreed to carry the government’s television “EDU TV” content for free so that families that have a TV do not have to pay to see the channel. Furthermore, programming was extended to run from Monday to Saturday from 6:40 AM to 7:00 PM, depending on the grade. Third, the digital site was enhanced to be device-neutral, “web-light”, and with enough capacity to support a large number of users at the same time.  Technology appropriateness As access to devices and connectivity needed for remote learning varies across the country: about 90 percent of households have access to a mobile phone, 71 percent possess a TV, 37 percent a radio, and only 17 percent have connection to the Internet at home. Kenya’s multimodal remote learning strategy seemed to have implemented the appropriate technologies to scale up remote learning solutions to reach a maximum number of students while schools were closed. However, a survey conducted by the  Kenya National Bureau of Statistics  found that  about 25 percent of households were not using any method to learn at home and an Uwezo report showed that only 22 percent of school-going children in Kenya were accessing digital resources for remote learning. Among these children, 42 percent accessed learning through a TV program, 27 percent through WhatsApp, 19 percent through a radio program, and 10 percent downloaded materials from the Kenya Education Cloud (Uwezo 2020). From the technology appropriateness aspect, take-up was low because the ways in which technologies were implemented were not suited for the context. While most Kenyans have mobile phones, only half can access the Internet that way and computers are available in just one-fifth of households. Thus, solutions offered through the Kenya Education Cloud only worked for a select group of students. Furthermore, only 44 percent of Kenyans said they receive reliable electricity from the national grid, making it challenging to access remote learning solutions through electronic means. Source: Interviews conducted by the World Bank EdTech team with Kenyan government officials as well as from Uwezo (2020). 37 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Evidence on the effectiveness of with differences being driven by factors such as remote learning during COVID-19 household’s socioeconomic background, and pupils’ and teachers’ digital divide (Tomasik et al. 2020). is mixed In the US, there is mounting empirical evidence on While limited, emerging research on the the negative impacts of schooling disruptions on effectiveness of remote learning during the learning outcomes. For early learners, the absence pandemic is mixed at best (See Annex Table of unstructured social activities that typically occur in A.1). Given the myriad ways in which the pandemic a preschool setting and the fact that remote learning has wreaked havoc on school systems, students does not adequately replace preschool activities and parents, this should come as no surprise. suggests that learning losses could be double what is expected during a typical summer break (Barnett Evidence from a study of Dutch students’ learning and Jung 2020). T his is in part due to the disruption outcomes before and after COVID-19 school of in-person schooling as well as a lack of parental closures suggests learning losses equivalent to activities to replace preschool learning. For older one-fifth of the school year (3 percentile points) students, disparities in the underlying schooling after a relatively short period of school closure (8 conditions are reflected in emerging assessment weeks), particularly for children in primary school. data. A recent study finds that students learned Researchers also find that losses were up to 60 “only 67 percent of the math and 87 percent of the percent larger among students with less-educated reading that grade-level peers would have typically parents. This evidence suggests that even children learned by the fall.” This suggests that children have studying in a technologically-advanced remote lost about 3 months of learning in mathematics learning setup made little or no progress on learning and 1.5 months in reading. These averages mask outcomes (Engzell et al. 2020). Another study in disparities by race: in schools that predominantly Belgium, focused on the Flemish region, found that serve non-white students’ scores were 59 percent mathematics and language scores decreased by of the historical average in math and 77 percent in 0.19 and 0.29 standard deviations respectively for reading (Dorn et al. 2020). students in grade 6 compared to previous cohorts. Moreover, the researchers found that inequality both Data from Sao Paulo, Brazil obtained at the within and between schools increased between 7 beginning of 2021 confirms previous trends percent and 20 percent for both subjects, varying by found in learning outcomes. Assessments of students’ socioeconomic status. This suggests that more than 20,000 students in grades 5, 9, and schools with a higher proportion of students from 12 in language and mathematics show a 10-year households with higher socioeconomic status may setback in learning outcomes in language when suffer lower learning losses from school closures comparing different cohorts of students in grade (Maldonado and De Witte 2020). 5. For mathematics students in the same grade, learning outcomes were comparable to the cohort Similarly, a study conducted in Germany, largely of fifth graders 14 years ago. The study concludes focusing on Baden-Württemberg, found that that COVID-19 caused children in fifth grade to children in primary education suffered from learning have an equivalent pre-pandemic learning level of losses in 2020 scores compared to previous years, a child in third grade. In the case of Chile, almost of around 0.07 standard deviation for reading and 1.9 million students from grades 2 to 12 were 0.09 for maths. Learning losses were more important assessed, equivalent to 81 percent of the total for lower-achieving students (Shult et al. 2020). enrollment. Results show that after a full academic A natural experiment in Switzerland comparing year under COVID-19, both girls and boys between learning progress 8 weeks before and during school grades 6 and 12 failed to reach 60 percent of the closures concludes that secondary school students minimum expected knowledge in language. This were able to sustain learning progress in a remote is a particularly alarming result considering that learning environment, while learning gains for the assessment was based on a prioritized subset primary school children slowed and became more of the general curriculum. Here too, the lowest varied depending on the remote learning setup. learning outcomes were found in mathematics. Children attending in-person instruction learned Students in grade 10 only reached 27 percent of twice as fast compared to remote instruction, 38 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW the minimum expected knowledge. Learning gaps literacy rate was nearly 70 percent as of 2018, were found between vulnerable and better off which can limit parental ability to utilize digital tools students, particularly among those above grade 6. and therefore their ability to help children with schoolwork (McClain-Nhlapo et al 2020). However, not all places experienced learning losses with remote learning during the pandemic. Recent Teachers will need tailored and data from Australia and Uruguay suggests that despite long periods of remote learning, on average, ongoing support if remote learning student test scores stayed the same. In Australia, is going to work recent student test scores were on par with results from 2019 (Karp 2021). Similarly, in Uruguay, a The variety of remote learning modalities national evaluation for third and sixth grades did not possible require different skills and proficiencies find relevant differences in students’ achievement from teachers. The complexity of remote learning between 2017 and 2020. Moreover, results show varies by the amount of learning resources used. promising evidence that, when controlling for For example, TV lessons with follow-up calls from socioeconomic context, student performance teachers will be more complex than delivering remote improves as the usage of digital platforms that allow learning through TV lessons alone, as the former for teacher-student interaction increases (INEEd would involve teachers to record the lesson and make 2021). follow-up calls. It is possible that more advanced technological systems (e.g., remote, collective, synchronous systems such as videoconference or What was a digital divide for some real-time collaboration online) may require more groups is now a digital chasm for advanced skills from the teacher, such as digital many literacy to use remote modalities, adapting teaching content and preparing lessons suitable to online Digital divides are more visible than before. delivery, keeping a remote classroom engaged, and COVID-19 has highlighted the growing gap between assessing progress and communicating feedback the underprivileged and privileged members remotely, among others. This is not only because of society in terms of access to infrastructure of the requisite technical skills needed to use these (electricity, reliable connectivity or devices), tools (a key condition), but also because of the variety resources (access to learning materials, textbooks, of actions teachers may need to take remotely to platforms, education software, among others), maximize learning; for example, managing time and capacities (technical skills to learn how to use usage, facilitating interactions, adjusting content different technologies, media literacy, higher-order depending on progress, mediating relationships, skills, among others). Data from Maths-Whizz, a managing student attention, providing support virtual tutoring platform, suggests that increases and personalized attention, to name but a few in the number of users between April and May (Wilichowski and Cobo 2020). In the case of South 2020 compared to before COVID-19 are higher Korea, the country swiftly increased the digitalization in countries with high levels of internet or mobile of education during the COVID-19 pandemic. phone access, such as New Zealand or the United However many teachers had not experienced online Kingdom. On the contrary, when partner schools teaching before and, facing the lack of skills to in rural Kenya closed due to COVID-19, student prepare online classes, had difficulties in supporting use of the platform fell to almost zero (Whizz children in the remote environment and providing Education 2020). This is likely due to the fact that interactive tasks. As support, the government put in only 51 percent of Kenyans have internet access place remote learning training videos for teachers, at home.3 School ICT labs are an important source a network of experienced teachers to support of internet access for many rural Kenyan students and guide other teachers in remote learning, and and school closures hinder digital remote-learning helplines to address any technical issues in online approaches. Similarly, Nepal introduced an online classes (UNICEF ROSA et al. 2021). portal for grades 1-10 and a YouTube channel to deploy remote learning. However, household The risk of burnout despite support is real. In Brazil, internet access in the country, as of 2017, was more than 60 percent of school principals across just 18 percent (ITU 2021). Furthermore, the adult the country felt insufficient training of teachers 39 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW was a key hurdle for implementing and developing Nepal showed positive evidence of increasing remote learning (Secretaria de Educação Básica foundational numeracy via phone calls and SMS and Ministério du Educação 2021). One study, messages during COVID-19 as revealed by a conducted by Instituto Peninsula found that 67 randomized trial including 3,700 households with percent of teachers were anxious about remote children in public school, grades 3-5. The study teaching, 38 percent felt tired, while less than 10 highlights that mobile phone-based tutoring (by percent were happy or satisfied (Instituto Peninsula reaching caregivers and students) led to a 30 percent 2020). In England, headteachers’ job satisfaction increase in foundational numeracy. This study has fallen because they are worried about school suggests that instructional support through mobile budgets, keeping staff and students safe from phones can be a high-access and low-cost approach COVID-19, and finding cover for teachers off sick or to providing instruction at scale (Radhakrishnan et self-isolating. In Chile, the quality of life perception al. 2020). In addition, the language of instruction among female teachers has been affected due to used for remote learning has not always been the COVID-19 confinement and several studies inclusive; only one in three of the 101 countries have reported that female teachers in the region that reported the languages of instruction of their do more hours of housework than male teachers. In remote learning curriculum have remote learning Peru, as of November 2020, 65 percent of teachers available in multiple languages (Center for Global were stressed and anxious due to remote learning Development 2020). See Box 4.2 for the Uruguayan work and administrative reporting. experience. Communication with parents and students is key for remote learning interventions to operate at scale Parents are important allies in ensuring take- up and effectiveness of remote learning. During COVID-19, the government of Pakistan set up a comprehensive remote learning program that involves content broadcast on TV (through a program called TeleSchool) along with SMS messaging to deliver schedules, updates, and quizzes. Yet, in Punjab, the most populous province of the country, only 30 percent of households were aware of remote learning opportunities and only 10 percent used these programs while schools were closed (Geven et al. 2020). At a national level, Brazil deployed remote learning through TV programs and pre-recorded and live classes, among other strategies. However, only 12 percent of 58,000 school principals reported that all students in their schools were engaging in remote learning (Secretaria de Educação Básica and Ministério du Educação 2021). In Bangladesh, from a sample of students who were eligible to receive poverty-targeted stipends, 86 percent were aware of “Sangsad”, a TV learning program implemented by the Ministry of Education, but only 43 percent had watched it in the past week (Biswas et al. 2020). In Sierra Leone, recorded lessons were aired daily at a fixed time, leaving out students who had household or labor responsibilities. 40 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Box 4.2 Uruguay: The Ceibal en Casa Program Context Uruguay’s government created Plan Ceibal in 2007 with the aim of supporting education with technology (Plan Ceibal 2021a). Since its implementation, every child who enters the public education system is given a computer for personal use with a free internet connection at school. Thus, over 80 percent of households in rural areas have a computer (Rieble-Aubourg et al. 2020)7 and the program has been successful in increasing digital inclusion in Uruguay, reducing the digital divide and implementing a comprehensive remote learning program while schools were closed during the COVID-19 pandemic (Caballero 2018). Equally important, Ceibal’s Mathematics Adaptive Platform has improved student test scores by 0.2 standard deviations prior to the pandemic. The Ceibal en Casa (Ceibal at home) program launched immediately after school closures were announced because it could draw on the pre-existing systematic deployment of Plan Ceibal’s digital resources (Ripani 2020). Even though Uruguay is the one of the few countries in the region where education technology is highly developed, the government continued to strengthen its technological infrastructure and remote services during the pandemic: telecommunication operators agreed to zero-rate access to the main education technology (EdTech) platforms, server capacity increased by 400 percent, computers were provided to students who lacked access to devices required for remote learning, and a help desk was established to address student and teacher concerns. Technology appropriateness Uruguay’s vast experience with EdTech platforms, the government’s investment in technological infrastructure in the last decade, and the selection of appropriate technologies and content to deliver remote learning while schools were closed during the pandemic allowed for high take-up. Student usage of Ceibal’s online platforms during COVID-19 increased 2,454 percent in March 2020 in comparison to March 2019. Average daily access to Ceibal’s online platform (CREA) was 206,000 users when schools were fully closed and remote learning was in place (March 15, 2020 – June 30, 2020). As schools partially reopened, average daily access to this online platform was 108,000 users (Figure 4.3, on next page). Use of the CREA platform has decreased as schools reopened given that teachers, students, and caregivers were “very exhausted” from remote learning and were in need of human interaction. That said, teachers believe that even when schools fully reopen, they will continue to use the CREA platform and other Ceibal’s EdTech resources. Students have also been engaging withonline formative assessments implemented by SEA (Sistema de Evaluación de Aprendizajes in Spanish). While schools were closed, 96 percent of primary school students voluntarily underwent formative online assessments through an evaluation platform that was implemented prior to the pandemic (Ripani 2020). Teacher effectiveness and learner engagement Uruguay’s Institute for In-service Teacher Training adapted a pre-COVID-19 coaching program that provides pedagogical support to teachers to an online format to continue supporting teachers (ANEP 2020). Uruguayan teachers have been able to access teaching resources such as discussion forums, virtual training, and guidelines for remote teaching through the CREA learning management system that teachers have been using for several years. Over 90 percent of Uruguayan teachers were satisfied or very satisfied with the remote training received during the pandemic. Thus, prior training and coaching have worked to pivot towards remote teaching-learning during COVID-19. Teachers were also expected to provide administrative information, but instead of requesting new information from them, the government decided to turn to GURI, a digital platform that has been used by Uruguayan teachers for over 10 years to report data such as student attendance and grades (Plan Ceibal 2021b). Moreover, the platform already contained parents’ contact information, allowing teachers to easily 41 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW reach out to them during school closures. Teachers have played a critical role in ensuring Ceibal’s online learning platforms are used effectively. Researchers have observed that students benefit more when the national Mathematics Adaptive Platform is used at a group level in the classroom, suggesting the critical role that both teachers and peers play when using digital platforms for learning (Perera 2018). Figure 4.3 Use of CREA Virtual Classrooms (2020 vs 2019) Remote learning during Blended learning during school closures partial school reopening Source: Interviews conducted by the World Bank EdTech team to Uruguay’s government officials as well as from the following secondary sources: Ripani (2020); ANEP (2020); Instituto de Formación en Servicio; and Plan Ceibal (2020a; 2020b). 42 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Notes 1. Challenges are most prevalent in regions where linguistic diversity is notable, such as Sub-Saharan Africa, and Asia and the Pacific. 2. U-Report is a social messaging tool and data collection system developed by UNICEF. The program sends SMS polls and alerts to its participants, collecting real-time responses, and subsequently publishes gathered data. Malawi information referenced here retrieved from https://ureport.mw/ opinion/4395/ 3. Gallup World Poll Data 2019. References Andrew, A., S. Cattan, M. C. Dias, C. Farquharson, L. Kraftman, S. Krutikova, . . . A. Sevilla. 2020. Learning During the Lockdown: Real-Time Data on Children’s Experiences During Home Learning. London: Institute for Fiscal Studies. Retrieved from http://dx.doi.org/10.1920/BN.IFS.2020.BN0288. ANEP. 2020. “Cursos e Inscripciones.” Instituto de Formación en Servicio. https://www.dgeip.edu.uy/IFS/ index.php?option=com_content&view=category&layout=blog&id=17&Itemid=196 Azevedo, J. P., D. Goldenberg, S. Montoya, R. Nayar, H. Rogers, J. Saavedra., and B. W. Stacy. 2021. Will Every Child Be Able to Read by 2030? Defining Learning Poverty and Mapping the Dimensions of the Challenge. Policy Research Working Paper 9588. Washington, DC: World Bank. Retrieved from http://gaml.uis.unesco.org/wp-content/uploads/sites/2/2021/03/Azevedo-et-al-2021_Will-Every- Child-Be-Able-to-Read-by-2030.pdf Azevedo J.P., M. Gutierrez, R. de Hoyos, J. Saavedra. 2022 “The Unequal Impacts of COVID-19 on Student Learning.” In: Reimers F.M. (ed) Primary and Secondary Education During Covid-19. Cham, Switzerland: Springer. https://link.springer.com/book/10.1007/978-3-030-81500-4#about Barnett, W. S. and K. Jung, 2020. “Understanding and responding to the pandemic’s impacts on preschool education: What can we learn from last spring?” The National Institute for Early Education Research. https://nieer.org/wp-content/uploads/2020/08/NIEER-Special-Report-July-2020-What-Can-We- Learn-From-Last-Spring.pdf Biswas, K., T. M. Asaduzzaman, D. K. Evans, S. Fehrler, D. Ramachandran, and S. Sabarwal. 2020. TV-Based Learning in Bangladesh : Is it Reaching Students? Washington, DC: World Bank. Retrieved from https://openknowledge.worldbank.org/handle/10986/34138 Bol, T. 2020. “Inequality in Homeschooling During the Corona Crisis in the Netherlands. First Results from the LISS Panel.” https://doi.org/10.31235/osf.io/hf32q Bonal, X. and S. González. 2020. “The Impact of Lockdown on the Learning Gap: Family and School Divisions in Times of Crisis.” International Review of Education 66, 635–655. https://doi.org/10.1007/s11159- 020-09860-z Caballero, S. 2018. “Una Revision Sistemática a 10 Años del Plan Ceibal en Uruguay.” Retrieved from http://didaskomai.fhuce.edu.uy/index.php/didaskomai/article/view/29 43 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Center for Global Development. 2020. “COVID education policy tracking open-access database.” https:// www.cgdev.org/media/covid-19-education-policy-tracker Dorn, E., B. Hancock, J. Sarakatsannis,and E. Viruleg. 2020. “COVID-19 and Student Learning in the United States: The Hurt Could Last a Lifetime.” Public Sector Practice. McKinsey & Company. Retrieved from https://www.mckinsey.com/industries/public-and-social-sector/our-insights/covid-19-and- learning-loss-disparities-grow-and-students-need-help# Engzell, P., A. Frey, and M. D. Verhagen. 2020. “Learning Loss Due to School Closures During the COVID-19 Pandemic.” SocArXiv. https://doi.org/10.31235/osf.io/ve4z7 Geven, K. and A. Hasan. 2020. Learning Losses in Pakistan Due to COVID-19 School Closures. Washington, DC: World Bank. Retrieved from https://openknowledge.worldbank.org/handle/10986/34659 Global Partnership for Education. 2020b. “Malawi: COVID-19 Response.” https://www.globalpartnership. org/where-we-work/malawi Gondwe, G. 2020. “Learning Through the Radio Amid COVID-19.” Web article. https://www.unicef.org/ malawi/stories/learning-through-radio-amid-covid-19 Instituto Nacional de Evaluación Educativa (INEEd). 2021. “Aristas 2020. Primer Informe de Resultados de Tercero y Sexto de Educación Primaria.” https://www.ineed.edu.uy/images/Aristas/Publicaciones/ Aristas2020/Aristas-2020-Primer-informe-de-resultados-de-tercero-y-sexto-de-educacion- primaria.pdf International Telecommunications Union (ITU). 2021. World Telecommunication/ICT Indicators Database (24th Edition/December 2020). Retrieved from https://www.itu.int/en/ITU-D/Statistics/Pages/ publications/wtid.aspx Instituto Peninsula Brasil. 2020. “Sentimento e Percepçao dos Professores Brasileiros nos Diferentes Estagios de Coronoavirus no Brasil.” Retrieved from https://institutopeninsula.org.br/wp-content/ uploads/2020/05/Covid19_InstitutoPeninsula_Fase2_até1405-1.pdf Karp, P. 2021, August 24. “’Reassuring’: Pandemic has not hurt children’s literacy and numeracy levels, Naplan data suggests.” The Guardian. https://www.theguardian.com/australia-news/2021/aug/25/ reassuring-pandemic-has-not-hurt-childrens-literacy-and-numeracy-levels-naplan-data-suggests Maldonado, J. and K. De Witte. 2020. “The Effect of School Closures on Standardized Student Test Outcomes.” Discussion paper series DPS20.17. https://bera-journals.onlinelibrary.wiley.com/doi/ abs/10.1002/berj.3754 McClain-Nhlapo, C. V., R. Kulbir Singh, A. H. Martin, H. K. Alasuutari, N. Baboo, S.J. Cameron, . . . M. Tucker. 2020. “Pivoting to Inclusion: Leveraging Lessons from the COVID-19 Crisis for Learners with Disabilities” (English). Retrieved from http://documents.worldbank.org/curated/en/777641595915675088/ Pivoting-to-Inclusion-Leveraging-Lessons-from-the-COVID-19-Crisis-for-Learners-with-Disabilities Organization for Economic Co-operation and Development (OECD). 2020. “Strengthening online learning when schools are closed: The role of families and teachers in supporting students during the COVID-19 crisis.” Retrieved from https://www.oecd.org/coronavirus/policy-responses/strengthening-online- learning-when-schools-are-closed-the-role-of-families-and-teachers-in-supporting-students- during-the-covid-19-crisis-c4ecba6c/ Plan Ceibal. 2021a. “What is Plan Ceibal?” Web site. Retrieved from About us: https://www.ceibal.edu.uy/ en/institucional 44 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Plan Ceibal. 2021b. “Qué es GURI?” Web site. Retrieved from https://www.ceibal.edu.uy/es/articulo/que- es-guri Radhakrishnan, K, S. Sabarwala; U. Sharma, C. Cullen, C. Crossley, T. Letsomo, and N Angrist. 2021. Remote Learning : Evidence from Nepal during COVID-19. Washington, DC: World Bank. https:// openknowledge.worldbank.org/handle/10986/36031 Rieble-Aubourg, S. and A. Viteri. 2020. “COVID-19: Are we prepared for online learning?” (CIMA Brief, No. 20). Washington, D.C. Retrieved from https://publications.iadb.org/publications/english/document/ CIMA-Brief-20-COVID-19-Are-We-Prepared-for-Online-Learning.pdf Ripani, F. 2020. “Uruguay: Ceibal en Casa (Ceibal at home).” Education Continuity Stories Series. Retrieved from https://oecdedutoday.com/wp-content/uploads/2020/07/Uruguay-Ceibal-en-casa.pdf Schult, J., N. Mahler, B. Fauth, and M.A. Lindner. 2021. Did Students Learn Less During the COVID-19 Pandemic? Reading and Mathematics. Competencies Before and After the First Pandemic Wave. Baden-Württemberg: Institute for Educational Analysis. https://psyarxiv.com/pqtgf/ Secretaria de Educação Básica, and Ministério da Educação. 2021. Questionario dos Diretores Escolares. Retrieved from Painel Educação Básica e o COVID-19: https://painelcovid-seb.mec.gov.br/ questionario/ Tomasik, M., L. Helbling, and U. Moser. 2020. “Educational Gains of In‐Person vs. Distance Learning in Primary and Secondary Schools: A Natural Experiment During the COVID‐19 Pandemic School Closures In Switzerland.” International Journal of Psychology. https://doi.org/10.1002/ijop.12728 United Nations Educational, Scientific and Cultural Organization (UNESCO). 2020b. UNESCO COVID-19 Education Response: How Many Students Are At Risk of Not Returning to School? Advocacy paper. Retrieved from https://unesdoc.unesco.org/ark:/48223/pf0000373992 United Nations Educational, Scientific and Cultural Organization (UNESCO), UNICEF, and World Bank. 2020. What Have We Learnt?: Overview of Findings from a Survey of Ministries of Education on National Responses to COVID-19. Paris, New York, Washington D.C.: UNESCO, UNICEF, World Bank. UNICEF ROSA, UNICEF EAPRO, United Nations Educational, Scientific and Cultural Organization (UNESCO) Bangkok, Cambridge Education. 2021. Republic of Korea Case Study. Situation Analysis of the Effect of and Response to COVID-19 in Asia. https://www.unicef.org/rosa/reports/situation-analysis- effects-and-responses-covid-19-education-sector-asia Uwezo. 2020. Are Our Children Learning? The Status of Remote-learning among School-going Children in Kenya during the Covid-19 Crisis. Nairobi, Kenya: Usawa Agenda. https://palnetwork.org/wp- content/uploads/2020/05/Usawa-Agenda-2020-Report.pdf. Whizz Education. 2020. “Whizz findings on learning loss.” Whizz Education. https://gola.education/wp- content/uploads/2020/10/Dr-Junaid-Mubeen-Director-of-Education-Whizz-Education.pdf Wilichowski, T. and C. Cobo. 2020. “From Coping to Improving and Accelerating: Supporting Teachers in the Pandemic and Beyond.” World Bank blog. https://blogs.worldbank.org/education/coping- improving-and-accelerating-supporting-teachers-pandemic-and-beyond World Bank. 2020a. “Mobile distance and Hybrid Education Solutions. A Knowledge Pack.” Retrieved from https://pubdocs.worldbank.org/en/685691598013656403/WorldBank-EdTech-Team-Knowledge- Pack-MobileDistance-HybridEducationSolutions-version2.pdf World Bank. 2020d. “COVID-19 High-Frequency Monitoring Dashboard.” Retrieved from https://www. worldbank.org/en/data/interactive/2020/11/11/covid-19-high-frequency-monitoring-dashboard 45 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW 5. FIVE PRINCIPLES FOR REIMAGINING LEARNING sharon-mccutcheon 46 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW 5. FIVE PRINCIPLES FOR REIMAGINING LEARNING Despite the challenges noted in this report, in conceptual framework of this report, which calls for the course of more than 18 months a variety of an alignment between effective teachers, suitable experiences have been documented, capacities technology, and engaged learners to harness the created, and new forms of education delivery untapped potential of remote learning. Given the adopted at a scale. This presents a number of ongoing nature of the pandemic, these principles possibilities for reimagining how education can be are intended as foundations on which to build offered and enriched in the years to come. This is reimagined learning. especially pertinent as governments around the world continue to implement school closures and Table 5.1 offers a forward-looking assessment are offering multiple remote learning strategies. of four hybrid learning strategies—combining In some countries, schools remain fully or partially in-person and remote learning—that can be closed. In others, schools are open in some regions, implemented during the process of school reopening for some grades, some of the time. Remote learning to increase resilience. These build on the conceptual remains a key means of ensuring continuity of framework in Figure 2.1 and the variety of remote learning as school systems return to increased in- learning strategies highlighted in Figure 3.3. Table person learning. 5.1 illustrates a range of combinations of in-person and remote learning and sketches the advantages The lessons of this pandemic provide countries and limitations of each. As this report underscores with an opportunity to refocus attention on an the effectiveness of hybrid learning does not stem urgent and as-yet incompletely answered policy from the mere availability of technology; rather it conundrum: What needs to be done to make is critically dependent on the quality of teacher- remote learning work for all? This section offers student interaction, how prepared teachers are to five principles for hybrid learning strategies that teach in these contexts, regular monitoring, and policymakers might use to reimagine learning going support from parents. forward. These principles are aligned with the 47 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Table 5.1 Hybrid Learning Strategies for Today and Tomorrow Strategy Description Advantages Limitations • Low barrier of adoption as students are • Both teacher-student and peer interaction are In-person learning in classrooms already familiar with paper, radio, and/or limited to in-person learning. TV. In-person learning following safety protocols combined with one-way interactions such as • Requires ministries of education to have partners • Low media literacy or digital skills + paper take-home packages, radio with adequate infrastructure and expertise to required. learning sessions and/or television deliver remote learning – especially in the realm of education programs that students • High accessibility through paper-based radio and/or TV media outlets. can use as a complement to what material or partnerships with national and Paper, radio and/ they learned in class. • Remote learning is limited to an asynchronous or TV lessons local community radios as well as with TV modality when students use radio and/or TV. broadcasters. • Low barrier of adoption. • Peer interaction is only limited to in-class sessions. In-person learning in classrooms • Low media literacy required. • Requires ministries of education to have partners In-person learning following safety protocols combined with adequate infrastructure and expertise to with one-way interactions such as • Basic digital skills required for interaction deliver remote learning, especially in the realm of + take-home packages, radio, and/ with teachers through mobiles. radio and/or TV media outlets. or television education programs. Additionally, to ensure two-way • High accessibility though partnerships • Remote learning is limited to an asynchronous interactions, teachers follow-up with national and local broadcasters. modality when students use radio and/or TV. Paper, radio and/ either through phone calls, SMS, or or TV lessons with WhatsApp messages. • Meaningful two-way teacher-student • Limited channels and options for remote teacher- mobile follow-up interactions. student interaction. In-person learning in classrooms • Basic digital skills required to use online • Peer interaction is only limited to in-class sessions. following safety protocols combined platforms to watch pre-recorded lessons In-person learning with one-way interactions such as • In low-income settings, families might not have and interact with teachers. pre-recorded lessons that can either access to devices required for digital learning. + be accessed by using YouTube or a • Online asynchronous approach allows government website. Also, teachers students to access learning sessions at • Remote learning is limited mostly to asynchronous provide feedback to students any moment (flexible use of time). modality when students use radio and/or TV. Pre-recorded to ensure meaningful two-way • Meaningful two-way teacher-student • Teachers need a set of digital and pedagogical skills online lessons with interactions either through an online to teach effectively in remote learning settings. mobile follow-up platform or via mobile messaging. interactions. In-person learning in classrooms • Meaningful two-way interactions are ensured both in-class and remotely with • Available only for contexts with high access to following safety protocols combined digital devices and Internet connectivity. with two-way digital learning teachers and peers. In-person learning solutions such as interactive and/ • Learning is mostly synchronous and allows • Costly to set-up and incorporate adaptive software or adaptive software for students for student learning. + for immediate feedback. to practice what was learnt during class and strengthen specific skills. • Adaptive software ensures personalized • Relatively advanced digital skills required from both Adaptive software Live online sessions can be used as learning for a heterogeneous group of students and teachers. and/or live online a complement to provide teacher- students and supports them to focus • Teachers need a set of digital and pedagogical skills support sessions student feedback and reinforce on areas of weakness with efficient meaningful two-way interactions. to teach effectively in remote learning settings. remediation. 48 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Adopt Suitable Technology 1. Ensure remote learning addressing this question, it is imperative that technology is fit-for-purpose decisionmakers consider the readiness of the teacher and the support available to them. Remote learning during COVID-19 has underscored For example, given the limited remote learning the fact that it is no longer sufficient to simply time, countries may choose to focus on core consider how many teachers and students have subjects, such as literacy and numeracy and, access to the Internet. Today’s policymakers must within the core subjects, focus on fundamentals aim for enabling “meaningful connectivity”, which that are prerequisites to developing strong means that students, teachers, and parents can critical thinking skills (World Bank 2021c). Now, use the Internet every day via an appropriate device more than ever, ensuring that all students learn with enough data and connection speed to enable must be prioritized over the need to complete learning. To reduce the risk of a “remote learning all material in the curriculum. paradox”, countries need to determine what is the minimum data consumption needed to enable As Sierra Leone has a low penetration of internet remote learning, with a special focus on getting and TV, policymakers prioritized technology that more women and girls online.1 As countries do so was fit-for-purpose by delivering remote learning and strive to strike a balance between in-person through printed material and a radio learning and remote-learning, some guiding questions program. The Ministry of Education partnered with policymakers must consider include: 12 community radio stations that retransmit the radio learning program to reach students in remote a What critical factors must be accounted for 1. communities. As this radio program was already in when deciding which technology is appropriate place during the Ebola crisis, existing content was to deliver remote learning? These include but curated and launched on April 6, 2020, less than are not limited to access and availability of the one week after schools were closed. New learning technology, the appropriate contents, teachers’ sessions were created with support from teachers technical competence, and pedagogical of top public and private schools, who were selected relevance. For example, Estonia has set up a based on assessment results. To allow for two- hotline that teachers can access to receive way interaction, a ‘live’ phone line is open to allow targeted support. In both Cambodia and India, children to call in with their questions and all calls to rural teachers have been provided with video the radio learning program are toll-free. lessons that provide a model for them to follow (World Bank 2021c). → From principle to action In order to ensure that learning technology is fit-for- b What strategies can be adopted to evaluate 2. purpose, better data about EdTech is needed. The quickly and efficiently whether an educational World Bank has developed an EdTech Readiness technology will be useful for the majority of Index (ETRI) to inform related policies. The ETRI goes students? An emerging body of evidence beyond measuring the availability of devices and the summarizes the state of cost-effective level of connectivity to capture key elements of the investments in education. This can provide a larger education-technology ecosystem in a country, useful start, especially in settings where context- guiding efforts to increase learning opportunities specific data are scarce (World Bank, FCDO, and and reduce inequalities. Building Evidence in Education 2020). c How can digital technologies be used not 3. only for passive content consumption but also to help students learn to think critically? In 49 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Prioritize Effective Teachers 2. Use technology to enhance for remote teaching that teachers can access for free through its CREA learning management system. teacher effectiveness Regardless of the learning modality and available → From principle to action technology, teachers will play a critical role. Teachers “Technology for Teaching” is the World Bank’s can motivate students to find positive value in the program to enhance and scale-up teacher learning process, provide incentives to perform, professional development (TPD) opportunities using give targeted feedback in the areas where students tech-based solutions. This initiative aims to support experience difficulties, and provide socio-emotional governments’ COVID-19 recovery and resilience support. This is especially true during the pandemic, efforts by: i) Utilizing existing research and anecdotes as many students have been exposed to an array of for development of a tool to help policymakers and challenging situations. The pandemic has reinforced practitioners assess and compare contextual and the need to better support teachers. Although this technical factors for successful implementation challenge is certainly not new, the importance of of TPD programs using technology; ii) Identifying teachers in high- and low-tech environments is more scalable and replicable TPD interventions that utilize evident than ever (and parents and families globally low and high-tech solutions to train teachers and are coming to realize this truth). The skills needed pedagogical leaders; iii) Developing global public to effectively teach remotely are not yet taught to all goods including technical and operational guidance teachers, everywhere. Doing so requires structural notes with lessons learned on how to design, changes to pre- and in-service teacher professional implement, and evaluate TPD approaches using development and would include pedagogical skills tech-based solutions; and iv) Providing technical for remote teaching alongside in-class pedagogical assistance to country teams involving hands-on skills. It would also build digital skills to allow support to countries implementing TPD. teachers to be native users of the remote learning technologies available to them. At the same time, it is critical to implement technologies for teaching, for professional development, and for tutoring and coaching as regular practices during and after the pandemic. Teachers could be supported through structured teacher guides and additional guidance to facilitate effective remote learning (World Bank 2021c). As has been done in Uzbekistan, teachers could be provided with free SIM cards to ensure that they can access educational material online and through media such as WhatsApp (World Bank 2021c). This crisis should be taken as an opportunity to reimagine with teachers their role in education for the years to come. Uruguay, for example, leveraged digital technologies to enhance the effectiveness of teachers. The Institute for in-Service Teacher Training has taken online a coaching program that was providing pedagogical support to teachers prior to COVID-19. The government has also enhanced a comprehensive toolkit of teaching resources such as discussion forums, virtual training, and guidelines 50 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Prioritize Effective Teachers 3. Establish meaningful two-way different approaches that employ digital technologies interactions to support learning at home or to assist educators in the classroom. For remote learning to be successful it needs to allow for meaningful two-way interaction between students and their teachers (Barron et al 2021). Such interactions can be created by using the most appropriate technology for the local context. Indeed, in many settings this does not necessarily mean that governments need to seek out the most advanced technology. Especially where connectivity is limited, the combination of low-tech modalities with devices that allow students to interact with each other and their teachers will likely be a more effective way of ensuring that remote learning is able to deliver on its promise. This is especially critical in remote and rural environments where two-way teaching and learning interaction is a privilege that not all can afford. Where feasible, technologies that allow students to interact with teachers via internet-enabled devices, if appropriate, should indeed be used. In Brazil, state education secretaries have combined low-tech and high-tech modalities to establish meaningful two-way interactions; for example, a mobile application has been developed to encourage teacher-student interaction for a limited amount of time after each TV learning session; telecommunication operators zero-rate this mobile application so that students and teachers access from any device, without paying for the bandwidth. This application is combined with printed take-home material. → From principle to action The World Bank’s Education Global Practice has: created short notes to curate evidence and examples and provide links to materials that countries have used to support various stakeholders (students, teachers, parents, school management) during both remote and in-person instruction; developed Excel-based costing tools to identify the resources required for delivering remote instruction, including social-emotional learning, and for implementing the activities implied by the joint framework; testing 51 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Ensure Learners are Engaged 4. Engage parents and students Lifelong Health has also created open source tip as partners in the teaching and sheets for parents in more than 100 languages. These resources cover a range of topics including learning process structuring a child’s day, stress management, and family budgeting. High levels of isolation, disconnection, and frustration at the start of the pandemic meant that In Peru, a remote learning program is constantly teachers needed to not only deliver their lessons communicated to ensure parents have the and provide cognitive support to their students, information needed during the distance learning but they were also the main form of social contact period. For example, guidelines for remote learning for many students. These “side effects” of remote and weekly schedules for a specific week of 2021 or hybrid learning will need to be addressed by can be found in the national remote learning the education community. For example, some platform. Moreover, the Ministry of Education has countries, such as Chile, have moved to emphasize been regularly supervising the adoption, perceived socio-emotional competencies in their curriculum effectiveness, and satisfaction of Peru’s remote to facilitate socio-emotional learning and well- learning program through phone calls with parents being of students (World Bank 2021c). Students once a month. According to the Ministry’s Monitoring also need help with overcoming the learning loss and Evaluation (M&E) Unit survey, 85 percent of they are experiencing. An individualized, paper- parents had received support from teachers at and-pencil self-learning program in Bangladesh least once in the past week, showing the Peru has was found to significantly improve students’ made efforts to engage parents as partners in the mathematical abilities. The program was designed teaching and learning process through constant to ensure that each student works at the level that teacher-parent communication. is appropriate for their individual skills, advancing and learning new concepts in small steps through easily understandable hints and examples. In terms → From principle to action of technology, examples from India and Uruguay The World Bank’s Read@Home has developed show that that computer-assisted instruction can global public goods to help countries select increase learning, with suggestive evidence of storybooks, engage parents to read with children positive impacts that were larger for students from at home, and use innovative approaches to deliver disadvantaged backgrounds (World Bank, FCDO, and procure books. The Read@Home initiative and Building Evidence in Education 2020). aims to deliver reading and learning materials to hard-to-reach homes, as quickly and efficiently as However, teachers and students are not in this possible, along with support for parents and other alone. Parents are willing and eager allies that can be caregivers to engage with children’s learning. enlisted to help support students as they continue Read@Home is working to support countries with: to engage with the teaching and learning process. (i) just-in-time technical assistance to complement As countries prepare for transitioning to a more country-efforts and operations to source and select consistently remote learning model, it is necessary quality reading and learning materials for children to prioritize design strategies that can secure the and accompanying materials for parents/caregivers social and emotional well-being of students and to support children’s learning; (ii) just-in-time and teachers. A host of resources exist to support this. longer-term technical assistance to complement Sesame Workshop, through Sesame Street, has country-efforts and operations to improve efficiency created materials (“Caring for Each Other”) for and reduce costs to procure and distribute learning parents of young children to engage their children materials; and (iii) funding to close gaps at a country in play-based learning activities and discuss level and incentivize resource allocations (on a emotions. Materials are currently available in Arabic, matching basis).  Bangla, English, Hindi, and Spanish. Parenting for 52 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Ensure Learners are Engaged 5. Rally all actors to cooperate between what the evidence suggests is effective for learning in promoting learning and what is happening in practice, and allowing governments to track This pandemic has made it clear that governments progress as they take action to close those gaps. in general and ministries of education in particular cannot operate in isolation. Given the interdisciplinary nature of the digital  education Despite the challenges that this report notes, endeavors it is critical that ministries instead of in the course of just 18 months, a variety leading each one of the critical services needed of experiences have been documented, (e.g., financing, deploying connectivity, acquisition capacities created, and new forms of of devices and learning materials, training teachers, monitoring, etc.), they work in close coordination education delivery adopted at a scale. with other entities (public, private, academic) This presents a number of possibilities for to effectively orchestrate different players and reimagining how education can be offered to secure the quality of the overall learning and enriched in the years to come. Remote experience. In some countries, the preferred learning remains a key means of ensuring institutional model for managing educational policy is defined as high levels of centralization, while in continuity of learning as school systems others the preference is for a more decentralized return to increased in-person learning while approach. Ministries of education need to become at the same time prepare for unplanned “learning organizations”, in order to distill what closures as the pandemic lingers. When has worked (and from  what hasn’t) during the its three complementary components– pandemic. This requires not only higher levels of teachers, students and technology–are well- adaptability but more effective monitoring and real-time assessment. In order to become “learning aligned, remote learning has the potential organizations”, ministries of education will need to address the inequality in learning that to build a variety of capacities including but not has held back global progress on learning limited to drawing on the consolidated strengths poverty for far too long. of their remote- and blended-learning technical teams, increasing the quality and relevance of their EdTech policies, and implementing a robust impact evaluation strategy. This will require convening key players (product and service providers) and supporting an inclusive ecosystem of stakeholders to both mobilize financial and human resources and to solve logistical problems. The challenges facing education systems during this global disruption can be considered an opportunity to better collaborate and liaise with local and international partners. → From principle to action The World Bank’s Education Global Practice supports countries in the deployment of the Global Education Policy Dashboard. This tool helps to identify priorities for investment and policy reforms that are suited to each country by highlighting gaps 53 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Note 1. See more information see the World Bank brief on minimum data consumption, Alliance for Affordable Internet, and Worldwide Web Foundation. References Barron, M., C. Cobo, A. Munoz-Najar, and I. Sanchez Ciarrusta. 2021. “What is Hybrid Learning? How can countries get it right?” World Bank blog. https://blogs.worldbank.org/education/what-hybrid- learning-how-can-countries-get-it-right World Bank. 2021c. “Policy Actions for School Reopening and Learning Recovery.” Retrieved from https:// www.worldbank.org/en/news/factsheet/2021/04/30/notes-on-school-reopening-and-learning- recovery World Bank, Foreign, Commonwealth and Development Office (FCDO), and Building Evidence in Education. 2020. “Cost-effective Approaches to Improve Global Learning: What Does Recent Evidence Tell Us Are “Smart Buys” for Improving Learning in Low-and Middle-income Countries? Retrieved from http://documents1.worldbank.org/curated/en/719211603835247448/pdf/Cost-Effective- Approaches-to-Improve-Global-Learning-What-Does-Recent-Evidence-Tell-Us-Are-Smart-Buys-for- Improving-Learning-in-Low-and-Middle-Income-Countries.pdf 54 16 Number of counbtries Number of countries Number Number ofof countries counbtries Number of counbtries Number Number of of counbtries countries -F Number of countries 0 10 20 30 40 50 0 1 2 3 4 5 6 7 8 9 0 5 10 15 20 25 0 5 10 15 20 25 30 35 40 6- eb- 25 Ma 202 16-Feb-2020 -M r-2 0 13 ar 02 6-Mar-2020 -A -2 0 2 pr 02 25-Mar-2020 21 -Ma -20 0 -M y- 20 Middle East and 13-Apr-2020a 20 ANNEX 2-May-2020 9- yo- 20 J 28 un 202 21-May-2020 -J -2 0 FullyNorth 17 un 02 9-Jun-2020 closed - -2 0 5- Jul 02 28-Jun-2020Au -2 0 24 g 0 - - 2 17-Jul-2020 Africa Fully closed 12 Feb 2020 Fully closed Fully closed 5-Aug-2020 -S -2 0 e 0 24-Aug-2020 1- p- 20 2 20 Oct 02 12-Sep-2020 -O -2 0 8- ct- 020 1-Oct-2020 27 No 202 20-Oct-2020 - v- 0 16 Nov 202 8-Nov-2020 Partially closed Partially closed -D -2 0 Partially closed Partially closed 27-Nov-2020 4- ec- 020 23 Jan 202 16-Dec-2020 - - 0 11 Jan 202 South Asia 4-Jan-2021 -F -2 1 23-Jan-2021 2- eb- 021 Fully open Sub-Saharan Africa Fully open Fully open 21 Ma 202 11-Feb-2021 -M r-2 1 0 2-Mar-2021 9- ar- 21 Fully open 2 55 28 Apr 02 21-Mar-2021 - 1 17 -Ap 20 Latin America and the Caribbean 9-Apr-2021 -M r-2 21 28-Apr-2021 5- ay- 021 24 Jun 202 17-May-2021 Academic break -J -2 1 Academic break Academic break 5-Jun-2021 13 un 02 -J -2 1 24-Jun-2021 1- ul- 021 A 20 ug 202 13-Jul-2021 -A -2 0 1-Aug-2021 8- ug- 021 Academic break 27 Sep 202 20-Aug-2021 -S -2 1 16 ep 02 8-Sep-2021 -O -2 1 27-Sep-2021 ct 02 -2 1 02 16-Oct-2021 1 Asia South Africa Africa and the and North Caribbean Fully open Middle East Fully closed Sub-Saharan Latin America Partially closed Academic break Figure A.1 Time-series of School Closure Status from February 2020 to October 2021, by Region REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW 16 - 1F Number Number of of countries 6e-F countries Number of countries Numberof countries Number of Number countries ofcountries b e 2-2 0 0 10 20 30 40 50 60 0 1 2 3 10 20 30 27 6-M -b 02 -2M 5- ar- 0 20 a 13 r- M 20 0 ar 2 by Region 6- -Ap2-0 2 0 M2- r- 0 M 220 a 21 ya 20 0 - 15 Ma 2 -y -2 20 -J9 yo 00 220 28 J u- -2 0 nu-n 02 25 -Ju 2-0 20 0 -1J7- n-222 00 5-uJ ul 02 - 3-24 g 00Alu 2-2 0 S - e -20 2200 12eF Fully closed Fully closed p-b- 20 Fully closed -S 20 20 13 ep -O 1- - 220 2 0 20cOct 02 22 8 Oc20- t- - 20 0 -N -N t-2 220 27o o 02 0 - v-v-2 0 16 No 2 0 20 1- -D v-0 Partially closed Partially closed Ja e 22 0 Partially closed 4- n c- 02 0 10 23 Ja-n 22 0 02 - -Ja 20 - 0 1F 1e 21 -Fb-n-2021 22 2 eb 20 2 1 0 Fully open Fully open -2M-Ma -2 21 Fully open North America 1-a r- 2 1 East Asia and Pacific M r- 0 2 Europe and Central Asia r-0 21 56 1- 9-Aa2 22 M 2 8a pr 0 1 - A y -2 2 1 -r2 02 10 17-M p -02 1 -J5 ay 0 21 u-Jn -2 2 1 2 u -2- 02 1 Note: The analysis covers schools from pre-primary to upper secondary level. Academic break 02 Academic break 20 4-J n Academic break un 0 22 -1J3u 1 -J -20 1 1 29 -A 2 l u - l-2 21 -2 A 0- ug- 0 02 20 uAg u 20 1 8- -g 2-2 21 8-27 Sep 0 0 O-S -2 22 11 16cte p 21 0 -O-2-0 20 ct 22 -2 1 1 0 21 Source: World Bank. Calculations based on UNESCO global monitoring of school closures caused by COVID-19. East Asia and Pacific Europe and Central Asia Fully open Fully closed North America Partially closed Academic break Figure A.1 (Continued) Time-series of School Closure Status from February 2020 to October 2021, REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Table A.1 Summary of Empirical Evidence on Learning Losses Closure Learning Source Country Education Level Subject Sample Size Length Loss Primary (Grade K–5) Math 33 percent Not specified Dorn et al., Not USA 2020 specified Primary (Grade K–5) Reading 13 percent Not specified Primary (Age 8) Math 0.063 SD 92 180 students Primary (Age 8) Reading 0.057 SD 76 397 students Primary (Age 8) Spelling 0.094 SD 90 403 students Primary (Age 9) Math 0.073 SD 93 417 students Primary (Age 9) Reading 0.098 SD 79 016 students Primary (Age 9) Spelling 0.071 SD 91 567 students Engzell et Netherlands 8 weeks Primary (Age 10) Math 0.094 SD 93 769 students al.,2020 Primary (Age 10) Reading 0.084 SD 68 412 students Primary (Age 10) Spelling 0.076 SD 91 315 students Primary (Age 11) Math 0.050 SD 73 263 students Primary (Age 11) Reading 0.074 SD 48 537 students Primary (Age 11) Spelling 0.076 SD 69 841 students Primary (Grade 4-7) Math, Spelling, Reading 0.08 SD 350 000 students Primary, Grade 6 Math 0.19 SD 856 students Primary, Grade 6 Dutch 0.29 SD 982 students Maldonado & De Witte, Belgium 9 weeks Primary, Grade 6 Social Science Insignificant 755 students 2020 Primary, Grade 6 Science 0.33 SD 588 students Primary, Grade 6 French 0.30 SD 880 students Primary (Grade 5) Math (operations) 0.09 SD > 80 000 students Schult et 8.5 Germany Primary (Grade 5) Math (numbers) 0.03 SD > 80 000 students al., 2021 weeks Primary (Grade 5) Reading 0.07 SD > 80 000 students Primary (Grade 3-6) Math, German 2X 13 134 students Tomasik et Switzerland 8 weeks al., 2020 Secondary Math, German Insignificant 15 551 students (Grade 7-9) 57 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Figure A.2 Learning Poverty and Households’ Internet Access, for Countries Providing Remote Learning Strategies Through Online Platforms 100 90 80 Learning Poverty (%) 70 60 50 30 40 20 10 0 0 10 20 30 40 50 60 70 80 90 100 Households with Internet Access at home (%) Countries offering Remote Learning through Online Platforms Linear Fit Source: World Bank. Calculations based on UNESCO, UNICEF, World Bank, and OECD (2021a) as of June 2021; Learning Poverty database from Azevedo et al. (2019) as of August 2021; and information on internet access is from Technologies database extracted from International Telecommunication Union (2021). Note: Sample size of 24 countries. Governments responded to the survey question “S4 Q1. Which distance learning solutions were or are being offered in your country during the pandemic in 2020 and /or 2021?”. It considers at least one of the education levels (pre-primary, primary, lower secondary, and upper secondary). The y-axis reports Learning Poverty, understood as the percentage of children of 10 years old that are unable to read and understand a short age-appropriate text. This global indicator combines the share of learners who haven’t achieved minimum reading proficiency and the share of out-of-school children, who are assumed as not being able to read proficiently (World Bank 2019a). The x-axis reports the estimated proportion of households with access to internet at home. The horizontal red line represents the learning poverty country median (percentage) for all countries with available data of learning poverty (Azevedo et al. 2019): countries falling above the median are classified as having relatively higher learning poverty; countries falling below the median are classified as having relatively lower learning poverty. The vertical red line represents the median for percentage of households with internet access for all countries with available data. 58 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Figure A.3 Children Engaging in Any Remote Learning Activity Since School Closures, by Learning-Adjusted Years of Schooling (LAYS) 13 12 11 10 9 LAYS ( years) 8 7 6 5 4 3 2 1 0 0 10 20 30 40 50 60 70 80 90 100 Households with children engaging in remote learning (%) Source: World Bank. Calculations based on High-Frequency Phone Survey (World Bank, 2020d) as of March 16, 2021, and LAYS database from WBopendata as of June 2021. Note: Sample size of 27 countries. 59 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Figure A.4 Children Engaging in Any Remote Learning Activity Since School Closures Began, by Remote Learning Strategies (Over Time) 60 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW Figure A.4 (Continued) Children Engaging in Any Remote Learning Activity Since School Closures Began, by Remote Learning Strategies (Over Time) Source: World Bank. Calculations based on the High-Frequency Phone Survey (World Bank, 2020d) as of March 16, 2021. Note: Survey responded at the household level. Responses reference 7 days when children were engaging in any learning activity. Responses from Wave 1 for Latin America and the Caribbean were removed to avoid a different reference window. 61 REMOTE LEARNING DURING COVID-19: LESSONS FROM TODAY, PRINCIPLES FOR TOMORROW References Azevedo, J. P., D. Goldenberg, S. Montoya, R. Nayar, H. Rogers, J. Saavedra., and B. W. Stacy. 2021. Will Every Child Be Able to Read by 2030? Defining Learning Poverty and Mapping the Dimensions of the Challenge. Policy Research Working Paper 9588. Washington, DC: World Bank. Retrieved from http://gaml.uis.unesco.org/wp-content/uploads/sites/2/2021/03/Azevedo-et-al-2021_Will-Every- Child-Be-Able-to-Read-by-2030.pdf Donnelly, R. and H. A. Patrinos. 2021. Learning Loss During COVID-19: An Early Systematic Review. COVID Economics. Vetted and Real Time Papers (77). https://assets.researchsquare.com/files/rs-518655/ v1/53d0e59a-9bfa-4a49-bce3-df3f46f5dacb.pdf?c=1631882790 International Telecommunications Union (ITU). 2021. “World Telecommunication/ICT Indicators Database (24th Edition/December 2020).” Retrieved from https://www.itu.int/en/ITU-D/Statistics/Pages/ publications/wtid.aspx United Nations Educational, Scientific and Cultural Organization (UNESCO), UNICEF, World Bank and Organization for Economic Co-operation and Development (OECD). 2021a. Survey on National Education Responses to COVID-19 School Closures, Round 3. Paris: UNESCO; New York: UNICEF; Washington DC: World Bank; Paris: OECD World Bank. 2020d. “COVID-19 High-Frequency Monitoring Dashboard.” Retrieved from https://www. worldbank.org/en/data/interactive/2020/11/11/covid-19-high-frequency-monitoring-dashboard 62 Supported with funding from