https://www.worldbank.org/en/topic/edutech #covidedtech Remote Learning and COVID-19 The use of educational technologies at scale across an education system as a result of massive school closings in response to the COVID-19 pandemic to enable distance education and online learning revised draft: 16 March 2020 Little research attention has been paid to documenting and analyzing attempts of education systems moving quickly and at scale to provide online learning when all or many schools are closed. Related 'good practices' are considered rare, and on the whole, activities and initiatives of these sorts are poorly documented, especially when it comes to the needs of learners and education systems across the so- called 'developing world' That said, it is possible to extrapolate from the existing knowledgebase about the use of educational technologies in general over past decades, as well as from consensus expert and practitioner wisdom and experience, to offer high-level guidance and 'rules of thumb' for policymakers forced to make related decisions in fast moving, very challenging circumstances with little guidance or relevant experience. This rapid response briefing note was prepared by the EdTech team of the World Bank’s Education Global Practice. Contents General comments and guidance …2 Students …3 Remote learning content and applications …4 Subjects to teach and activities at a distance …6 Teachers and teaching …7 Parents and families …8 Policymakers …8 Notable examples to learn from …9 Vendors … 10 Annex 1: Simplified high-level matrix of options … 11 Caveats and a way forward … 12 Rapid response briefing note: Remote Learning & COVID-19 Outbreak (revised draft: 16 March 2020) p. 1 https://www.worldbank.org/en/topic/edutech #covidedtech Remote Learning and the COVID-19 Outbreak Distance education and online learning in response to pandemic disease What we know, what we believe -- what we don't Important caveat: What follows is not meant to be an exhaustive catalog of everything that is known (or has been debated) about this topic. Rather, it is a quick attempt to summarize and give shape to emerging knowledge, expert opinion, and practical know-how of potential relevance to decision makers making related decisions in real time. General comments and guidance Transitioning to online learning at scale is a very difficult and highly complex undertaking for education systems, even in the best of circumstances Few (if any) education systems, even the most high performing, are well equipped to offer online learning for all students at scale, quickly. 'Failure' is common, and success is often a result of experience and learning from past failures. Technological advances often outpace the ability of decision makers to keep up. Costs, especially capital costs, are usually quite high. Providing sufficient infrastructure is often seen a primary hurdle to be overcome. While infrastructure is certainly important, and expensive, much greater challenges relate to supporting teachers so that they can in turn support learners in a new learning environment; offering high quality, curriculum-relevant digital learning content and assessment tools; promoting the development of a variety of digital skills to enable students to be able to use technology effectively in support of their learning; implementing supportive data and information management systems; monitoring and evaluating what is happening, and its impact; and enacting enabling policies. Moving to online learning at scale raises profound equity concerns (!!!) While many large scale investments in educational technologies are accompanied by rhetoric touting the potential for technology use to close various education gaps related to access and performance between different student groups, in practice the move to online learning at scale typically disproportionately benefits students already advantaged in various ways (e.g. rich over poor, urban over rural, high-performing over low-performing, student in highly educated families over students from less educated families). The particular and individual needs of students with disabilities and other special education needs -- a group that can often greatly benefit from the use of assistive technologies of various sorts -- are often not sufficiently considered when deploying online learning environments at scale. Where online learning is already widespread, 'success' is more likely Where online learning, and tools to support online learning, are already a constituent part of what an education system (or an individual school) is able to provide, the potential for success is greater. Good schools, in good education systems, are most likely to do the best Very good, well prepared schools already using technology reasonably effectively are those most likely to navigate the transition to online learning most effectively and efficiently. Often, these are located in well resourced, reasonably affluent communities. Conversely, students and teachers in schools and Rapid response briefing note: Remote Learning & COVID-19 Outbreak (revised draft: 16 March 2020) p. 2 https://www.worldbank.org/en/topic/edutech #covidedtech education systems that are under-resourced school and/or in poor communities are typically much less able to benefit from online learning opportunities. Students Most online learners will experience difficulties Most children will have great difficulty accessing online learning, the impact of which will probably be of limited value for most of them. This is especially true for children in poor communities, in households where Internet access is poor (or non-existent), who have little prior experience with online learning, and/or are subject to numerous other disadvantages. Highly motivated learners, especially those with previous experience in online learning, are the most likely to take the most advantage of online learning opportunities Where education systems (or schools) are not able to support online learning opportunities at scale, some highly motivated students with access to sufficient bandwidth, connected devices, and ability to learn independently may be able to take advantage of online learning resources offered by companies and non-profit groups. Where education systems are unable to provide such online learning opportunities themselves, there is value in alerting students to the availability of such resources. The 'best' students in off-line learning environments tend to perform better than their peers in online learning environments as well -- differentially so The students best placed to make best of online learning will be those already competent and well versed in using technology tools to support their learning, including and especially online, who have sufficient access to good bandwidth and connected devices, as well as support from their family and peers. Online learning for young learners is more difficult -- older students do better There are few successful examples of sustained online learning at scale for young students. Where such examples exist, online learning sessions are typically of shorter duration, are led by highly competent teachers, feature engaging content, class sizes are very small, access to bandwidth and connected devices is very good, and the process is supported by a very involved caregiver. In such contexts, interactions with learning content on devices is most impactful in short increments, with the support of a caregiver. The use of educational radio or television, and not online learning at scale, may be more relevant for many young learners. Targeting older learners is a more viable option. When first going online, education systems (and parents) should expect dips in student achievement The transition to fully online, virtual learning almost always results in lower educational outcomes in the short term. This occurs for a number of reasons, including a lack of familiarity with the tools and process; a lack of a conducive environment at home to support online learning (including insufficient access to bandwidth and devices); differential impact related to a range of equity issues; and a lack of congruence between what is taught in classrooms and what is taught online. Motivation to continue with online learning can present a real challenge for both students and teachers, especially over time. Rapid response briefing note: Remote Learning & COVID-19 Outbreak (revised draft: 16 March 2020) p. 3 https://www.worldbank.org/en/topic/edutech #covidedtech Remote learning content and applications Providing a consolidated, one-stop-shop for access to online learning opportunities is strongly advised Where an education system is unable to provide access to online learning opportunities itself in the event of school closings, a central online portal can provide a consolidated listing of available content, tools, apps and platforms, together with support materials and guidance for students, teachers and parents. More capable or advanced education systems can go one step further and provide single-sign access to a set of online learning opportunities, tools, platforms and content, in order to simplify the related user experience; this can also help education systems monitor usage and provide insight into what might be going wrong -- or right. Where providing a single consolidated online reference or portal website is beyond the capability of an education system, sub-regional educational authorities, developmental partners or civil society groups, or even individual schools, may wish to provide this instead. Creating an inventory of existing learning content ready to be deployed via remote learning is necessary, as well as a plan on how to make available additional content A quick inventory of content that can be easily made available via remote learning is a first order of business. Such content may come from multiple sources; many gaps should be expected, as well as potential duplication. Freely available, ‘open education resources’ are plentiful in some languages; in languages where digital learning content is limited, translating existing open education resources from other languages may be worth considering, together with partners. Educational publishers may have existing content that is not yet in widespread use within an education system but which might be very relevant and useful; in such cases, it may be useful to explore short-term partnerships with such publishers. Organizing digital educational content to align with existing curricula can be critical in providing users and teachers with a way to ensure that the learning opportunities provided correspond to broader educational objectives within an education system Simply pointing students and teachers to large online repositories, while potentially useful, is typically not enough. Where content is from a wide variety of sources, how can teachers and students know what to access, and in what order? Ministries of education and their partners can play a useful role in helping to organize existing learning content so that learners, their caregivers, and teachers can understand what content is available and the sequence in which it should be used, in line with existing curricula. Making content available on a wide variety of devices -- and mobile friendly -- is critical In school settings, education systems can largely dictate the specific types of edtech devices and technologies used, and can permit or block access to specific applications, platforms, and learning materials. Outside of such controlled environments, anything often goes. As a result, decision makers should take care to ensure that online learning tools and platforms are available on a wide variety of devices, running a variety of operating systems and software applications. Given the near ubiquity of mobile phones in many communities and households, and the low availability of other types of computing devices (especially desktop and laptop computers), ensuring that online learning opportunities can be accessed using mobile devices can be critical in ensuring access by the widest possible user base. Rapid response briefing note: Remote Learning & COVID-19 Outbreak (revised draft: 16 March 2020) p. 4 https://www.worldbank.org/en/topic/edutech #covidedtech Supporting the use of low bandwidth (including offline) solutions is key Mandating that online learning opportunities be optimized for low bandwidth and poor latency conditions is advisable in most all contexts. Promoting the availability of off-line tools and approaches and downloadable can be similarly quite useful, especially where there is time before schools close to convert educational content into digital formats and load them onto devices. Videos can offer valuable learning resources when schools are closed, providing that there is sufficient available bandwidth, the content is engaging, and production values are of sufficient quality Many education systems looking to go online when schools are closed initially consider the use of education videos to substitute for classroom lectures. Education systems planning to simply record a teacher giving a long lecture and make it available online for students often find that only the most highly motivated and engaged students are able to absorb the content, with limited impact. Short online video tutorials can be especially useful (e.g. of the sort offered by Khan Academy), especially in support of self-study and where an education system has the capacity to procure such content, or develop it itself. Best practice holds that, generally speaking, shorter content is more easily accessed and absorbed by students. An education systems may be challenged to host video content itself, and may need to rely on an external provider to provide hosting and related support services. The cost of producing video content can vary widely, and depends on many factors. Educational radio and television are viable options, especially in low resource environments In many developing countries where broadband access to the Internet is not widely available or where online learning is simply not a viable option in the near term, the use of more traditional distance education approaches, such as educational radio and television, is potentially very relevant, especially in countries where such channels are already in use. Even where such channels are not in use, countries have demonstrated that it is possible to debut educational radio and television programs at relatively short notice, although little is known about the impact of such programs on learning outcomes, and student engagement and motivation can be a very real challenge, especially once the novelty effect wears off. In some contexts, the use of podcasts should be considered, as supplement or to substitute for radio content. Providing supplemental guidance and support on how to use and access remote and online learning content can be critical Simply making content available is not enough. Users need to be able to easily understand how to access and use it. Before schools close, providing written guidance to families and teachers on how to access content, what related expectations are, and where to find additional information, can be critical. After schools close to students, it may be possible for teachers to remain in schools for related training and to help prepare for the first weeks of remote instruction. Once schools are closed, teacher peer support groups (e.g. via texting, WhatsApp, WeChat, Facebook) can provide invaluable mechanisms for both informal support and official information sharing, in addition to an official online portal (such as a ministry of education web site). Help desks (via chat, text, phone, message boards, radio or TV call in programs) can also be considered. It may be more difficult to utilize existing learning management systems (LMS) designed to support in class instruction for use in exclusively online environments than it may first appear Education systems that already make widespread use of digital learning management systems in Rapid response briefing note: Remote Learning & COVID-19 Outbreak (revised draft: 16 March 2020) p. 5 https://www.worldbank.org/en/topic/edutech #covidedtech support of teaching and learning as content repositories and in tracking assignments may find that these tools do not easily lend themselves to support online instruction beyond self-study. Using multiple media channels to share information about remote and online learning opportunities can be very useful Even if learning materials are only offered in one way (e.g. printed materials, radio or TV, online learning content, phone-based apps), multiple media channels can be employed to alert learners, families and communities groups to the existence of the materials and where to find additional support or guidance. What to subjects to teach and which activities lend themselves to remote learning Some academic subjects are easier to move online than others Generally speaking, academic subjects that are largely lecture-based and/or lend themselves to self- study are easier to move online quickly. Subjects in which learning content has already been digitized, especially where it is explicitly aligned with official school curricula, offer better candidates for online learning in the short term than those that do not. Many in-school activities and approaches do not translate easily to online environments Much of what happens in schools cannot be easily transferred online. Instructional approaches, content, pacing, interaction models, and assessment may all need to be adapted when transitioning to online learning. While creative uses of new technologies can support student learning in 'non-core' subjects like music, art, and physical education, there are few successful models of doing so at scale for all students. The move to online schooling is typically limited to providing substitute opportunities for academic teaching and learning activities -- other aspects of the schooling experience are left out Schools are about much more than just student instruction and assessment. Many important characteristics of well-functioning schools and the services they provide, such as developing socialization skills (with peers and adults), access to counseling and health services, school feeding programs, the promotion of good citizenship and instruction in ethics, as well as many others, are typically outside the scope of what is offered when schools go 'online'. Test prep, especially when is revolves around large scale memorization and working through problem sets, is particularly well suited in many instances to move to an online environment, but brings with it profound implications for equity Online learning can be usefully deployed to support test preparation in many instances, especially toward the end of the academic year when instruction periods are complete, or almost complete, and where independent self-study using high quality materials available in digital formats is common. The existence of external tutoring firms, a persistent (and often malignant) characteristic of many education systems, can suggest places where online test prep may be potentially viable, especially where such firms offer related online programs. That said, online test prep carries with it profound equity implications, and education systems will need to seriously consider related mitigating factors, should it decide to move forward in support of such activities. Rapid response briefing note: Remote Learning & COVID-19 Outbreak (revised draft: 16 March 2020) p. 6 https://www.worldbank.org/en/topic/edutech #covidedtech Student assessment, especially high-stakes assessment, can be particularly problematic in online environments Education systems will struggle to offer highs takes examinations online, as may be the case where school closures are expected to run through the end of an academic year and threaten existing exam schedules. While technologies do exist to allow for automated proctoring of online exams, they are not in widespread use in existing education systems and bring with them various challenges related to reliability and privacy. As a result, there are not a viable option, outside of (perhaps) a few niche cases. Teachers and teaching Few teachers are able to easily transition to online learning environments quickly and effectively Teaching online differs greatly from teaching in the classroom. While the subject matter and learners may be the same, teaching exclusively online requires a different skill set than teaching face-to-face. Even teachers well experienced in the use of educational technologies to support student learning in classroom environments, and whose students regularly use educational technologies outside of class, can struggle when operating in a wholly online environment. Teachers working online need to be trained and supported Teachers who do not have access to sufficient broadband and a connected device at home will (obviously) not be able to support student learning online. Where such support is required, education systems will need to make available related infrastructure. Very few classroom teachers have received training on online instructional approaches and tools. If they are to support online learning by their students while schools are closed, they will need to be prepared to do so before schools are closed. Where this is not possible, education systems should not develop an approach to online learning that relies on teacher instruction or involvement. The existence of peer support groups, especially where they are already enabled through the use of technology tools such as email, online message boards, Facebook and WhatsApp, can be invaluable when teachers are forced to work fully online. Teachers are often parents too, and may need to assist their own children with online learning In times of massive school closures, teachers expected to support students through online learning may have to support the online learning of their own children, compromising their availability to teach and support their students. Some pedagogical approaches can be more easily translated to online learning and distance education environments than others Instructional practices that rely heavily on teacher lectures, or teacher assignment of self-study materials, are more easily transferable to online learning environments than more sophisticated pedagogical approaches such as those typically found where students are engaged in more learner- centered or project-based approaches. 'Champion', highly capable online teachers will emerge, but they will be the exception, not the rule A subset of highly motivated, highly competent teachers, especially those comfortable with using new technologies and reliable access to the Internet, will be able to support many of their students online in various ways, despite the lack of official support from their schools and education systems. This is especially true where teachers have prior experience, as learners or especially as teachers, operating in Rapid response briefing note: Remote Learning & COVID-19 Outbreak (revised draft: 16 March 2020) p. 7 https://www.worldbank.org/en/topic/edutech #covidedtech online learning environments. While much can be learned from these teachers, they will be the exception. While many useful insights can be drawn by studying such teachers, who can serve as best practice examples for their peers, education systems would do well *not* to promote the use of online learning activities that assume that such teachers are the norm. Parents and families Parents and caregivers are a critical piece of the puzzle Parents and other caregivers play critical roles in supporting students learning online when schools are closed. This is especially true for younger students, lower performing students and for students with special education needs. Even in the best of circumstances, most parents are ill-equipped to do so effectively, especially where they themselves are not technologically savvy, may need to support multiple children within the household as they learn and make decisions about how to allocate existing devices and bandwidth between siblings. Few parents themselves will have had much experience learning online themselves. Education systems should consider dedicated outreach to parents and caregivers explaining what is expected of children learning outline, suggestions for how parents can best provide related support and, where possible, should consider providing dedicated resources (websites, online help desks, call centers, texting campaigns). Policymakers The menu of potential options for policymakers to consider largely depends on two factors: the readiness of the existing system to support online learning at scale (including the local 'ecosystem' of edtech providers and vendors, and the availability of existing educational content, platforms and distribution channels) as well as how much time there is to prepare before schools close. [Please see Annex 1 for a simplified high-level matrix of options] There is not enough experience of funding of large scale online learning for students when schools are closed Perhaps because of the ad hoc nature of most activities of this sort to date, there is minimal documentation of costs related quickly rolling out online learning across an entire education system while schools are closed, as well as related funding mechanisms. When negotiating with vendors to help provide access to online learning while schools are closed, policymakers are typically in a position of great disadvantage In times of crisis, vendors can often have the upper hand when negotiating with government and school officials (and parents as well) to provide access to online learning opportunities at scale. In most countries, there exists a great information asymmetry between vendors and government on related topics, and vendors often function as de facto informal advisors to senior decision makers. Issues of expediency can dictate that planning and deployment processes occur very quickly. That said, policymakers would do well to consider issues related to data privacy, security, and the potential for sort term technology choices to result in vendor lock-in to specific tools over a longer period of time. In order to enable access to remote learning opportunities, policymakers can consider ways to enable connectivity and access to connected devices Rapid response briefing note: Remote Learning & COVID-19 Outbreak (revised draft: 16 March 2020) p. 8 https://www.worldbank.org/en/topic/edutech #covidedtech Policymakers may explore tapping into national Universal Service Funds to quickly pay for schemes to enable increased Internet access for learners, caregivers and teachers. Zero rating by mobile providers can enable access to specific online educational content (e.g. a national education portal, the use of specific educational apps) for free. At a school and national level, laptop and desktops in schools can be pre-loaded with educational content and loaned out. The inclusion of edtech-competent staff in planning and implementation decisions related to online learning is critical Where the use of online learning is being considered in the event of large scale school closings, decision makers should include people with practical knowledge of the use of educational technologies at scale in related discussions. This should be self-evident, but it is not always not the case. Existing bureaucratic processes and hierarchies may conspire against the inclusion of people with experience on edtech topics in related planning processes, especially where they are happening very quickly, include discussions with vendors, and where large sale procurements are being considered. Experience suggests that circumstances where people with edtech experience are seen as relevant only to the process of implementation of approaches to online learning at scale, and not of related planning, are likely to falter, quickly. Ensuring student safety online Where student use of technology tools is mandated, schools and education systems can often strictly monitor and control how such tools are used within school settings, and when connecting to school resources from outside of school. When schools are closed and online learning platforms, applications and tools are used by students at scale outside of school, education systems typically have much less control. Additional actions to help ensure student safety and security in such scenarios, especially where students are using tools provided by third parties with little oversight from education systems and/or where there are little or no relevant policy guidance or legal protections in place, can be critical. Notable examples to learn from Emerging approaches to the use of online learning at mass scale during the current COVID-19 outbreak in China offer many potential models for policymakers to consider and learn from China is in many ways unique in its ability and capacity to transition its schools quickly to online learning, given strong government support for the digitization of learning materials in the past, a comparatively high level of device ownership and broadband connectivity among learners, especially in richer, more urban areas of the countries; the high priority placed on education by Chinese parents and in Chinese society; a vibrant local ecosystem of edtech companies, many of them quite large and sophisticated; traditional pedagogical approaches that often times reward memorization, rote learning and self-study which can be relatively easily duplicated online; and experiences almost two decades ago under the SARS epidemic. That said, and for these and other reasons, emerging practices in China to support online learning at scale during the COVID-19 pandemic represent a leading edge laboratory for experimentation and quick implementation that should be of relevant interest for policymakers around the world. Initiatives to utilize education radio in West Africa during the Ebola epidemic point to useful models for many less developed countries to consider Rapid response briefing note: Remote Learning & COVID-19 Outbreak (revised draft: 16 March 2020) p. 9 https://www.worldbank.org/en/topic/edutech #covidedtech Lecture-based educational content is especially well suited for adaptation and delivery using radio and television. Many countries have longstanding educational radio and/or television programs that can be quickly scaled up during times of crisis. Even where such programs are not in widespread use, experience from Liberia, Sierra Leone and Guinea during the Ebola outbreak suggest models of how educational radio programs can be quickly developed and delivered, with the benefit that content can often be developed and delivered in multiple local languages. While quality, impact on educational outcomes, and learner engagement is typically not very high in such initiatives, such offerings are potentially more impactful than the alternative: do nothing. As more education systems in highly industrialized nations transition to online learning during school closures, they will offer many related models and insights of potential relevance to policymakers in other countries In most such systems, educational technologies have been deployed at scale for many years, often decades, technical infrastructure is quite good, and systems, tools and platforms are already in widespread use to support digital learning within schools, and in some cases outside schools as an extensive of the learning environment in schools. Initiatives to address equity issues related to technology use in education have been well considered in many such systems, which offers the potential for more equitable approaches to emerge, although it should be well noted that related rhetoric has typically far outpaced related action. In addition, experience with online-only 'virtual schools', and with homeschooling in the United States (which is in many ways a good analogue for the type of online learning proposed in many education systems in response to the COVID-19 pandemic), offer many practical examples of good (and bad) practices from which policymakers can draw useful operational insights. Vendors Providers of online learning products and services would do well to consider not only short term gain, but also the potential for long term partnerships The foundation of many fortunes can be made during times of crisis. Edtech vendors can be very aggressive in marketing their goods and services to policymakers (and parents) who are poorly equipped to understand what is being offered to them, and what is needed. The transition to online learning as a result of widespread closing of schools presents not only an opportunity for vendors to make money (it is possible to do well while doing good), but also to increase brand recognition; build market share; better understand the needs of education systems, policymakers, administrators, teachers, parents and learners; and engage in ethical corporate responsibility in a time of crisis. Rapid response briefing note: Remote Learning & COVID-19 Outbreak (revised draft: 16 March 2020) p. 10 https://www.worldbank.org/en/topic/edutech #covidedtech Annex 1: Simplified high-level matrix of options Supporting online and remote learning at scale when schools are closed System readiness Highly ready Utilize and explore how Disseminate support Single sign-on access to the use of existing online materials and guidance online learning platforms learning platforms can be for teachers and students expanded Provide training for Identify all students and teachers and implement ↑““ teachers facing access remote support challenges mechanisms ↑ “ “& →” “ Agreements with ISPs to enable Internet access for students facing access challenges ↑ “ “ & →” “ Moderately ready Take stock of existing Establish relationships Institute limited provision online learning tools with vendors to provide of connected devices access to existing online preloaded with learning Create central learning platforms materials for some informational web page students facing access (with links) ↑ “ “ & →” “ challenges ↑““ ↑ “ “ & →” “ Minimally / not ready Use existing mass media →” “ →” “ channels like radio and TV to provide limited Acquire / repurpose Develop additional access to remote learning existing radio / TV content and establish content in some subjects content operating team to for some students oversee process for more Conduct complementary students in more topics media outreach campaign to schools, teachers and Consider use of SMS to parents (including SMS) provide limited set of learning materials in some subjects to caregivers and students Schools already closed Schools closing Schools may close imminently soon (days, weeks) (weeks, months) Time to schools closure and initial implementation Key: ↑ “ “ = including options from cell below →” “ = including options from cell to the left Rapid response briefing note: Remote Learning & COVID-19 Outbreak (revised draft: 16 March 2020) p. 11 https://www.worldbank.org/en/topic/edutech #covidedtech For more information Large-scale, national efforts to utilize technology in support of remote learning, distance education and online learning during the COVID-19 pandemic are emerging and evolving quickly. For updated versions of this draft briefing note, supporting documentation, and other related documents, please visit the EdTech & COVID-19 resource on the World Bank’s education web site, https://www.worldbank.org/en/topic/edutech/brief/edtech-covid-19. Rapid response briefing note: Remote Learning & COVID-19 Outbreak (revised draft: 16 March 2020) p. 12