Mobile Distance & Hybrid Education Solutions A Knowledge Pack Last updated: November 11, 2020 Author: Marisa Steinmetz, World Bank, EdTech Team Overview: What does the World Bank and its Global EdTech team do? How does this Knowledge Pack fit in? Background o World Bank’s goals o World Bank Education Technology team’s vision o World Bank’s 5 EdTech Principles o World Bank’s EdTech Approach o Overview of this Knowledge Pack on Mobile Distance & Hybrid Education Solutions Click on any hyperlink to jump directly to the section. What are the World Bank’s goals? 03 The World Bank Group has two goals: To end extreme poverty and promote shared prosperity in a sustainable way. Back to section overview What is the World Bank’s Education Technology team’s vision? 04 The World Bank’s Education Technology (EdTech) team’s vision is to: Reimagine Human Connections to Transform Teaching and Learning for All Back to section overview What are the World Bank’s 5 EdTech principles? 05 1 ASK WHY: EdTech policies and projects need to be developed with a clear purpose, strategy and vision of the desired educational change. 2 DESIGN AND ACT AT SCALE FOR ALL: The design of EdTech initiatives should be flexible and user-centered, with an emphasis on equity and inclusion, in order to realize scale and sustainability for all. 3 EMPOWER TEACHERS: Technology should enhance teacher engagement with students through improved access to content, data and networks, helping teachers better support student learning. 4 ENGAGE THE ECOSYSTEM: Education systems should take a whole-of-government and multi-stakeholder approach to engage a broad set of actors to support student learning. 5 BE DATA DRIVEN: Evidence-based decision making within cultures of learning and experimentation, enabled by EdTech, leads to more impactful, responsible and equitable uses of data. Back to section overview What is the World Bank’s EdTech approach? 06 To operationalize the 5 EdTech principles, the World Bank focuses on: discovery, deployment and diffusion of new technologies. Back to section overview What is a Knowledge Pack? Who does it aim to serve? 07 What is a Main Target Audience Knowledge Pack? A series of short, pragmatic guides on individual topics within EdTech World Bank staff, institutions and that supports the target audience individual decision-makers beyond to make informed yet quick the World Bank especially those who decisions about EdTech support education ministries. interventions in their work, especially with education ministries. Back to section overview Overview of this Knowledge Pack 01. Why use Mobiles For Education? 04. Who can we learn from? a. Definition & Use Cases a. India Pratham b. Regional Mobile Statistics b. Zimbabwe Viamo c. Evidence Scan c. Peru Ministry of Education d. Nigeria, Edo State Ministry of Education 02. Should we use Mobile in our context? e. World Bank Projects with Mobile components a. Enabling Conditions for Success 05. Which Mobile Software exists for our Use Cases? b. Decision Tree c. Budgeting Guide a. Feature Phones b. Smartphones 03. How to implement a Mobile Solution? 06. Where can we find Additional Information? a. Inclusive Planning b. Institutional Setup 07. Acknowledgements c. Maximizing Access d. Intervention Rollout 08. Connect with the World Bank’s EdTech Team e. Sustainable Iteration f. Risks, Challenges & Tradeoffs Click on any hyperlink to jump directly to the section An overview of this Mobile Knowledge Pack Click on any hyperlink to jump directly to the section. Back to main overview Context 10 Amid the Covid-19 pandemic, distance & hybrid education solutions became no longer optional, but essential. The pandemic triggered an education emergency of unprecedented scale: in May 2020, nationwide school closures in 188 countries peaked at 1.6 billion children being out of school, equivalent to 91% of enrolled learners. With a vaccine projected to be available only in 2021, this is the “new normal”. Students are at risk of losing more than a year of learning, or dropping out for good, thereby diminishing a generation’s Those countries that are reopening schools opportunities for a lifetime. Fiscal challenges are often do so in a hybrid model, so that in- compounding academic ones. school time is minimized. There is no time to be lost - and innovative, mobile solutions are crucial to overcoming this challenge. Back to section overview Key Questions to be Answered 11 Can mobile be an effective tool for supporting distance learning or hybrid learning in my context? How can we effectively design, deploy and continuously improve mobile interventions? How should we leverage mobile solutions in the short-term versus in the long-term? Who has implemented such interventions and who can we speak to in order to learn about lessons learned? How can mobile help us assess if learning is taking place from a distance? Back to section overview Highlights & Target Audience 12 Mobiles are becoming increasingly ubiquitous and represent an effective 2-way communication tool too valuable for education to pass up. 96% of Leaders should leverage this “new the world’s population has access to a mobile; normal” to learn about & integrate 70% of that lives in low-resource settings. mobile solutions more permanently into their education systems. Highlights in this deck include: ❏ A detailed 14-phase implementation guide ❏ A Strategy Decision tree ❏ A list of ~100 feature & smartphone solutions ❏ Case studies from 4 contexts ❏ A checklist of Enabling Conditions This Knowledge Deck is intended to provide a basic overview of Mobile Distance & Hybrid Education Solutions to those who are considering mobile strategies, including: ❖ Ministries of Education (MoEs) in low- and middle-income countries ❖ Task Team Leaders (TTLs) at the World Bank ❖ Private sector actors ❖ Third sector actors Back to section overview 1. Why use Mobiles for Education? Click on any hyperlink to jump directly to the section. Back to main overview Back to main overview How can we leverage Mobile for Education? 14 Mobile has 9 Use Cases - consider ALL of them! Consider ALL configurations 2.1 Content Delivery (static) Mobile can be used on its own or - ideally - Mobile delivery of “one-way” content that as a way to complement other educational 1. Communication For Coordination doesn’t respond to student interaction but is 2.2 Content Delivery (Interactive) Mobile communication between parents, purely for consumption (sending of files, e.g. Mobile delivery of “two-way” content which media such as TV, radio and print teachers, principals & government with video, audio, images, voicenotes, podcasts, changes in response to student (interactive symbiotically. Evaluate mobile’s particular explicit objective of coordinating stakeholders links, documents, m-books, print2screen, voice response (IVR), chatbots) mobile apps value-add from an angle of unique to support student learning; e.g. weekly fixed e-learning courses) (incl. educational games), e-books with text- advantages over other tools, such as its 2- learning schedules, homeschooling guides, etc. 2-speech, adaptive assessments, adaptive way communication ability that enables content delivery, LMS) assessment, M&E, authentication, peer- 9. Digital Credentialing collaboration, etc - which can’t be done via Authentication of student ID, e.g. for high- 3.1 Synchronous Instruction (Parent) TV or radio. The exact use case should stake exams; via fingerprint scanner, face recognition, voice biometrics, etc Conducted by the parent at home, as 1-on- always depend on context and need; see 1 or 1-to-many, guided by instructions the decision tree for guidance. received via mobile 8. Authentication Authentication of student ID, e.g. for high- Consider ALL Phone Types 3.2 Synchronous instruction (Teacher) stake exams; via fingerprint scanner, face Conducted remotely/in hybrid mode by recognition, voice biometrics, etc teacher, 1-on-1 or 1-to-many, via text-, Feature Phones audio-, video- or LMS-based mobile 1.8 to 2.8-inch LCD screens (color/B&W), SD 7. Monitoring & Evaluation solutions (audio/voice call, SMS/MMS/ IM, card, GPS, camera, buttons-based input, Mobile data collection (usage data, surveys, voice notes, mobile radio, etc) torchlight; voice calls, SMS, MMS, basic voice calls, IVR, browser-based) on inputs, browser, FM, media player; no wifi outputs & outcomes 4. Peer-2-Peer Collaboration Smart Feature Phones Collaboration between students via simple Feature Phones with web-browser capability 6. Content Creation mobile communication tools (voice call, enhanced via KaiOS to run smartphone apps Teachers or students use mobiles (only SMS/MMS, IM) or more interactive, mobile, smartphones) to create educational digital online tools (online forums, LMS, cloud- (YouTube, Google maps, Whatsapp, etc) 5. Assessment content (video recordings, edited video Formative (in-class checks for understandings , based collaborative file editing, etc) compilations, images with explanations homework, surveys, quizzes, exit slips, etc; Smartphones drawn onto them, explanatory docs, etc) Interim (every 6-8 weeks, helps predict Touchscreen, 2 cams, micro-SD; mobile OS, performance on summative exams) & apps, advanced browsing & UI, etc summative (end-of-year high-stake exams) Back to section overview Is Mobile an Option in our Context? 15 Mobile might be the most equitable option Households with access to different media (DHS 2011-18) How much data can 2% of avg. local monthly income buy? Mobile penetration is actually at times higher than that of TV, radio or computers, specifically among the poor. For example, in households of primary-aged students in Africa, 46% of poor households own mobiles, while that number is only 30% for radio, 4% for TV, 1% for computers and 0.3% for the internet. Similarly, in the Nigerian state Edo 91% have access to a mobile phone, yet only 69% to TV and 46% to a radio. Household ownership of course does not imply that the child owns or has access to the phone; the extent of phone access at home always needs to be evaluated in-depth. Furthermore, while ownership of mobiles is high, penetration and Greater than 1GB at 2% GNIC e.g. South Africans with affordability of cellular & internet subscriptions still vary. incomplete formal schooling To view country-level data, visit the World Bank database, ITS Less than 1GB at 2% GNIC need 10% of their monthly Current trends however clearly justify investing in mobile expenditure to afford 1GB. statistics portal or USAID’s DHS data portal. sooner than later - especially in Sub-Saharan Africa and the MENA region, adoption is rapidly on the rise. Africa is now a bigger mobile phone market than Latin America or USA and in Current Mobile Access by Region (GSMA, 2018) Growth in Subscriptions & Internet Access by region (GSMA) the next few years will surpass Europe. Currently, subscriptions are reaching on avg. min. 44% of people across regions. In the short-term, focus on offline, low-bandwidth mobile solutions, but rapidly enhance mobile internet access to advance to sophisticated mobile solutions long-term. “Offline” strategies like SMS, voice calls & IVR are still the most equitable avenue for large-scale mobile learning. Some mobile solutions are more inclusive than others: for example IVR is accessible to the blind, physically disabled & dyslexic, offline, feature-phone compatible, configurable as multilingual & soon personalized with AI NLP. Back to section overview How effective are Mobile Solutions for Education? 16 Communication For Coordination Overall, existing evidence shows With parents: Communicating formative mathematics With principals: In Peru, school managers were nudged With teachers: There seems to be no research on how extreme promise of mobile usage for assessment results of rural Chinese primary school via SMS alerts to use funds appropriately and be effective mobile communication is between principals and education. However there still are children to parents produced 0.3 SD improvements, accountable for completing maintenance work on time. teachers, or teachers and the government. With regards huge literature gaps and a dire need especially for left-behind children. Similarly, when Chilean The study found that sending SMS increased the to their coordination with parents, research affirms that for more research (and in particular parents received regular SMS about their child’s likelihood that maintenance managers performed their teachers need quality training and constant more RCTs) in mobile-based education attendance, behavior and mathematics test scores, the activities at the appropriate time, reducing the encouragement to display regular, effective use of mobile probability that a student attends school sufficiently to noncompliance rate in reporting by more than 15%. It messaging with parents (& students). Plus, inconsistent solutions, especially in low-income permit grade progression increased by 6.6%, and the was also found that the impact of the messages is take-up of mobile communication platforms tends to be settings. Note that this is just an likelihood that they passed the grade by 2.9%, virtually different depending on the content: the impact was due to, in part, administrators’ failure to establish school- evidence scan rather than a full- eliminating grade failure among marginal students; the greater with “social norm” messages that mentioned wide norms about adopting one communication platform fledged review and is far from positive effects were larger when parents were sent more that most of their nearby peers completed the work, & a set of communication practices. exhaustive - this does not cover all messages. In the US, effects of such interventions were leveraging reputational and peer pressure. Content Delivery educational levels nor all student even larger. With students: SMS were sent to remind college- groups and contexts; it is supposed to intending high school students of required pre- SMS-based: Primary: M-stories increased student Parents often don’t have accurate information about their be illustrative. children’s effort and performance, and in particular low- matriculation tasks and to connect them to counselor- literacy in primary school students in Papua New Guinea based support; this increased student enrollment by 4-7%. and in Zambia via 50 maternal language stories. income parents hold mistakenly optimistic beliefs about However there is no question that The effects were largest for students with less clearly Secondary: m-learning showed significant positive effects their children’s learning & attendance; providing that formulated college plans and less access to help from for language learning (specifically vocabulary retention) mobile can be effective in supporting information can enable parents to better support their other sources. SMS nudges also reduced dropout levels of & math. University: Programming students receiving education outcomes; the devil is, of children. The risk is that low-resource parents at times US STEM community college students; 72% of the SMS as learning support improved their performance. course, in the details: Implementation reallocate resources away from lower-performing students who participated in a nudges trial decided to Adult Education: SMS lead to significant and long-term fidelity, content selection, even the children toward their higher-performing children, continue on in STEM courses after their first semesters, gains in literacy, numeracy and retention. General: SMS- reinforcing inequalities in performance. This can be framing of SMS, etc. remedied possibly via additional info; explicitly explaining compared with just 56% of the students who opted to not based delivery - compared to print - can enhance learner receive nudges. Similarly, reminders to apply for financial creativity, learner flexibility & self-image. the earnings returns of secondary education to parents in As mobile is under-researched, and to the Dominican Republic led to a 0.2 to 0.35-year increase aid for their second year, where recipients were about 12‒14 % more likely to remain enrolled in the next two IVR-based: There is no direct evidence on IVR some extent always will be given its in the years of schooling completed. semesters. Interestingly, when scaled nationally, the same interventions for younger children, but audiobooks made fast pace of development, highly agile, a group of 2nd grade students outperform the control However mobile-only communication with parents also strategy appeared less effective; this was explained via a iterative, and data-driven project lack of personalization. Even small changes in the framing group by 3x in reading comprehension, 7x in 2nd grade management must fill the knowledge has a number of challenges that should be addressed in of information may change or de-bias behaviour because vocabulary, and 4x in reading motivation in an RCT the design phase: they’re prone to misuse, conflicts due gaps on the go to constantly learn from to misunderstandings, engaging on out-of-hours, harming of cognitive and attentional limitations; e.g. not providing (however with a limited sample size). For youth and success and failures in implementation. how-to instructions reduces effectiveness of nudging. adults, 42% of users demonstrated increased confidence the school climate & minimizing face-to-face We encourage you to document & Of course, primary and ECCE students can be expected to in using English after consuming BBC Janala, which communication. Determining & enforcing group rules, benefit less from mobile communication for coordination reached 28 million Bangladeshis (80% of its users were share your lessons learned with us! informing group members about the aims of the group, than secondary and postsecondary students. from rural areas) with its 3-min audio lessons, of which 7 only group admins texting the group and not abandoning million were accessing via dial-in on their feature phones face-to-face communication can minimize those issues. Back to section overview followed by SMS-based quizzes. [continues on next slide] How effective are Mobile Solutions for Education? 17 Content Delivery (continued) when parents of preschool children were given an Synchronous instruction - Teachers Assessment: While mobile P2P instruction seems to have electronic reading application that audio- and video- positive effects, evidence on P2P assessment seems less App-based: Adaptive smartphone based learning records them as they read to their child; they were SMS: Medical students who were taught via SMS had higher promising. e.g. university students had negative apps have proven effective in supporting early asked to set goals for the amount of time they test scores than peers who were taught via traditional lecture experiences with IM‐assisted peer assessments. literacy & numeracy in low-income students without would spend reading to their child in the coming (however they had lower satisfaction rates), similarly to teacher guidance (KitKit, OneCourse, Homer) and as week. They were reminded by SMS to read in order residents who were taught via SMS compared to those who General: Researchers have identified design principles to supplement. App-assisted EFL showed promising to reach the goal & received a congratulatory SMS read the same content in books. follow for effective tech-assisted P2P learning. Mobile apps results for adult immigrants as a supplement, not as a non-monetary reward upon reaching it. They can also be leveraged to optimize scheduling and peer core instruction. And STEM apps have been shown were also provided with information about the IM: Instruction via WhatsApp was more effective in matching. to enhance conceptual understanding (physics, importance & benefits of parental involvement. SMS supporting vocabulary acquisition of 14-year old Iranian EFL geography) & support collaborative, inquiry & tips can help also reduce parental stress and learners than traditional classroom instructions; pictorial Assessment - Formative & Summative problem-based learning. However even the much increase parent-adolescent communication & annotations were crucial for effectiveness. Similarly, more basic, browser-based learning apps accessible parental competence, thereby enhancing parent- Whatsapp-based instruction was effective in improving Mobile assessments can have multiple formats, from semi- via smart feature phones (ie. browser-enabled student interaction and family peace & can improve critique writing skills in EFL students. automatic (assessment conducted by software, but grading feature phones) can support learning: Nokia’s free how they manage student health functions crucial & feedback done by teacher) to fully automatic (no teacher Mobile Math app helped reduce nearly 2,000 grade for performance, e.g. sleep. Video: No difference in performance was found between involvement needed), and their impact on student learning 10 South African students’ mathematics attainment university students who participated in face-to-face will vary depending on their regularity, quality, and what is decline by 3.5% (statistically significant); students IVR: There is little research on how well parents supplemental instruction versus in online instruction. done with assessment data, ie. to what extent data is used were using the app on a voluntary, supplemental respond to IVR-based instructions in the to tailor instruction to that student & whether that basis rather than as part of the formal school day. homeschooling realm, however there is plenty in the tailoring is done by a teacher or an ALS.. medical that indicates that parents respond well to Peer-2-Peer Learning Considerations: Evidence on mobile content delivery IVR-based guidance. For example, parents of Evidence on the effectiveness of mobile assessments is still sparse, and existing evidence is mainly from toddlers who received IVR-based instructions on SMS/IM instruction: has been shown to increase student supporting student learning - especially remotely - is settings where mobile learning was optional; questions to ask during pediatric visits and follow-up motivation and to be at times even more engaging than video- sparse. However secondary students improved exam effectiveness depends on many factors incl. content questions were more likely to report discussing based P2P learning. performance in terms of their factual knowledge via quality, subject, student age & implementation important issues, and 100% of clinicians reported formative mobile micro-assessments which followed approach and fidelity. Plus SEL factors like growth that PHP improved the quality of their care. This Video instruction: In an online, synchronous online micro-learning units. Most mobile assessment studies mindset and perceived autonomy & content dynamic is transferable to virtual parent-teacher mathematics peer-tutoring program over 7 weeks as well as a focused on formative assessments with elementary relatedness are a significant factor in student meetings as well as responding to IVR-based whole year for low-achieving third graders, students in the students in STEM subjects. Most of the articles reported a performance in m-learning, too. homeschooling tips and other information. experimental group had a very significant gain in mathematics significant positive impact on student learning learning compared to their control group. Elementary school performance, motivation & attitudes.. Synchronous instruction - Parental App-based: While there is little research on how and students in an online peer‐assisted learning group online even which apps most effectively support parental outperformed the face‐to‐face group on reading skills. Literature identifies some best practices: Establishment of SMS-based: Leveled m-stories sent to parents to be homeschooling, simple access to an educational app a classroom culture that encourages regular use of read at home by children with their caregivers in can reduce parental anxiety about their students Hybrid: P2P learning in the classroom that paired high- assessment tools; use of varied approaches to assessing Zambia produced an effect of 0.3 on students’ oral performance; this anxiety tends to be negatively achieving students with lower achieving students and that was students; tracking of individual student progress toward reading fluency; prior to the project, few parents or correlated with elementary student achievement (in assisted by a mobile app resulted in the treatment group goals; providing clear guidance on grading. Trust in the caretakers read with their children at home. math) and is hence beneficial to eliminate. outperforming the control group. source of the assessment is also crucial. Gamification can Similarly, Parental reading time was doubled Back to section overview increase the participation in digital formative assessments 2. Should we use Mobile in our context? Click on any hyperlink to jump directly to the section. Back to main overview 19 Which Enabling Conditions are crucial for Success? ❏ Relevance: Mobile solution adresses a clear need that is sharply Important: you do not need to be in a context where ALL ❏ Financial: Budget is at the very least sufficient for low-tech felt by potential users; addresses issues important to these items are checked - those contexts mostly don’t exist mobile interventions (e.g. SMS-based); systems are in place for Development Objectives, National & Local policy priorities yet. This checklist describes, the IDEAL context, and is hence safely and transparently distributing funding to mobile solutions a guideline for what context you should build out OVER TIME ❏ Superiority: Need can only be addressed or be best addressed via time, in phases. You only need a few (underlined) to begin. ❏ Technical: Mobile penetration is high (90% among feature mobile solution(s); current solutions for this issue are considered phones), at-home mobile / broadband internet access & school inadequate; Superior effectiveness to other innovative models connectivity are high; a dedicated, organization-internal IT team established or at least strong indications for it is well-equipped, Government has a digital M&E system in place ❏ Effectiveness: Sufficient (case studies) or strong evidence (RCTs / ❏ Informational: Strong Knowledge & Information Exchange (KIX) Independent external evaluation / meta-analyses) from multiple structures are in place; KIX helps identify promising, contextually settings that the solution is effective in this / similar contexts relevant mobile solutions and analyze implementation fidelity, impact effectiveness & cost effectiveness of mobile solutions. ❏ Interoperability: Implementable within existing systems with few Easy-to-use online communication platforms/ offline forums components easily added; represents a manageable departure MOBILE enable 1) parents & teachers to find info on mobile solutions from current practices & behaviors of target population and SOLUTION (effective products/services, how-to guides), 2) private & third current practices and cultures of adopting organization(s) INFRASTRUCTURAL sector organizations on how to collaborate with the government CAPACITY for mobile solutions and why, 3) researchers & leaders to ❏ Cost-effectiveness: Appropriate cost-effectiveness for the context; compiles emerging best practices, 4) entrepreneurs to high public ROI (over time) or viable business model (pilots might understand the struggles that the public education sector needs need subsidy but financially viable over time) solutions for & on curriculum standards; Regular KIX meetings are held between stakeholders, 5) online training resources, etc. ❏ Political: A clear EdTech Policy with an explicit mobile component ❏ Skillfulness: Beneficiaries have the digital literacy to use the is in place that outlines the short-term (3-24 months) and ideally mobile solution/ can be easily upskilled to do so; intermediaries also long-term (up to 10 years) vision for edtech in the country have technical skills to select, develop & deploy the solution(s) and the particular role that mobile has to play in that vision. There INSTITUTIONAL HUMAN is political buy-in and support for that strategy across parties. ❏ Managerial: Leadership has a clear strategy for mobile but is CAPACITY CAPACITY able to adopt an agile / adaptive and data-driven approach to ❏ Legal: A policy is in place to cybersecurity and data protection. implementation given the fast-evolving nature and still limited evidence on best practices for mobile. Across levels, there are ❏ Partnerships: Strong technical / financial / informational / clearly assigned individuals whose role & responsibilities focus implementation partnerships for the R&D / development / on mobile solutions management. Accountability is maintained marketing / sales / deployment / M&E of mobile solutions exist via high standards & close tracking of results. Performance between iNGOs, NGOs, unions, donors, governments (incl. IT management is maintained, rewarding those who succeed & teams, M&E teams, leadership, etc) & corporations (mainly supporting those who struggle MNOs, tech companies etc), social enterprises & startups; partnerships are leveraged for effective piloting / scaling Country Context ❏ Cultural: Mobile phones are accepted as a tool that can serve educational purposes; women and girls have the same access to Back to section overview mobile phones as men and boys do. 20 How should we start? How to prioritize Use Cases? Needs Assessment Coordination Content Delivery M&E Synchronous Instruction Remaining Use Cases & Content (Parental) Delivery Upgrade to ‘interactive’ Use mobile for Peer-2-peer Learning Use mobile for Use for Mobile Step 3: Use Synchronous Synchronous Assessments Yes mobile for M&E Instruction Instruction (formative, Are you offering (Parental) (Teacher) summative) educational content Content delivery already (or Delivery soon) via TV or radio (interactive) for basic learning continuity? Supplement with Add smartphone No feature phone static & then Step 2: Use Mobile Yes Does more than static & then interactive Yes For (2-way) Are you Step 3: Do 40% of your interactive content delivery Communication communicating content delivery population have content delivery (browser-based For Coordination for coordination via radio / TV access to a basic, (mlearning & & mobile apps) (SMS, voice call, with Students, first feature phone or IVR) IMs, mobile Parents, surveys) smartphone? Yes No Prioritize print, Do you have Teachers and Principals via TV TV & radio primary data on No Consider or Radio? solutions; only current needs of Step 2: Start with integrating use mobile for your students, Communication advanced core content delivery parents, teachers For Coordination curriculum Step 1: Use mobile if all other and principals? via Radio & TV Adaptive (voice call, SMS; mobile use No Learning browser-based cases (M&E etc) Solutions surveys) for your are already once you have needs assessments’ leveraged to built out all primary data their maximum other solutions collection Back to section overview 21 How should be budget a Mobile Intervention? Key Cost Components Sample unit costs 1. Capital expenses 1. Devices a. Smartphone min. $50; $214/avg. a. Hardware b. Feature Phone min. $10 i. Smartphones / feature phones c. Smart Feature Phone min. $11 (KaiOS) ii. Microservers d. Microserver ~$500 (Rachel 3) iii. Main servers 2. Content b. Content a. Content management platform $73,000 (EdoBEST@Home) i. Content development b. Content License $234,000 (EdoBEST@Home) → subject matter experts → UI/UX designers → both of the above are rates charged by Bridge Academies International → Videographers/Voice artists → Graphic Designers 3. Network a. SMS rates $0.01-0.08 /SMS ii. Licenses for Purchased content b. Airtime rates $0.01-0.19 /minute c. Networks → View additional surveys & sample country-specific rates from Viamo here i. national airtime rates ii. number of users (more users = more airtime) 4. Server/Hosting costs iii. minutes/SMS per user iv. number of interactions a. AWS rates: View calculator here to calculate costs based on your specific needs. v. Hosting / Servers vi. mobile internet rates 1. Services a. Mobile Survey Design & Setup $5,000 /Survey (Viamo) 2. Human resources b. Call Center Set-up $5,000 /Project (Viamo) a. Services c. Call Center Training $2,000 /Project (Viamo) b. Training d. Remote Training Technical Design & Setup $5,000 /Round (Viamo) e. Design and Implementation Team $300 /day (Viamo) 3. Time f. IT support (for content platform) $67,000 /project (EdoBEST) a. Duration of the project Back to section overview 3. How to strategically implement Mobile Solutions? Click on any hyperlink to jump directly to the section. Back to main overview Implementation - Overview of the 14 Phases 23 Note: While this implementation guide has been designed with the current Phase 01-04: Inclusive Planning Covid-19 pandemic in mind, the overall steps apply to all mobile interventions, independent of context. 3. Conduct a Rapid 4. Create Initial 1. Establish a 2. Conduct a Rapid Internal & External Implementation Representative Target Group Capacity Plan & Risk Working Group Needs Assessment Assessment Analysis Phase 05-10: Setup & Rollout 9. Maximize Access 10. Intervention 5. Ramp up 6. Ramp up 7. Activate Public 8. Set up the Initial (to Mobile phones Rollout (depends Internal Capacity External Capacity Communication M&E structure & the Internet) on use case) Phase 11-14: Sustainable Iteration 11. Ongoing 13. Hybrid internal & external 12. Ongoing 14. Post-Covid: The Reopening of KIX, L&D and agile Capacity Building “New Normal” Schools Iteration Back to section overview Implementation - Short term (Phase 1-4): Inclusive Planning 24 Back to section overview Phase 1 Phase 2 Phase 3 Phase 4 Establish a Representative Conduct a Rapid Target Group Conduct a Rapid Internal & Create Initial Implementation Working Group (WG) Needs Assessment External Capacity Assessment Plan & Risk Analysis Objective The situation is evaluated from all The initial Plan has agile design that lives up to The needs of your affected target group The financial, technological, human, possible stakeholder perspectives and the dynamic situation, integrates best practices (students - incl. adult learners-, parents, political, legal & informational capacity of solutions are co-designed to maximize gleaned from other contexts, meets standards, teachers, principals) are truly understood. both internal & external stakeholders & buy-in, inclusion, as well as prevent is scalable & includes contingency plans for Beyond educational needs, analysis should target group, as well as areas for redundancy with what other early-stage risks. In the short-term, goal is cover contextual factors like health, safety, partnerships, are clear. Identify strengths stakeholders are doing. learning continuity & engagement for dropout financial and mental well-being. & weaknesses in all to optimize symbiosis. prevention, less so academic outcomes. Share the rapid assessment insights among Setup Invite national public, private & third Use both qualitative & quantitative data; Create a simple rapid capacity evaluation stakeholders & create agile / adaptive plan sector, intersectoral & international start with secondary data, then collect rubric that lists all the above capacity outline (context, needs, SMART objectives, actors into the WG, either as core team primary data and centralize it. Use a areas, further segmented into strengths & intervention(s) design, timeline, risks, members or as advisors; incl. especially framework to guide data collection. weaknesses. Interview samples of all responsibilities, budget incl. gaps, KPIs/M&E); teachers & parent associations, MNOs, Segment & quantify subgroups in your stakeholders, from MNOs to community invite stakeholders to contribute (digitally). tech firms, community leaders, think target group e.g. by mobile access, etc. leaders. Do phone calls, mobile surveys Consult with countries in similar situations; tanks, NGOs, religious orgs, donors. Truly LISTEN to your target groups’ needs. (e.g.Google Forms) & smartphone-based include L&D plan. Use 10 Digital Principles. , incl. digital literacy assessments (e.g.Pwc app). designing with users. Hold 2 reviews for approval. Establish clear & inclusive objectives Analyze targets’ groups needs to identify Human: in orgs, staff’s expertise, skills & Pick your battles, think multimodal and use Execution for the WG. Both short-term & long- which mobile use cases would be most access to training channels plus ability to hire insights from other orgs/ countries. Prioritize use term objectives should cover the valuable & whether those needs are best external consultants; in target group, (digital) cases that cannot be covered by TV/radio (e.g. categories of equity, inclusion, and most equitably served via mobile or literacy & ability to handle educational M&E, P2P-collaboration, assessment, 2-way quality, transparency, agility, other media. For the latter, analyze target concerns amid crisis. Tech: hardware / communication) rather than just content delivery. accountability, & cost-effectiveness. groups’ access to mobile versus other software infrastructure, e.g. cellular internet Prioritize solutions that maximize equity & fast The equity lens should generally media (TV, radio, print), as well as types of vs router access (sample survey in Appendix implementation. Ensure that stakeholders are include those with no access to devices (basic, feature, smartphone). For C), electricity/battery charging opportunities, aligned, well-informed, bought-in & agree on devices and/or internet, but also IDPs, primary data collection, leverage mobile types of devices, actual daily mobile access comms structures (regularity, medium, points of refugees, women, special needs (incl. phone calls) and be mindful of for student (in mins), etc. Info: databases, contact, etc). Consider multimodal options, e.g. students, language minorities, creating representative samples. Leverage intra-& inter-stakeholder channels. Legal: weekly SMS alerts 30 mins prior to a broadcast nomads, SES & rural-urban divides. community leaders and NGOs. cybersecurity & procurement laws. boosts radio campaign listenership by up to 20%. Implementation - Short term (Phase 5-7): Institutional Setup 25 Back to section overview Phase 5 Phase 6 Phase 7 Phase 8 Ramp up Internal Capacity Ramp up External Capacity Activate Public Communication Set up the Initial (Remote) M&E structure Organization-internal capacity is rapidly ramped Students, parents, teachers & principals are well- Internal M&E and external M&E structures are setup Objective Capacity of students, teachers, principals, parents, and up to meet the moment; including financial, informed on the intervention & expectations and quickly and designed in a relevant, agile & cost- partner organizations is increased rapidly to enable human, technological & informational capacity. kept up to date about changes effective manner. them to cooperate productively. Have a holistic, agile M&E Plan. Organization-internal Financial: Tap national Universal Service Funds & Financial: Ensure that parents have basic financial Based on info gathered during needs assessments, & external, beneficiary / partner-focused KPIs. Keep Setup Covid Emergency Funds. Mobilize private & Third security despite crisis in order to have the mindspace draft clear PR documents which describe the why & processes short (sample design, surveys, QA, etc). sector Partnerships to mobilize funding; e.g. Brazil’s to assist their child in mobile learning process. Ensure how of the intervention and where to go for FAQs, Consider scale-based & MCQ surveys to maximize São Paulo mobilized $40 million in funding for their that teachers & principals have access to funds needed and set expectations that are to be met. Have a responses. Define data protection rules, e.g. for phone Covid education response from existing partners. to e.g. get SIM cards to communicate with students. dedicated PR team with school district members. numbers, recording of calls. Include a baseline. Make Ensure that partner orgs can channel money Review PR documents with stakeholders involved in samples representative (rural-urban, SES, language Human: Based on the results of your internal needs effectively to where it’s needed for the intervention. implementation prior to publication. Develop message minorities, etc) & disaggregated. Adapt existing M&E. assessment, proceed with upskilling as needed development protocols to enable prompt drafting and (digital literacy, design thinking for redesign of Informational: Provide clear instructions on how the approval of statements so that communications can Pick only crucial internal & external KPIs at the processes for rapid deployment, agile project mobile intervention will work & what the reason for it be completed in a timely way. beginning, ie. those that will allow to make better management, family engagement, etc). Hire is, using TV & radio (e.g. caregivers need to know that decisions, fast. While KPIs should cover inputs, consultants/staff where upskilling isn’t an option. they will receive an IVR call and how that works) Set up a helpdesk toll-free hotline & a website that activities, outputs, outcomes (learning continuity, Leverage community leaders and local NGOs to further people can visit for FAQs & to fact-check mental well-being) & impact (learning outcomes, Informational: Designate clear communication spread the word. Put in place communication misinformation. Allow people to send questions via student retention), focus on KPIs like engagement & structures (regularity, POCs) for the intervention; structures with external partners for coordination. various media, incl. Twitter, Facebook, Email, SMS. service satisfaction rates, mental health, etc. that act assign an L&D unit to capture & disseminate best Send updates via multiple channels every 1-3 days. as early warning systems when things go off-track. practices on the go; ensure that local & central Human: Partner with MNOs & tech companies to units are clear on communication structures & provide digital & mobile literacy training to external Be multimodal in your PR. Publish press release on Consider multi-modal remote mobile M&E avenues. intervention design. Leverage community leaders, stakeholders, incl. users & intermediaries. Ask School website;share it with media & partner organizations Phone calls, SMS-based, IVR-based, browser-based principals, parent associations & MNOs for data Management Committees (SMCs) (if in place) & (NGOs, Labor Associations, private companies esp. surveys, offline surveys, sending of videos/photos via collection (e.g. numbers). Map school connectivity. community leaders to support coordination in villages. MNOs, etc). Make PR announcement via radio & TV. If Whatsapp - it depends on your users access situation. Share guidance with parents on how to prepare their within budget & of sufficient reach, run awareness Technological: If needed, expand IT infrastructure, kids for distance learning, e.g. how to behave during campaigns via social media. Post content regularly on Mobilize multiple stakeholders to help with data e.g. server capacity. Adapt existing digital tools video-based instruction. Leverage free online M&E social media pages; Dr. Shawqi, Egyptian Minister of collection, but don’t rely much on volunteers. e.g. ask rather than create new ones where possible. courses & how-to-guides to upskill M&E staff, etc. Education, earned praise for tweeting a 6-min video teachers to reach out to each of their students 1/week Design with the user, & design for scale to the where he explained to students how to register for the to identify student needs under remote instruction. extent that time & resources permit. Put in place Technological: Create data & server capacity sharing Edmodo platform. Print street ad banners. Then share Ask MNOs to track & share relevant mobile (GPS)data. QA processes to ensure that newly added capacity agreements with partners. Put data sharing & the announcements via mobile (SMS/Whatsapp). works well & that issues are flagged before they cybersecurity agreements in place with organizations. Consider regular blasts via robocalls as well. Be strategic in your M&E staff training. Train your cause problems. Address data privacy & security M&E staff with short (e.g. 2-day) workshops. Implementation - Short term (Phase 9): Maximizing Access 26 Back to section overview Phase 9.1 Phase 9.2 Phase 9.3 Phase 9.4 Increasing Internet access Increasing access to Mobile Optimizing software for Selecting hardware for Phones maximum access maximum access Encourage communities to systematically share General questions to ask: Can the software be used offline? At what Whether you are looking at main Ask MNOs to “zero-rate” educational websites & apps; i.e., phones among families. If you are considering mobile network speed does it function? Is it multilingual / allows for easy no data charges apply when these resources are accessed servers, microservers, or devices, content delivery, provide instructions on how many localization? Is it purpose-built for underserved populations or just hardware has to meet the following (e.g. South Africa). Be mindful of resource bandwidth hours each child will need access to a phone to be able adapted from a high-income context? If the latter, has it been used in requirements. Request that MNOs give preferential access to criteria: 1) optimized for low to have a positive learning experience. low-income contexts before? At what maximum scale has the bandwidth, 2) offline capacity, ie. scarce bandwidth for education-related data & service platform been used in the past and how easy is it to scale it further in (“bandwidth-shaping”). able to cache info and queue online Provide access to installments-based phone purchase case additional capacity is needed? Is it customizable, if yes, to what sync requests for the rare times schemes to credit-unworthy low-income communities. degree? How easy is UX / does it require only minimal digital when the device is online, 3) high Make available free SIM cards for use by teachers, students literacy? What is their data privacy policy? Can you try before you & parents, with expedited registration procedures, coupled offline storage capacity, 4) low Consider phone loan schemes. Smartphones /Feature buy? Compatibility with in-house software? power consumption, 5) ability to with special data plans. Or distribute full-on smartphones with phones could be loaned to families for the preloaded content and data plans already installed. draw energy from renewable (mainly duration,based on a needs assessment, with the Key questions to ask when procuring messaging software: Does the solar), 6) chargeable on 12V/DC, 7) provision of a solar lamp that enables at-home USB platform allow for sending multilingual SMS/USSD incl. easy instant powersmart (e.g. powers down at Set up free public wifi spots & movable hotspots. e.g. via charging. Phones could be tagged with transponders or translation features? What real-time data shows up on the data data buses / Google’s 4G Loon balloons. Hotpots cause gatherings night), 7) no AC/DC-inverters, 8) high enabled to “find my device” in case the item is dashboard (e.g usage, message delivery, customer satisfaction,etc)? battery life, 8) high CPU & RAM, 9) that harm social distancing; tell people to keep 2m apart. misplaced. Provide info on care for the phone (e.g. low cost, 10) no moving parts, 11) that they cannot be washed with water and soap are Key questions to ask when procuring IVR software: How easily does uses passive cooling (fans suck in Consider reverse-billed SMS messaging, unbanning VoIP, or important) and asked to sign a loaner agreement. the software integrate existing contact data from one’s CRM? How “data-free” IM apps like Moya to enable free communication. dust, insects & humidity), 12) complex of an interaction can the software handle? Does the resistant to voltage swings/dips, Partner with MNOs and Phone producers to get platform layer technology such as AI, automatic speech recognition, brownouts, dips, 13) UX friendly (ie. Utilize bluetooth or offline File Transfer Apps to pass on files (MNO-branded) devices into the hands of learners & text-to-speech, call recording, SMS, voice biometrics, transcription, within communities. Bluetooth’s radius range is on avg. 10m. not complex for user to use, min. teachers, incl. procurement & delivery. voicemail detection? Do they have different voice artists who can number of buttons etc), 14) hard to File transfer apps are 200x faster than Bluetooth. deliver messages in appropriate tone (serious, fun, etc)? Does it work break / durable ( resistance to water, Leverage a pre-loaded advertising application on only via VoIP, or in-country network infrastructure (former is more humidity, dust, dirt, and extreme Boost Wifi signal available in schools. Some schools have smartphones to subsidise the cost of a handset by expensive)? How do they pilot IVR content prior to full delivery? Does local area networks /wifi hotspots that could be boosted to heat), 15) parts should ideally be ~$49 for low-income users. (Social Eco) it allow to speak to a live operator eventually when needed? How do recyclable, 16) screens need to be enable the surrounding homes to access the network. they optimize for multiple calling attempts? legible in direct sunlight 17) Negotiate discounted bulk purchase prices for feature maintenance is low cost & requires Install local plug-and-play microservers that create local or smartphones with mobile phone manufacturers. Key questions when procuring software for smartphones: Which OS intranet, accessible via mobiles to view preloaded content only minimum skills (tech that can’t Sign up a sufficient number of parents who are version(s) is the software compatible with? How fast does it consume be locally maintained, supported / (RACHEL, Kolibri, OLX, Kiwix, BluPoint, BRCK, Snappbox). interested in a reduced phone price prior to this phone battery? How much CPU/RAM does it take up? How much repaired is unsustainable), 19) ideally negotiation. As an MoE, consider slashing import taxes storage space does it need? If it requires internet, how much can be built from scratch using local MNOs can host offline content platforms on local servers to & duties temporarily; they can reach as high as 50% of bandwidth does it consume? Is it accessible for special needs users connect to their base stations, enabling access to the server components, 20) interoperability, 21) the total device cost in some African countries. (audio instructions, text2speech, adjustable text size and font, etc). reliability, 22) parts are recyclable. anywhere a 3G connection is connected to the tower. Implementation - Short term (Phase 10): Intervention Rollout 27 Back to section overview Use Case 1 Use Case 2 Use Case 2 Use Case 4 Use Case 3 Communication For Mobile M&E Synchronous Parental Content Delivery Synchronous Teacher Coordination Instruction Instruction Setup Survey a representative sample of Pick your mobile M&E medium based on Survey parents to understand their literacy Analyze capacity assessment data that you Based on the capacity & needs parents / teachers / principals/ Phase 2&3 data (user needs & capacity levels, digital literacy, most common collected on your students to understand 1) assessment data (digital literacy, students to identify the regularity, assessment), ie. choose among USSD/ SMS mobile apps used, and how much time in a how many minutes/day on avg. students internet access, phone ownership etc) format & topic they want for /voice calls / IVR / IMs etc. Create day they can and want to devote to have access to phone 2) with what internet you have on students & teachers, pick communication. Pick your mass protocols for data collection, analysis & conducting educational activities with their speed & bandwidth 3) what their digital the tools that will provide most communication medium - voice reporting. Common actionable KPIs: child. Pick your mobile delivery mode for literacy is, 4) whether access is via feature or equitable access or create sub-segments calls, SMS, VoIP, IM, file sharing - activity (usage of solution); learning instructions/ activities that are to be sent to smartphone. Accordingly pick content of students by tool/device. Provide based on users’ capacity & needs. outcomes; early warning indicators for parents based on survey insights. Invite avenue (SMS, IVR, IM, website, LMS, etc). standards on distance grading practices Set fixed times for outreach to students at risk of dropout / not graduating caregivers to share evidence of their child’s Make lessons short given likely limited & update or simplify assessment rubrics create a routine for users & (attendance, (mental) health, homework learning, e.g. by sending a video / picture phone access. to reflect the current situation. maximize uptake. submissions, GPA, etc) /audio. Procurement Start with simple, human-led voice View checklist here. Key considerations: Leverage mobile communication channels Pick content authoring tool & choose For video-conferencing tools, consider calls & SMS, then procure mass Does your internal policy allow data to be that parents are already comfortable with, 1) required bandwidth, 2) required content source based on content rubric, messaging/IVR/ chatbot platforms. stored on external servers (ie. cloud) or not e.g. Whatsapp, rather than asking to install internet speed (2G, 3G, 4G, LTE, internal capacity & urgency: 1) existing in- Content should be developed in- due to sensitivity? Which survey answer new apps. Procure existing activities from broadband, dial-up), 3) participant county content, 2) OER, 3) free online house, regarding: for all: distance types need to be supported? Do you need other organizations; activities just need to number, 4) cybersecurity, 5) data policy content, 4) content from proprietary learning info, health advice, digital real time synchronization from the be implementable around the home, with 6) pricing, 7) features & relevant format publishers, 5) developing content in-house. access initiatives, financial support, field/field office? In what formats do you minimal parental effort, with an element of options (webcast, webconference, Local existing content is preferable. Hire responsibilities, salary/fee payment, need the data to be exported (Excel, fun & which can be shared in 64-character and/or webinar) 8) ease of UX, 9) quality instructional designers & Subject Matter FAQs, mental health; for teachers: STATA, etc)? What should the data limitations of an SMS. Universal Design for of technical customer support, 9) tool experts (SMEs). Design for universal access; instructional & training resources; dashboard look like? Which language(s) do minorities (language, location, SES, etc) & compatibility with relevant OS (likely e.g. audio, text-to-speech. Include non- for parents: learning goals, grades, you need supported? Which admin-portal low-literacy parents. In PR, highlight Android), 10) Vendor stability & cognitive content (meditation sessions, absences & missed assignment, skill features do you need & should they be importance of parental engagement. references from past clients digital safety content) & analogue activities. gaps & learning successes role-based? Weekday evening calls tend to be the best Share activities via SMS (How To Guide) & Always pilot & validate content with a Train teachers in the tool(s) in 2-day Deployment Standardize message structures, to get a high response rate. To keep up a small sample prior to large-scale workshops/month & cover where to find topic range, regularity & which Whatsapp. Encourage parents to conduct groups of contacts each type of response rate over months, consider learning activities via social media hashtags deployment to make final tweaks as shareable MoE resources/OER & how to message is sent out to. 1/week is a introducing mobile money based where they can share posts of them necessary. Ensure that content approval conduct formative tests using SMS/ minimum. Put QA processes in place incentives or vouchers. If number conducting activities(e.g. #LearningAtHome). procedures are fast if they require the Whatsapp/ voice (notes) & attitudinal for development, approval and database is incomplete, consider random For ECCE, ideally share activities as videos MoE’s approval. Put in place tight feedback barriers (growth mindset). Clarify remote send-off of messages. Include M&E number dialing, using a quick screening where teacher models the exercise. Share a mechanisms from students, teachers & teaching M&E/ teacher performance mechanisms (how many people questionnaire to identify whether the PDF Guidebook to enhance overall parental parents on content quality & relevance to structures/incentives Provide teacher FAQ “read” an IM, etc). Establish “office respondent is from a relevant group. engagement/ family relationships & reduce improve content constantly and leverage & hotlines. Share behavioral norms with hours” for educator check-ins violence at home in these stressful times. assessments too. Optimize costs over time. students & parents for video conferences. Implementation - Long-term (Phase 11-14): Sustainable Iteration 28 Back to section overview Phase 11 Phase 12 Phase 13 Phase 14 Ongoing internal & external Ongoing Capacity Building Hybrid Reopening of schools Post-Covid: The New Normal KIX, L&D and agile Iteration Objective The mobile intervention is constantly improved Capacity of stakeholders continues to be built Mobile interventions are smoothly adapted / Due to the increasing ubiquity of mobile phones over time based on data-driven insights on the fly, throughout. Human capacity is built transitioned into (post-peak!) reopening rather & internet access as well as the Covid-19 documented from formal M&E & stakeholder via internal & external upskilling, technological than discarded. Given necessary strategies for behavioral shift towards digital learning, mobile progress review meetings and Knowledge via continuous upgrades of software, hardware Covid (split schedules/class rotation to reduce education strategies become an integral part of Innovation Exchange (KIX) structures. Curation and general IT; informational via improved class sizes & need for remediation), the urgency “normal” education&enhance it. Personalization, of & access to lessons learned is UX-friendly; communication structures; etc. Schools / for mobile 2-way communication & M&E, communication, authentication, dynamic how to use database is clearly communicated individuals that are succeeding get recognized, personalized mobile instruction remains. Users’ content creation & delivery for education are across stakeholders. those that aren’t get extra support. behavioral shift is capitalized on. maximized thanks to mobile. Integrate in policies. Agile iteration is a constant cycle of Regularly update your capacity mapping (every Design a hybrid learning model (distance & in- Do a stocktake of effective mobile strategies Setup improvement in products / services / policy 6 months); this incl. defining desired vision, person instruction) which compensates for lost acquired during school closures & do a capacity based on prototyping & testing insights rather outcomes and KPIs together with stakeholders, instructional time (remediation), enables and needs assessment to evaluate the extent to than unachievable perfect information at the and mapping across all stakeholders, capacity smooth class rotation of students, frees up which - and how fast - additional layers of mobile outset.: 1) Define Initial Strategy via latest data areas (technological, human, political, legal, etc) teachers from admin, PD to strengthen teacher solutions can be integrated. Take into 2) Define Solution Requirements 3) Develop and spheres/levels (system, org., individual/ capacity to work with blended digital consideration again all the possible mobile use Prototype/MVP, 4) Internal & user testing, 5) community level). Have an M&E system in place pedagogy. Content must include knowledge on cases listed in this deck, look at your context’s Bug fixing & 1st iteration via test insights, 6) to track progress in capacity development (CD). disease transmission & prevention. Use mobiles CAGR in access & affordability rates & think about Wider testing, 7) 2nd iteration based on test2 Assess the incentives for stakeholders that will to further strengthen communication & what students & workforce need to survive in the insights, 8) Beta Field Release / soft launch to ensure follow-through of CD. Update capacity coordination mechanisms that promote 21st century. Consider investing in custom smaller segment, 9) Alpha release to all & building interventions accordingly. engagement with communities. software (takes ~1yr development, 6mo. piloting). scaling, 10) Start again Start designing your system around 21st century Execution Maximize Knowledge & Innovation Exchange Be multimodal in your capacity building. Formal Continue as many mobile uses cases as pedagogies by leveraging mobile: flipped by documenting & sharing best practice & case training, coaching, mentoring, peer exchange possible, adapted for the partial in-school classrooms, blended learning, project-based study files, hosting/sharing webinars, creating networks, resource hubs, institutional twinning, context. Continue content delivery & learning, inquiry-based learning, student-driven collective databases & collaborating to create consultative support, financial injections, virtual synchronous remote instruction (AI/teacher) for learning, etc. Leverage mobile apps to expand upskilling courses (across countries). Have a conferences - they can all be mixed and matched remedial purposes. Continue communication access via mobile to 21st subject topics that are single, user-friendly, searchable software where to meet the moment. Have an assigned capacity for coordination to keep parent participation hard to make time for during standard school you store all your knowledge. Identify topics of development workgroup in your organization. levels high. Continue remote assessments and days (e.g. life skills, socio-emotional learning, interest for KIX via regular stakeholder surveys. Review CD M&E indicators regularly. general M&E to track behaviors, best practices, entrepreneurship, coding, digital literacy, global Foster a knowledge-sharing,agile culture; ensure learning outcomes and identify students at risk citizenship, etc) & to free up teachers from daily workflows have built-in mechanisms for Regularly advocate for CD. Share success stories of dropping out or repeating the year. repetitive admin tasks (attendance taking, basic easily sharing ideas & information. to ensure it‘s top of mind for stakeholders. Encourage content creation among teachers grading, salary pickup when mobile money is an Engage the media in covering efforts, to further and students to maximize OER. option instead etc), or to improve M&E (e.g. encourage & recognize stakeholder efforts. better dropout warning systems, attendance, etc) Implementation - Phase 13: Hybrid Reopening (in-depth) 29 Hybrid Learning is the only option Challenges to address upon reopening Hybrid learning means that 25-75% of traditional face-to-face time is replaced by digital 1. A widened achievement gap not necessarily along not necessarily geographically or school instruction. A combination of in-school and at-home distance learning, rather than a full type specific ways, but likely along SES and depending on whether the family was directly return to face to face classes, will have to be the required reopening strategy for multiple affected by Covid or not. Given schools’ varied capacity in deploying distance learning, reasons: some students will come back to school having experienced little to no distance learning. 1. Reopening of schools will likely be staggered, resulting in students still 2. Rampant mental health issues: A study of 2,300 Chinese elementary school students found depending on distance learning solutions part-time. starting with the younger K- that 23% reported having depressive symptoms during the shutdown, a 35% jump from the 8 students (since they require more face-to-face time to learn effectively) and norm. A recent poll of American teens found that more than 20% report feeling involving smaller class sizes via rotating class schedules (half-day or half- disconnected from their school community, 25% reported that they lost sleep due to worry, week,with either shortened or extended school days until 5pm), possibly even and 30% that they feel unhappy or depressed. And PTSD seems prevalent, too. a school year extension oto still to still achieve sufficient instructional hours; desks spaced six feet apart, separated by plexiglass while teachers rotate 3. Continued digital divides: Given a continued need for distance learning, In the ideal case, between classrooms & staggered drop-off and pick-up times. by the beginning of the school year, all students should have the device and connectivity they need to access learning at home. Until that is accomplished, print learning packets will 2. You might face limited in-school staff, especially if a large proportion of your be necessary to bridge the divide. teaching force is over 50, that segment might not be able to return to the classroom safely, as well as educators with pre-existing conditions might have 4. Overwhelmed teachers: Students’ learning loss and its high variation will demand an to remain at home and teach remotely. Similarly, at-risk students and extensive amount of highly personalized remediation. Plus, teacher and admin staff layoffs administrators should not return to school. caused by budget crunches combined with at-risk teachers staying at home will worsen student-teacher ratios. Teachers might need to teach wearing PPE and disinfect regularly, 3. Localized, 14-to-28-day rolling closures triggered by new outbreaks could inhibiting their focus which will already be limited by stress and burnout. cause full shifts back to distance learning. In Tel Aviv, schools had to shut down again due to a renewed outbreak just after reopening. 5. A need to simplify curriculum. Given the school reopening context limiting instructional time due to likely involving reduced staff, massive variation in student learning losses, time 4. Due to variations in learning loss, you will need an adaptive, personalised lost to additional health protection procedures, limited classroom time for students and system that can base student progression on demonstrated mastery of instability caused by infection resurgences, it’s absolutely crucial to focus on the basics and competencies, rather than on seat time. cut out from the curriculum elements that aren’t essential. Plus, PE & SEL will have to take up more time in the curriculum to ensure that students’ mental health is addressed. 6. Flexible procedures: Given the dynamic situation, clear protocols are needed for all kinds of scenarios: from school reopening to temporary school closures, all requiring fast communication structures & ways to practice protocols. Back to section overview Implementation - Phase 13: Hybrid Reopening (in-depth) 30 01: Needs & Capacity Assessment 02: Communication for Coordination 03: Assessment - Interim 04: Content Delivery - Mental Health 05: Content delivery - Remedial → Assess students’ and teachers’ social, → Clearly announce reopening Prior to reopening, conduct a student Prior to & throughout resuming school, → Reduce the mandatory curriculum for emotional, and mental health after this dates & how reopened schools will benchmark assessment to understand extent & address students’ mental well-being, the academic year to only core topics, as period of isolation. Student and teacher variation of learning losses: else their ability to learn will be most of the year will go into remediation / function via SMS, IVR, voice calls, stress may mutually reinforce each other. impaired. catch-up programs/ALPs for missed content IMs to principals, parents (also see → Send out clear instructions to parents on & mental health recovery. Inform teachers. → Identify the most vulnerable students 1, 2, 3, 4, 5), min. 1 month prior. the assessment date & ideal setup at home → Send out simple mental health (homeless, disabled, affected by COVID-19 Announce benchmark assessments, (quiet room, access to phone, etc) advice, reminders & activities via SMS → Inform teachers & principals about the through family death / hospitalization, remediation systems, instructional / to parents & teachers; gather feedback ‘real’ grade levels of their students and offline) & prioritize them in support mental health / financial / health → Conduct a short oral student benchmark on their usefulness to iterate their share ability grouping instructions. Follow- efforts support structures, and key hotlines assessment using IVR (15 mins max.) for core design. Provide resources to deal with up with principals/community leaders via subjects like math, language, and STEM, and a anxiety, isolation, grief and trauma. SMS/calls to confirm that ability grouping / FAQ websites for parents & → Check-in with parents via phone calls / short SMS-based live, timed student quiz using Share guidelines with parents on how has taken place. admins a mass messaging platform as proxy for a to protect children’s as well as their IVR to assess how much learning support they were able to provide to their child written assessment own mental health. Making children → Prepare a prioritized & sequenced (regularity & quality) and listen to their → Announce in relevant languages feel safe, regular play, enough sleep, digital content repository containing small concerns (be it academic, health-wise, and both in text and audio to cater → Categorize students in a database into their limiting exposure to news (2 times/ group instruction materials; static content financial, etc) to evaluate their capacity to to illiterate parents / caregivers \ real grade levels; combine that data with day), family harmony & exercise are all (SMS-length activities & lesson plans, digital support their child during remedial efforts additional M&E data about students’ helpful for reducing stress. worksheets, audio & video files) & /parents’/teachers’ extent of usage of distance interactive content (IVR, apps, browser- → Explain how hybrid learning → Check-in on UX with deployed distance learning to inform remediation strategy → Set up mental health hotlines (1-on- based quizzes & MOOCs, etc). Design works & communicate 1 counselors) & IVR solutions for learning solutions - update your data on content sufficient for 2 months, then use their access, usability, and effectiveness to expectations for what (measurable) → Conduct interim assessments every 1-2 teachers & students & parents. Ideally feedback to improve the next round inform remedial / hybrid learning strategy learning targets are to be met under months after school reopening; provide have mental health counselors check-in these unusual circumstances, and benchmark targets to teachers & parents (e.g. with students once every two weeks. → Share content via SMS, IVR, IMs or → Internally, check if you have experts in what role parents / caregivers, words/minute). Tests should be administered offline P2P sharing apps to community mental health & instructional designers teachers, community leaders play in by the teacher in-class. Tests should be short → Set up SMS- and IM-based peer leaders/ teachers/parents /principals. with expertise in remedial instruction, (half-page) paper-based benchmark group chats for students, parents and ensuring that students reach those mental health and socio-emotional assessments and instructions for teachers teachers and regularly share prompts → Explain tiered instruction to teachers targets simple; then make teachers share results via to make them discuss their mental learning, as that content will be crucial for and to adjust ability grouping every 2 addressing student learning losses. Hire SMS to one centralized number and enter the wellbeing challenges. months when benchmark results are out. short-term consultants to fill gaps in → Manage expectations; make results into the central student database. e.g. if a grade 3 student fell to grade 2, he expertise if necessary. Consult with other clear that operations might not be → Include mental health indicators in gets grade 2 content until the 2nd countries and organizations that have smooth from the start, and that → Report results of interim assessments to your M&E and conduct regular SMS- benchmark says he‘s reached grade 3, then already deployed comparable solutions to constant feedback will be crucial for parents via SMS so that parents know when to based mental health check-ins with content should be grade 3 again. see if content can be shared and localized. increase their support if needed (or to feel students success of the intervention rewarded/proud when the student improves). Back to section overview Implementation - Phase 13: Hybrid Reopening (in-depth) 31 Back to section overview 06: Synchronous Teacher instruction - 07: Synchronous Parental 08: Assessments - Formative 09: P2P-collaboration 10: M&E Hybrid / Remedial instruction - Remedial To rapidly fill learning gaps, students need → Design low-resource Hybrid Learning → Design low-resource remedial Leverage mobile for peer support to reduce → Leverage mobile to gather data via SMS or regular, useful feedback on learning & (likely scripted) lesson plans based on activities which parents can the burden on teachers & administrators. check-in calls from teachers / principals on key teachers need to adapt instruction. Formative reduced curriculum; this ensures that conduct at home with their KPIs, e.g. 1) Outcomes: a) % of students below assessments combined with tiered instruction teachers are clear on how to keep a children; share them both via SMS → Pair students (grade 3 upwards) who have grade-level according to latest benchmark enable that. continuity between what happens in the & IVR (to cater to illiterate parents) access to a mobile phone for peer learning assessments, b) % of students who moved up by a classroom & during distance learning. and make teachers share peer exercises / grade level according to latest benchmark → Design low-resource formative → Parent-led activities need to be prompts for peer collaboration via SMS/IM; assessment, c) % of students who made learning assessments for each grade that can be → Design remedial lesson plans (low- designed for varying student establish regularity and timing of peer progress since last benchmark assessment; 2) Early shared with teachers via SMS / as part of resource) which assume that students learning loss, and then activities interaction (e.g. 2x/week for 30mins). Warning Indicators: a) % of students in a school Lesson Plan PDFs via IMs. Provide a good mix have been grouped by ability in-class or matching the grade level that the Encourage students to help each other with with low attendance, b) % of teachers with low of CFUs, exit slips, short essay topics, quizzes, across classes student receded to due to learning assignment questions, both via SMS and in- attendance, c) % of students with low mental well- self-& peer assessments, etc; for pre-, during loss (based on info gathered during person during school hours.Explicitly tell being, d) % of students who are stagnant or still & post-class. → Share these lesson plans weekly via the original needs assessment); students that message exchange should be regress in their learning after 2 benchmark SMS/IM with teachers/via a browser- content should be aligned with the limited to academic purposes to prevent assessments. 3) Activities: a) % of teachers who → Provide simple decision trees for teachers based LMS; if possible & appropriate, remediation content used by excessive charges. Inform teachers on how participate in ongoing training on ability grouping & for which resources from the content provide teachers with zero-rated access teachers; while there is no and when to shuffle student pairing. Try out remedial instruction, b) % of principals, admins & repository to give to students based on to MoE-compiled content repository guarantee that all parents will be both cross-age and same-age peer systems, as SMCs trained to support & monitor teachers. formative test results. websites. willing and able to support their well as groups of up to 5 peers. child, those who do will help reduce → Ensure that sufficient staff is available to → Make it mandatory for teachers to share → Provide hotlines, group chats & the burden on teachers. → Create local mobile peer communities of analyze data & to conduct follow-up calls with the results from formative assessments in coaching voice calls to support teachers practice which can support each other on an underperforming schools. Leverage community SMS format to one government number. in multiple modalities for adoption of → Provide guidelines to parents on academic & mental health level, as students, leaders and SMCs (if in place ) to do in-person lesson plans; incl. mental health check- how to structure their children’s parents, teachers & admins currently face follow-up, incl. school visits to observe teachers. → Ensure that teachers follow basic ins day; children should continue to similar struggles. Limit parent / teacher / formative assessment best practices: 1) have a regular school day routine at student groups to 50 members each; provide → Reward & support. Publicly recognize the schools clarify learning objectives & rationale to → Text parents reminders on home on the days when students codes of conduct for groups to prevent /teachers who are showing high fidelity students, 2) explain to students how to gauge cybersecurity rules for the mobile aren’t in school, ie. regular start & message flooding. Provide guidance on which implementation and/or manage to produce results. progress, 3) ask high quality follow-up distance learning component. finish times, regular recess, & type of information should/should not be Similarly recognize students & parents. Give extra questions to the assessment to identify consistent 45-min. learning slots for shared in groups; encourage sharing of text- support to those who struggle, via training, financial student misconceptions or redirect their → Make principals check in min. 1/week the students. only information to minimize data or infrastructural support. thinking, 4) if necessary, re-explain by with each teacher. In Peru, principals consumption. If you can’t zero-rate access, presenting concepts in a new way (different discuss: 1) ways to evaluate student choose SMS-based groups over IMs, as the → Identify best practices. Identify what schools format, etc), 5) share feedback that makes participation & performance, 2) status latter consume precious data. Assign 1 who are steadily succeeding are doing and share students reflect on performance & provides updates on student work completed, 3) moderator/group. Use group messaging tools their practices widely, since those practices will be next steps, 6) give students a 2nd chance to support for development of activities. that hide member numbers/ images. relevant to your particular context. demonstrate success (in other format). Implementation - Phase 13: Reopening: Adaptive Remediation 32 Back to section overview Numerous countries are considering Adaptive Evidence Review ALS on Mobiles / Tablets at home (self-directed) Conditions for Success Learning Systems to remedy the vast differences in learning losses. These slides Pedagogical interventions that tailor teaching to Tablet: In Tanzania, 100 students used Kitkit School’s Success: In the short-term, successful remediation despite provide an intro. Relevance varies by context. student learning levels - either teacher-led or numeracy and literacy tablet app over 3 months in 2 varied learning losses is success; however over time, ALS should ? facilitated by ALS - are effective at improving student test groups: 1) 6-10-year-olds in a community center, 2) enable enhanced equity and changing the role of teachers from test scores, as is individualized, repeated teacher a treatment group of Grade 1-3 students in a rural imparting knowledge (which can be done mostly by ALS) to Adaptive Learning systems / software (ALS) / training often associated with a specific task or tool primary school. The out-of-school children’s post-test teaching higher-order, applied skills. ALS can teach basic content technology (ALT) guides a student through (Evans & Popova 2016). The pooled effect size scores became comparable to the baseline scores of the pre-class, provide remedial instruction post-class & monitor instruction, remediation & assessments that are associated with adaptive instruction is 0.42 standard in-school group: children in-school averaged 53% on the student progress, freeing teachers up to provide targeted continuously calibrated to each student’s deviation, while that of programs with non-adaptive literacy and 48% on the math baseline test, while out- support, and do more project-/inquiry-based learning in-class. evolving proficiency level. The ALS dynamically instruction is about one-quarter that, at only 0.12 SD. of-school children achieved a 52% average on the adjusts the learning experience in real time based literacy post-test, and a 48% average on the math post- on data points created by students’ interaction ALS on computers in After-School Centers test. On average, they showed a 15% improvement in with it. This data is processed by an algorithm that literacy scores and 20% improvement in math. In alters instructional goals, content delivery Learning gains using an ALS (Mindspark) for students Rwanda, nearly 700 primary school students improved systems, or curriculum, usually keeping students (grade 6-8) in after-school centers was at least 2x as their literacy in 3 months via 30-mins of KitKit each in their Zone of Proximal Development. much as students in the comparison group. 90 days of weekday (voluntary). In Kenya, refugee students used attendance (equivalent of half a school year at 80% KitKit for 30 minutes/day for 40 sessions throughout 8 The types of algorithms are (simple to advanced): attendance) would deliver 0.59 standard deviations of weeks and saw improvements of 32-46% in literacy and Decision Tree ALS → Rules-Based ALS → Advanced learning in math and 0.37 in Hindi (avg. attendance 14-36% in numeracy. nSimilarly, Onebillion’s OneCourse Algorithm ALS → Machine-Learning-Based ALS among lottery winners was 58% (~50 days) The RCT created learning gains in Brazil, Malawi (incl. special was conducted with 619 students from public middle needs students) & Tanzania. The sophistication of so-called ‘adaptive’ learning schools in low-income neighborhoods in Delhi; systems varies significantly. It is hence important student proficiency was highly heterogeneous, Mobile: Homer, With just 15 minutes of practice a day, Conditions to analyze in detail what exactly a vendor means spanning 5-6 grade levels in each grade. six 90-minute HOMER increased early reading scores among low- Human: when they say that their product is ‘adaptive’. A slots per week outside of school hours 45 mins were income 4-5-year-olds by 74% (Test of Preschool Early 1. Stakeholder buy-in & shift towards innovation rudimentary system e.g. only provides one computer-based instruction (a mix of math, Hindi & Literacy TOPEL); improvements showed especially in 2. Sufficient digital infrastructure in-school/at home benchmark assessment at the beginning and English across days); the other 45mins were print knowledge, phonological awareness & letter 3. Ongoing staff training & professional development hence works with a static learner profile, rather instructor-led in groups of 12-15 students, providing sounds. The randomized, 6-week study sampled 95 4. Emotional, financial support for students/ teachers than constantly update that profile throughout homework completion help, preparation for school disadvantaged (eligible for free/reduced lunch) students 5. Clear expectations management & communication usage; it also often adapts just on right or wrong exams and instruction on topics of broad relevance. across 7 Head Start classrooms in Brooklyn, who mostly 6. Incentives for students to complete course answers. More sophisticated algorithms adapt The cost per student was ~US$3/month. The ALS had little or no previous exposure to touchscreen 7. Sufficient technical expertise & coordinating ability based on the specific (false) answer chosen, since enabled to cater the varying proficiencies seamlessly; devices (Neuman, 2014). in implementing agent (government / organisation) each answer choice usually represents a where group-based instruction has had any impacts in ALS: misconception and 2 students at the same level the past, it required making student groups more Other: In an ALEKS pilot in Ecuador 800 students 1. Alignment of content to curriculum & exams may be stuck for different reasons. They consider homogeneous (ability grouping for remedial work / increased curriculum knowledge by 10%/month (vs. US 2. Quick & direct feedback on student performance factors incl. time to complete exercises, past re-grouping classes). Gains in math scores were seen results). i-Ready created gains of 38-46% in math & ELA 3. 24/7 Technical support services performance of students with a similar learner on below grade-level questions (which is what the CAL in grades K-8 (sample of 860,000 students). Knewton at 4. structured curriculum, with clear goals & real-life profile, or learning style preference (if students software taught), not on grade-level questions (which ASU improved college course completion. applications based on students’ interests perform better by watching a video vs reading weren’t taught by the CAL software). 5. Supports diverse learning (individual/ peer/team) some text) to optimize content adaptation. Implementation - Phase 13: Reopening: Adaptive Remediation 33 Implementation Steps How to procure ALS? Back to section overview What adaptive learning software exists out there? View extensive guide here & implementation case studies here. Send out an RFI (Request for Information). See an extensive sample ALS RFI here and a framework to evaluate courseware 1. Assess Need: validate rationale with teachers, parents, Subject Level Smartphone App Browser-based (with content) here. Some basic questions to ask are: students & regional education officials; is an ALS really what they need to progress in school / employability? 1. Content: Does the platform come with preloaded content? If Math ECCE ABCMouse, ToDoMath, ALEKS, Pearson Success Maker 2. Assess Capacity: Evaluate technical infrastructure, student yes, is the content supposed to be a study aid, provide Hatch /parent / teacher digital literacy, staff’s attitude toward supplemental instruction, or is it a whole course? Can the innovation, regional governmental capacity to coordinate algorithm be layered onto our existing content? implementation & maintain IT, etc. 1-12 ST Math, Byju’s, Mindspark, ALEKS, LearnBop, 3. Choose Learning Modality: Will a school-based, after-school or 2. Student UX: Ease of navigation? How advanced is the Mathspace, Mangahigh, Imagine Math, Dreambox, Redbird home-based ALS (or both) be best-suited? If school-based, will feedback a student gets for right/wrong answers (none, text/ Levered Advanced Learning, Pearson you use a station rotation or lab model? img/ video)? What format do hints have (text, image, video, Success Maker, KnowRe, Khan 4. Evaluate Cost-effectiveness: Mindspark in Delhi cost live tutors)? Does it have accessibility features? Academy $3/child/month, but costs can be higher. Main cost drivers are on-the-ground & remote support as well as hardware 3. Infrastructure: Does it work offline (with periodic sync)? Is purchases, servers, cloud services, and maintaining IT systems. Higher Ed Scootpad Pearson MyLab there a mobile app? How much bandwidth does this ALS Compare that to take-home print materials, etc. consume? Any LTI integrations? API availability? 5. Ensure buy-in: Install/reinforce incentives for instructors to participate & change via training from pure content delivery & Literac ECCE Homer, Hatch, OneCourse Lexia 4. Customization: Is the knowledge map accessible so that one seat-time-based learning to targeted support & competency- y can verify & edit it to one’s liking? e.g. intro to biology can be based learning; assign responsibilities; sensitize communities taught from macro to micro (from biomes to DNA) or vice 6. Carefully select ALS: Choose ALS based on rationale for ALS, 1-12 Homer, Scootpad, i-Ready, Lexia Redbird Advanced Learning, versa; depends on instructor/curriculum. How customizable is infrastructural limitations, budget, HR capacity etc iStation (ELL), Freadom Pearson Success Maker the content (some configurability; off-shelf course content / 7. Design intervention: Design pilot together with teachers, authoring offered as service / Open authoring platform)? parents, IT teams, vendor, regional educational officials, How much (if any) control does a teacher have over content? community leaders & headmasters from the start; interview all STEM 1-12 Ck-12 Platform, Byju’s, stakeholders about concerns, goals, etc Tappity 5. Support: How does onboarding and setup work? How big is 8. Capacity ramp up: install solar charging stations for in-school / the team on the provider’s end that would be at disposal for take-home devices; distribute smartphones via Field Assistants teacher training, parent questions, student support? Do they on motorbikes; train staff (incl. using OER); etc Higher Ed Pearson Mastering provide ongoing support beyond setup? Cybersecurity? 9. Pilot ALS: Define KPIs for pilot success; test for human and technical capacity issues; document lessons learned. Ensure 6. Analytics: What does the student data analytics page look sufficient time on task for students. Ensure that teachers check Various WileyPlus, Desire2Learn, Waggle, McGrawHill Connect, like for students vs admins vs parents? student analytics boards min. 1/week. Cengage MindTap, Acrobatiq, Realizeit, SmartSparrow, 10. Iteration, Scaling and M&E: Track student ALS usage & Knewton Alta Cogbooks, Fulcrum Labs, Sapling, 7. Adaptivity: Is just the instructional or the assessment assessment performance; teacher engagement with analytics Course Connect, Kidaptive, Cerego, component internally adaptive, or both? How regularly is the & ability grouping / personalization in-class Waymaker (OER), EdReady student assessed and the content adapted? 11. If early results are promising, continue to advocate for ALS & celebrate small successes to maximize motivation of staff 8. Effectiveness: Any effectiveness studies/ case studies? Risks & Challenges to account for in each Use Case 34 Back to section overview Synchronous Instruction Assessments General Risks Content Delivery Monitoring & Evaluation (Teacher) Cybersecurity is a standard risk if students With mobile M&E, there is always a risk Safety: During school closures, children’s If the state doesn’t provide continuation Summative, high-stakes assessments go online to access content; protecting of selection bias, as not everyone has exposure to harm is increased, reducing of salaries, some teachers may move require ideally equal access to testing personal data & ensuring internet access is access to a mobile phone and the digital their mental well-being & ability to focus on from their posts to their hometowns or software & an equally peaceful limited to educational content is crucial, as literacy to navigate an m-survey; multi- academics; from malnutrition (given lack of seek other work. assessment environment to enable the much as protecting platforms from hacking. modal mobile M&E (IVR, SMS, etc) can school meals), to risks of abuse/ FGM/ child to focus; having a clear PR & access reduce this bias. Low response rates are pregnancy, and child labor/early marriage Teachers might lack motivation to learn strategy in place is crucial to ensure that System overload leading to an inability to another risk. to supplement family’s crisis-reduced the skills needed to navigate the digital students have equal chances during high- income. Ministries of Education, health, deliver content is a large risk; large file tools needed to teach remotely; using stakes exams. sizes of content can clash with bandwidth Data Security is a major issue, especially if gender, social protection, etc. have to tools that they already use on a regular limitations, especially when combined with you involve Third Party Monitors, and cooperate and provide stipends. basis (e.g. Whatsapp) can reduce the Online assessment systems might break the surge in mobile network activity (China requires strict legal contracts, incl. data barrier to behavioral change, as well as down due to system overload when so & Italy saw a surge of 70% as a result of sharing consent practices. Human: MNOs might not cooperate for micro-certificates linked to career many students login simultaneously; zero-rating, and are usually reluctant to Covid, which no mobile operator would progression to incentivize course taking. accordingly capacity has to be built up in plan for, causing internet shutdowns); Siloed data & interoperability risk; zero-rate “heavy” content like videos; focus advance and backup systems/contingency including a request to the public in PR to standardization & communication to on low bandwidth solutions. Governmental Not engaging teachers soon enough in plans put in place, plus communication use internet responsibly and mainly for prevent different definitions /indicators/ capacity & institutional path dependence the crisis can lead to a sense of lethargy structures to tell users via SMS when the educational purposes is hence crucial, as datasets is crucial. might seriously inhibit ability for agile and irrelevance, with teachers not getting system will be back up again. response. Schools might have to conduct well as compressing files. an active role soon enough, and just e.g. Sensitive, personal survey questions can layoffs to save money. Parents are worried passing on content; it’s important to Cheating is of course a real risk; exam If not universally designed, content will create distrust in users and reduce about their livelihoods & hence have engage them fast and motivate them. proctoring softwares can help address exacerbate equity gaps; e.g. by being participation / usage. limited capacity to support their child’s that issue, but often require smartphones learning. Teachers/principals are faced unilingual, or not offering adjustments to Communication (fingerprint scan, video/photo, etc) for special needs students. Furthermore, a Staff’s availability to cope with the steep with countless questions from parents/ verification. Formative assessments are hastily put together, bad online learning Parents & teachers might perceive calls learning curve of learning, installing, students. Unequal access to mobile phones lower risk, but require regular effort from experience could turn students off from and texts as spam at first. Hence announce testing & navigating M&E ICTs is a risk. - along Gender/rural-urban/ SES divides, etc teachers/MoE side to maintain & evolve. future online learning experiences; focusing on radio/TV when to expect outreach & can exacerbate inequalities if unaddressed. Synchronous Instruction Content Creation on quality of content and UX as much as how to distinguish spam from real content. Avoid inflated expectations; at such an continuity is hence crucial. (Parental) early stage hype can cause disappointment. Quality control is crucial to make sure that If messaging isn’t clear, parents won’t Given varying digital literacy and general content created on mobile is both safe Keep content holistic, not just academic. understand the why/how of engagement Financial: private & public school budgets literacy among parents, homeschooling and of high pedagogical value. Make content fun to maximize engagement, efforts and disengage. Parental Literacy & will be strained due to purchase of new could exacerbate learning differences and cover topics like mental health to language & self-confidence barriers also materials, software & bandwidth needed between students of different SES Peer2Peer Collaboration maximize focus and coping skills. need considerations. for distance education solutions, combined backgrounds; it is therefore crucial to keep with economic downturn. Parents might activities simple, low-resource, not very The risk of cyberbullying & tech-savvy Don’t make content entirely screen- Information that isn’t clear such as “nearly request their school fees to be refunded. time-consuming, and instructions need to students hacking the platform exists; given dependent. Create analogue content that meets standard” can mislead parents into public school budgets could be slashed Will be provided both via audio as well as in the current bigger crisis however this has screen-free activities, too. believing children are on track when they hence likely need a huge injection of cash. writing to not inhibit illiterate parents. should be minimal. are not. Actionable clear feedback is crucial. 4. Whom can we learn from? Click on any hyperlink to jump directly to the section. Back to main overview Case Studies - Overview 36 Pratham, one of India’s largest education NGOs at 10,000 staff, managed to pivot Viamo specializes in scalable feature phone engagement solutions using IVR, SMS, & scale its predominantly face-to-face programs in 5,000 villages to mobile IM bots, Apps & Web. Globally their feature phone engagement solutions reach distance learning in 11,000 villages just two weeks into the national quarantine. >100,000 people/day in Africa & Asia and >10 million people from 2012 to 2017. Their agile response and ability to repurpose staff, re-design their extensive They have implemented mobile education solutions across Africa, from content content for SMS and IVR delivery, and partner extensively with both the Indian delivery to assessments, coordination and M&E, with proven learning outcomes. state governments and other NGOs to further scale their reach, is worth learning One of their Covid response projects is an ECCE IVR pilot in Zimbabwe for 10,000 from. The case study covers the uses cases content delivery, parental students. Viamo’s ability to rapidly deploy solutions at scale - up to 9 million users - , synchronous instruction, communication for coordination, synchronous as well as their close partnership with MNOs and specialized expertise in feature instruction by teachers, and M&E; incl. feature phone and smartphone solutions. phone solutions make them an organization worth learning from. Peru’s MoE completed nearly 20,000 M&E surveys of parents, teachers and The Moe of the Nigerian State Edo managed to pivot its effective public in-school principals in just a few weeks into school closures and created a publicly initiative EdoBEST - conceived in 2018 together with Bridge Academies accessible M&E dashboard with the collected data, with less than 50 full-time International - to an EdoBEST@Home mobile distance education version to staff and an additional 300 call center staff hired. This, among other aspects, support over 20,000 primary school students at home, and is now being scaled helped better understand the usage and effectiveness of Peru’s Aprendo en Casa to all 250,000 Edo State students, incl. ECCE, secondary & private school students. Covid distance learning solution. The initiative is an example of an excellent public initiative to keep learning going, The MoE team responsible for this was incredibly agile in its ability to redesign as well as a long-term perspective on a continuous transition to mobile-asssisted processes for mobile M&E and shorten processes that usually take e.g. 3 months education even once schools reopen. Case study includes content delivery, to a week, without however foregoing too much QA in the process. They also have formative assessment, M&E and synchronous instruction by teachers / parents. plenty of practical advice to learn from. Back to section overview 37 Case Study - Lessons Learned: Pratham (India) Back to section overview Feature & Smartphones Content Delivery (static) Parental Instruction M&E Teacher Instruction Communication Capacity Repurpose staff flexibly & keep content simple: Video teams were asked to write SMS lessons; Level Grades 1-8, TVET Ramp-up directors to map curricula to more state boards; staff visiting villages now do M&E calls instead. Have an organization-wide call for tasks & give responsibility generously. Make 1 SMS cover 2-3 age groups. Org. Capacity 10,000 staff; founded 1994. Private, NGO. In March, they had a content repository of 4000 digital stories, 3600 videos & 300 HTML educational games (in 11 Indian languages), and plenty scripted Build Community Capacity: Take the saying “it takes a village to raise a child” literally; leverage lessons/activities. Had experience in IVR programs pre-crisis. Ran F2F programs in 3-5k villages. community members. They’ll also create youth leadership in villages to act as coordinators/mobilizers. Partner extensively (outside education silo, too): Shared their digital & SMS content with 14 state govs, Intervention(s) Content Delivery & synchronous instruction (by parents): For Grades 1-8, send daily activities as SMS 230 nonprofits; made content available on web platform DIKSHA (mobile app) & created guide for (165 characters) to parents’ phone which contain hands-on, home- or outdoor-based learning activities in language, math, English & science plus art, music & theatre; send text/ video/ audio via developing educational SMS content. Non-education orgs (SEWA) also a good way to access parents. Whatsapp; also set up an IVR+SMS program for nursery to Grade 8 for the Delhi government. Leverage Communities for rapid phone number collection: Within 2 weeks they identified community Synchronous Instruction (by Teachers): In TVET, shifted from a primarily center based model to an leaders/teachers/well-connected volunteers in each of the 11k villages and got their phone numbers; online instruction model. Also switched to multi-skilling since crisis makes job outlook unclear. those then collected the community numbers in 3 days (now have total of 85k numbers). Intervention Reaching children in 11,000 communities across 20 states in India (as of June 2020). In Delhi alone, Coordination Nudge parents via the power of imitation: a video of a parent doing one of Pratham’s activities with his ~600,000 students from nursery to grade 8 were sent activities over SMS or IVR on the first day. child went viral among the communities; parents then recorded themselves doing this activity & sharing Scale it. Leveraging social recognition effects can be more effective than just telling people what to do. Outcomes The ”some fun + learning” approach spurred much creativity, students sent e.g. 5,000 photos of a bangles creation activity (samples here & here & here). TVET webinars had 500 attendees. Send multi-age group SMS at the same time every day. Children begin to expect these messages and are in a learning mindset at the time. Challenges Transforming their content into SMS-length format, and make it accessible to communities via SMS Leverage Free SMS packages: kept SMS free of cost by making their points of contact get new SIM cards & Whatsapp. Plus people weren’t sharing devices due to social distancing. Cost which offered “first 100 SMS free”; POCs receive the content via Whatsapp, then distribute content via Reduction SMS (& Whatsapp) to community using SIM. If >100 SMS needed/person, find new person & new SIM. Needs Start with simple check-ins via phone: 1,500 staff called up TVET students when Covid hit to Assessment. check on each individual's’ situation - how they were feeling, their salary situation, etc. Incentivize volunteers with an “Education for Education” model: They reward volunteers for their support during this educational crisis with access to online educational content & MOOCs (TVET etc) Planning Be agile & iterative: They make quarterly & regionally varied plans to stay dynamic & relevant. Only Maximize hardware sharing: A village member started using loudspeakers on temples so that everyone took them 5 days to make IVR content for Delhi as they created only initial content & published. could listen to the voice notes. In M&E calls, they ask village volunteers to pass on their phone. At first, prioritize student engagement over academic outcomes: recognize the duress under which Have a simple but holistic question protocol. For SMS content, 4,000 staff touches base with parent min. students & parents are. Intentionally designed a program that was both “A little fun” and “a little M&E 1/week (14 calls/day/person). “Did you receive the activity? What will your child learn from this activity? study”; most students are glad to be out of school, parents have bigger concerns than academics What more do you want/need? How was your day / how are you feeling? Any challenges?” They also try to talk to the child. For IVR, child keeps a project book which teacher checks at reopening. Case Study - Sample Images of SMS Lessons by Pratham (India) 38 On the right you see sample SMS content (translated into English for the reader’s benefit) for Early Childhood. To view Pratham’s complete guide to developing SMS-based educational content, click here. In Early Childhood, Pratham emphasizes these six main developmental categories: Cognition, Physical, Creative, Socio-emotional, Early Language, & Early Math development. The aim is to stimulate a love for learning and design activities that are so fun the child forgets that it is learning something. Given the importance of parental involvement in early childhood activities, Pratham usually follows up with illiterate mothers via a phone call to explain the activity to them. Back to section overview Case Study - Sample Images of SMS Lessons by Pratham (India) 29 Sample SMS Activities - Lower Primary Types of & Sample SMS Activities - Upper Primary (Science) Back to section overview Case Study - Lessons Learned: Viamo (Zimbabwe/across Africa) 40 Back to section overview Feature Phones Content Delivery (interactive) Parental Instruction M&E Teacher Instruction Communication Level ECCE (and Primary, Secondary, TVET, Postsecondary, ABE, Professional Development) Needs Understand your users phone access, usage & edtech familiarity. Prior to launch, ask: “How frequently do assessment you access your phone? Have you used a phone for learning prior? if yes, did you enjoy it?” 200 staff; founded 2012. Social enterprise. Offer mobile technology services for Interactive Voice Org. Capacity Do your due diligence. Viamo does a connectivity test first, always, as well as a capacity analysis of their Recognition & SMS. An intervention can be designed & deployed in <3 weeks. Present in > 20 partners. Field testing of content before offering content at scale. markets in Africa & Asia; implemented mobile education solutions across Africa, from content delivery to assessments, coordination and M&E. Worked with the World Bank (DRC), UN, USAID, Capacity Ministries, etc. Content expertise is brought by clients or partner organisations; Viamo only has Bump up server capacity if you expect large call numbers. The capacity ceiling for Viamo is millions; 10 Ramp-up million was the maximum that they handled at once. They can always increase server capacity. technical expertise (turning curricula into IVR / SMS, voice recording, deployment via their technical infrastructure, translation, etc). Global reach is >100k people/day; >10 million people from 2012-17 Communicate clear info to users about the initiative to maximize uptake. The more sensitized users are to PR the fact that they will receive a call the higher uptake is. Also provide a clear audio intro for the call to Intervention(s) ECCE literacy & numeracy content delivery (in minority language Ndau) & formative assessments explain that this is a call by MoE (not a spam call) & that they should call their child over for the lesson. via IVR in Zimbabwe for rural students (5-year-olds) at 10mins/week; World Vision, Aktion Deutschland Hilft, Save The Children, ECHO & MoE are partners for curriculum development & Setup speed depends on technical infrastructure already in place. Viamo already had infrastructure set up coordination. Technical Intervention Zimbabwe: pilot phase with 10,000 students, potentially to be scaled up in the fall. Other projects Setup in Zimbabwe: an E1/SIP agreement with MNO; they just need access to a server, then software connects to Scale scaled to 200,000 callers in 1 year (Tanzania) or a volume of 9.5 million calls & SMS (Pakistan) Viamo server/cloud. Without that, it would take min. a few months to set up. VoIP is a way to instantly set things up but that’s a lot more costly. Viamo is unique in that they have in-country infrastructure. This project is only about to be deployed, hence no data yet; however in a past project in Zambia Outcomes (local language stories via SMS+IVR+ radio), they saw 1 grade level-improvement in oral proficiency Send out calls on varying times and days until people pick up. Viamo’s software does that automatically. Negotiate airtime costs with MNOs. Service is freely accessible for the users, cost to organizers depends on Challenges → This project is one of the first times that they are using IVR for ECCE; unsure about uptake levels Cost multiple factors: national airtime rates, number of users (because more users = more airtime), minutes per → Across projects, mobile access gender divide: usually the man owns the phone (60% male) Reduction user, duration of the project (Viamo’s fee is monthly ), number of interactions. Airtime usally the most Design IVR modules short & fun. They try to keep IVR to 5min-modules and assessments to max. underestimated cost. 3-2-1 is their cheapest service: it’s a one time fee (30msg content bundle = $40k) and Planning 20mins. Take into consideration tone of voice (e.g. serious vs fun). Ultimately, if users don’t enjoy then no airtime costs because MNOs cover the cost (as usually people then stay with that MNO). Cost pp. the content or find it useful, the intervention will fail. Gamify participation; e.g. when kids answer a can be from $2.5 at large scale to $30 due to smaller scale. Full project management & design fees:~$30k. lot of SMS questions correctly, they can win an automated call back from one of the show’s characters (users started averaging 72 questions/day as they wanted to win more phone calls). M&E Keep IVR formative assessments & retention surveys simple. e.g. a nursery rhyme about what sounds do animals make, and then they ask “what sounds does this animal make” so they check via these mini Consider Master Service Agreements for international projects. It speeds up the process as due assessments if students has listened and is still present. Satisfaction survey: e.g. “Were you happy with the diligence & pricing negotiations take place at the HQ level (they had an agreement with e.g.UNICEF) lesson? Would you like to hear more lessons?” Plan for enough time for content development and clarify data collection rules. Developed Create a UX-friendly, digital M&E dashboard. Dashboard of for client organizations shows how people are content usually has to be approved from MoE, which can be time-consuming. And not all ministries performing on retention surveys; demographic breakdown by region & income bracket, which stems from a are comfortable with the idea of data being collected on their students/parents/teachers. survey from the beginning, ie. self-reported data. Case Study - Lessons Learned: Ministry of Education, Peru 41 Back to section overview Feature Phones M&E Capacity Shorten & adapt existing processes. They adapted/shortened every single part of their process; e.g. they Level K-12 reduced QA duration but hired more QA staff, learned to do segmentation, sample design & data entry Ramp-up faster (in 3 days). Assembled 1st survey in just 1 week with the help of IPA (usually takes 3 months) Org. Capacity 50 staff from MoE (but not all full-time), an additional 330 people hired for M&E call center Don’t be frugal with your M&E staff training. They switched from 1-day training workshops (goals of Intervention(s) Voice-call-based mobile M&E surveys to better understand stakeholder needs & effectiveness of survey, tricky questions, FAQs, etc) to 2 days to prevent issues down the line. Started ensuring 1 day rest Peru’s Covid remote learning solutions, incl. TV & Radio and its online platform Aprendo En Casa between each new survey design to allow teams to mentally rest and develop better, thoughtful surveys. Intervention To date more than 20k surveys carried out, covering teachers, principals and families from both Partner across sectors. OSIPTEL (Supervisory Agency for Private Investment in Telecommunications) Scale public and private schools, with about 30% being from rural areas. helped with matching the ID numbers to current mobile number which ensured trustworthy contact data. Enseña Peru (TFA) helped with getting 422 volunteers. Regional & district education offices (DREs) helped They learned just how big the demand for info was. Satisfaction rate with the online platform was getting teachers principals & parents onboard. Non-profit association (IPA) helped design 1st survey. Outcomes higher than with TV, and that Whatsapp is the preferred medium across all users (sending homework, talking to students, etc) hence recommended that the MoE give training to teachers in Whatsapp for Distance Education. ⅔ of population access internet via data plans Technical Adapt existing tools as much as possible. They adapted their tools which were usually already used to Setup collect data for school inspections. "Semáforo Escuela" tool will be adapted to mobile format. Challenges → Lacked personnel to do enough surveys; trying to see if they can use families to help with data collection; either way they can’t survey everyone, especially not in remote areas Create an official number verification mechanism for users. Within gov. website you can verify that the → Non-response rate was 75%; they didn’t expect response rate to be so low number someone called you from is a legitimate gov. number. IT office in MoE helped set that up in 2 days. → Lots of scams through telephone calls started throughout this M&E time → Coordination with other ministries & internally; the MoE & its data collection office had a way of Keep questions simple. e.g. “How often has the teacher communicated with the student over the last doing things, was hard for them to adapt to this more agile situation; had to change mindset week?” “What things do you have access to in your house? (ie. internet, radio, mobile data network etc)” → Data security; they are conscious that they have a lot of info from a large number of people; (not even their national statistics institute has data on whether families have data plan or router access) even though they signed confidentiality agreements, they decided that they didn’t want to work with volunteers anymore in order to have better data control within the ministry Deploymen Expect 5 calling attempts until you get an answer. Conduct 5k calls/day to get ~1k to pick up phone. Survey t the parent most involved in the child’s education. They ask to speak to the parent most involved with the Needs Define decision-making KPIs. They asked MoE: “What numbers do you need to make decisions?” child’s education, to get reliable information; it’s hence often the women who they talk to. Each week, assessment survey different groups. e.g. this week they are surveying parents. Not every week is the same in terms of Planning Be open to adjust processes and criteria over time. They learned to reduce surveys with principals volume & response rates; e.g. when they survey urban groups they get more response. as those are overloaded with requests from stakeholders; now a stronger focus on teachers & families. They will not sample by school, so they can then verify if things that teachers are saying are Make your dashboard public: publicly accessible M&E dashboard facilitates data- and insights-sharing. true (since the’ll be able to match teachers to schools). In July they are going to go back to their Cost Don’t rely on volunteers unless you have to. They used volunteers for 1st round due to initial lack of regular data collection strategy called "Semáforo Escuela". They adjusted margins of error over Reduction budget; faced issues, volunteers weren’t eager to call each person 5x; once results proved M&E value, time; for normal M&E they have 5% but now they added a bit more (8.5%), to maintain 5% would gov. gave funding. Cost of M&E was $200k, however without salaries (330 staff in call center + 50 MoE). be too expensive since they would have to do a lot more surveys 42 Case Study - Sample Images of M&E Dashboard by MoE, Peru Back to section overview On the left you see the main sidebar menu of Peru’s Covid-19 distance learning M&E On the left you see one dashboard. small extract from the M&E dashboard with June At the top, the three icons allow 2020 data collected via you to choose between “Home” mobile by Peru’s MoE. It (where you can pick the month clearly shows that among you want to see data for), the distance learning “Teacher M&E Data” and “Parent media for their “Aprendo M&E Data” en Casa” initiative, TV is the most accessed at 46%, Below that, you can filter the radio the least at 8%, and data by school level, region, the website at 24% district, and intervention type. predominantly by smartphones. While Below you can choose the kind of mobile access might only data you want to see: coverage the the 2nd most common, of the multimodal “Aprendo en Service satisfaction is the Case” (Learning at Home) highest with the website. initiative, access (by medium), You can also see that the usage patterns, regularity of mothers are the remote interaction with predominant caregiver teachers, how well parents are who gives the child access coping with homeschooling (and to the phone / who where they say they need help), supports the phone during well-being of students & families, phone use. 16% of and if parents are willing to send students have no back their child to school or not. accompaniment. If you wish to see screenshots of The dashboard was all parental dashboard M&E created using Microsoft pages, please click here. For PowerBI. teacher dashboard, click here. Case Study - Sample Images of M&E Dashboard by MoE, Peru 43 The other interesting set of data the Peruvian MoE collected is the extent and format of interaction that teachers have with students. It shows that the predominant medium - by far - is Whatsapp at 95% and SMS just 0.1%. It also shows that teachers predominantly interact with the student (40%) and even more so with a parent (44%) 1- 2/week. An impressive 35% interact with the student every single day. While 97% of students seem to have a “portfolio” of assigned work, a disconcerting 15% haven’t had any contact with the teacher in the last week. In terms of feedback, 56% of students seem to have received feedback in the form of grades of “good /bad / room for improvement”, while 37% have also received full-on explanations as to what exactly they have done well and where they need to improve; however 7% of students say their teacher only confirms receipt of the messages rather than providing feedback. Back to section overview 44 Case Study - Lessons Learned: EdoBEST@Home, Nigeria Back to section overview Feature & Smartphones Content Delivery (static) Parental Instruction Teacher Instruction Communication M&E Plan beyond school closures. They intend to keep the online component (EdoBEST@Home) even when Level ECCE, Primary, Secondary Planning schools reopen, e.g. for remedial online tutoring, but also long-term to to an advanced LMS. EdoBEST began in 2018 as a PPP between MoE, Bridge Academies International & the World Bank; Org. Capacity Be frugal. The total cost of EdoBEST@Home is just $376,000; cut out everything non-essential from F2F all teachers were given 5x10 days of training & tablets where lesson plans were periodically uploaded; usage of LPs was tracked; did across-grade ability grouping; program was effective, even Zero-rate access to your educational website. MNO MTN Nigeria zero-rated website access & 2-way-SMS caused a 17% increase in public school enrollment with parents moving their kids from private to Technical public schools; 270k parent phone numbers had been collected by headteachers already using Setup Upgrade your cybersecurity. They have an ICT agency that they work with routinely; agency upgraded the tablets/smartphones; when Covid hit, EdoBEST covered 80% of schools in Edo, learning materials & security certificate, firewall etc. lessons plans were already digitized, 280,000 students had experienced tech-based learning model, 11,000 (head)teachers were comfortable with use of edtech. 91% of Edo households own a mobile. Capacity Ensure that your staff has the technical equipment to conduct work effectively. They’re upgrading 65% of ramp-up staff with high-end laptops; currently staff are mainly using inexpensive phones/paper/desktops. Pivoted to EdoBest@Home for primary: multi-channel, curriculum aligned remote learning, 4h of Intervention(s) daily content; created 7000 virtual, Zoom-based classrooms (30mins daily); 20min math & literacy Provide pedagogical & technical coaching to your teachers for the switch to online. L&D Supervisors & QA digital self-study activity packets & storybooks for independent study; homeschooling support via officers provided coaching remotely (via Zoom etc). Technical field officers conducted technical onboarding 45m-min learning guides with parent-led activities, distributed via WhatsApp & online. Formative (ensuring that all teachers bought a smartphone, how to operate the virtual classroom, etc). quizzes via Whatsapp/USSD (for feature phones). Students can ask teachers questions via Whatsapp. Due to many offline users also IAI radio lessons. Create online P2P forums for teachers. Ongoing partnership with Facebook; they use its Workspace which Intervention The program reaches 20,000 primary school students daily and 930 primary schools have been allows teachers to collaborate online; 2000 teachers are already on there (drains lots of battery though). Scale connected online (out of 1,000 schools) to support students remotely. The bank is processing a $75m P4R operation to scale up the program; 1 million website hits in 1 month Ensure that parents use the most powerful mobile network. Some parents were struggling to access the virtual classrooms due to unstable network; staff guided them on which SIM card provider to switch to. Outcomes Realized that the virtual classroom functions so well they could continue this even once schools Local govs also download a copy of website content and distribute it locally. reopen; unexpectedly high engagement rates of teachers; unexpectedly low from urban parents; Many people outside of Edo state from across Nigeria are visiting the website, incl. private schools; Start community sensitization early to prevent skepticism. Initial outreach calls to parents were met with → Not enough data yet on whether the program is producing sufficient learning outcomes skepticism (“How did you get my number, is this a scam, will you charge for this?”). Announced initiative Challenges → Breaking the culture of falsifying data; putting in place QA processes via a jingle in 9 languages on radio, TV, social media; shareable audio/videos explain EdoBEST@Home. → How to onboard more pupils into the program as fast and effectively as possible? (JSS next) Internal Encourage internal knowledge sharing. They started internal webinars hosted by staff members, teachers, → Low parental engagement in urban areas (even compared to remote); much more community KIX principals & guests to speak about their particular work & remote learning experience; currently a lack of sensitization still needed (physically going to marketplaces etc); but it’s eating into their resources. collaboration, hence try to instill more effective communication via these ‘socratic’ conversations. When schools resume they will really have to strengthen their urban school-based committees. → Some teachers are still not participating; unsure about the reasons, need more data M&E Conduct daily classroom check-ins. They took their usual offline M&E and adapted it to the online → Hackers trying to bring down the website experience. Using classroom observations form, field officers now randomly select a few virtual classrooms → Some people tried to take content from their website and sell it despite it being proprietary and login to observe the teacher-student interaction; then randomly select the parents of those kids and → Parents complained content was too easy/hard; created content above & below grade level asked them questions; to see if kids are learning, they ask them quick CFUs.Data shows up on a dashboard 45 World Bank FY 2021 Projects with a Mobile Component Project Use Case Project Location Project Total Commitments (in USD) Project ID M&E (textbook distribution) Ethiopia (refugees) 60 million P168411 Content delivery, synchronous Ghana 26 million P173282 instruction (teacher), Assessment Content delivery Mauritania 40 million P163143 Content delivery Mali 80 million P164032 Content delivery Somalia 40 million P172434 M&E Sierra Leone 50 million P167897 Back to section overview 5. Which relevant Mobile Software exists for our use cases? Click on any hyperlink to jump directly to the section. Back to main overview Market Scan: Feature Phone Solutions (offline & online) 47 Back to section overview Viamo Arist Cell-Ed Eneza Remind Other Content delivery, Assessment, M&E, Content delivery, Assessment Content delivery, Assessment, Content delivery, Assessment, Communication for Coordination Content Delivery Use Cases Communication for Coordination Synchronous Instruction by Teacher Synchronous Instruction (by Teacher) RapidPro SMS (& Facebook Messenger, Customizable,>1,000 hours of ready- Revision materials via SMS/USSD (& Two-way SMS (&smartphone) Product IVR & SMS platform WhatsApp) messaging & authoring to-go SMS/ call-in lessons for Web & Android) & ability to ask platform for parent-teacher-student Smart TXTBKS platform literacy, numeracy, language & job teachers questions live via SMS comms. Opt-in via SMS with code. upskilling Ustad Levels All levels All levels Adult Primary, Secondary All Authentication Languages All languages All languages English, Spanish English, French, Swahili Messages can be translated into over 70 languages; Platform is in: EN, Boloro Other Reach more than 100k people/day - Certified to protect user data well Trained over 50, 000 workers in 14 Reached over 6mn offline users at ES, FR, DE, POR, CHN countries with 50+ partners 8,000+ schools in Kenya, Ghana & 35mn (parents, students, teachers) use Instruction more than 10 million from 2012-17 (ISO & GDPR compliant) (Colombia, Chile, Ghana, India, etc) the Ivory Coast it; 2.5mn messages sent/day; good (Parent) privacy policy Dost Education Founded 2012 2018 2014 2014 2011 Communication / M&E Ubongo Learning M-Shule World Reader Funzi KaiOS rapidSMS Use Cases Content Delivery Synchronous Instruction (AI), Content Delivery Content Delivery Versatile Assessments FrontlineSMS SMS adaptive literacy & math 12,000 e-books for literacy, language Free life skill, entrepreneurship & A mobile OS that makes no-touch Product Literacy & numeracy content; IVR feature phones which have a web Classpager instruction system using AI & & social studies that can be accessed Covid lessons via internet-enabled version via KaiOS & Viamo’s service browser smart, enabling 300+ apps chatbots; student data is shared with on a browser-enabled feature phone (feature) mobiles(browser-based). Classparrot (investing into more IVR & SMS now) e.g Whatsapp, YouTube (KaiStore) parents & teachers. Opt-in via SMS. Levels ECCE, Primary, Secondary ECCE, Primary Secondary / Adult Smart IVR Grades 4-8 All Arabic, Dari, EN, FIN, Somali, Sorani, TalkingPoints Languages English, French, Hausa, Kikuyu, Swahili, English 52 languages Swahili. Translations upon request. All Kinyarwanda, Luo, Swahili, Yoruba M&E Culturally relevant, curriculum 5 million users Other TV/radio version had 24% numeracy $1/month/student; >12k primary & aligned books; tracks time spent 150 million users, primarily from EduTrac improvement versus peers adult learners India, Indonesia, Rwanda, Nigeria reading & reading engagement. Founded 2017 RapidPro 2013 2017 2010 2015 Market Scan: Smartphone & Smart Feature Phone Solutions 48 Back to section overview Searchable Database of Communication Remind, Bloomz, eKool.eu, Edmodo, SchoolCNXT, Otus,engageSpark, Skooly, Telebu, SimpleTexting Mobile Solutions by the EdTech Team ClassFlow, TES Teach with Blendspace, Screencastify, Loom, LessonUp, Explain Everything Whiteboard, Educreations, Edpuzzle, PlayPosit, Screencast-O-Matic, Nearpod, Please view our database here. Content Creation Kaltura, Pear Deck, ThingLink, Buncee, Squigl, LessonUp, Claned, EdVisto, Seppo, Audacity, Blender, GIMP, Snap Collage, Canva Filter by language, grade, subject, offline access, cost, etc. Content Delivery Other Repositories Offline file sharing: ShareIt, Send Anywhere, Files Go by Google, Zapya, (static) IM Offline: Moya, Bridgefy, FireChat, Briar, Rumble; IM Online: Whatsapp, Signal, Telegram, Facebook Messenger 589 mobile, edu-resources by Learning Keeps Going ECCE: Literacy: Homer, Google Primary: Math: Prodigy, Elephant Learning Secondary: MOOCs: Future Learn, iCourse, Mental Health: Mightifier, Calm, Filter by language (English, Content Delivery Sanvello, Headspace, French, Spanish, Chinese, Bolo / Read Along, Reading Eggs, Math Academy, Illustrative Mathematics, Nafham, Udemy, EdX, Alison, Coursera, Edraak, (interactive) Storyline, Audible; Numeracy: DoodleMaths, Khan Academy, Buzzmath, Khan Academy, Mawdoo3, AplusClick, Breathe2Relax, Mindshift, VA Korean, etc), grade, availability Montessori Geometry, OneCourse; Twelve a Dozen, Splash Math, Operation BioInteractive, HippoCampus, KialoEdu, Lynda; Mental Health Apps, WorryWatch outside of US, cost, use case Arts: Drawn to Discover, Bandimal; Math, Motion Math, DragonBox, 7generation, Math: yMath, Mathspace, eMathStudio, STEM: Codeable Crafts; Simple Zearn, Prodigy, Math snacks; Literacy: Google Mathletics, Yup; Language Arts: Project Special Needs: Bookshare, Dyslexia Teacher-approved Android apps Machines; PE: GoNoodle, Bolo, EkStep, PBSKids, Newsela, Poio,African Gutenberg, Language Learning: Akelius Academy, SpeakTX by Google GoNoodle Videos Other: Mobile Storybook, Lexi’s world, Storyweaver, (English, French, Greek), Duolingo, Lingvist, Visit Google Playstore, go to the Montessori apps OneClass; STEM: CodeSpark Academy Arts: Speakly; STEM: Tynker; Arts: Google Arts & “Kids” tab” & look for the METKids; All: Brainpop, IXL, Learning Passport Culture; All: Byju’s, Listenwise, Unacademy, etc. “Teacher-approved” badge. 4,700 educational apps Instruction (teacher) Synchronous Teaching: Slack/Zoom/Hangouts/Skype/Trueconf/Whatsapp/Dingtalk/Jitsi/Webex, ClassIn. Classroom Management: Microsoft Teams, Gooru, GoClass, Verso, by Common Sense Media Edmodo, ActivelyLearn, Kiddom, Skooler, Schoology, Seesaw, ClassDojo, Classmax, EquityMaps - Chart Dialogue, FreshGrade, GradeCraft, Sanako, Eko, Kiddom, ClassPlus 2,100 Android apps; Filter by age, cost, subject, skill, genre; Instruction (parent) ECCE: Tinkergarten@Home, PBS For Parents, extensive reviews for each app Planning: Homeschool Panda; Device Safety management: Qustodio Parental Control, Family Time from educational angle; material for homeschooling (K-5, 6-12) P2P Learning VoiceThread, Peergrade, Storillo, Parlay, SammTalk, Google Classroom, OneClass 645 Digital Learning Solutions by UNHCR Assessment Spiral, Kahoot!, Quizlet, Classkick, SeeSaw, StickPick, ClassroomQ, Dugga. Autograding: Bakpax, Moby.Read, Questbase Excel sheet categorized by subject, level, language, cost KoBo Toolbox, LogAlto, Granity, Magpi, Mobenzi, Survey CTO, Twilio, engageSpark, FastfieldForms 400 educational apps M&E 67 educational mobile apps Digital Credentialing Credly, Accredible by EdSurge Authentication Educational Resources Duo, Thales Group, Appdome by EducationNation 6. Where can we find additional information? Click on any hyperlink to jump directly to the section. Back to main overview Additional Resources 49 Covid-19 Education Response Strategy • Global Education Cluster Education Crisis Management Toolkit • INEE International Emergencies in Education Network Resource Database • EU Education in Emergencies Practical Framework • UNICEF Guidance on Distance Learning Modalities • UNESCO Guidance on Active Learning at Home during Educational Disruption • UNESCO The Chinese Experience in Maintaining Uninterrupted Learning in Covid-19 Outbreak • UNHCR Education during Covid-19: Emerging Promising Practices • US Washington State Continuous Learning 2020 Plan Scaling Solutions Effectively • R4D Taking Innovations To Scale: Methods, Applications and Lessons • Journal of Educational Change Scaling up successfully: Lessons from Kenya’s Tusome national literacy program Online Educational Resource Databases • ISTE Learning Keeps Going: Free Tech for Learning • Education Above All Free Education Resources For Homeschooling - Internet Resource Bank • UNESCO Distance Learning Solutions • CrowdED Learning Teacher Tools Repository Back to section overview 7. Acknowledgements Click on any hyperlink to jump directly to the section. Back to main overview 52 Acknowledgements Our sincerest ‘thank you’ to everyone who shared their Dr. Madhav Chavan, Co-Founder and President Stephen Meyer, Director of Strategic Partnerships time and expertise with us Rachael Alldian, Director Of Program Operations and Nishant Baghel, Director Technology Innovations for the case studies in this Global Projects Lead Knowledge Pack - especially during these incredibly busy, strenuous, uncertain times! We encourage everyone to take a closer look at the great work all of these organizations are doing. Dr. Joan Osa Oviawe, Executive Chairperson of the Edo Veronica Diaz Hinostroza, Consultant State Universal Basic Education Board (SUBEB) Andrew Ragatz, Senior Education Specialist, World Bank Back to section overview Connect with the World Bank’s EdTech Team @WbEduTech Website & Resources Medium Posts (Weekly/Monthly mailers) Blogs Apple Podcasts | Spotify | Anchor @EDTECH Internal: EdTech_Core_Team@worldbank.org External: iciarrusta@worldbank.org Back to main overview