Education system capacity Technological and logistical considerations Dimensions Document Overview: This template is useful to assess implementation prerequisites and enabling conditions for phone-based formative assessment solutions. The assessment contains two broad areas: (a) Education system capacity (b) Technological and logistical considerations Education system capacity Column C- Indicator - Outlines the areas that might be helpful to understand the context of the country and readiness of the country to implement remote assessment solutions. Column D - Evidence - Mentions the details supporting the existence of information corresponding to Column C. Column E - Source - Mentions the source from where the information in Column D was obtained. Column F - Data - Mentions whether the information for the corresponding indicators in Column C exists or not. Columns G-H-I - Relevance for solution (SMS, IVR, live calls) - Mentions how relevant the information provided under Column D is relevant for the implementation of three assessment solutions. Columns J - Description/Comments - It provides further details about the indicators in Column C to accurately fill out the information in Column D. Technological and logistical considerations Column C- Indicator - Outlines the areas that might be helpful to understand the context of the country and readiness of the country to implement remote assessment solutions. Column D- Evidence - Mentions the details supporting the existence of information corresponding to Column C. For some Dimensions such as demographics, technology and financial, this column provides information of the scale/unit to provide the information. Column E- Source - Mentions the source from where the information in Column D was obtained. For some Dimensions such as demographics, technology etc., this column provides relevant links from where the ifnromation can be obtained. Column F- Data - Provides the information found using the link in Column E. Columns G-H-I- Relevance for solution (SMS, IVR, live calls) - Mentions how the information provided under Column D is relevant for the implementation of three assessment solutions. Columns J - Description/Comments - Provides further details about the indicators in Column C to accurately fill Dimensions Rows 11-19- Alignment with existing learning content - Captures the existing and upcoming remote learning initiatives that the World Bank (WB) and/or the Ministry of Education (MOE) are involved in. Rows 21-24- Human - Captures teachers' ability to participate in remote learning/assessmnet initiatives and their perceptions about it. Rows 26-28- Demographics - Captures the urban population share, literacy rates and language diversity in the country. Rows 30-37- Technology - Captures how widely teh etchnology is available in the country using parameters such as mobile. penetration rates, mobile cellular subscriptions, access to internet, etc. Rows 39-46- Service providers - Captures the capacity of mobile service providers using indicators, such as existence of mobile aggregators, two-way SMS etc., relevant to implementation of three technology solutions. Rows 48-51- Financial - Captures the cost of various indicators such as cost per SMS using a Mobile Aggregator, cost of mobile data for 1Mb (Prepaid card), etc. Rows 53-57- Time - Captures the indicators that may affect the timeline of the project, such as the existence of a database of phone numbers to ease the collection of phone numbers. Rows 59-63- Local implementor capacity - Captures the capacity of the implementation firm using indicators Picture CHECKLIST OF PRECONDITIONS FOR PHONE-BASED FORMATIVE ASSESSMENT INPUT PERCEIVED SCORE OF COUNTRY NAME IMPORTANCE (LOW; MEDIUM; HIGH; N/A) N/A = Not Applicable Blank = No data found RELEVANCE FOR SOLUTION DESCRIPTION / COMMENTS DIMENSION INDICATOR EVIDENCE SOURCE/REFERENCE DATA SMS IVR LIVE CALL The World Bank is currently engaged w ith a YES / NO distance learning initiativ e in the country Countries that focus on teacher-guided home learning modalities where teachers have remained in contact with students could have higher adoption rates in a phone-based The Ministry of Education is currently formative assessment, vs. modalities that just rely on self-learning, especially if the teacher- implementing (or implementing in the near future) YES / NO guided modalities involve any of the reviewed technologies (SMS, IVR, Live Calls). A second one or more distance learning strategies to layer is the level of technology used in distance learning. A hybrid technology approach support educational deliv ery during COVID-19 could contribute to a broader reach and, therefore, increase the likelihood of a successful intervention. Current (or soon to be implemented) MoE distance learning approaches are exclusiv ely YES / NO based on Self-Learning (or assisted learning w ith E the support of parents/guardians D In addition to Self-Learning approaches, the MoE U has implemented (or w ill implement in the near YES / NO C future) Teacher-Guided approaches A ALIGNMENT WITH T EXISTING LEARNING CONTENT Current (or soon to be implemented) MoE I distance learning approaches use SMS YES / NO O technology to reach students N Current (or soon to be implemented) MoE S distance learning approaches use IVR technology YES / NO Y to reach students S Current (or soon to be implemented) MoE T distance learning approaches use Liv e Calls YES / NO E technology to reach students M General perception regarding the accessibility and C reach of educational resources prov ided by the Perceptions around accessibility can influence the adoption of an assessment. If the content YES / NO A MoE in response to the pandemic is mostly has not reached a critical mass, the assessment will be less likely to be successful. P positiv e A C General perception regarding the quality of Perceptions around quality can influence the adoption of an assessment. If the content is I educational resources prov ided by the MoE in YES / NO considered of low quality, the recipient's participation in an assessment can be affected. T response to the pandemic is mostly positiv e Y The crucial human aspects to be considered relate to teacher's ability to participate in the Enough teachers hav e access to a smartphone intervention. The two main components are access to smartphones and access to the internet. in order to participate from a phone-based YES / NO Such relevance is due to the fact that using a smartphone/mobile app to aggregate content formativ e assessment for dissemination will heavily influence the time they spend on the tasks related to the assessment. HUMAN Enough teachers hav e access to the internet to YES / NO connect v ia mobile dev ices or any other means Teachers are motiv ated to support remote YES / NO learning assessment. The country has at least one nationally Existence of nationally representative learning assessment in grades 2 or 3 of primary YES / NO representativ e learning assessment education. Data 2017-2018 Urban Population Share % of total population DataReportal Share of population (%) living in urban areas. DEMOGRAPHICS % of the population of a given age group that can read and write. (The emphasis on Adult Adult Literacy Rates % of total population DataReportal literacy rates relates to the importance of parent involvement in the formative assessment process). Are there multiple national w idely spoken Language considerations are important for access/costs/ and time. Voice options are Ethnologue languages in the country? recommended in a multilingual setting. Mobile penetration % of Population DataReportal The estimated unique subscribers of mobile services in a country (per total population). The % of a country's population that is covered by at least one mobile network operator's Netw ork cov erage (GSM) Total of suscriptions GSMA network. TECHNOLOGY International Internet fixed broadband % of Population Speedtest Global Index International Internet bandwidth (kb/s) per Internet user. Mobile cellular subscriptions PPP $/min Our World in Data Mobile phone subscriptions, measured as the number per 100 people. The number of fixed landline telephone and mobile cellular phone subscriptions, measured Fixed landline telephone subscriptions # per 100 people Our World in Data per 100 people. Mobile connections that are pre-paid # per 100 people DataReportal Percentage of mobile connections that are prepaid. Mobile accounts for financial transactions World Bank Global Findex The percent of adults who used mobile phones to conduct a financial transaction Access to electricity % of Population GSMA Proportion of the population with access to electricity. T E (Africa only) A Guide to Using Mobile At least one mobile aggregator w orks in the At least one mobile aggregator is the minimum viable option for an implementation. If there C Aggregators to Deliver NGO Services at YES / NO country are more, they will be considered in procurement processes post-design. H National Scale N To assess the potential of a Mobile Aggregator it is necessary to review their capabilities, their O Av ailable mobile aggregators in country hav e YES / NO presence in a country (or multiple countries, and other elements to be included in the L SMS capabilities guidance note). O G Av ailable mobile aggregators in country hav e YES / NO I Voice capabilities (IVR/Calls) C There is a mobile aggregator that has both A YES / NO SERVICE PROVIDERS capabilities SMS and Voice capabilities L (Africa only) A Guide to Using Mobile A Tw o-w ay SMS is av ailable in country Aggregators to Deliver NGO Services at YES / NO N National Scale D (Africa only) A Guide to Using Mobile Are short codes av ailable in country? Aggregators to Deliver NGO Services at YES / NO L National Scale O G Toll-free number setup is av ailable in country YES / NO I The country rarely suffers from internet Frequent internet shutdowns due to infrastructure issues or government decisions can affect the S YES / NO outages/shutdow ns service providers' ability to provide an effective service and compromise the intervention. T I C Av erage cost per SMS using a Mobile Aggregator USD$ Twilio Cost per individual SMS in USD A L Cost per 1MB of mobile data in USD. (Accessibility of Mobile Data is important for teacher's Av erage cost of mobile data for 1Mb (prepaid USD$ Visual Capitalist card) use of software tools for managing SMS or calls). C FINANCIAL O What is the cost of 1 minute call time out of N USD$ Twilio Cost per 1 minute of call time in USD netw ork using a Mobile Aggregator? S I Average per-minute cost of different types of mobile cellular calls (PPP $) (Data is PPP D Prepaid mobile cellular tariffs, PPP $/min. USD$ WB TCdata360 (2012-2016) adjusted. Accessibility to mobile services is important for teacher's use of IVR or calls) E R A MoE or school access to robust databases for receivers can have an impact in rollout times for MoE or schools manage a parents and students T YES / NO implementation. However, participation can also be linked to a marketing campaign for parent phone number database I enrollment in the service, which is why relevance remains as "Medium". O N Distance learning w ill persist throughout the first The persistence of distance learning throughout 2021 has a direct impact on the relevance of YES / NO S semester of 2021 establishing remote formative assessments. TIME This item is based on a projected SMS startup timeline * The following items are directed at Implementation timeline allow s for a 3-month YES / NO the LeAp team and WB country offices. If there is an urgency to start piloting, a longer startup startup phase might be a deterrent from using a specific type of technology. Implementation timeline allow s for a 4-month YES / NO This item is based on a projected IVR startup timeline startup phase Implementation timeline allow s for a 5-month YES / NO This item is based on a projected Live Calls startup timeline startup phase The LeAP team or the WB country offices hav e identified potential implementing partners Having a potential implementing partner can help the WB accelerate the implementation w orking on current distance learning initiativ es process, but is not crucial. The guidance note will contain a short section profiling "The right YES / NO or w ith prev ious phone-based proj ect Implementing Partner." A previously identified implementing partner for a remote learning implementation experience (NGO, Dev elopment intervention can become a potential partner for a PFA. Contractor) The implementing partner(s) hav e prev ious experience implementing phone-based YES / NO dev elopment proj ects LOCAL IMPLEMENTER CAPACITY The implementing partner(s) hav e prev ious experience w orking w ith MNOs and mobile YES / NO serv ice prov iders/v endors (e.g. aggregators) Potential partners hav e or can access an ICT team to set up and test the system, configure YES / NO procured phones or carry out any additional ICT- related activ ities The country's MoE has literacy, math, and other education experts av ailable to design the YES / NO assessments in the official language(s) Picture Picture Picture