SABER IN ACTION SABER SERVICE DELIVERY JUNE | 2017 SABER Service Delivery (SABER SD) is an initiative by the Education Global Practice to uncover bottlenecks that inhibit student learning in low income and middle income countries. This school survey collects strategic information on the school inputs and processes that produce learning outcomes. The measure aims to uncover the extent to which policies translate into implementation and practice. As a global initiative, SABER SD provides data for the new global lead indicator on learning, which makes it easier to monitor the Sustainable Development Goal of achieving universal primary education. Going Beyond Basic Indicators: SABER Service Delivery SABER Service Delivery (SABER SD) is a new initiative built upon SABER policy and implementation tools and the World Bank’s Service Delivery Indicators (SDI), which was originally developed to assess the efficacy of service delivery in Africa. SABER SD contributes to the growing SABER evidence base by capturing policy implementation measures from several of SABER’s core domains, including: School Autonomy and Accountability, Teachers, Finance, Education Management Information Systems, and Education Resilience. The SABER SD instrument collects data at the school level and asks questions related to the roles of all levels of government (including local and regional). The SABER SD tool will provide comprehensive data on teacher effort and ability; principal leadership; school governance, management, and finances; community participation; and student performance in math and language and includes a classroom observation module. In addition, SABER SD:  Provides policy-makers with a quick,comprehensive overview of how their education system is currently functioning;  Complements existing SABER domain research to deepen the knowledge and understanding of education systems; saber.worldbank.org SABER IN ACTION: JUNE SABER SERVICE DELIVERY 2017  Identifies breakdowns in policy implementation at various levels of government;  Provides metrics to assess the quality of education service delivery in a country;  Links gaps in policy and policy implementation to real consequences in the classroom;  Offers information and accountability for the education system as a whole;  Provides data which could help bridge dialogue among financiers, policy-makers, and frontline education service providers; and  Generates information on the implementation of country policies benchmarked by SABER. Piloting the Tool To date, SABER SD has informed operations in Laos, Punjab (Pakistan), and Afghanistan. The tool is also being used to monitor three operations in Latin America that aim to improve teacher classroom practices. SABER SD is designed to support operations geared toward improving the quality of education service delivery and, ultimately, learning outcomes. In Laos SABER SD directly supports the Second Global Partnership for Education Project which focuses on strenghtening school-based management and improving the teaching and learning of reading. The survey instrument is enabling the collection of baseline and end-line data that measure whether schools meet the minimum education quality standards. The SABER SD team worked with the government, a local survey firm, and educators to collect data from a random sample of 200 schools nationwide. Detailed information was collected from four categories of schools: urban and rural, public and private, single grade and multi-grade, and those located in priority and non-priority districts of the GPE II project. PHOTO: ANAHITA H MATIN / WORLD BANK saber.worldbank.org P2 SABER IN ACTION: JUNE SABER SERVICE DELIVERY 2017 The data collection process in Laos allowed SABER SD to explore and set firm protocols for data collection. In an effort to streamline data collection and reduce error SABER SD piloted the use of tablets with questionnaires in Computer-Assisted Personal Interviewing (CAPI) form. Enumerators captured data directly into the tablets, which were able to instantaneously report data to the central server for review through mobile internet connection. SABER SD expects to publish the results of the Laos application later this year.In addition to Laos, SABER SD is serving as a monitoring tool for the Third Punjab Education Sector project in Punjab, Pakistan and as a diagnostic tool for an upcoming operation in Afghanistan. PHOTO: PHOTO: ANAHITA H MATIN / WORLD BANK MARIA QURESHI / WORLD BANK In Pakistan, the SABER SD team carried out an extensive pilot across 20 primary schools. A wide-ranging sample of public, private, urban, rural, co-ed, and single-sex schools was selected. Some of the innovations included the use of: 1) the Development World Management Survey (DWMS) to evaluate school management, 2) a high inference classroom observation tool with potential for implementing it as the government’s primary observation protocol, 3) IEA’s Literacy and Numeracy Assessment (LaNA) f or pupil assessment 4) teacher assessment with questions on pedagogical content knowledge, content knowledge, perceptions and background, and 5) a parent involvement and socioeconomic status questionnaire. The SABER SD team will be using feedback from focus groups and pilot data to revise and further adapt the Service Delivery tools for the field work from October to December 2017. In Afghanistan, there is scarce evidence as to the type of instruction and/or learning that is happening in the classroom. The SABER SD team worked with the Ministry of Education, a local survey firm, principals, and teachers to address this need by incorporating the Stallings classroom observation method into the tool. The field team is collecting data from 200 random schools nationwide, from which 170 of the schools are public and 30 are community-based education schools. The SABER SD survey is being implemented on pen and paper due to the insecure nature of certain regions of Afghanistan. SABER SD report on Afghanistan will be published later this year and is expected to be used as a diagnostic tool in Education Quality Reform in Afghanistan operation project. saber.worldbank.org P3 SABER IN ACTION: JUNE SABER SERVICE DELIVERY 2017 Components of SABER SD SABER SD Survey Modules Module Title Description School Collects information on school infrastructure, state of facilities, number of students, 1 Information number of shifts, and number of grades taught. Collects data on teachers and measures principal and teacher absence from school 2 Teachers day and from designated class time; collects information on teacher remuneration, professional development opportunities and requirements, and teacher motivation. Collects information on school governance, management, and financing. We have piloted different versions of the module to capture different dimensions of these School 3 concepts from target setting to community/parental participation and use of Management information for decision making. The instrument will be aligned to the SABER domains and Stanford’s World Management Survey. Measures teacher practices and classroom behaviors with the use of an open source tool designed to measure the quantity (i.e. time-on-task) and quality of 4 Classroom instruction (i.e. quality of teacher feedback, ability to convey knowledge to students, Observation behavior management). This is supplemented with an optional student report on their perceptions of teacher practices. Measures student learning outcomes in mathematics and reading in the language of 5 Pupil instruction; captures student non-cognitive skills such as the “Big 5” and grit. Assessment Aligned to international student assessments to report on the new Global Lead Indicator (UIS, 2016). Assesses teacher knowledge of mathematics and the language of instruction; 6 Teacher includes a written test that measures both content knowledge and pedagogic Assessment content knowledge. Captures information on the home environment, parental involvement, and socio- *Parent 7 economic background of students. Interview * This module was only piloted in Punjab and is optional. What’s Next for SABER SD? The SABER SD survey is still being piloted and improvements will be made to the tool as a result. Analysis of survey data will help countries identify gaps in policy implementation and pinpoint bottlenecks or breakdowns in the quality of education service delivery. This information will help countries and partners identify where to channel education resources and support. saber.worldbank.org P4