Policy Research Working Paper 10253 Capturing the Educational and Economic Impacts of School Closures in Poland Tomasz Gajderowicz Maciej Jakubowski Harry Anthony Patrinos Sylwia Wrona Europe and Central Asia Region Office of the Chief Economist December 2022 Policy Research Working Paper 10253 Abstract The effect of school closures in the spring of 2020 on the that the learning loss was equal to more than one year of math, science, and reading skills of secondary school stu- study. Assuming a 45-year working life of the total affected dents in Poland is estimated. The COVID-19-induced population, the economic loss in future student earnings school closures lasted 26 weeks in Poland, one of Europe’s may amount to 7.2 percent of Poland’s gross domestic longest periods of shutdown. Comparison of the learning product. outcomes with pre- and post-COVID-19 samples shows This paper is a product of the Office of the Chief Economist, Europe and Central Asia Region. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at hpatrinos@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Capturing the Educational and Economic Impacts of School Closures in Poland Tomasz Gajderowicz University of Warsaw, Poland Maciej Jakubowski University of Warsaw, Poland Harry Anthony Patrinos World Bank Sylwia Wrona University of Warsaw, Poland JEL Classification: I21, I24 Keywords: COVID-19, school closure, learning loss, Poland */ All views expressed here are those of the authors and should not be attributed to the World Bank Group. 1. Introduction COVID-19 led to strict lockdown measures, which included school closures in most countries. As a result, more than 1.5 billion students were out of school for weeks or months (UNESCO, 2022). The loss of schooling is expected to negatively impact children's cognitive development, even if distance learning modes are enacted. The loss of in-person teaching could also lead to inequality since the only remaining relevant input is parental involvement during school closures (Agostinelli et al., 2022). Most studies document significant learning loss. In Europe, the average learning loss is almost a quarter of a school year, but the estimates are available mainly for Western European countries (Donnelly and Patrinos, 2021). Worldwide the loss is even greater, especially in lower-income countries (Patrinos et al., 2022). Poland is an interesting case because it represents countries in Eastern Europe where school closures lasted longer, and research on learning loss is scarce. 2. Measuring learning loss using PISA as a benchmark Most countries do not have standardized assessments that allow comparison of student results over time. To measure learning loss in Poland, we implemented an assessment of students in mathematics, science, and reading, that reports results on a scale comparable to OECD's PISA international study. The results are based on a representative random sample of students in 2021 from grades 10 to 12 and compared to PISA results that are available for a random sample of students from Warsaw collected between 2003 and 2018. In each comparison, the sample sizes are greater than 1,000 students (Jakubowski et al., 2022). 2 Two major changes could affect students' results in 2021 compared to previous years. First, it was the first assessment after schools reopened after the closures. All schools in Poland were closed for at least 26 weeks. This was longer than most Western European countries, which were also better prepared for digital instruction. Second, structural changes implemented in 2016 shortened the period of general academic education for all students, switching the 6+3+3 grade structure with the 8+4 system, and replacing national examinations after the 6th and 9th grades with the 8th-grade exam. These changes reversed the 1999 education reform, which successfully raised student achievement (Jakubowski et al., 2016) and improved labor market outcomes (Drucker et al., 2022; Liwiński, 2021). To properly estimate the effect of school closures and to distinguish it from the effect of the 2016 structural changes, we compare the expected and actual achievement of three cohorts of students in secondary schools. We assume students should gain a minimum of 0.1 standard deviation (SD) during one year of education. That is a safe assumption but in line with previous studies comparing 15, 16, and 17-year-old student results on the PISA scale in Poland (Jakubowski et al., 2022). International evidence indicates the gains should be larger, around 0.2 SD, which would make our results more significant as they increase the expected achievement (Avvisati and Givord, 2021). We also assume students tested in autumn (10th grade in TICKS 2021) have a similar achievement to those tested one grade below in the spring (9th-grade assessment in PISA 2003-2018). Assuming any achievement progress between spring and autumn makes our results even more significant. Figure 1 shows a significant achievement decline among 10th-grade students as their actual results are lower than expected by approximately 0.4 SD in mathematics and almost 0.6 SD in reading and science. For the 11th grade, mathematics and reading results are lower than expected by 3 approximately 0.3 SD, and in science, by almost 0.4 SD. For the 12th grade, the gap is the smallest, around 0.2 SD, and is not statistically different from zero. Figure 1. The actual and expected achievement of secondary school students in Warsaw 10th grade 11th grade 12th grade math actual math actual math actual math expected math expected math expected science actual science actual science actual science expected science expected science expected reading actual reading actual reading actual reading expected reading expected reading expected 460 480 500 520 540 560 500 520 540 560 580 520 540 560 580 Source: Jakubowski et al., 2022. We separate the effects of the pandemic and the structural reform by comparing achievement changes in the cohorts affected by both events (10th and 11th grade) to changes in the 12th grade affected by the pandemic only. We regress the PISA 2003-2018 scores on the results from 2021, adjusted for the expected minimum achievement gain over one year of education. The regression equation is given by: 𝑌𝑌𝑃𝑃𝑃𝑃𝑃𝑃𝐴𝐴 = 𝛽𝛽0 + 𝛽𝛽1 𝑌𝑌1011 + 𝛽𝛽2 𝑌𝑌101112 + 𝛽𝛽3 𝐺𝐺𝐺𝐺𝐺𝐺𝐺𝐺𝐺𝐺𝐺𝐺 + 𝛽𝛽4 𝑆𝑆𝑆𝑆𝑆𝑆 The structural reform effect is captured by 𝛽𝛽2, which measures the difference between students who followed the new and the old school structure. The estimate of the learning loss is given by 𝛽𝛽1. We also control for student gender and socioeconomic status to minimize individual variance 4 in student outcomes for the main results. We obtain similar results with added controls for test mode (paper vs. computer) or using multiple imputations to include non-complete student responses (for more details, see Jakubowski et al., 2022). The learning losses due to the pandemic are around -0.3 SD in mathematics and science and are greater than those caused by structural changes (around -0.2 SD). In reading, the learning loss is smaller, around -0.2 SD, and statistically insignificant in most specifications. 3. Economic consequences of the learning loss The learning loss consequences can be assessed both by looking at macroeconomic outcomes and individual-level productivity reflected in wages. On the macro level, we used the growth coefficient of 1.74, estimated by Hanushek and Woessmann (2010) for OECD countries. Taking a lower bound estimate of the learning loss, the loss of 0.2 SD in achievement would translate into -0.35 percentage points in GDP growth. A more realistic 0.3 SD learning loss would translate into the effect of -0.52 percentage point on GDP growth. To estimate the individual-level impact of remote education on wages, we follow Psacharopoulos et al. (2021), where the loss in wages is given by: 𝐿𝐿 = 𝑃𝑃𝑃𝑃 (𝑌𝑌 ∙ 𝛼𝛼 ∙ 𝑟𝑟) ∙ (𝑆𝑆 ∙ 𝛽𝛽 ) In the equation, L is the total loss in wages. The first factor is the product of the mean annual earnings Y, the adjustment factor to account for the period in which schools were closed α, and the rate of return of one year of schooling r, all estimated in their present value (PV). The second factor is the product of the number of students S and the efficiency of distance learning β indicates how many students can switch to remote learning without disruptions. The rate of return of one year of 5 schooling r is equal to 7.8%, following the meta-analysis for Central and Eastern European countries (Wincenciak et al., 2022). Using this value and accounting for the 26 weeks of school closures, the value of the adjustment factor to account for the period in which schools were closed α is equal to 0.52. According to the CSO data, the mean annual earnings Y in 2021 are PLN 72,000. Following Psacharopoulos et al. (2021), the present value is calculated over a student's working life period of 45 years, assuming a 3% discount rate. The number of students S equals 6 million (UNESCO, 2022). Finally, the efficiency of distance learning β was set at 0.6 following the Polish school closures duration (UNESCO, 2022) and arguments of Kuhfeld et al. (2020) and Psacharopoulos et al. (2021). The yearly economic loss at the individual level equals PLN 2,927. The present value for the 45 years of working life amounts to a loss of PLN 74,693 for each student. It sums up to PLN 179 billion for the affected population, which equals 7.2% of the 2021 GDP. Conclusions The Polish success story of rapid social and economic progress relied strongly on human capital improvement. Unfortunately, this factor is now under significant distress. Significant learning losses have been experienced by Polish students due to the COVID-19-induced school closures. In mathematics and science, the learning losses are equal to more than a year's worth of schooling, even though schools were closed for only part of an academic year. In addition, we show that the 2016 reforms also had a negative impact on student learning. These skills losses are likely to affect the future economic success of the students as well as the country as a whole. Future earnings are projected to decline by PLN 74,693 (more than US$15,000) per year for the affected students. The country would then lose the equivalent of 7.2% of GDP over time. 6 Schools have reopened, and students are back in class. The task of learning recovery must commence, both to recover some of the losses and to accelerate learning. Accelerated programs must include tailored responses for those that fell behind the most, such as tutoring (Fryer and Howard-Noveck 2020), which has proven to be successful before the pandemic and during the school closures, when it was implemented cost-effectively online (Carlana and La Ferrara 2021). Teachers need to be supported to address the challenge with additional resources. Education systems need to be made more resilient to face future shocks. 7 References Avvisati, F., Givord, P. (2021). How much do 15-year-olds learn over one year of schooling? An international comparison based on PISA. OECD Education Working Papers No. 257. Carlana, M., La Ferrara, E. (2021). Apart But Connected: Online Tutoring and Student Outcomes During the COVID-19 Pandemic. CEPR Discussion Paper No. DP15761. Donnelly, R., Patrinos, H. A. (2021). Learning loss during COVID-19: An early systematic review. Prospects 1-9. Drucker, L.F., Horn, D. & Jakubowski, M. (2022). The labour market effects of the polish educational reform of 1999. Journal of Labour Market Research 56, 13. Fryer Jr, R.G., Howard-Noveck, M. (2020). High-dosage tutoring and reading achievement: evidence from New York City. Journal of Labor Economics 38(2): 421-452 Hanushek, E. A., & Woessmann, L. (2010). The high cost of low educational performance: The long-run economic impact of improving PISA outcomes. OECD Publishing, France. Jakubowski M., Gajderowicz T., Wrona S. (2022). Achievement of Secondary School Students after Pandemic Lockdown and Structural Reforms of Education System. Evidence Institute and City of Warsaw research report. Jakubowski M., Patrinos H., Porta E., Wisniewski J. (2016), The Effects of Delaying Tracking in Secondary School: Evidence from the 1999 Education Reform in Poland. Education Economics 24(6). Patrinos, H.A., Vegas, E., Carter-Rau, R. (2022). An Analysis of COVID-19 Student Learning Loss. Policy Research Working Paper No. 10033, World Bank. Psacharopoulos, G., Collis, V., Patrinos, H.A. and Vegas, E. (2021). The COVID-19 cost of school closures in earnings and income across the world. Comparative Education Review 65(2): 271-287. 8 UNESCO (2022). UNESCO map on school closures. Retrieved at https://covid19.uis.unesco.org/ on March 2022. 9