Policy Research Working Paper 10308 The Effects of Differential Exposure to COVID-19 on Educational Outcomes in Guatemala Andres Ham Emmanuel Vazquez Monica Yanez-Pagans Education Global Practice February 2023 Policy Research Working Paper 10308 Abstract This paper studies the effects of differential exposure to show that municipalities with a higher warning index had COVID-19 on educational outcomes in Guatemala. The significantly larger dropout, lower promotion rates, and a government adopted a warning index (ranging from 0 to greater share of students switching from private to public 10) to classify municipalities by infection rates in 2020, schools. These effects were more pronounced during the which was then used by the Ministry of Education in 2021 first year of the pandemic. The findings show differential to establish a “stoplight” system for in-person instruc- effects by the level of instruction, with greater losses for tion. Using administrative panel data for all students in younger children in initial and primary education. The Guatemala, the study employs a difference-in-differences results are robust to specification choice, multiple hypoth- strategy that leverages municipal differences over time in esis adjustments, and placebo experiments, suggesting that the warning index to estimate the effects of the pandemic the pandemic has had heterogeneous consequences. on dropout, promotion, and school switching. The results This paper is a product of the Education Global Practice. It is part of a larger effort by the World Bank to provide open access to its research and make a contribution to development policy discussions around the world. Policy Research Working Papers are also posted on the Web at http://www.worldbank.org/prwp. The authors may be contacted at a.ham@uniandes.edu.co, evazquez@cedlas.org, and myanezpagans@worldbank.org. The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team The Effects of Differential Exposure to COVID-19 on Educational Outcomes in Guatemala1 Andres Ham2 Emmanuel Vazquez3 Monica Yanez-Pagans4 Keywords: education, dropout, promotion, school switching, government policy, COVID-19. JEL Classification: H12, I21, I24, I25, I28. 1 This research was carried out as a background paper for the 2022 Central America Human Capital Review by the World Bank (P177403). The World Bank provided access to anonymized data to conduct the analyses in this paper. 2 Associate Professor, School of Government, Universidad de los Andes. Email: a.ham@uniandes.edu.co. 3 Senior Researcher, Center for Distributive, Labor and Social Studies (CEDLAS), Instituto de Investigaciones Económicas, Facultad de Ciencias Económicas, Universidad Nacional de La Plata. Email: evazquez@cedlas.org. 4 Senior Economist. Education Global Practice. The World Bank. Email: myanezpagans@worldbank.org. 1. Introduction The COVID-19 pandemic led many education systems to interrupt in-person instruction, moving towards virtual learning during the first year of the health emergency in 2020 and to hybrid instruction in 2021 and 2022. Generalized school closures and interruptions due to the pandemic have been shown to have had a negative impact on the global stock of schooling and learning measures since the beginning of the health crisis (García, 2020; Azevedo et al., 2021; Halloran et al, 2021). Disruptions to schooling are expected to affect student attendance, performance, and overall educational attainment in both the short and long runs (Marcotte and Hemelt, 2008; Alexander et al., 2016; Meyers and Thomasson, 2017). However, while losses are expected generally, there remains much to learn about the differential effects of the pandemic on educational outcomes. This paper estimates the differential effects of higher exposure to COVID-19 infections on dropout, promotion, and school switching in Guatemala. Like many Latin American countries, in- person schooling was interrupted to help contain the spread of the virus. The Guatemalan government developed a municipal-level warning index to monitor the spread of COVID-19. Over time, lockdown measures were relaxed as vaccines became readily available. Starting in 2021, the Ministry of Education used the aforementioned warning index to implement a “stoplight” system in schools across the 341 municipalities in the country.5 Districts classified in green could carry out in-person instruction without restrictions; those in yellow and orange were obliged to use hybrid instruction, with the main difference being the social distancing requirements between 5 Ministry of Health’s Ministerial Agreement No. 300-2020 related to the Sanitary Guidelines for the Prevention and Control of SARS-CoV-2 infections and other epidemics in schools from the national education system published on December 22, 2020. 2 yellow (2.5 square meters between students) and orange (4 square meters between persons); and finally, municipalities classified in red were only allowed to conduct virtual schooling activities. We employ a difference-in-differences strategy given that we have administrative records for all registered students in the education system over time. Therefore, we construct a panel that tracks the same students over 2018-2022, providing two pre-COVID-19 and two post-COVID-19 years. These data contain information on students, their schools, and their municipality of attendance. We believe these records are thus well-suited to estimate the effects of differential municipal exposure to COVID-19 infections on student-level educational outcomes in the short- term. Results show that that locations with greater exposure to COVID-19 infections had greater educational losses. Higher values of the COVID-19 warning index increased dropout rates between 0.7-0.8 percentage points across the Guatemalan education system, an increase of about 8.1% compared to the pre-pandemic average. We also see a reduction of 2.4% in effective promotion rates6 and an increase in movers from private to public schools of 10% compared to pre-pandemic levels, with a corresponding reduction in the other direction (public to private schools) of 6.3%. We explore further heterogeneity in two dimensions: over time and by educational level. Effects were stronger during the first year of the pandemic: dropout increased by 10.5% in 2020 and 5.8% in 2021; promotion fell by 3.4% and 0.7%; switching from private to public schools increased by 0.2% in 2020 and was unchanged in 2021, while the reverse direction shows a decrease of 0.2% in both years. We reject the null hypothesis that estimated effects are equal in 2020 and 2021 across outcomes and specifications. We also find greater losses for younger children from higher COVID-19 exposure. Effects for most outcomes are significant for initial 6 Effective promotion is a binary indicator that is equal to one if a student is observed today in a higher grade than they were registered in the previous year and zero otherwise. By definition, this variable excludes dropouts. 3 (pre-primary) and primary schooling but insignificant for students in lower and upper secondary schools. Estimates using the time spent by each municipality in different color categories provide qualitatively similar results compared to findings using values of the warning index: an increase in dropout, reduction in promotion, and an increase in student switching from private to public schools when comparing municipalities in orange and red with those in green and yellow. We cannot reject that effects for municipalities that spent more time in the orange and red categories are statistically identical, suggesting no distinction between the highest warning levels. Our results depend on the assumption that outcome trends in municipalities with greater and lower warning indices were parallel before the pandemic. While we cannot test this directly because the warning index did not exist before the pandemic, we carry out a placebo experiment to verify our parallel trends assumption. First, we drop observations from 2021. Then, we assign warning index values from 2020 to 2019 and indices from 2021 to 2020, as if the pandemic had begun one year earlier. We estimate differential effects of the warning index by year (for 2019 and 2020). The parallel trends assumption holds for dropout but is satisfied in part for other outcomes. However, pre-existing differences in trends are in the opposite direction to our effect estimates and economically small in magnitude, which lends some support that our main results are adequately approximating the differential effects of exposure to COVID-19 on educational outcomes. We expect the results from this study to contribute additional evidence to the growing literature quantifying the short-term human capital costs from the COVID-19 pandemic in developing countries (Tadesse and Muluye, 2020; World Bank, 2021; Bundervoet et al., 2022). While previous studies have documented the generalized negative consequences of the crisis (Moscoviz and Evans, 2022), we argue that the pandemic did not affect all children equally. We 4 also expect our results to contribute to the conversation on the costs of the pandemic not only across space, but educational levels. Our findings indicate that the youngest children were the most affected. Given the importance of early childhood and primary education for human capital development, this evidence provides input for policy makers to design interventions that could help mitigate the medium- and long-term consequences of the pandemic (Fuchs-Schündeln et al., 2022). Therefore, we hope that this research will be a timely guide for educational policy in Guatemala and other developing countries with similar education systems. While we are unable to study the consequences of the pandemic on learning, documenting that some students were more affected than others is a necessary step to consider when defining recovery policies. The remainder of this paper is organized as follows. The next section describes trends in educational outcomes in Guatemala before and after the COVID-19 pandemic and introduces the warning index for infection rates developed by the government and used by the Ministry of Education to determine school restrictions on in-person instruction. Section 3 describes the administrative panel data and our selected educational outcomes. Section 4 discusses our identification strategy to estimate the effects of differential exposures to COVID-19. Section 5 presents our findings and tests their robustness. We conclude in Section 6. 2. COVID-19 and educational outcome trends in Guatemala 2.1. Changes in educational outcomes before and after the pandemic Estimates using Guatemalan administrative records indicate that between-year school dropout increased between 2018 and 2021.7 While 7.8% of school-age children dropped out in 2019, this 7 We define between-year school dropout when a student is observed in one school year but does not appear in the records for the following academic year. This differs from within-year dropout when a student does not finish the current school year. Throughout the remainder of the paper, we refer to between-year dropout solely as “dropout”. 5 percentage increased to 8.7% in 2020 and further to 11.6% in 2021 as Figure 1 shows. The figure also indicates an increase in effective promotion rates during the pandemic (from 81 to 87 percent),8 reflecting the automatic promotion policy implemented in the country during the duration of the crisis.9 We also find evidence of greater movements from private to public schools, with a corresponding reduction in the opposite direction. These changes in the composition of the student body provide insights into household behavior during a crisis, since parents’ may consider that private schooling no longer provides greater net benefits, and thus move their children from these institutions into lower cost or free public institutions when there are fewer available resources due to economic shocks (Shafiq, 2010; Azevedo et al., 2021). However, these are not the only stylized facts that appear in these data. World Bank (2022) provides an in-depth overview of trends in educational outcomes in Guatemala and other neighboring countries, highlighting that Guatemala remains behind many other countries with respect to human capital outcomes. The report finds differences in outcome trends across student characteristics, with greater dropout rates for boys, rural students, those in low-income households, and older children who attend lower and upper secondary schooling. This last stylized fact is important because educational attainment in Guatemala is below the Latin American average, with about 73% completing primary compared to more 91.6% for the region (SEDLAC, 2022). Most 8 Effective promotion is a binary variable that identifies whether a student is observed in a higher grade the following school year. By definition, it does not include dropouts nor graduating students with no next grade. 9 The promotion policy in the country during the pandemic was regulated by the Ministry of Education’s Ministerial Agreement No. 2690-2020 published on September 22, 2020, which provided the guidelines for students’ learning assessments, enrollment, promotion, and certification for school year 2020. For pre-primary education, the promotion was automatic as it had traditionally been in the past before the pandemic. For primary and secondary education, students could be promoted to the next grade if they scored a minimum of 60 points, calculated based on the assessments implemented and the activities conducted during face-to-face instruction and distance learning. In practice though there was a lot of controversy, in 2020, about the use of this evaluation method as the grading methodology allowed everyone to get at least 60 points. In 2021, these guidelines were revised, and the Ministerial Agreement No. 3474-2021 published on December 7, 2021 regulated the implementation of learning recovery agenda and established that students from primary and secondary who did not score the minimum grade will have up to two assessment opportunities to be promoted to the next grade. 6 of the observed dropout occurs during the transition from primary to secondary schooling (Adelman et al., 2018). Thus, the consequences of the pandemic on educational outcomes are likely to vary across student attributes and, additionally, may also differ across locations since not all districts were exposed to the same levels of infections during the pandemic. In fact, World Bank (2022) shows that department-level dropout rates ranged between 6.6% and 11.1% in Guatemala. Most research on the consequences of the pandemic has focused on estimating the generalized consequences on the entire population without considering that some locations were more exposed to the infection than others. Our objective here is to determine whether greater exposure to COVID-19 had larger effects on educational outcomes compared to lower levels of exposure. Figure 1. Changes in educational outcomes from 2018-2021 Source: Authors’ calculations from Guatemalan educational administrative records. Notes: The figure presents raw means by year for the entire sample in all educational levels. 7 2.2. Differential exposure to COVID-19 by municipality The Guatemalan government implemented lockdown measures on March 16, 2020. Throughout the pandemic, the government utilized a municipal-level warning index to classify municipalities according to the presence of the virus in each area. During 2020 and 2021, this index ranged between 0 and 10 and was updated on a biweekly basis by the Ministry of Health, which computed the index based on the number of new COVID-19 cases per 100,000 inhabitants and positivity rates in all municipalities. Higher values of this index denoted greater exposure to the virus in each municipality. Municipalities were classified as “yellow” when this index was between 2 and 5, “orange” when this index was greater than 5 and lower than 8, and “red” when the index was greater or equal to 8. This warning index and the “stoplight” system guided certain policy decisions throughout the crisis, including how schools were allowed to operate. Schools had been in session for about 40 days when lockdown began. During 2020, there was no in-person instruction, regardless of the warning index in each municipality. Given developments in knowledge on COVID-19, vaccine availability, and other factors, complete lockdown was relaxed beginning in 2021 according to the warning index. Ministerial agreement No. 300-2020 “Norma sanitaria para la prevención y control de infecciones por SARS-CoV-2 y otras epidemias, para los centros del sistema educativo nacional” (Ministerio de Salud, 2020) and its reform Ministerial Agreement No. 69-2021 “Reforma al Acuerdo Ministerial Número 300- 2020” (Ministerio de Salud, 2021), implemented a new system that set different conditions for schooling restrictions depending on the level of perceived risk in each municipality. Perceived risk was based on the district’s warning index and took the form of a “stoplight” system. 8 Table 1. Municipality-level school restrictions from warning index Color 2018 2019 2020 2021 Red n.a. n.a. Virtual Virtual Orange n.a. n.a. Virtual Hybrid Yellow n.a. n.a. Virtual Hybrid Green n.a. n.a. Virtual In person Source: Authors’ elaboration from Guatemalan legislation Notes: n.a.-Not applicable. Districts classified in green could resume in-person instruction without any distancing restrictions; those in yellow and orange were obliged to use hybrid instruction, with the main difference being the social distancing requirements between yellow (2.5 square meters between individuals) and orange (4 square meters between persons); and municipalities classified in red were only allowed to conduct virtual schooling activities. Table 1 summarizes these measures and how they have changed over time. Figure 2 maps the average warning index by municipality for pre-COVID-19 years (2018 and 2019) and post-COVID-19 years for which we have available data (2020 and 2021). Given that the index was unavailable before the pandemic, we assign a value of zero for pre-COVID-19 years. We observe that infection rates varied across the country, with some districts more exposed than others to infections from the virus. Mean warning levels increased across the entire country from 2020 to 2021. While 86 municipalities had a low warning index in 2020 (below 5), this number was only 4 in 2021. On the opposite end, 2 municipalities had a warning index above 8 in 2020, which increased to 31 in 2021. We leverage this district-level variation over time to estimate whether students in districts with greater warning indices have different outcomes than students in municipalities with lower indices, estimating the differential effects of exposure to COVID-19. 9 3. Data The Ministry of Education (MOE) in Guatemala has been constructing a comprehensive database of its students since 2009 (Adelman et al., 2018). Currently, all schools are required to provide complete lists of all students who are enrolled in each grade and the MOE assigns each an individual identifier to track them over time. Thus, these records are a longitudinal census of all students in the country for as long as they remain in the education system. For the purposes of this study, these records enable the identification of students who drop out and the year in which they leave school. The data also include student-level demographics (gender, age, location); academic information (level, grade, final results of their academic year: promoted, not promoted, or withdrew); as well as school attributes, such as sector (public; private; cooperative; municipal), teaching modality (bilingual; monolingual), and school identifiers. We also have information on the municipality and department where the student attends school. There are available data for two pre-COVID-19 years (2018 and 2019) and two post-pandemic years (2020 and 2021), which corresponds to the most recent and complete school year data available from the Guatemalan MOE. We believe these records are well-suited to estimate the effects of differential municipal exposure to COVID-19 infections on educational outcomes for all students. Table 2 shows descriptive statistics before and after the pandemic began. We have over 16 million observations across these four years. The table confirms the outcome trends shown in Figure 1, with higher dropout, promotion, and switching from private to public schools after the pandemic. Dropout is defined as between-year dropout: a student is observed in one school year but does not appear in the records the following academic year. Effective promotion is a binary indicator that is equal to one if a student is observed today in a higher grade than they were in the previous year and zero otherwise. School switching is a binary indicator equal to one if a student 10 was in a private school the year before but in a public school today (switched to public) or if a student was in a public school the year before but in a private school today (switched to private). Approximately half the students are male, are 10 years old on average, less than half reside in urban areas and about 4% were graduating from upper secondary in the year we observe them in the panel. Among the students, 14.6% are in initial or pre-primary schooling, 57% in primary (grades 1-6), 18.6% in lower secondary (grades 7-9), and 9.8% in upper secondary (above 9th grade). About 25% attend bilingual schools, which may provide instruction in both Spanish and an indigenous language. Just over 22% of students attend a private institution, and we can also see that the COVID-19 positivity rate after 2020 was on average 18% but ranged between 0 and 41 percent across municipalities. While having longitudinal data on students is ideal to observe changes over time, there are limitations due to their annual frequency. In an ideal situation, we would like to follow the same students on a weekly or monthly basis to better gauge the changing conditions with respect to the spread of COVID-19. However, the data are unable to provide this ideal setting that would better allow capturing the dynamics of school outcomes during the pandemic. This is not only an issue in Guatemala, but with most administrative educational records in developing countries. In the next section, we discuss how we reconcile the rapidly changing nature of infection rates with the available annualized administrative data and define an identification strategy to approximate the causal effects of differential COVID-19 exposure on educational outcomes in Guatemala. 11 Figure 2. Evolution of COVID-19 warning index in Guatemala from 2018-2021 Source: Authors’ calculations from Guatemalan municipal-level data. Notes: The figure presents the average warning index by municipality in each year, with comparable intervals over time. We do not have available information on the warning index for municipalities in grey (denoted as “No data”). Table 2. Descriptive statistics from educational administrative records Pre (2018-2019) Post (2020-2021) Mean (SD) Mean (SD) Outcomes Dropout 0.086 (0.281) 0.102 (0.302) Effective promotion 0.802 (0.398) 0.823 (0.381) Switched to public school 0.010 (0.098) 0.011 (0.103) Switched to private school 0.032 (0.176) 0.023 (0.149) Student attributes Male 0.516 (0.500) 0.513 (0.500) Age 10.722 (4.145) 10.953 (4.409) Lives in urban area 0.414 (0.493) 0.407 (0.491) Graduating 0.041 (0.198) 0.041 (0.199) Educational level Initial 0.146 (0.353) 0.148 (0.355) Primary 0.570 (0.495) 0.576 (0.494) Lower secondary 0.186 (0.389) 0.179 (0.383) Upper secondary 0.098 (0.297) 0.097 (0.295) School attributes Bilingual instruction 0.245 (0.430) 0.251 (0.433) Private institution 0.228 (0.420) 0.221 (0.415) Municipality attributes COVID-19 positivity rate 0.000 - 18.837 (5.028) Observations 8,199,383 8,259,519 Source: Authors’ calculations from Guatemalan educational administrative records. Notes: This table shows pre- and post-COVID averages for students and their standard deviation in parentheses. 4. Identification strategy and empirical approach Given the available data, we estimate results using a difference-in-difference approach with two- way fixed effects. Since we have two measures of the intensity of COVID-19 exposure, we propose two forms of estimation. The first uses the average warning index for the municipality in each year as the main explanatory variable and the second uses the amount of the year spent in different color categories, mainly orange and red. Therefore, the base category is green and yellow. Figure 3 shows the distribution of the warning index in post-COVID-19 years (2020 and 2021) and the amount of time each municipality spends (as a percentage of fortnights) in yellow, orange, and red (see Table A.1 for descriptive statistics on the warning index and the share of time in each category). Figure 3. Distribution of warning index and time spent in each color category Source: Authors’ calculations from Guatemalan municipal-level data. Notes: The figure presents the distribution of the warning index and time spent in each color category (as a fraction of the school year) in post-COVID years, 2020 and 2021. The dotted vertical line shows the mean. The average warning index in 2020 was 5.73, which increased to 6.94 in 2021; with standard deviations of 0.970 and 0.865, respectively. During 2020, municipalities spent 15.4% of the total school year in green (pre-pandemic weeks), 28.1% in yellow, 24.6% in orange, and 31.9% in red. In 2021, these shares shifted to 0, 32.2, 25, and 42.8 percent for green, yellow, orange, and 14 red; respectively.10 As stated beforehand, we set the warning index equal to zero in pre-COVID- 19 years and consider that municipalities were all in the green category before the crisis began. Our strategy assumes that municipalities with a higher warning index or greater exposure to COVID-19 infections had parallel trends in dropout, effective promotion rates, and school switching before the pandemic. We formally test whether this assumption holds in the next section, conducting placebo experiments given data availability and other empirical constraints. Our first estimation strategy involves running the following regression: = + + + {,} + + (1) where represents the outcome for individual i in school s living in municipality m at time t. is the municipality-specific warning index that varies over time and therefore estimates how an increase of one unit in the warning index affects the selected outcome. is a matrix that includes student- and school-level controls such as gender, urban residence, level of schooling, grade, and whether the school provides bilingual instruction. We also control for secular time trends with year fixed effects. We include fixed effects at two levels of aggregation: municipality (m) and schools (s). We do not include a specification with student-level fixed effects because it may result in sample selection that may exacerbate biases due to sample selection.11 10 Figure A.1 in the Appendix shows a municipal-level map of time spent in each color category during 2020 and 2021. 11 For instance, some students appear only once in the data because they drop out and the system does not follow them after they exit the school system. In this case, including student-level fixed effects in a regression would remove these observations from the data, only keeping students who are observed multiple times throughout the period. Given that observing an individual multiple times means lower dropout, we would be biasing our estimates by including only “survivors” across the education system over time. Hence, the gains from controlling for student-level fixed effects do not outweigh the costs in our application to the Guatemalan data. 15 Our second strategy tries to determine whether the amount of time spent in certain color categories determined by the warning index has different effects on educational outcomes: = + 1 + 2 + + {,} + + (2) where in this case 1 estimates the effect of 1 percent more time in the year spent in the orange category on educational outcomes and 2 the same effect but in the red category. We highlight that these coefficients must be compared to the time spent in green and yellow categories. This decision is taken because no municipalities were in the green category during 2021, so we are unable to use that category as the single comparison point throughout the selected period. We also include fixed effects by municipality or school, as well as year fixed effects in this specification. All regressions have clustered standard errors by municipality since the warning index and the time spent in each color category varies at this level (Abadie et al., 2022). We also follow the best practices in the difference-in-difference literature (Roth et al., 2022), although we note that even though the warning index may vary within each year, our data does not allow taking these changes into account. Even if we wanted to control for differential changes in exposure to COVID- 19, as suggested by Goodman-Bacon (2021), we are unable to do so with the available data because they are annual records that cannot be disaggregated by either month, week, or days. To further ensure the robustness of our results, we apply multiple hypothesis corrections to avoid any potentially misleading findings due to having multiple outcomes, specifications, and parameters of interest, which leads to estimating several coefficients using a single source of variation. We present q-values in addition to standard p-values in all regression tables, calculated using the method proposed by Benjamini and Hochberg (1995) that controls for the false discovery 16 rate (FDR) described in Anderson (2008). We also conduct placebo experiments to lend further support that any results we find from applying our empirical strategy are due to differences in exposure to COVID-19 and not any other pre-existing factors that differ across municipalities. 5. Results The results of estimating Equation (1) using the municipal-level warning index are shown in Table 3. Each column reports a different specification, which varies in whether we use controls or not and the level of fixed effects that we employ. The table presents the coefficient of interest, which captures how a change of 1 point in the warning index affects educational outcomes: dropout, effective promotion, switching from private to public school and vice versa (public to private). We also include the pre-COVID-19 means to provide a relative measure of the estimated effects. Our findings show that higher values of the warning index increased dropout rates between 0.7-0.8 percentage points, an effect consistent across specifications with and without individual and school-level controls and including either municipality or school fixed effects. Compared to the pre-COVID-19 mean of 3.2 percent, this represents an increase of 8.1% in between-year school dropout. We also see a reduction in effective promotion rates of around 2 percentage points, a fall of 2.4% with respect to the pre-pandemic mean. There is an increase in movers from private to public schools of 10% compared to pre-pandemic levels and a reduction in the other of 6.3%. All coefficients are robust to multiple hypothesis adjustments, since they remain significant after adjusting for the 16 hypotheses tested in Table 3. These results suggest that municipalities with greater exposure to COVID-19 infections had greater educational losses in Guatemala. 17 Table 3. The effect of COVID-19 exposure on educational outcomes (1) (2) (3) (4) A. Dropout Warning index 0.008 0.007 0.007 0.007 (0.001)*** (0.001)*** (0.001)*** (0.001)*** [0.001] [0.001] [0.001] [0.001] Mean outcome 0.086 0.086 0.086 0.086 2 Adjusted R 0.007 0.079 0.090 0.128 Observations 16,458,902 16,394,979 16,458,819 16,394,924 B. Effective promotion Warning index -0.020 -0.019 -0.019 -0.019 (0.002)*** (0.002)*** (0.002)*** (0.002)*** [0.001] [0.001] [0.001] [0.001] Mean outcome 0.802 0.802 0.802 0.802 2 Adjusted R 0.022 0.068 0.095 0.118 Observations 13,337,281 13,301,355 13,337,055 13,301,145 C. Switched to public school Warning index 0.001 0.001 0.001 0.001 (0.000)*** (0.000)*** (0.000)*** (0.000)*** [0.001] [0.001] [0.001] [0.001] Mean outcome 0.010 0.010 0.010 0.010 Adjusted R2 0.005 0.013 0.101 0.100 Observations 16,458,902 16,394,979 16,458,819 16,394,924 D. Switched to private school Warning index -0.002 -0.002 -0.002 -0.002 (0.000)*** (0.000)*** (0.000)*** (0.000)*** [0.001] [0.001] [0.001] [0.001] Mean outcome 0.032 0.032 0.032 0.032 2 Adjusted R 0.003 0.053 0.066 0.100 Observations 16,458,902 16,394,979 16,458,819 16,394,924 Controls No Yes No Yes Year effects Yes Yes Yes Yes Fixed effects Municipality Municipality School School Source: Authors’ calculations from Guatemalan educational administrative records. Notes: The table reports difference in difference estimates of the effects of differential exposure to COVID-19 (see Section 4 for details). Each column presents results from a separate regression for each outcome. Clustered standard errors by municipality are shown in parentheses. Columns 1 and 3 estimates Equation (1) with no controls, Columns (2) and (4) include controls for gender, urban residence, level of schooling, grade, and whether the school provides bilingual instruction. We present q-values that adjust for multiple hypothesis testing in brackets, calculated using the method by Benjamini and Hochberg (1995) that controls for the false discovery rate described in Anderson (2008). Significance levels: ***p<0.01; **p<0.05; *p<0.10. 18 We explore whether these estimated education losses differ over time, by interacting the warning index with year dummies for 2020 and 2021. This procedure allows determining whether the effects in 2020 and 2021 were similar or different. The results for these regressions are presented in Table 4. Effects were higher in the first year of the pandemic for all outcomes. Dropout increased by 10.5% in 2020 and 5.8% in 2021; while promotion fell by 3.4% and 0.7%, respectively. Switching from private to public schools increased by 0.2% in 2020 and was unchanged in 2021, while the reverse direction shows a decrease of 0.2% in both years. The estimated coefficients are once again robust to multiple hypothesis testing as evidenced by the estimated q-values in Table 4. We also test for the equality between coefficients, rejecting the null hypothesis that effects are equal in both post-pandemic years across outcomes and specifications. While our results suggest that greater exposure to COVID-19 had larger educational costs in municipalities with higher infection rates and effects were greater in 2020 but not 2021, we are estimating an average effect by pooling students across all educational levels, from pre-primary to secondary schooling. We therefore choose the last specification, which includes individual- and student-level controls with school fixed effects, to estimate separate regressions by educational level: initial (pre-primary), primary, lower secondary, and upper secondary in Table 5. 19 Table 4. The effects of COVID-19 exposure on educational outcomes by year (1) (2) (3) (4) A. Dropout Warning index x 2020 0.009 0.009 0.009 0.009 (0.001)*** (0.001)*** (0.001)*** (0.001)*** [0.001] [0.001] [0.001] [0.001] Warning index x 2021 0.005 0.005 0.005 0.005 (0.001)*** (0.001)*** (0.001)*** (0.001)*** [0.001] [0.001] [0.001] [0.001] Pr(2020=2021) 0.006 0.005 0.007 0.009 Mean outcome 0.086 0.086 0.086 0.086 Adjusted R2 0.007 0.079 0.090 0.128 Observations 16,458,902 16,394,979 16,458,819 16,394,924 B. Effective promotion Warning index x 2020 -0.029 -0.028 -0.027 -0.027 (0.003)*** (0.003)*** (0.003)*** (0.003)*** [0.001] [0.001] [0.001] [0.001] Warning index x 2021 -0.006 -0.006 -0.007 -0.006 (0.003)** (0.003)** (0.003)** (0.003)** [0.035] [0.054] [0.027] [0.035] Pr(2020=2021) 0.000 0.000 0.000 0.000 Mean outcome 0.802 0.802 0.802 0.802 Adjusted R2 0.022 0.069 0.095 0.119 Observations 13,337,281 13,301,355 13,337,055 13,301,145 Controls No Yes No Yes Year effects Yes Yes Yes Yes Fixed effects Municipality Municipality School School (Continues on next page) 20 Table 4. The effects of COVID-19 exposure on educational outcomes by year (continued) (1) (2) (3) (4) C. Switched to public school Warning index x 2020 0.002 0.002 0.002 0.002 (0.000)*** (0.000)*** (0.000)*** (0.000)*** [0.001] [0.001] [0.001] [0.001] Warning index x 2021 -0.001 -0.001 -0.000 -0.000 (0.000)*** (0.000)*** (0.000) (0.000) [0.001] [0.001] [0.131] [0.130] Pr(2020=2021) 0.000 0.000 0.000 0.000 Mean outcome 0.010 0.010 0.010 0.010 Adjusted R2 0.005 0.013 0.101 0.100 Observations 16,458,902 16,394,979 16,458,819 16,394,924 D. Switched to private school Warning index x 2020 -0.002 -0.002 -0.002 -0.002 (0.000)*** (0.000)*** (0.000)*** (0.000)*** [0.001] [0.001] [0.001] [0.001] Warning index x 2021 -0.001 -0.001 -0.002 -0.002 (0.000)*** (0.000)*** (0.000)*** (0.000)*** [0.001] [0.001] [0.001] [0.001] Pr(2020=2021) 0.000 0.000 0.026 0.014 Mean outcome 0.032 0.032 0.032 0.032 Adjusted R2 0.003 0.053 0.066 0.100 Observations 16,458,902 16,394,979 16,458,819 16,394,924 Controls No Yes No Yes Year effects Yes Yes Yes Yes Fixed effects Municipality Municipality School School Source: Authors’ calculations from Guatemalan educational administrative records. Notes: The table reports difference in difference estimates of the effects of differential exposure to COVID-19 (see Section 4 for details). Each column presents results from a separate regression for each outcome. Clustered standard errors by municipality are shown in parentheses. Columns 1 and 3 estimates Equation (1) with no controls, Columns (2) and (4) include controls for gender, urban residence, level of schooling, grade, and whether the school provides bilingual instruction. We present q-values that adjust for multiple hypothesis testing in brackets, calculated using the method by Benjamini and Hochberg (1995) that controls for the false discovery rate described in Anderson (2008). Significance levels: ***p<0.01; **p<0.05; *p<0.10. 21 We observe greater losses for the youngest children from higher COVID-19 exposure. In terms of dropout, effects are significant for initial and primary schooling levels, indicating an increase of 18.2% for the former and 6.8% for the latter due to the pandemic. Effects for lower and upper secondary are statistically insignificant when looking at q-values. The same pattern carries over when observing effective promotion, although we do not have information for children in pre-primary education as the promotion outcome measure that we are using requires having data for the student from the prior year to understand if s/he was promoted to the next grade or not. Grade promotion falls by 2.2% for students in primary education and we find no effect for those attending secondary. School switching behavior also shows that the effects are concentrated among the youngest. Changes from private to public schools increased 44.4%, 12.5%, and were unchanged for students in initial, primary, and secondary education, respectively. In turn, changes from public to private schools fell across all educational levels, by 16%, 9.1%, 1.2%, and 11.1% for initial, primary, lower secondary, and upper secondary, respectively. Most effects we find are robust to multiple hypothesis adjustments, reinforcing our finding that younger children were more affected by the COVID-19 pandemic in educational terms. Given that the warning index was used by the government to create color categories, we also explore whether time spent in the classifications with a greater percentage of COVID-19 infections (orange and red) had different effects compared to time spent in green or yellow (lower infection rates) classification. We estimate Equation (2) for the four specifications used throughout the paper in Appendix Table A.2. Figure 4 summarizes the results graphically. 22 Table 5. The effect of COVID-19 exposure on educational outcomes by level of instruction Initial Primary Lower Upper secondary secondary A. Dropout Warning index 0.006 0.005 0.004 0.001 (0.001)*** (0.001)*** (0.002)* (0.002) [0.001] [0.001] [0.113] [0.783] Mean outcome 0.033 0.073 0.176 0.070 2 Adjusted R 0.040 0.146 0.167 0.086 Observations 2,412,052 9,416,097 2,981,696 1,585,079 B. Effective promotion Warning index - -0.018 0.005 -0.000 (0.003)*** (0.003)* (0.005) [0.001] [0.112] [0.930] Mean outcome 0.815 0.754 0.837 Adjusted R2 0.107 0.156 0.133 Observations 9,413,743 2,969,515 917,887 C. Switched to public school Warning index 0.004 0.001 0.000 -0.000 (0.001)*** (0.000)*** (0.000)** (0.000) [0.001] [0.001] [0.065] [0.153] Mean outcome 0.009 0.008 0.018 0.004 2 Adjusted R 0.142 0.114 0.067 0.015 Observations 2,412,052 9,416,097 2,981,696 1,585,079 D. Switched to private school Warning index -0.004 -0.002 -0.001 -0.001 (0.000)*** (0.000)*** (0.001)** (0.000)** [0.001] [0.001] [0.058] [0.065] Mean outcome 0.025 0.022 0.082 0.009 2 Adjusted R 0.069 0.092 0.165 0.030 Observations 2,412,052 9,416,097 2,981,696 1,585,079 Controls Yes Yes Yes Yes Year effects Yes Yes Yes Yes Fixed effects School School School School Source: Authors’ calculations from Guatemalan educational administrative records. Notes: The table reports difference in difference estimates of the effects of differential exposure to COVID-19 (see Section 4 for details). Each column presents results from a separate regression for each educational level using specification (4) in Table 3. Clustered standard errors by municipality are shown in parentheses. We present q-values that adjust for multiple hypothesis testing in brackets, calculated using the method by Benjamini and Hochberg (1995) that controls for the false discovery rate described in Anderson (2008). Significance levels: ***p<0.01; **p<0.05; *p<0.10. 23 Figure 4. The effects of COVID-19 exposure on educational outcomes by color Source: Authors’ calculations from Guatemalan educational administrative records. Notes: The figure presents the estimates for 1 and 2 and their 95% confidence intervals from Equation (2) that capture the effect of more time spent in the orange and red warning categories, respectively. We present results for all four specifications considered throughout the paper and which may be found in Appendix Table A.2. We find that effects are of the same sign as when using the warning index in Table 3. That is, being in the orange or red category has a negative effect on educational outcomes compared to time spent in green and yellow. However, these estimates capture the effect of an additional 1 percent of time spent in each category, so are not directly comparable to the results using the warning index. Our interest lies in determining whether the educational consequences of time spent in orange and red categories differ between themselves. We find that the effects of time spent in orange and red categories are statistically identical for all outcomes and specifications. We also explore differential effects by year and color in Appendix Table A.3, confirming that effects seem to be larger during the first year of the pandemic (2020) than the second year 24 (2021). However, there remains no visible difference between time spent in orange and red categories compared to green and yellow within each year. Results by educational level in Appendix Table A.4 confirm our previous results that younger children were more affected, but once again we find no statistically significant differences of longer time spent in either the orange or red category in dropout. While we find some differential effects in other outcomes, we are cautious with their interpretation given the number of estimated parameters and hypothesis tests carried out. Considering the complete set of results that differentiate by color category, we interpret the findings as suggesting that greater exposure maintains its negative effects on educational outcomes in Guatemala, but there is no difference between the two highest infection categories. The results presented in this section depend on the assumption that outcome trends in municipalities with greater and lower warning indices were parallel before the pandemic. In order to test the validity of this assumption, we carry out a placebo experiment. The values of the warning index are equal to zero before 2020 and we only have this information for 2020 and 2021. One way to test whether pre-period trends are parallel is to assume that the pandemic began in 2019. For this procedure, we drop data from 2021. Then, we assign the warning index values from 2020 to 2019 and the indices from 2021 to 2020, as if the pandemic had begun one year earlier. We then estimate differential effects of the warning index by year (for 2019 and 2020, respectively). If parallel trends hold, the coefficient on the interaction should be statistically insignificant or if trends were reversed, they should be of the opposite sign than our estimates in previous tables. Table 6 presents the results of this placebo experiment. The coefficients for the interaction between the “placebo” warning index and 2019 for dropout are all insignificant, suggesting that trends in this outcome were parallel between municipalities with higher and lower warning indices. The coefficient on the warning index with 2020 confirms the increase in dropout we observe in 25 Table 3, with a similar coefficient and percentage change with respect to the 2018 mean. For effective promotion, we find that trends were different before the pandemic, but they were moving in the opposite direction. That is, municipalities with higher COVID-19 exposure were slightly increasing their promotion rates between 0.5-0.6%. While not fully parallel, these differential trends are small and were reverted due to the pandemic as the interaction with 2020 shows. For school switching, we also find statistically significant trends which are small in magnitude. While different from zero, the actual trends report changes close to zero with respect to the mean. While this placebo exercise does not completely rule out parallel trends in school switching outcomes, we believe that the pre-existing difference in trends is economically irrelevant, and we can be confident that our estimates are attributable mainly to the pandemic. We conduct a second placebo experiment, whose results are shown in Appendix Table A.5. The exercise is similar in that it shifts when the pandemic began but changes the post-COVID-19 year we use from 2020 to 2021. That is, we first drop 2020 data from the estimation sample. Then, we assign the warning index values from 2020 to 2019 and the warning indices from 2021 remain the same, as if the pandemic began one year earlier. We then estimate differential effects of the warning index by year (for 2019 and 2021, respectively). The results are almost identical to those shown in Table 6, with dropout having parallel trends, effective promotion reverting those trends, and mixed results for school switching but still with effect magnitudes close to zero. 26 Table 6. Placebo effects of COVID-19 exposure on educational outcomes (1) (2) (3) (4) A. Dropout Warning index x 2019 0.000 0.001 0.001 0.000 (0.001) (0.001) (0.001) (0.001) [0.670] [0.626] [0.626] [0.626] Warning index x 2020 0.008 0.008 0.007 0.007 (0.002)*** (0.002)*** (0.002)*** (0.002)*** [0.003] [0.002] [0.001] [0.002] Pr(2019=2020) 0.000 0.000 0.000 0.000 Mean outcome 0.086 0.086 0.086 0.086 Adjusted R2 0.006 0.079 0.097 0.136 Observations 12,342,065 12,294,413 12,341,959 12,294,343 B. Effective promotion Warning index x 2019 0.005 0.005 0.005 0.004 (0.002)** (0.002)** (0.002)** (0.002)** [0.043] [0.046] [0.039] [0.046] Warning index x 2020 -0.025 -0.025 -0.025 -0.025 (0.005)*** (0.005)*** (0.004)*** (0.004)*** [0.001] [0.001] [0.001] [0.001] Pr(2019=2020) 0.000 0.000 0.000 0.000 Mean outcome 0.802 0.802 0.802 0.802 Adjusted R2 0.021 0.066 0.096 0.120 Observations 10,014,671 9,988,171 10,014,417 9,987,940 Controls No Yes No Yes Year effects Yes Yes Yes Yes Fixed effects Municipality Municipality School School (Continues on next page) 27 Table 6. Placebo effects of COVID-19 restrictions on educational outcomes (continued) (1) (2) (3) (4) C. Switched to public school Warning index x 2019 -0.001 -0.000 -0.000 -0.000 (0.000)*** (0.000)*** (0.000)*** (0.000)*** [0.004] [0.012] [0.004] [0.004] Warning index x 2020 0.002 0.002 0.002 0.002 (0.000)*** (0.000)*** (0.000)*** (0.000)*** [0.001] [0.001] [0.001] [0.001] Pr(2019=2020) 0.000 0.000 0.000 0.000 Mean outcome 0.010 0.010 0.010 0.010 Adjusted R2 0.006 0.014 0.107 0.105 Observations 12,342,065 12,294,413 12,341,959 12,294,343 D. Switched to private school Warning index x 2019 0.000 0.000 0.000 0.000 (0.000) (0.000) (0.000) (0.000)* [0.130] [0.146] [0.130] [0.087] Warning index x 2020 -0.003 -0.003 -0.003 -0.003 (0.000)*** (0.000)*** (0.000)*** (0.000)*** [0.001] [0.001] [0.001] [0.001] Pr(2019=2020) 0.000 0.000 0.000 0.000 Mean outcome 0.032 0.032 0.032 0.032 Adjusted R2 0.003 0.053 0.069 0.103 Observations 12,342,065 12,294,413 12,341,959 12,294,343 Controls No Yes No Yes Year effects Yes Yes Yes Yes Fixed effects Municipality Municipality School School Source: Authors’ calculations from Guatemalan educational administrative records. Notes: The table reports placebo estimates of the effects of differential exposure to COVID-19. For this procedure, we drop data from 2021. Then, we assign the warning index values from 2020 to 2019 and the warning indices from 2021 to 2020, as if the pandemic began one year earlier. Each column presents results from a separate regression for each outcome. Clustered standard errors by municipality are shown in parentheses. Columns 1 and 3 estimates Equation (1) with no controls, Columns (2) and (4) include controls for gender, urban residence, level of schooling, grade, and whether the school provides bilingual instruction. We present q-values that adjust for multiple hypothesis testing in brackets, calculated using the method by Benjamini and Hochberg (1995) that controls for the false discovery rate described in Anderson (2008). Significance levels: ***p<0.01; **p<0.05; *p<0.10. 28 6. Conclusion This paper estimates the differential effects of exposure to COVID-19 on educational outcomes. We take advantage of the municipal-level warning index that the Guatemalan government developed, with lower values denoting fewer COVID-19 cases and higher values greater infections, to implement a difference-in-differences strategy using student-level administrative panel data. We are therefore able to estimate how the pandemic affected dropout, effective promotion, and school switching behavior (private to public and vice versa) across the entire school system. Results show that municipalities with a higher warning index had significantly larger dropout, lower promotion rates, and a greater share of students switching from private to public schools. These effects are more pronounced during the first year of the pandemic. We also find differential effects by level of instruction, with greater losses for the youngest children who are enrolled in initial and primary education. These results are robust when using the warning index and time spent in different color categories, and are also consistent across specifications, still significant after multiple hypothesis adjustments, and mostly validated by placebo experiments. The literature on the consequences of the pandemic has grown over the past few years, especially in terms of quantifying differences among population groups. While this study takes a step in this direction with respect to extensive margin educational outcomes, we do not have information to determine effects on intensive margin results such as the quality of education. Some research has already found generalized learning losses (García, 2020; Tadesse and Muluye, 2020; Bundervoet et al., 2022; Marín et al., 2022; Singh et al., 2022), but a similar analysis would be warranted to better understand other impacts from the pandemic. These potentially unequal effects are especially relevant in terms of education since they have immediate impacts as well as 29 potentially longer-term consequences on the accumulation of human capital. Further evidence is required to better determine policy responses to reduce long-term losses. As the world turns the page on the pandemic, new challenges will arise. Further research is required in different settings to better understand how these past few years have affected different domains of life. Education is important due to its relationship to development over the life cycle and its correlation with improved living standards. However, education is multidimensional and involves understanding its extensive and intensive margins for different populations, since as we show here, the effects in Guatemala were not equal across the school system. Additionally, other important aspects such as mental health have become a growing concern during and after lockdowns. Gathering an evidence base on these matters is pivotal to define the way forward and guarantee long-term recovery in all countries and for all peoples. 30 References Abadie, A., Athey, S., Imbens, G. W., Wooldridge, J. (2022). When should you adjust standard errors for clustering? (No. w24003). National Bureau of Economic Research. Adelman, M., Haimovich, F., Ham, A., Vazquez, E. (2018). Predicting school dropout with administrative data: new evidence from Guatemala and Honduras. Education Economics, 26(4), 356-372. Alexander, K., Pitcock, S., Boulay, M. C. (2016). The Summer Slide: What we know and can do about Summer Learning Loss. Teachers College Press. Anderson, M. L. (2008). Multiple inference and gender differences in the effects of early intervention: A reevaluation of the Abecedarian, Perry Preschool, and Early Training Projects. Journal of the American Statistical Association, 103, 1481–1495. Azevedo, J. P., Hasan, A., Goldemberg, D., Geven, K., Iqbal, S. A. (2021). Simulating the potential impacts of COVID-19 school closures on schooling and learning outcomes: A set of global estimates. The World Bank Research Observer, 36(1), 1-40. Benjamini, Y., Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), 57, 289–300. Bundervoet, T., Dávalos, M. E., & Garcia, N. (2022). The short-term impacts of COVID-19 on households in developing countries: An overview based on a harmonized dataset of high- frequency surveys. World Development, 105844. Fuchs-Schündeln, N., Krueger, D., Ludwig, A., & Popova, I. (2022). The long-term distributional and welfare effects of Covid-19 school closures. The Economic Journal, 132(645), 1647- 1683. García, S. (2020). COVID-19 and primary and secondary education: the impact of the crisis and public policy implications for Latin America and the Caribbean. UNDP Covid-19 Policy Document, 20. Goodman-Bacon, A. (2021). Difference-in-differences with variation in treatment timing. Journal of Econometrics, 225(2), 254-277. Halloran, C., Jack, R., Okun, J. C., Oster, E. (2021). Pandemic schooling mode and student test scores: Evidence from US States (No. w29497). National Bureau of Economic Research. Marín, L., Pico, M. R., Maldonado, D., García, S. (2022). Desigualdad en el aprendizaje durante el COVID-19: evidencia para estudiantes de secundaria en Colombia (No. 020157). Marcotte, D., Hemelt, S. (2008). Unscheduled School Closings and Student Performance. Education Finance and Policy 3(3), 316-338. Ministerio de Educación (2020). Ministerial Agreement No. 2690-2020 “Autorizar el proceso de evaluación, registro, promoción y certificación de los aprendizajes para el ciclo escolar 2020 del Sistema Educativo Nacional”. Government of Guatemala. 31 Ministerio de Salud (2020). Ministerial Agreement No. 300-2020 “Norma sanitaria para la prevención y control de infecciones por SARS-CoV-2 y otras epidemias, para los centros del sistema educativo nacional”. Government of Guatemala. Ministerio de Salud (2021). Ministerial Agreement No. 69-2021 “Reforma al Acuerdo Ministerial Número 300-2020”. Government of Guatemala. Moscoviz, L., & Evans, D. K. (2022). Learning loss and student dropouts during the covid-19 pandemic: A review of the evidence two years after schools shut down (pp. 1-24). Center for Global Development. Meyers, K., Thomasson A. M. (2017). Paralyzed by Panic: Measuring the Effect of School Closures during the 1916 Polio Pandemic on Educational Attainment. NBER Working Paper 23890. https://www.nber.org/system/files/working_papers/w23890/w23890.pdf . Roth, J., Sant’Anna, P.H.C., Bilinski, A., Poe, J. (2022). What’s Trending in Difference-in- Differences? A Synthesis of the Recent Econometrics Literature. SEDLAC (2022). Socio-Economic Database for Latin America and the Caribbean (CEDLAS and The World Bank). February 2023. Shafiq, M. N. (2010). The effect of an economic crisis on educational outcomes: An economic framework and review of the evidence. Current Issues in Comparative Education, 12(2), 5-13. Singh, A., Romero, M., Muralidharan, K. (2022). Covid-19 Learning Loss and Recovery: Panel Data Evidence from India. NBER Working Paper 30552. https://www.nber.org/papers/w30552. Tadesse, S., & Muluye, W. (2020). The impact of COVID-19 pandemic on education system in developing countries: a review. Open Journal of Social Sciences, 8(10), 159-170. World Bank. (2021). Learners with Disabilities and COVID-19 School Closures: Findings from a Global Survey Conducted by the World Bank’s Inclusive Education Initiative. World Bank. World Bank (2022). Central America Human Capital Review. Latin America and the Caribbean Region. Report. 32 Online Appendix Figure A.1. Time spent in each color category during the pandemic Source: Authors’ calculations from Guatemalan municipal-level data. Notes: The figure presents the average time (in percent) spent in each color category by municipality in each year, with comparable intervals over time. We do not have available information on the warning index for municipalities in grey (denoted as “No data”). 33 Table A.1. Descriptive statistics for warning index and restrictions 2018 2019 2020 2021 Mean (SD) Mean (SD) Mean (SD) Mean (SD) Warning index 0.000 - 0.000 - 5.726 (0.969) 6.940 (0.865) Differential restrictions Time in green category 1.000 - 1.000 - 0.154 (0.000) 0.000 (0.000) Time in yellow category 0.000 - 0.000 - 0.281 (0.196) 0.322 (0.192) Time in orange category 0.000 - 0.000 - 0.246 (0.140) 0.250 (0.131) Time in red category 0.000 - 0.000 - 0.319 (0.211) 0.428 (0.183) Source: Authors’ calculations from Guatemalan municipal-level data. Notes: This table shows yearly averages and their standard deviation in parentheses. 34 Table A.2. The effects of COVID-19 exposure on educational outcomes by color (1) (2) (3) (4) A. Dropout Time in orange 0.033 0.030 0.032 0.033 (0.004)*** (0.005)*** (0.005)*** (0.006)*** [0.001] [0.001] [0.001] [0.001] Time in red 0.041 0.039 0.037 0.038 (0.006)*** (0.006)*** (0.005)*** (0.005)*** [0.001] [0.001] [0.001] [0.001] Pr(Orange=Red) 0.053 0.019 0.146 0.213 Mean outcome 0.086 0.086 0.086 0.086 Adjusted R2 0.007 0.079 0.090 0.128 Observations 16,458,902 16,394,979 16,458,819 16,394,924 B. Effective promotion Time in orange -0.057 -0.053 -0.055 -0.053 (0.015)*** (0.015)*** (0.015)*** (0.015)*** [0.001] [0.001] [0.001] [0.001] Time in red -0.096 -0.092 -0.093 -0.091 (0.014)*** (0.013)*** (0.013)*** (0.013)*** [0.001] [0.001] [0.001] [0.001] Pr(Orange=Red) 0.000 0.000 0.000 0.000 Mean outcome 0.802 0.802 0.802 0.802 Adjusted R2 0.022 0.068 0.095 0.118 Observations 13,337,281 13,301,355 13,337,055 13,301,145 Controls No Yes No Yes Year effects Yes Yes Yes Yes Fixed effects Municipality Municipality School School Source: Authors’ calculations from Guatemalan educational administrative records. Notes: The table reports difference in difference estimates of the effects of differential exposure to COVID-19 by color category: time spent in orange and red categories compared to time spent in green and yellow (see Section 4 for details). Each column presents results from a separate regression for each outcome. Clustered standard errors by municipality are shown in parentheses. Columns 1 and 3 estimates Equation (2) with no controls, Columns (2) and (4) include controls for gender, urban residence, level of schooling, grade, and whether the school provides bilingual instruction. We present q-values that adjust for multiple hypothesis testing in brackets, calculated using the method by Benjamini and Hochberg (1995) that controls for the false discovery rate described in Anderson (2008). Significance levels: ***p<0.01; **p<0.05; *p<0.10. 35 Table A.2. The effects of COVID-19 exposure on educational outcomes by color (continued) (1) (2) (3) (4) C. Switched to public school Time in orange -0.003 -0.003 -0.000 -0.000 (0.001)** (0.001)* (0.001) (0.001) [0.026] [0.055] [0.847] [0.855] Time in red 0.004 0.004 0.006 0.006 (0.001)*** (0.001)*** (0.001)*** (0.001)*** [0.001] [0.001] [0.001] [0.001] Pr(Orange=Red) 0.000 0.000 0.000 0.000 Mean outcome 0.010 0.010 0.010 0.010 Adjusted R2 0.005 0.013 0.101 0.100 Observations 16,458,902 16,394,979 16,458,819 16,394,924 D. Switched to private school Time in orange -0.004 -0.004 -0.008 -0.008 (0.001)*** (0.001)*** (0.001)*** (0.001)*** [0.002] [0.001] [0.001] [0.001] Time in red -0.010 -0.009 -0.011 -0.011 (0.001)*** (0.001)*** (0.001)*** (0.001)*** [0.001] [0.001] [0.001] [0.001] Pr(Orange=Red) 0.000 0.000 0.005 0.004 Mean outcome 0.032 0.032 0.032 0.032 Adjusted R2 0.003 0.053 0.066 0.100 Observations 16,458,902 16,394,979 16,458,819 16,394,924 Controls No Yes No Yes Year effects Yes Yes Yes Yes Fixed effects Municipality Municipality School School Source: Authors’ calculations from Guatemalan educational administrative records. Notes: The table reports difference in difference estimates of the effects of differential exposure to COVID-19 by color category: time spent in orange and red categories compared to time spent in green and yellow (see Section 4 for details). Each column presents results from a separate regression for each outcome. Clustered standard errors by municipality are shown in parentheses. Columns 1 and 3 estimates Equation (2) with no controls, Columns (2) and (4) include controls for gender, urban residence, level of schooling, grade, and whether the school provides bilingual instruction. We present q-values that adjust for multiple hypothesis testing in brackets, calculated using the method by Benjamini and Hochberg (1995) that controls for the false discovery rate described in Anderson (2008). Significance levels: ***p<0.01; **p<0.05; *p<0.10. 36 Table A.3. The effects of COVID-19 exposure on educational outcomes by color and year (1) (2) (3) (4) A. Dropout Time in orange x 2020 0.066 0.064 0.053 0.055 (0.021)*** (0.020)*** (0.015)*** (0.016)*** [0.003] [0.003] [0.001] [0.002] Time in orange x 2021 0.016 0.012 0.020 0.021 (0.006)*** (0.005)** (0.004)*** (0.004)*** [0.010] [0.025] [0.001] [0.001] Time in red x 2020 0.054 0.052 0.048 0.049 (0.008)*** (0.008)*** (0.007)*** (0.007)*** [0.001] [0.001] [0.001] [0.001] Time in red x 2021 0.025 0.023 0.024 0.025 (0.005)*** (0.005)*** (0.005)*** (0.005)*** [0.001] [0.001] [0.001] [0.001] Mean outcome 0.086 0.086 0.086 0.086 Adjusted R2 0.007 0.079 0.090 0.128 Observations 16,458,902 16,394,979 16,458,819 16,394,924 B. Effective promotion Time in orange x 2020 -0.148 -0.142 -0.129 -0.128 (0.050)*** (0.047)*** (0.041)*** (0.041)*** [0.005] [0.004] [0.003] [0.003] Time in orange x 2021 -0.007 -0.005 -0.014 -0.012 (0.013) (0.013) (0.011) (0.011) [0.579] [0.676] [0.222] [0.297] Time in red x 2020 -0.151 -0.147 -0.144 -0.142 (0.019)*** (0.018)*** (0.017)*** (0.017)*** [0.001] [0.001] [0.001] [0.001] Time in red x 2021 -0.025 -0.022 -0.028 -0.026 (0.014)* (0.014)* (0.014)** (0.014)* [0.076] [0.109] [0.051] [0.065] Mean outcome 0.802 0.802 0.802 0.802 Adjusted R2 0.022 0.069 0.095 0.119 Observations 13,337,281 13,301,355 13,337,055 13,301,145 Controls No Yes No Yes Year effects Yes Yes Yes Yes Fixed effects Municipality Municipality School School (Continues on next page) 37 Table A.3. The effects of COVID-19 exposure on educational outcomes by color and year (continued) (1) (2) (3) (4) C. Switched to public school Time in orange x 2020 0.004 0.005 0.004 0.004 (0.002)** (0.002)** (0.002)** (0.002)** [0.033] [0.033] [0.025] [0.025] Time in orange x 2021 -0.007 -0.007 -0.003 -0.003 (0.001)*** (0.001)*** (0.001)*** (0.001)*** [0.001] [0.001] [0.001] [0.001] Time in red x 2020 0.010 0.011 0.011 0.011 (0.001)*** (0.001)*** (0.001)*** (0.001)*** [0.001] [0.001] [0.001] [0.001] Time in red x 2021 -0.006 -0.005 -0.001 -0.001 (0.001)*** (0.001)*** (0.001)** (0.001)** [0.001] [0.001] [0.025] [0.025] Mean outcome 0.010 0.010 0.010 0.010 Adjusted R2 0.005 0.013 0.101 0.100 Observations 16,458,902 16,394,979 16,458,819 16,394,924 D. Switched to private school Time in orange x 2020 -0.009 -0.009 -0.009 -0.008 (0.003)*** (0.003)*** (0.002)*** (0.002)*** [0.004] [0.002] [0.001] [0.001] Time in orange x 2021 -0.002 -0.002 -0.008 -0.008 (0.001) (0.001) (0.001)*** (0.001)*** [0.237] [0.118] [0.001] [0.001] Time in red x 2020 -0.012 -0.012 -0.012 -0.012 (0.002)*** (0.001)*** (0.001)*** (0.001)*** [0.001] [0.001] [0.001] [0.001] Time in red x 2021 -0.006 -0.006 -0.009 -0.009 (0.001)*** (0.001)*** (0.001)*** (0.001)*** [0.001] [0.001] [0.001] [0.001] Mean outcome 0.032 0.032 0.032 0.032 Adjusted R2 0.003 0.053 0.066 0.100 Observations 16,458,902 16,394,979 16,458,819 16,394,924 Controls No Yes No Yes Year effects Yes Yes Yes Yes Fixed effects Municipality Municipality School School Source: Authors’ calculations from Guatemalan educational administrative records. Notes: The table reports difference in difference estimates of the effects of differential exposure to COVID-19 by color and year (see Section 4 for details). Each column presents results from a separate regression for each outcome. Clustered standard errors by municipality are shown in parentheses. Columns 1 and 3 estimates Equation (2 with no controls, Columns (2) and (4) include controls for gender, urban residence, level of schooling, grade, and whether the school provides bilingual instruction. We present q-values that adjust for multiple hypothesis testing in brackets, 38 calculated using the method by Benjamini and Hochberg (1995) that controls for the false discovery rate described in Anderson (2008). Significance levels: ***p<0.01; **p<0.05; *p<0.10. 39 Table A.4. The effects of COVID-19 exposure on educational outcomes by level of instruction and color Initial Primary Lower Upper secondary secondary A. Dropout Time in orange 0.041 0.033 -0.020 -0.009 (0.012)*** (0.004)*** (0.010)* (0.016) [0.002] [0.001] [0.099] [0.637] Time in red 0.037 0.028 0.013 0.004 (0.008)*** (0.004)*** (0.010) (0.014) [0.001] [0.001] [0.270] [0.799] Pr(Orange=Red) 0.623 0.209 0.007 0.295 Mean outcome 0.033 0.073 0.176 0.070 Adjusted R2 0.040 0.146 0.167 0.086 Observations 2,412,052 9,416,097 2,981,696 1,585,079 B. Effective promotion Time in orange - -0.058 0.069 0.079 (0.018)*** (0.016)*** (0.016)*** [0.003] [0.001] [0.001] Time in red - -0.089 0.039 0.030 (0.014)*** (0.013)*** (0.021) [0.001] [0.005] [0.225] Pr(Orange=Red) 0.032 0.098 0.000 Mean outcome 0.815 0.754 0.837 Adjusted R2 0.107 0.156 0.133 Observations 9,413,743 2,969,515 917,887 Controls Yes Yes Yes Yes Fixed effects School School School School (Continues on next page) 40 Table A.4. The effects of COVID-19 exposure on educational outcomes by level of instruction and color (continued) Initial Primary Lower Upper secondary secondary C. Switched to public school Time in orange 0.007 -0.002 -0.002 0.000 (0.004)* (0.001) (0.001) (0.001) [0.099] [0.270] [0.169] [0.899] Time in red 0.018 0.006 0.001 -0.001 (0.003)*** (0.001)*** (0.001) (0.001) [0.001] [0.001] [0.299] [0.433] Pr(Orange=Red) 0.000 0.000 0.000 0.078 Mean outcome 0.009 0.008 0.018 0.004 Adjusted R2 0.142 0.114 0.067 0.015 Observations 2,412,052 9,416,097 2,981,696 1,585,079 D. Switched to private school Time in orange -0.016 -0.011 0.009 0.001 (0.003)*** (0.002)*** (0.005)* (0.001) [0.001] [0.001] [0.124] [0.319] Time in red -0.021 -0.009 -0.004 -0.002 (0.002)*** (0.001)*** (0.003) (0.001) [0.001] [0.001] [0.295] [0.225] Pr(Orange=Red) 0.050 0.259 0.000 0.005 Mean outcome 0.025 0.022 0.082 0.009 Adjusted R2 0.069 0.092 0.165 0.030 Observations 2,412,052 9,416,097 2,981,696 1,585,079 Controls Yes Yes Yes Yes Fixed effects School School School School Source: Authors’ calculations from Guatemalan educational administrative records. Notes: The table reports difference in difference estimates of the effects of differential exposure to COVID-19 by color (see Section 4 for details). Each column presents results from a separate regression for each educational level using specification (4) in Table A.2. Clustered standard errors by municipality are shown in parentheses. We present q-values that adjust for multiple hypothesis testing in brackets, calculated using the method by Benjamini and Hochberg (1995) that controls for the false discovery rate described in Anderson (2008). Significance levels: ***p<0.01; **p<0.05; *p<0.10. 41 Table A.5. Placebo effects of COVID-19 exposure on educational outcomes (1) (2) (3) (4) A. Dropout Warning index x 2019 0.000 0.001 0.001 0.000 (0.001) (0.001) (0.001) (0.001) [0.005] [0.003] [0.005] [0.004] Warning index x 2021 0.008 0.008 0.007 0.007 (0.002)*** (0.002)*** (0.002)*** (0.002)*** [0.001] [0.001] [0.001] [0.001] Pr(2019=2021) 0.000 0.000 0.000 0.000 Mean outcome 0.086 0.086 0.086 0.086 Adjusted R2 0.006 0.079 0.097 0.136 Observations 12,342,065 12,294,413 12,341,959 12,294,343 B. Effective promotion Warning index x 2019 0.005 0.005 0.005 0.004 (0.002)** (0.002)** (0.002)** (0.002)** [0.943] [0.907] [0.943] [0.898] Warning index x 2020 -0.025 -0.025 -0.025 -0.025 (0.005)*** (0.005)*** (0.004)*** (0.004)*** [0.004] [0.007] [0.004] [0.005] Pr(2019=2021) 0.000 0.000 0.000 0.000 Mean outcome 0.802 0.802 0.802 0.802 Adjusted R2 0.021 0.066 0.096 0.120 Observations 10,014,671 9,988,171 10,014,417 9,987,940 Controls No Yes No Yes Year effects Yes Yes Yes Yes Fixed effects Municipality Municipality School School (Continues on next page) 42 Table A.5. Placebo effects of COVID-19 restrictions on educational outcomes (continued) (1) (2) (3) (4) C. Switched to public school Warning index x 2019 -0.001 -0.000 -0.000 -0.000 (0.000)*** (0.000)*** (0.000)*** (0.000)*** [0.263] [0.125] [0.091] [0.090] Warning index x 2021 0.002 0.002 0.002 0.002 (0.000)*** (0.000)*** (0.000)*** (0.000)*** [0.002] [0.005] [0.519] [0.519] Pr(2019=2021) 0.000 0.000 0.000 0.000 Mean outcome 0.010 0.010 0.010 0.010 Adjusted R2 0.006 0.014 0.107 0.105 Observations 12,342,065 12,294,413 12,341,959 12,294,343 D. Switched to private school Warning index x 2019 0.000 0.000 0.000 0.000 (0.000) (0.000) (0.000) (0.000)* [0.748] [0.698] [0.263] [0.131] Warning index x 2020 -0.003 -0.003 -0.003 -0.003 (0.000)*** (0.000)*** (0.000)*** (0.000)*** [0.001] [0.001] [0.001] [0.001] Pr(2019=2021) 0.000 0.000 0.000 0.000 Mean outcome 0.032 0.032 0.032 0.032 Adjusted R2 0.003 0.053 0.069 0.103 Observations 12,342,065 12,294,413 12,341,959 12,294,343 Controls No Yes No Yes Year effects Yes Yes Yes Yes Fixed effects Municipality Municipality School School Source: Authors’ calculations from Guatemalan educational administrative records. Notes: The table reports placebo estimates of the effects of differential exposure to COVID-19. For this procedure, we drop data from 2020. Then, we assign the warning index values from 2020 to 2019 and the warning indices from 2021 remain the same, as if the pandemic began one year earlier. Each column presents results from a separate regression for each outcome. Clustered standard errors by municipality are shown in parentheses. Columns 1 and 3 estimates Equation (1) with no controls, Columns (2) and (4) include controls for gender, urban residence, level of schooling, grade, and whether the school provides bilingual instruction. We present q-values that adjust for multiple hypothesis testing in brackets, calculated using the method by Benjamini and Hochberg (1995) that controls for the false discovery rate described in Anderson (2008). Significance levels: ***p<0.01; **p<0.05; *p<0.10. 43