Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) Report Number: ICRR0023076 1. Project Data Project ID Project Name P133195 AR Second Rural Education Improvement Country Practice Area(Lead) Argentina Education L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-84520 31-Dec-2019 241,740,322.94 Bank Approval Date Closing Date (Actual) 02-Dec-2014 22-Dec-2021 IBRD/IDA (USD) Grants (USD) Original Commitment 250,500,000.00 0.00 Revised Commitment 245,729,367.12 0.00 Actual 245,729,367.12 0.00 Prepared by Reviewed by ICR Review Coordinator Group Judyth L. Twigg Salim J. Habayeb Eduardo Fernandez IEGHC (Unit 2) Maldonado 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to the Loan Agreement (p. 5), the project's objective was "to support the Borrower to reduce repetition rates in primary education and increase enrollment in, and completion rates of, secondary education, all in the Borrower's rural areas." Three outcome targets were revised to be less ambitious at a 2018 restructuring. This validation will perform a split rating for the second and third objectives, for which the revision of targets had an effect on the original Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) versus revised outcome ratings. At the time of this restructuring, 53.3 percent of Bank funds had been disbursed. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 15-Nov-2018 c. Will a split evaluation be undertaken? Yes d. Components According to the Project Appraisal Document (PAD, p. 6), the project was to prioritize schools located in dispersed or remote rural areas, where nearly 40 percent of the population has at least one unmet basic need. The project included three components: 1. Reducing repetition rates in rural classrooms (appraisal: US$59 million; adjusted at 2018 restructuring: US$44 million; actual: US$41.23 million). This component had two subcomponents. The first, on improving teaching and school management, was to support the recently approved National Plan for Teacher Training by designing and implementing an in-service training strategy for all pre-primary, primary, and secondary rural schools for teachers, school directors, and supervisors. The second, on strengthening the link between schools and families, was to assist school management to work collaboratively with parents, former students, local technical experts, and neighbors, among others. Under this subcomponent, small funds were to finance such activities as sharing of school management and teachers' performance experiences at the level of the school cluster, conducting simple preventive maintenance of schools, and purchasing needed school equipment. Access to these funds was subject to the development of school improvement plans at each school. At the 2018 restructuring, three new activities were added: design and pilot implementation of a new national secondary school model that would revamp the curriculum to focus on competencies and project-based learning; training of school directors in use of a national standardized student assessment to improve learning in schools in vulnerable areas of the Province of Buenos Aires (PBA); and designing and piloting an intervention that combined intensive math tutoring with cognitive- behavioral therapy (CBT) to reduce dropout among at-risk students in PBA. The scope of the CBT pilot was reduced at a 2019 restructuring. 2. Increasing enrollment and completion of rural secondary schools (appraisal: US$180 million; unchanged at 2018 restructuring; actual: US$182.73 million). This component was to expand the supply of secondary education services in rural areas, including infrastructure for new schools, increased capacity for existing schools, school maintenance, learning materials, and the supply of qualified teachers. As a predecessor Bank-financed project had focused on primary and lower secondary education, this component was to tackle primarily remaining coverage gaps at the upper secondary level. To address Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) infrastructure shortages, a map of underserved areas was to be developed in order to locate demand and help determine the best service delivery models for each area. At the 2018 restructuring, the planned number of civil works was reduced significantly. 3. Strengthening project management, monitoring, and evaluation (appraisal: US$11.5 million; adjusted at 2018 restructuring: US$26.5 million; actual: US$17.84 million). This component was to support project management, which was to be based on existing structures within the national and provincial education ministries, and monitoring and evaluation (M&E), including studies and evaluations conducted by the National Directorate of Information and Evaluation of Educational Quality (DiNIECE). At the 2018 restructuring, new activities were added to support the use of standardized learning assessments (APRENDER) as diagnostic tools to improve the quality of learning; this intervention were particularly important to the project because rural areas had not previously been included in national assessments. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates The project was to be fully financed by an International Bank for Reconstruction and Development loan of US$250.5 million. Individual component costs were adjusted at the 2018 restructuring, with the scope of the first component's original activities reduced and that of the third component expanded. US$241.8 million was actually disbursed. US$4.77 million was cancelled, primarily because pandemic-related country-wide constraints on the government's ability to import goods made it impossible to use project funds to purchase netbooks (with digital connectivity) as planned. The remaining difference between original and actual costs was due to exchange rate fluctuations. The government contributed an unplanned US$11.2 million for netbooks and other computer equipment, nine infrastructure works that were not completed during the project period, and COVID-19-related infrastructure works. The project was approved on December 2, 2014, became effective on May 7, 2015, and closed on December 22, 2021. It was restructured five times:  November 14, 2018: Baseline data, which had originally been formulated using information as of 2012, were updated to 2014 values. Outcome targets were revised to align with the updated baselines and to take observed trends from 2014-2016 into account. Component costs were adjusted.  July 11, 2019: The closing date was extended by twelve months, from December 31, 2019 to December 31, 2020, to allow for completion of civil works and other activities.  November 19, 2020: The closing date was extended by an additional ten months, to October 31, 2021, to ensure completion of planned infrastructure works and to take into account delays and challenges caused by the COVID-19 pandemic and fluctuations in the exchange rate.  August 13, 2021: The closing date was extended a final time, to December 22, 2021, to allow for completion of procurement of netbooks that would ensure digital connectivity of schools during the pandemic. Because of import challenges, this equipment was eventually purchased with government funds.  April 27, 2022: Four months after closure, the government requested partial cancellation of the undisbursed balance of the project. Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) 3. Relevance of Objectives Rationale The project's objectives were highly relevant to country conditions. In the years leading up to project appraisal, Argentina had made important progress in the education sector. Investment per student was the highest in the Latin America and Caribbean Region in 2010, the education budget rose from 4.2 percent to 6.1 percent of gross domestic product from 2006 through 2012, and coverage rates at all education levels improved significantly. A Bank-financed First Rural Education Improvement Project (2005-2014) had supported a national rural education policy aimed at achieving universal education through context- appropriate approaches tailored to the needs of those living in rural areas, with focus at the primary education level. Despite progress in closing rural-urban gaps, however, inequalities between urban and rural areas persisted. These gaps began with the first grades of primary education, where repetition rates in rural areas at appraisal were about twice those in urban centers, and continued through the secondary level. In 2010, up to half of secondary students in rural areas were over-age for their grades, compared with a third in urban centers, and individuals in urban areas were twice as likely as their rural peers to have completed secondary school. Boys were disproportionately represented among repeaters in primary school and dropouts at the secondary level. Low secondary education coverage rates were attributable partly to factors of a structural nature beyond the boundaries of the education sector (relative deprivation of basic needs), but some factors were inherent to the education sector: insufficient supply of secondary institutions and qualified teachers, inadequate set-up of rural education environments, low incentives for parent and community involvement in education processes, and lack of adequate M&E. The objectives were also tightly aligned with government strategy. At the time of project appraisal, the government's main priority was to re-establish economic growth, which had decelerated after a decade of strong performance, while prioritizing equity and inclusion (ICR, p. 5). The government's National Education Plan at appraisal, covering 2012-2016, contained two lines of action: improve the quality of education services, and universalize education services with a special focus on the inclusion of marginalized groups. The strategy called for expanding the supply of secondary education services, pedagogical interventions, and learning materials to support students at risk of repeating early primary grades, also for an ambitious teacher training plan. Finally, the project was relevant to Bank strategy at appraisal through closing. The Bank's Country Partnership Strategy (CPS) with Argentina at appraisal (FY2015-18) had a results area on improving the employability of Argentina's youth and an outcome indicator on secondary school completion rates in rural areas, identical to one of this project's outcome indicators. The Country Partnership Framework at closing (CPF, FY2019-22) had a focus area to address key institutional constraints for better governance and service delivery, an objective to build better skills for the future, and an indicator on number of provinces implementing a national program to reduce dropouts in secondary schools. The Performance and Learning Review of the CPF added an objective to improve the human capital of vulnerable populations, further aligning the CPF with the project's focus on rural areas. Rating Relevance TBL Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Reduce repetition rates in primary education in rural areas Rationale The theory of change for this objective held that support for rural schools based on production and implementation of school improvement plans would lead to increased parent and community participation and engagement in student learning; that in-service training of rural teachers, school directors, and supervisors would lead to improved teaching and school management practices; and that provision of learning equipment and materials in rural schools would increase access to these materials. In turn, these changes were expected to contribute to reduced repetition rates for students at the primary level in rural areas; the project's theory of change did not specify intermediate steps between these outputs and outcomes. This theory of change rested on the assumption that fluctuations in macroeconomic variables would not significantly alter construction and materials costs. Outputs All rural schools developed school improvement plans, meeting the target. At least one teacher completed a training in 25.9 percent of rural schools, not reaching the target of 70 percent. The principals of 24.3 percent of rural schools completed a training, reaching neither the original target of 70 percent nor the revised target of 50 percent. The ICR (pp. 44-45) stated that there were 964,644 total project beneficiaries, not meeting the target of 1.14 million. Of those, 470,728 (48.8 percent) were female, falling slightly below the target of 50 percent. The target did not take into account migration to urban areas during the project period. The indicator measured only students who were currently enrolled in rural schools, not capturing students who transited through the rural education system during the project implementation period or the teachers, principals, and supervisors who also benefited from the project's interventions. The project team later added that in 2020 and 2021, as schools were reopened in rural areas following the COVID-19 pandemic, the project supported works to ensure access to safe water, sanitation, ventilation, and hygiene; different forms of digital connectivity to rural areas to support hybrid learning; and the printing and distribution of materials to support remote and hybrid learning strategies. Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) Outcomes The reported average repetition rate in primary schools in rural areas increased from 3.8 percent in 2014 (adjusted from the original 2012 baseline of 6.7 percent) to 4.1 percent in 2020, not reaching either the original target of four percent or the revised target of 3.5 percent. This result was likely influenced by a national regulation enacted in June 2012 that forbid repetition of first grade. That policy had the effect of immediately reducing repetition rates for first grade but increasing repetition rates for second through sixth grades over subsequent years, until overall primary repetition trends re-stabilized in 2017. As the ICR (p. 19) explains, the policy likely led to an inflated baseline in 2014, meaning that, although the outcome target was not achieved by 2020, actual movement on the indicator was likely not in the wrong direction. The project team later provided information for the 2021 school year showing that repetition rates had dropped to 2.1 percent, exceeding both the original and revised targets. Given trends prior to the pandemic and the likely influence of project-supported activities in 2020 and 2021, the positive reported outcome for 2021 is likely attributable, at least in part, to the project's interventions. Achievement of this objective is therefore rated Substantial. Rating Substantial OBJECTIVE 2 Objective Increase enrollment in secondary education in rural areas (original outcome target) Rationale In addition to the logic guiding the first objective, the theory of change for this objective held that building of new schools and refurbishing of existing schools in rural areas, as well as undertaking of minor civil works following the COVID-19 outbreak to comply with safe school protocols, would increase access and improve the physical environment for learning for rural secondary education. In turn, these changes would lead to increased enrollment in rural secondary education institutions. This theory of change rested on the additional assumptions that eligible students would be stimulated to enroll in secondary education if there were improvements in the learning environment, and that increased access to secondary education would result in higher numbers of enrolled students despite declining trends in the overall rural population. Outputs 183 schools were built or refurbished, not meeting the original target of 360 schools, but exceeding the revised target of 160 schools. Civil works were completed in 139 secondary schools, not meeting the original target of 360 schools, but exceeding the revised target of 110 schools. Investments were geographically allocated according to eligibility criteria that covered unsatisfied demand, poverty levels, and the proportion of the population with unsatisfied basic needs. During the COVID-19 pandemic, 218 schools received upgrades to ensure access to safe water, sanitation, ventilation, and hygiene through a project-supported COVID-19 Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) Infrastructure Fund; materials were printed and distributed to support remote learning; and different modes of digital connectivity were supported for rural areas (the ICR did not provide detail on this support). Between 2015 and 2019, 5,190 locally based productive school projects (PBLs) that incorporated project- based learning with engagement of local specialists were carried out; none were conducted in 2020 or 2021. At project completion, 55.5 percent of rural schools offered lower secondary education with a PBL, not reaching the target of 90 percent of schools. Outcomes The number of students entering the first year of secondary school in rural areas as a proportion of students who finished primary school in rural areas increased from 74.4 percent in 2014 (adjusted from the original 2012 baseline of 72.1 percent) to 77.7 percent in 2020, not meeting the original target of 85.6 percent. The ICR presented an analysis showing that transition rate trends from primary to secondary education in rural areas were consistently positive from 2012 through 2019, prior to the COVID-19 pandemic, and that achievement for this indicator would have reached 80.2 percent in 2020 had these trends continued. This adjusted result still does not meet the original outcome target. The project team later provided information that the actual transition rate in school year 2021 was 79.7 percent. Because the original outcome target was not met, achievement under the original target for this objective is rated Modest. Rating Modest OBJECTIVE 2 REVISION 1 Revised Objective Increase enrollment in secondary education in rural areas (revised outcome target) Revised Rationale The number of students entering the first year of secondary school in rural areas as a proportion of students who finished primary school in rural areas increased from 74.4 percent in 2014 (adjusted from the original 2012 baseline of 72.1 percent) to 77.7 percent in 2020, not meeting the revised target of 80 percent. However, the adjusted achievement of 80.2 percent in 2020 meets the target, as does the actual achievement for school year 2021 (79.7 percent). Achievement of this objective under the revised outcome target is therefore rated Substantial. Revised Rating Substantial Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) OBJECTIVE 3 Objective Increase completion rates of secondary education in rural areas Rationale In addition to the logic driving the first two objectives, the theory of change for this objective held that carrying out studies on issues regarding rural education and mapping underserved areas to identify demand, as well as supporting standardized assessments of learning outcomes in rural areas, would lead to informed decision-making based on evidence from adequate M&E services. With coverage gaps concentrated at the upper secondary level (PAD, p. 8), a focus on infrastructure improvements at that level would facilitate secondary education completion as students moved from lower to upper secondary. In addition, a pilot intervention to reduce dropping out from secondary school in PBA would enhance understanding of the factors leading to student dropout. Together, these changes would contribute to increased completion rates in rural secondary education institutions. Outputs Pedagogical materials were prepared for lower and upper secondary education, as planned. The number of upper secondary schools as a proportion of the number of lower secondary schools increased from 61 in 2015 to 75 in 2020 (2,702 upper secondary and 3,600 lower secondary), meeting the target. The project supported the implementation of APRENDER, which had first been carried out in 2016, at the primary level in 2018 and the secondary level in 2019. One national report, 24 provincial-level reports, 20,819 school-level reports, and several technical documents were produced on the basis of these assessments, and a report on student learning in rural primary schools was prepared. The percentage of rural schools benefiting from the use of APRENDER as a diagnostic tool increased from 22.4 percent in 2017 to 82.9 percent in 2020, exceeding the target of 60 percent of schools. The percentage of rural schools in which the principal accessed school reports for each edition of APRENDER increased from 23 percent in 2014 to 56.3 percent in 2020, not reaching the target of 80 percent; neither the indicator nor the ICR measured what principals actually did with the accessed information. 2019 school reports were postponed due to the prioritization of a 2020 educational continuity assessment carried out in a sample of schools. Didactic materials were designed for a CBT pilot, as planned, and the pilot was implemented in 29 schools, fewer than the 100 schools that were initially planned. Outcomes The percentage of the total number of students enrolled in the first grade of upper secondary education who completed the last year of upper secondary education (twelfth grade) in rural areas, including students who repeat, increased from 63.3 percent in 2014 (adjusted from the original 2012 baseline of 63.4 percent) to 68.3 percent in 2020, not reaching the original target of 72 percent. The project team later added that the twelfth- Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) grade completion rate for school year 2021 was 74.1 percent, exceeding both the original and revised (68 percent) targets. The number of students in rural areas enrolled in tenth grade as a proportion of students who completed ninth grade in rural areas increased from 91.6 percent in 2014 (adjusted from the original 2012 baseline of 92.1 percent) to 95.2 percent in 2020, almost reaching the original target of 96 percent (which remained unchanged).The project team later added that the ninth-to-tenth grade transition rate for school year 2021 was 96.5 percent, again meeting the target. With both outcome targets met, achievement of this objective is rated Substantial. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale With achievement of two objectives rated Substantial and one rated Modest under the original outcome targets, efficacy under the original targets is rated Substantial. Overall Efficacy Rating Substantial OBJR1_TBL OVERALL EFFICACY REVISION 1 Overall Efficacy Revision 1 Rationale With achievement of all three objectives rated Substantial under the revised outcome targets, efficacy under the revised targets is rated Substantial. Overall Efficacy Revision 1 Rating Substantial 5. Efficiency The PAD's careful and well documented economic and financial analysis (Annex 6, pp. 57-63) found that project-supported increases in upper secondary school completion rates in rural areas would raise the incomes of rural households by 1.2 percent per year. A lower-bound estimate found economic benefits to 2035 of Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) US$643 million, excluding broader benefits to improved labor productivity, higher female labor participation rates, better health, and enhanced democratic participation. The ICR (Annex 7, pp. 75-78) updated that analysis using a similar methodology but also calculating the net present value (NPV) and internal rate of return (IRR). It found, under a conservative scenario, an NPV of US$3.13 million and an IRR of 5.1 percent, noting that "rural education projects are far costlier per beneficiary than urban education projects" (p. 25). It also noted that the benefits used in the estimate were a lower bound, as a range of potential social and economic benefits were not included. Institutional and implementation arrangements built on those established under the predecessor project. The project adapted nimbly during the COVID-19 pandemic by financing infrastructure work to support student safety and virtual teacher training. The average construction cost per square meter of new schools was US$891, lower than the approximately U$1,000 under the predecessor project. Nonetheless, significantly fewer schools than planned were built or refurbished under the second component, even though spending remained at originally planned levels. The ICR (p. 31) noted that key decisions like the number and location of school refurbishments and other infrastructure works were not taken until well into the project implementation period, reducing efficiency. Furthermore, as the ICR (p. 26) noted, there were long delays between preparation (October 2012) and approval (December 2014) due to credit restrictions to the country that stretched the timeline for appraisal and negotiation; significant implementation delays (due in part to changes in government administration); and construction bans and school closures during the initial COVID-19 outbreak. The ICR did not clarify how project-supported activities in the urban areas of the PBA, added at the 2018 restructuring, contributed to the project's focus on rural areas. The project team later added that these pilot activities covered both rural and urban school settings, that special attention was paid to opportunities for learning specifically about rural schools, and that the decision to cover both types of schools under the pilot was made for reasons of administrative efficiency. However, to the extent that these activities covered urban schools, the project spent funds on activities that were not aligned with its objectives and the theory of change. Overall, efficiency is rated Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 100.00 ICR Estimate  5.10  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) 6. Outcome Under both the original and revised outcome targets, the project had high relevance, substantial efficacy, and modest efficiency. Overall outcome is rated Moderately Satisfactory, representing moderate shortcomings in the project's preparation, implementation, and achievement. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The ICR (p. 28) noted that some of the staff trained in provincial education ministries have been permanently hired by the ministries to continue support for rural education, and experienced project implementation staff continue to work in the central Ministry of Education, supporting continuity of achieved outcomes. The National Institute for Teacher Training will continue to support training activities. Infrastructure works that were not completed under the project were financed using national funds, demonstrating the commitment of the national government to the project's objectives. However, national elections are scheduled for October 2023, and a new government's priorities may differ. In addition, there are risks related to the maintenance of infrastructure in rural communities. A follow-on Bank-financed Program for Results operation, Improving Inclusion in Secondary and Higher Education (2019-2023, US$341 million), supports efforts to reduce dropout rates in basic and higher education among vulnerable populations. Other donors are also committed to continued progress in rural education (ICR, p. 39). 8. Assessment of Bank Performance a. Quality-at-Entry The Bank team worked closely with the government to design the project, which had clear, relevant objectives. Key lessons were learned from the predecessor project, including that context-appropriate service delivery models are required at the provincial level in order to pre-empt misconceptions about the adequacy of those models; strong and accessible support for teachers is critical to ensuring their motivation and performance; a focus on province-level goals and objectives is key to innovation and success; and greater attention is required to teaching Spanish as a second language in indigenous schools, as Spanish-language skill is most likely to determine the future trajectories of students (PAD, pp. 10-11). Overall risk was assessed as moderate, with particular emphasis on capacity risks stemming from the complex operational architecture inherent in the project's federated nature, as well as weak capacity and high rotation of staff at the provincial level. Mitigation measures included the placement of training teams at the subnational level and hiring of new staff to strengthen the capacity of the Project Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) Implementation Unit's (PIU's) financial management team (PAD, p. 12 and Annex 4, pp. 50-53). The PAD included a detailed implementation support plan (pp. 54-56). However, there were significant shortcomings in project design. Infrastructure planning was inadequate (see Sections 5 and 9a), specifically for the location and costing of planned works. Deficiencies in infrastructure planning had added significance, as infrastructure was a core aspect of the project, and two-thirds of project funds were allocated to civil works (ICR, p. 37). The ICR (p. 31) stated that it was unknown where infrastructure works would be carried out, how many new schools would be built, and how many schools would be refurbished. Initial indicators, baselines, and targets were not realistic: "Using 2012 data on a project that became effective in May of 2015 generated problems in setting realistic targets" (ICR, p. 38). Although, as the ICR explained, these baselines and targets were set using education system reporting that took place on one- to two-year delays, the project could have sought alternative sources of more timely data in order to avoid costly later restructurings. Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision Supervision missions were undertaken every six months, and Implementation Status and Results Reports were timely and complete. The Bank team processed restructurings as needed in response to new data and changes in circumstances, and it was particularly nimble in adjusting to external shocks such as a currency devaluation and the COVID-19 outbreak. Although there were five different task team leaders during the project's lifetime, the ICR (p. 38) noted that continuity was ensured by having periods of overlap in each case. Procurement, financial management, and safeguards were adequately supervised. However, the restructuring to accommodate updated baseline data took place relatively late in the project period. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design Project results monitoring was intended to strengthen national systems wherever possible. Outcome indicators were already tracked and reported on annually by DiNIECE. The project was to complement official system-wide information with just-in-time studies. An evaluation plan was to be developed under the project's third component. The Bank was also to maintain a robust technical cooperation program with Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) provinces and non-governmental actors such as academics, think tanks, and non-governmental organizations (NGOs) The results framework was clear and straightforward, though there were shortcomings. The project's theory of change lacked some intermediate steps; for example, it was assumed that teacher and principal training would lead directly to improved student enrollment and retention, without indicators measuring the quality of that training or the extent to which it was applied in classrooms. In addition, the ICR did not explain why the setting of outcome indicators and targets did not take several predictable exogenous factors into account, including trends in migration from rural to urban areas and the 2012 national policy prohibiting repetition of first grade. Lack of clarity during preparation on the number and location of schools to be built and refurbished, as well as their estimated costs, prevented the adequate setting of targets. b. M&E Implementation Project monitoring reports were created every six months using regular administrative data from the Ministry of Education. The ICR (p. 34) noted that data collection and processing was not always timely due to delays in processing the ministry's annual report. None of the planned impact evaluations were conducted. c. M&E Utilization According to the ICR (p. 34), M&E data were monitored and updated regularly by the PIU, allowing the Bank team to review progress toward achievement of objectives. The ICR did not provide further information on use of M&E results to adjust project implementation, though it noted that "the continued lack of timely and updated data complicated the evaluation of the interventions financed with project funds." Although the ICR was able to present sufficient data to assess achievement status of planned outcomes, project M&E experienced significant shortcomings in design and during implementation. M&E quality is therefore rated Modest. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified under Environmental Assessment category "B" and triggered two safeguard policies: OP/BP 4.01 Environmental Assessment and OP/BP 4.10 Indigenous Peoples. OP/BP 4.04 Natural Habitats and OP/BP 4.11 Physical Cultural Resources were preventively activated. The project's physical interventions included construction, rehabilitation, and refurbishing of schools. Potential adverse impacts were associated with the construction stage of civil works of small to medium Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) scale. Environmental impacts were therefore expected to be neither significant nor irreversible. However, given the targeting of rural areas and the prioritization of locations in the most dispersed or rural remote areas, it was considered possible that works would be located in natural habitat zones or areas with cultural values. An Environmental and Social Management Framework (ESMF) was developed by the Ministry of Education in consultation with education community representatives, national environmental authorities, and the NGO Rural Community Network. The framework was disclosed as required. During project implementation, the management of environmental safeguards was downgraded because of lack of institutional capacity to manage the ESMF and asbestos problems found in three schools; after implementation of an action plan to address these issues, the environmental management rating was upgraded to moderately satisfactory, where it remained until closure. The education ministry maintains a permanent dialogue process with representatives of indigenous peoples from each Argentine province. This process led to the creation of the Autonomous Indigenous People's Education Council (CEAPI) in December 2007. CEAPI examined project documents in 2013 and 2014, leading to key adjustments to project design: strengthened links between schools and families to foster local participation in school improvement plans; changes in the design of those plans to adjust to the context of each school, taking into account inputs from students, teachers, and other members of local communities; and enhancement of indigenous teacher training. An Indigenous Peoples Planning Framework (IPPF), incorporating feedback from CEAPI, was finalized and disclosed. After initial challenges with preparation and implementation of IPPFs at different levels in each province, including their limited one-year original time frames, implementation of IPPF-related activities was smooth and was rated moderately satisfactory from February 2017 through closure. The ICR (p. 35) stated that the natural habitats and physical cultural resources safeguards policies were not triggered due to lack of need. b. Fiduciary Compliance Procurement and financial management arrangements were adequate, with ratings of satisfactory or moderately satisfactory throughout implementation. PIU staff had prior experience with Bank-financed projects. The last procurement review found issues with lack of documentation, some instances of low contracting capacity with provincial units, and an issue with the standard document for contracting of works (it included a clause not acceptable to the Bank). These problems were rectified through training sessions and amendment of documents. Throughout the project's lifetime, there were "significant delays" in updating contract execution information in the Systematic Tracking of Exchanges in Procurement (STEP) system, which delayed procurement reviews (ICR, p. 36). Financial management shortcomings included inadequate documentation of retroactive expenses, especially for transfers to rural schools; this issue persisted through project closing. External audits were found acceptable by the Bank. c. Unintended impacts (Positive or Negative) None reported. Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Shortcomings in M&E design, implementation, timeliness of Quality of M&E Substantial Modest reporting, and use of data and analysis. Quality of ICR --- Substantial 12. Lessons The ICR (pp. 40-41) presented a series of insightful lessons, including:  Accurate and timely baseline data are essential to good target-setting. In this case, reliance on out-of-date baselines necessitated revisions of indicators and downscaling of outcome targets relatively late in the project period, with negative implications for the project's outcome rating.  Aligning a project with overarching strategic priorities in a sector can help it withstand shifts in political economy over its lifetime. In this case, the project supported high-level government priorities to enhance both access to secondary education and all dimensions of rural education. This congruence helped the project's core priorities remain relevant through three changes of government, five different education ministers, and internal unit reorganizations in the education ministry.  Although some exogenous factors affecting the cost of works cannot be foreseen, the impact of variables related to geographic location, scope, and price variation can be minimized through exhaustive prior identification and planning. In this case, significant modification of goals related to infrastructure was required because of inadequate prior planning, including estimates, as well as major currency devaluation and inflation. 13. Assessment Recommended? No 14. Comments on Quality of ICR Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review AR Second Rural Education Improvement (P133195) The ICR followed established guidelines. It carefully outlined the assumptions underlying the project's theory of change and included a literature review supporting the project's basic logic (Annex 5, pp. 69-71). Its discussion of the exogenous factors affecting some observed outcomes was detailed and carefully sourced. It provided useful annexes on project restructurings. Much of its discussion was clear and candid. However, it did not adequately explain several of the project's activities, including the locally based productive school projects and the national standardized student assessments. It was lengthy, at over 35 pages of main text, with some repetition. a. Quality of ICR Rating Substantial Page 16 of 16